Commit Graph

264 Commits

Author SHA1 Message Date
Simon
d8a165ac6f fix #2450 (#2493)
* fix #1441

* support custom datax config

* support datax custom config

* support datax custom config

* support datax custom config

* fix #2450

Co-authored-by: 张世鸣 <zhangshiming@cvte.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
2020-06-29 17:38:53 +08:00
songgg
b06c118af6 [optimization]:prevent repeated database updates (#2396)
* sqlTask failed to run

* prevent repeated database updates

* prevent repeated database updates

* prevent repeated database updates

Co-authored-by: songqh <songquanhe@foxmail.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
2020-06-29 17:38:37 +08:00
qiaozhanwei
eecf992432 update UT in pom.xml 2020-06-29 17:38:29 +08:00
BoYiZhang
bcdf58969c Fix document errors (#2382)
* Remove invalid code

* fix document errors

Co-authored-by: zhanglong <zhanglong@ysstech.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
2020-06-29 17:38:20 +08:00
songgg
0ad610a0f0 [optimization]:remove duplicate code (#2377)
* sqlTask failed to run

* remove duplicate code

* remove duplicate code

* remove duplicate code

* remove duplicate code

* remove duplicate code

Co-authored-by: songqh <songquanhe@foxmail.com>
2020-06-29 17:36:42 +08:00
itbasketplayer
d03f9daa0e .gitignore remove taskInstance/index.vue(#2300) and no need to findProcessDefineById 2 times(#2373) (#2374)
* some optimization

* revert

* ProcessDefinitionService#updateProcessDefinition findProcessDefineById 1 time
2020-06-29 17:36:42 +08:00
BoYiZhang
7fa0c0f79a Remove invalid code (#2342)
Co-authored-by: zhanglong <zhanglong@ysstech.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
2020-06-29 17:36:41 +08:00
songgg
d8c7a44e82 some annotation optimizations (#2333)
Co-authored-by: songqh <songquanhe@foxmail.com>
2020-06-29 17:36:38 +08:00
Rubik-W
668b86f12b fix:Optimize content returned by interface (#2294) 2020-06-29 17:36:37 +08:00
lgcareer
82d6dc9e52 new feature for #404 add resource tree function (#2323)
* add create resource directory

* add create resource directory

* update the resource test

* add upgrade sql in version 1.2.2

* Adding request parameter id to update queryResourceListPaging

* set isDirectory value is false default

* add full name to update updateResource

* remove request parameter isDirectory to update createResource method

* update queryResourceListPaging with change get to post

* update updateResource method with remove fullName

* File management list modification (#1976)

* add resource component

* add resource tree visitor

* return json string

* update queryResourceList

* upload file need fullName

* add method rootNode

* Shell task resources and authorization resources (#1989)

* File management list modification

* Shell task resources and authorization resources

* download resource when execute task

* download resource when execute task

* update authorization type

* download resource when execute task

* Spark task resource changes (#1990)

* File management list modification

* Shell task resources and authorization resources

* Spark task resource changes

* download resource when execute task

* update udf function service

* add resource type in ResourceComponent

* UDF resource tree and change DAG style (#2019)

* File management list modification

* Shell task resources and authorization resources

* Spark task resource changes

* UDF resource tree and change DAG style

* add deleteIds method in ResourceMapper and ResourceMapperTest

* Add comments on class and method

* add queryResourceByName method in controller

* update verify-name with change name to full name

* update queryResource with add parameter pid

* update queryResource with add parameter pid

* add resource ids in process definition and delete resource need judge whether it is used by any process definition

* Breadcrumb development (#2033)

* File management list modification

* Shell task resources and authorization resources

* Spark task resource changes

* UDF resource tree and change DAG style

* Breadcrumb development

* Breadcrumb development

* Resource tree bug fix (#2040)

* File management list modification

* Shell task resources and authorization resources

* Spark task resource changes

* UDF resource tree and change DAG style

* Breadcrumb development

* Breadcrumb development

* Resource tree bug fix

* update resource service test

* Fix github action rerun failed

* add status of PARENT_RESOURCE_NOT_EXIST

* Fix github action rerun failed (#2067)

* update resource service test

* Fix github action rerun failed

* add status of PARENT_RESOURCE_NOT_EXIST

* Change crumb position

* Change crumb position (#2068)

* build resource process definition map

* UDF changed to multiple choice

* UDF changed to multiple choice (#2077)

* Change crumb position

* UDF changed to multiple choice

* build resource process definition map (#2076)

* update resource service test

* Fix github action rerun failed

* add status of PARENT_RESOURCE_NOT_EXIST

* build resource process definition map

* update resource name also need update all the children full name

* need add queryResource

* update resource name also need update all the children full name (#2096)

* update resource service test

* Fix github action rerun failed

* add status of PARENT_RESOURCE_NOT_EXIST

* build resource process definition map

* update resource name also need update all the children full name

* need add queryResource

* Limit customization file content to no more than 3000 lines

* Limit customization file content to no more than 3000 lines(#2128)

* Limit customization file content to no more than 3000 lines(#2128) (#2140)

* Change crumb position

* UDF changed to multiple choice

* Limit customization file content to no more than 3000 lines

* Limit customization file content to no more than 3000 lines(#2128)

* add queryResourceJarList

* add queryResourceJarList

* add queryResourceJarList

* add queryResourceJarList (#2192)

* update resource service test

* Fix github action rerun failed

* add status of PARENT_RESOURCE_NOT_EXIST

* build resource process definition map

* update resource name also need update all the children full name

* need add queryResource

* add queryResourceJarList

* add queryResourceJarList

* add queryResourceJarList

* Modify the main jar package

* Modify the main jar package (#2200)

* Change crumb position

* UDF changed to multiple choice

* Limit customization file content to no more than 3000 lines

* Limit customization file content to no more than 3000 lines(#2128)

* Modify the main jar package

* add resource filter in order to get filtered resource

* add comments of resource filter

* update list children by resource

* choose main jar with resource tree (#2220)

* update resource service test

* Fix github action rerun failed

* add status of PARENT_RESOURCE_NOT_EXIST

* build resource process definition map

* update resource name also need update all the children full name

* need add queryResource

* add queryResourceJarList

* add queryResourceJarList

* add queryResourceJarList

* add resource filter in order to get filtered resource

* add comments of resource filter

* update list children by resource

* Return null if query resource list is empty

* update queryResource method change parameter pid to id

* getResouDelete checksum and modify parameter namerceId

* revert .env

* remove parameter pid

* Delete request interface

* go back to the last page

* jar interface call

* Fix issue #2234 and #2228

* change resource name with full name

* Fix issue #2234 and #2228 (#2246)

* update resource service test

* Fix github action rerun failed

* add status of PARENT_RESOURCE_NOT_EXIST

* build resource process definition map

* update resource name also need update all the children full name

* need add queryResource

* add queryResourceJarList

* add queryResourceJarList

* add queryResourceJarList

* add resource filter in order to get filtered resource

* add comments of resource filter

* update list children by resource

* Return null if query resource list is empty

* update queryResource method change parameter pid to id

* revert .env

* remove parameter pid

* Fix issue #2234 and #2228

* change resource name with full name

* Fix list query value error

* remove unauth-file with authorize-resource-tree

* Repair data cannot be echoed

* Repair data cannot be echoed

* execute mr and spark task need query resource name before

* Authorized resource interface replacement

* Authorized resource interface replacement

* Filter UDF resources

* Change parameters

* need query all authorized directory children when create task

* Change normalize.scss import method and animation.scss license modification

* Delete file list update processing

* It's fixed that resource not deleted in hdfs when delete it.

* add tooltips

* add tooltips (#2310)

* Echo workflow name and modify udf management name

* [new feature]add resource tree function

* revert front code in order to be same as dev branch

* revert front code in order to be same as dev branch

* revert common.properties and application.properties

* add super method

* update flink parameter test

* update flink parameter and unit test

* update resource service test

* If resource list is empty,need init it

* update flink parameter test

Co-authored-by: break60 <790061044@qq.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>
2020-06-29 17:36:36 +08:00
gabry.wu
f9c91b8cbe Adapting partial code(file name start with S #2) to the sonar cloud rule (#2270) 2020-06-29 17:36:36 +08:00
gabry.wu
30b98b89a2 Adapting partial code(file name start with T) to the sonar cloud rule (#2271) 2020-06-29 17:36:33 +08:00
gabry.wu
a1f7614476 Adapting partial code(file name start with W) to the sonar cloud rule (#2287) 2020-06-29 17:36:33 +08:00
WeiYe
6cca69369b Update test code logger class name (#2303) 2020-06-29 17:36:33 +08:00
gabry.wu
afa9a6467a Adapting partial code(file name start with Q) to the sonar cloud rule (#2242) 2020-06-29 17:36:29 +08:00
gabry.wu
e1771de45a Adapting partial code(file name start with R) to the sonar cloud rule (#2243) 2020-06-29 17:36:29 +08:00
jin
24fa31939b feat:optimizing code (#2245)
* feat:optimizing code

* add licence and  format code
2020-06-29 17:36:29 +08:00
Simon
5c30205574 support custom datax configuration (#2224)
* fix #1441

* support custom datax config

* support datax custom config

* support datax custom config

* support datax custom config

Co-authored-by: 张世鸣 <zhangshiming@cvte.com>
2020-06-29 17:36:29 +08:00
Rubik-W
9fe04524ae fix: Api server startup fail (#2198) 2020-06-29 17:36:28 +08:00
t1mon
55cc1c7c5b Replace Charset.forName("UTF-8") with StandardCharsets.UTF_8. (#2193) 2020-06-29 17:36:28 +08:00
gabry.wu
7f7f50a828 Adapting partial code(file name start with L) to the sonar cloud rule (#2171) 2020-06-29 17:36:28 +08:00
gabry.wu
610c95a470 use static method in static class JSON (#2156) 2020-06-29 17:36:27 +08:00
tswstarplanet
89c0a796f9 fix field access modifier (#2150) 2020-06-29 17:36:22 +08:00
gabry.wu
6a8cef08e4 Adapting partial code(file name start with E) to the sonar cloud rule (#1999)
* Adapting partial code(file name start with E) to the sonar cloud rule

* remove isEmpty invoke

* resolve conflicts
2020-06-29 17:36:13 +08:00
Tboy
7ed28af4fe
Upgrade curator version (#2083)
* upgrade curator version
issue: https://github.com/apache/incubator-dolphinscheduler/issues/2020
curator issue: https://github.com/apache/curator/pull/297

* add DefaultEnsembleProvider override

* add some unit test

* add License
2020-03-05 19:00:06 +08:00
qiaozhanwei
d5c795bd45
fix ugly logger usage (#2063)
* fix ugly logger usage

* remove space
2020-03-05 11:10:48 +08:00
qiaozhanwei
3e5a939913
delete unused import (#2066) 2020-03-05 11:07:44 +08:00
qiaozhanwei
b8014d91cf
add workflow id, name for email content (#2071)
* add tenantService UT

* fix spell

* #464 email content

* modify mysql1.2.2 ddl
2020-03-05 11:04:00 +08:00
Rubik
1ea48375f4
Fixbug i18n #2008 (#2035)
* fix:add chinese message

* fix:add chinese message

* fix:'getLocale()' can return not null

* fix:optimize message content
2020-03-02 10:29:42 +08:00
Yelli
84409b57ee
fix taskinstance.isSubprocess misjudgment (#2007)
* add processInstanceService UT

* fix taskInstance.isSubProcess
add processInstanceService UT

* add taskInstance.isSubProcess UT

* modify taskInstance.isSubProcess UT

* add license

* remove author&date
2020-02-28 16:15:21 +08:00
gabry.wu
5ca4862ea9
remove String.format for org.slf4j.Logger (#2001) 2020-02-25 14:19:26 +08:00
Yelli
619fb30c50
#1310 add executor user to the process instance & task instance pages (#1973)
* add executor name in processDefinitionInstance & taskInstance

* modify processInstanceTest

* modify processInstanceTest

* #1310 add executor user to the process instance & task instance pages

* add processInstanceUT & taskInstanceUT

* add processInstanceUT & taskInstanceUT

* modify processInstanceTest & taskInstanceTest

* Remove duplicate code

* add userServiceUT
2020-02-20 13:25:41 +08:00
dailidong
24f22974a2
solve jar conflict, delete servlet-api 2.5 jar (#1966)
* update README about DolphinScheduler

* Update issue templates

* update

* regularize api pom xml
update rpc maven compile to 1.8

* regularize api pom xml

* change commons.lang3.StringUtils to common.utils.StringUtils

* update pom.xml

* update

* correct equals method

* jasper-runtime is needed when api server start

* jasper-runtime jar is needed when api server start

* combine logback config of master/worker/alert/api server to one logback.xml

* remove tomcat runtime jar

* add UT

* add license

* remove jasper-runtime jar, not need anymore

* sovle jar conflict, remove servlet-api 2.5 jar

* remove servlet 2.5

Co-authored-by: DS <escheduler@outlook.com>
2020-02-17 10:16:40 +08:00
qiaozhanwei
9b75bebf1b
1,add sendAsync method 2,refactor LoggerClient (#1965)
* 1,remove dolphinscheduler-rpc module  2,add dolphinscheduler-remote module 3,add dolphinscheduler-service module 4,refactor LoggerServer module (#1925)

* 1,remove dolphinscheduler-rpc module
2,add dolphinscheduler-remote module
3,add dolphinscheduler-service module
4,refactor LoggerServer module

* ProcessUtils modify

* Refactor architecture (#1926)

* move version to parent pom

* move version properties to parent pom for easy management

* remove freemarker dependency

* delete CombinedApplicationServer

* #1871 correct spelling

* #1873 some updates for TaskQueueZkImpl

* #1875 remove unused properties in pom

* #1878
1. remove tomcat dependency
2. remove combined_logback.xml in api module
3. format pom.xml for not aligning

* #1885 fix api server startup failure
1. add jsp-2.1 dependency
2. remove jasper-runtime dependency

* add stringutils ut (#1921)

* add stringutils ut

* Newfeature for #1675. (#1908)

Continue to finish the rest works, add the cache feature for dependence,mr,python,sub_process,procedure and shell.

* Add modify user name for process definition (#1919)

* class overrides equals() and should therefore also override hashCode()

* #1862 add modify user in process difinition list

* #1862 add pg-1.2.2 ddl.sql

* modify ScriptRunnerTest

* add updateProessDifinition UT

* modify updateProcessDifinition UT

* modify updateProcessDifinition UT

* modify mysql 1.2.2 ddl.sql&dml.sql

* add scope test to mysql in pom

* modify pg-1.2.2 ddl.sql

* refactor module

* updates

Co-authored-by: khadgarmage <khadgar.mage@outlook.com>
Co-authored-by: zhukai <boness@qq.com>
Co-authored-by: Yelli <amarantine@my.com>

* dolphinscheduler-common remove spring (#1931)

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* SpringApplicationContext class title add license (#1932)

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* add license (#1934)

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* move datasource classes to dao module

* fix send4LetterWord bug

* Refactor architecture (#1936)

* move datasource classes to dao module

* fix send4LetterWord bug

* exclude jasper-compiler in case of runtime conflict

* LoggerServiceTest remove ProcessDao (#1944)

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* LoggerServiceTest remove ProcessDao

* exclude jasper-compiler in case of runtime conflict (#1938)

* move datasource classes to dao module

* fix send4LetterWord bug

* exclude jasper-compiler in case of runtime conflict

* DataAnaylysisServiceTest and ProcessDefinitionService modify

* remote module add comment

* OSUtilsTest modify

* add finally block to close channel

* add finally block to close channel (#1951)

* move datasource classes to dao module

* fix send4LetterWord bug

* exclude jasper-compiler in case of runtime conflict

* add finally block to close channel

* refactor log client service

* add sendSync method

* 1,quartz.properties add conf category
2,dolphinscheduler-daemon.sh modify

* dolphinscheduler-binary.xml modify

* add sendAsync method (#1962)

* add sendAsync method

* add sendAsync method

* 1,add sendAsync method
2,refactor LoggerClient

* LogClientService modify

Co-authored-by: Tboy <technoboy@yeah.net>
Co-authored-by: khadgarmage <khadgar.mage@outlook.com>
Co-authored-by: zhukai <boness@qq.com>
Co-authored-by: Yelli <amarantine@my.com>
2020-02-16 19:12:01 +08:00
gabry.wu
99ac739851
rename vaild to valid (#1961) 2020-02-15 18:55:16 +08:00
Yeleights
87cb33b233 deal with conflict 2020-02-14 22:46:45 +08:00
qiaozhanwei
c1ee133392
Refactor Architecture Basic modification #1658 (#1946)
* 1,remove dolphinscheduler-rpc module  2,add dolphinscheduler-remote module 3,add dolphinscheduler-service module 4,refactor LoggerServer module (#1925)

* 1,remove dolphinscheduler-rpc module
2,add dolphinscheduler-remote module
3,add dolphinscheduler-service module
4,refactor LoggerServer module

* ProcessUtils modify

* Refactor architecture (#1926)

* move version to parent pom

* move version properties to parent pom for easy management

* remove freemarker dependency

* delete CombinedApplicationServer

* #1871 correct spelling

* #1873 some updates for TaskQueueZkImpl

* #1875 remove unused properties in pom

* #1878
1. remove tomcat dependency
2. remove combined_logback.xml in api module
3. format pom.xml for not aligning

* #1885 fix api server startup failure
1. add jsp-2.1 dependency
2. remove jasper-runtime dependency

* add stringutils ut (#1921)

* add stringutils ut

* Newfeature for #1675. (#1908)

Continue to finish the rest works, add the cache feature for dependence,mr,python,sub_process,procedure and shell.

* Add modify user name for process definition (#1919)

* class overrides equals() and should therefore also override hashCode()

* #1862 add modify user in process difinition list

* #1862 add pg-1.2.2 ddl.sql

* modify ScriptRunnerTest

* add updateProessDifinition UT

* modify updateProcessDifinition UT

* modify updateProcessDifinition UT

* modify mysql 1.2.2 ddl.sql&dml.sql

* add scope test to mysql in pom

* modify pg-1.2.2 ddl.sql

* refactor module

* updates

Co-authored-by: khadgarmage <khadgar.mage@outlook.com>
Co-authored-by: zhukai <boness@qq.com>
Co-authored-by: Yelli <amarantine@my.com>

* dolphinscheduler-common remove spring (#1931)

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* SpringApplicationContext class title add license (#1932)

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* add license (#1934)

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* Refactor architecture (#1936)

* move datasource classes to dao module

* fix send4LetterWord bug

* LoggerServiceTest remove ProcessDao (#1944)

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* LoggerServiceTest remove ProcessDao

* exclude jasper-compiler in case of runtime conflict (#1938)

* move datasource classes to dao module

* fix send4LetterWord bug

* exclude jasper-compiler in case of runtime conflict

* DataAnaylysisServiceTest and ProcessDefinitionService modify

* remote module add comment

* OSUtilsTest modify

* add finally block to close channel (#1951)

* move datasource classes to dao module

* fix send4LetterWord bug

* exclude jasper-compiler in case of runtime conflict

* add finally block to close channel

Co-authored-by: Tboy <technoboy@yeah.net>
Co-authored-by: khadgarmage <khadgar.mage@outlook.com>
Co-authored-by: zhukai <boness@qq.com>
Co-authored-by: Yelli <amarantine@my.com>
2020-02-14 22:00:20 +08:00
Jave-Chen
cdd53a7673
Fix socket bug: read before sock connect. Add UT for FourLetterWordMain. (#1954)
* Fix socket bug: read before sock connect. Add UT for FourLetterWordMain.

* Fix code smell

   Remove  @Ignore and  @Test from testCmd().

* correct comment
2020-02-14 10:39:02 +08:00
Yeleights
6c90575a39 add process null check
Remove duplicate code in processDefinitionService
add processDefinitionService UT
2020-02-13 15:39:30 +08:00
gabry.wu
44b76468a8
Adapting partial code(file name start with C) to the sonar cloud rules (#1918)
* Adapting partial code(file name start with C) to the sonar cloud rules

* add some UT

* add some UT(rollback JSONUtils.checkJsonVaild)

* add some UT(fix OSUtils.exeCmd UT)
2020-02-13 12:42:08 +08:00
Yelli
768b5d9dfd
add state null check (#1939)
add process definition UT
2020-02-12 15:05:21 +08:00
lilin
735034c72f modify DataAnalysisServiceTest by mock 2020-02-12 10:44:44 +08:00
Jave-Chen
6a94f2d3d2
Merge pull request #1916 from samz406/dev
Modify LoggerServiceTest UT by mock
2020-02-11 09:31:06 +08:00
lilin
1172abee44 add comment 2020-02-11 09:00:56 +08:00
Yelli
d937a6ae94 Add modify user name for process definition (#1919)
* class overrides equals() and should therefore also override hashCode()

* #1862 add modify user in process difinition list

* #1862 add pg-1.2.2 ddl.sql

* modify ScriptRunnerTest

* add updateProessDifinition UT

* modify updateProcessDifinition UT

* modify updateProcessDifinition UT

* modify mysql 1.2.2 ddl.sql&dml.sql

* add scope test to mysql in pom

* modify pg-1.2.2 ddl.sql
2020-02-10 16:37:51 +08:00
lilin
69fa28357b modify @RunWith(MockitoJUnitRunner.class) 2020-02-10 15:33:37 +08:00
khadgarmage
c447bb489f add stringutils ut (#1921)
* add stringutils ut
2020-02-10 13:52:32 +08:00
lilin
b3976da5b0 modify LoggerServiceTest ut 2020-02-10 11:35:16 +08:00
gabry.wu
b77dddfeef Adapting partial code(file name start with B) to the sonar cloud rules (#1915)
* Adapting partial code(file name start with B) to the sonar cloud rules

* add @Ignore to BaseTaskQueueTest class
2020-02-09 17:56:55 +08:00
lilin
e00ef4915d modify loggerService UT by Mock 2020-02-09 16:32:10 +08:00
zhangzy
61593cfe6b ut change 2020-02-07 17:35:12 +08:00
zhangzy
9d944d3f76 Optimize duplicate code
Use public methods to simplify duplicate code
2020-02-07 15:47:07 +08:00
zhangzy
662233e0dd Remove ScheduleUtil,use the existing CronUtils
CronUtils is a public quartz tools, then the project dolphinscheduler-api can remove dolphinscheduler-server
2020-02-07 10:15:42 +08:00
dailidong
18750dfc08
remove jasper-runtime jar, not need anymore (#1900)
* update README about DolphinScheduler

* Update issue templates

* update

* regularize api pom xml
update rpc maven compile to 1.8

* regularize api pom xml

* change commons.lang3.StringUtils to common.utils.StringUtils

* update pom.xml

* update

* correct equals method

* jasper-runtime is needed when api server start

* jasper-runtime jar is needed when api server start

* combine logback config of master/worker/alert/api server to one logback.xml

* remove tomcat runtime jar

* add UT

* add license

* remove jasper-runtime jar, not need anymore

Co-authored-by: DS <escheduler@outlook.com>
2020-02-06 18:52:44 +08:00
dailidong
c63cd3e237
combine logback config of master/worker/alert/api server to one (#1898)
* combine logback config of master/worker/alert/api server to one logback.xml
2020-02-06 11:08:06 +08:00
dailidong
dbbc6f58da
jasper-runtime jar is needed when api server start (#1896)
* update README about DolphinScheduler

* Update issue templates

* update

* regularize api pom xml
update rpc maven compile to 1.8

* regularize api pom xml

* change commons.lang3.StringUtils to common.utils.StringUtils

* update pom.xml

* update

* correct equals method

* jasper-runtime is needed when api server start

* jasper-runtime jar is needed when api server start

Co-authored-by: DS <escheduler@outlook.com>
2020-02-05 09:21:21 +08:00
Tboy
42b3e5bd10
fix api server startup failure (#1886)
* move version to parent pom

* move version properties to parent pom for easy management

* remove freemarker dependency

* delete CombinedApplicationServer

* #1871 correct spelling

* #1873 some updates for TaskQueueZkImpl

* #1875 remove unused properties in pom

* #1878
1. remove tomcat dependency
2. remove combined_logback.xml in api module
3. format pom.xml for not aligning

* #1885 fix api server startup failure
1. add jsp-2.1 dependency
2. remove jasper-runtime dependency
2020-02-03 11:06:31 +08:00
Yelli
5169689c35
class overrides equals() and should therefore also override hashCode() (#1882) 2020-02-03 10:24:24 +08:00
Tboy
6b0debec79
some updates for pom (#1879)
* move version to parent pom

* move version properties to parent pom for easy management

* remove freemarker dependency

* delete CombinedApplicationServer

* #1871 correct spelling

* #1873 some updates for TaskQueueZkImpl

* #1875 remove unused properties in pom

* #1878
1. remove tomcat dependency
2. remove combined_logback.xml in api module
3. format pom.xml for not aligning
2020-02-01 16:10:17 +08:00
Jave-Chen
ff9009a5ae #1864 fix dependency and createCommand date check (#1865) 2020-01-22 23:16:13 +08:00
gabry.wu
a1b918c364 [Feature] createSchedule should return schedule's Id (#1861)
issue: https://github.com/apache/incubator-dolphinscheduler/issues/1860
2020-01-20 16:35:17 +08:00
liwenhe1993
30db5c1467 Api module relp server module/1756 (#1766)
* Remove server module on api module

1. Remove server module and add dao module.
2. Move quartz in server module to dao module.
3. Update config and class.


to resolve conflicts,I deleted CombinedApplicationServer.java

Co-authored-by: dailidong <dailidong66@gmail.com>
2020-01-20 16:26:23 +08:00
Jave-Chen
16913bacd7 fix return, obj error (#1855) 2020-01-17 21:55:50 +08:00
Jave-Chen
d371745ddf Fix bug: Use try-with-resources or close this "Socket" in a "finally" clause. (#1716)
* #1714
2020-01-17 19:17:30 +08:00
Jave-Chen
c522ea7ebb Fix bug: NullPointerException etc. (#1721)
* #1720 Fix bug: NullPointerException etc.

* fix code smell

* Update DateInterval.java

refactor equals method

Co-authored-by: dailidong <dailidong66@gmail.com>
2020-01-17 16:55:01 +08:00
老佛爷
1c77708a89 Supplementary data by schedule (#1830)
* Supplementary data by schedule

* fix sonar check bug

* fix code duplicated blocks

* ut

* loop by day

* MasterExecThread test

* test add licene

Co-authored-by: dailidong <dailidong66@gmail.com>
2020-01-16 15:33:00 +08:00
qiaozhanwei
53814425b7 revert #1686 (#1840)
* remove LogViewServiceGrpc.java file and pom modify

* remove kazoo

* remove kazoo

* remove kazoo

* remove common monitor package

* add license

* remove kazoo modify

* remove kazoo modify

* remove kazoo modify

* remove kazoo modify

* remove kazoo modify

* remove kazoo modify

* install.sh remove python kazoo

* add system param whether repeat running

* remove kazoo modify

* BusinessTimeUtils remove whther repeat running inner param

* add AccessTokenMapperTest UT

* CI UT yml modify,start postgresql and zookeeper by default

* add AlertGroupMapperTest UT in github action

* Conflicts reslove

* AlertMappert UT modify

* AlertMappert UT modify

* AlertMappert UT modify

* CommandMapperTest UT modify

* DataSourceMapperTest UT modify

* fix #1775 bug,delete process definition when process instance is running

* verifyTenantIsNull remove process definnition id print log

* revert #1686

* revert #1686
2020-01-16 15:09:13 +08:00
samz406
c558da0b75 add BaseServiceTest BaseDAGServiceTest UT (#1835)
* modify UdfFuncServiceTest UT

* fix get tenantCode my NPE

* add  ResourcesServiceTest

* add error log

* add BaseServiceTest BaseServiceTest UT

* update
2020-01-16 11:25:40 +08:00
Yelli
8399ab4718 add null check in markdownTable & modify EnterpriseWeChatManager logger class (#1809) 2020-01-14 14:36:38 +08:00
Yelli
38f30b5c4d add null check when import process has sub-process (#1806)
* fix bug: zk hasTask method NPE

* add retMap null check for AlertSender

* add null check when import process has sub-process
2020-01-13 10:11:35 +08:00
Yelli
1c5c8b75b9 refactor import process (#1804)
* refactor import process

* add refactor import process UT

* add import process UT

* add null check UT for import process metadata

* add UT for import process

* modify dependentparam UT

* modify testAddImportDependentSpecialParam

* modify DependentParamTest

* modify processDefinitionService UT
2020-01-11 15:05:30 +08:00
Tboy
3cf21dcac0 delete CombinedApplicationServer (#1763)
* move version to parent pom

* move version properties to parent pom for easy management

* remove freemarker dependency

* delete CombinedApplicationServer
2020-01-09 10:40:03 +08:00
Yelli
7dfa9976a0 refactor export process (#1772)
* refactor process export

* comment & null check for export process

* add License

* add javadoc

* add DataSourceParamTest and DependentParamTest

* refactor process export & add exportProcessMetaDataStr UT
2020-01-09 10:35:52 +08:00
Yelli
23cec993f9 fix bug: fix ResourcesServiceTest (#1759) 2020-01-08 09:59:49 +08:00
Tboy
074fd41aa6 remove freemarker dependency (#1757)
* remove freemarker dependency
2020-01-07 22:33:31 +08:00
Jave-Chen
4b027fbf02 Fix SonarCloud bug “Resources should be closed” (#1667)
#1666 Fix SonarCloud bug “Resources should be closed”
2020-01-07 21:53:55 +08:00
Yelli
8125974c61 ds.server regularize log print (#1751)
* ds.server regularize log print

* log print modify

* log print modify

* modify lang3 to lang
2020-01-07 17:29:19 +08:00
elonlo
8ae401e3d7 Refactor login verification process (#1543)
* Refactor login verification process

* Delete application-combined.properties

application-combined.properties doesn't needed

Co-authored-by: dailidong <dailidong66@gmail.com>
2020-01-07 16:23:23 +08:00
Yelli
7762a62270 add null check in SubProcessTaskExecThread (#1737)
* modify FileUtils.readFile2Str

* #1300 Add right alignment function in sql email content

* cancel formatted for alert_mail_template.ftl

* #747 sql task password Log desensitization

* cancel mail_temple

* edit ExcelUtils

* modify test method name

* #747 sql task password Log desensitization

* #1544 workflow import

* Constants add DATASOURCE_PASSWORD_REGEX

* #747 sql task password Log desensitization

* deal with import project have sub process

* modify export process addTaskNodeParam method name

* add testAddTaskNodeSpecialParam UT

* add ProcessDefinitionServiceTest-ut to pom

* add testImportSubProcess in ProcessDefinitionServiceTest

* add testImportSubProcess in ProcessDefinitionServiceTest

* add testImportProcessDefinition

* fix sonar bug: not enough arguments

* fix sonar bug: change condition & not enough arguments

* fix bug sonar: add null check in daghelper && add hashcode method in ProcessData

* fix bug sonar: add null check in daghelper

* CollectionUtils.isEmpty()

* add hashcode() && null check

* merge if

* remove hashcode

* remove e.getMessage
2020-01-07 11:07:23 +08:00
Tboy
4cc058c931 move version properties to parent pom for easy management of api module (#1736)
* move version to parent pom

* move version properties to parent pom for easy management
2020-01-07 10:22:11 +08:00
dailidong
0628aec535 regularize api module pom.xml and rpc module compile to 1.8 (#1733)
* update README about DolphinScheduler

* Update issue templates

* update

* regularize api pom xml
update rpc maven compile to 1.8

* regularize api pom xml

Co-authored-by: DS <escheduler@outlook.com>
2020-01-06 22:53:38 +08:00
samz406
15ae4dfa74 fix get tenantCode my NPE (#1684)
* modify UdfFuncServiceTest UT

* fix get tenantCode my NPE

* add  ResourcesServiceTest

* add error log
2020-01-06 19:47:29 +08:00
Tboy
94fa1adf9b move version to parent pom (#1722) 2020-01-06 11:35:29 +08:00
qiaozhanwei
1f7db8f797 Revert "move version to parent pom (#1712)" (#1715)
This reverts commit 87e8f0d83b.
2020-01-04 23:38:08 +08:00
Tboy
87e8f0d83b move version to parent pom (#1712)
* remove commons-lang3 dependancy

* remove commons-lang3 dependancy

* remove unused import

* all the submodule should not contain version, so move version to parent,
2020-01-04 23:11:13 +08:00
Tboy
4d1abb3fc7 remove commons-lang3 dependancy (#1709)
* remove commons-lang3 dependancy

* remove commons-lang3 dependancy

* remove unused import
2020-01-04 18:01:10 +08:00
Yelli
7ecd38f7d6 fix sonar bug: change condition & not enough arguments (#1692)
* modify FileUtils.readFile2Str

* #1300 Add right alignment function in sql email content

* cancel formatted for alert_mail_template.ftl

* #747 sql task password Log desensitization

* cancel mail_temple

* edit ExcelUtils

* modify test method name

* #747 sql task password Log desensitization

* #1544 workflow import

* Constants add DATASOURCE_PASSWORD_REGEX

* #747 sql task password Log desensitization

* deal with import project have sub process

* modify export process addTaskNodeParam method name

* add testAddTaskNodeSpecialParam UT

* add ProcessDefinitionServiceTest-ut to pom

* add testImportSubProcess in ProcessDefinitionServiceTest

* add testImportSubProcess in ProcessDefinitionServiceTest

* add testImportProcessDefinition

* fix sonar bug: not enough arguments

* fix sonar bug: change condition & not enough arguments
2020-01-04 11:32:07 +08:00
Tboy
884237487c remove commons-lang3 dependancy, using common-module instead for server module (#1698)
* remove commons-lang3 dependancy

* remove commons-lang3 and commons-lang dependancy

* remove commons-collections4/commons-logging/commons-io dependancy

* tmp add commons-lang3 for compile

* remove commons-lang3 dependancy, using common-module instead

* remove common StringUtils instead of lang3

* add tmp lang3 for compile
2020-01-03 19:23:05 +08:00
Yelli
216154dc06 fix sonar bug: null check & clean throw exception (#1690)
* fix sonar bug: null check & clean throw exception

* fix sonar bug: null check & clean throw exception
2020-01-03 11:46:51 +08:00
Yelli
a80e90276f fix sonar bug: not enough arguments (#1687)
* modify FileUtils.readFile2Str

* #1300 Add right alignment function in sql email content

* cancel formatted for alert_mail_template.ftl

* #747 sql task password Log desensitization

* cancel mail_temple

* edit ExcelUtils

* modify test method name

* #747 sql task password Log desensitization

* #1544 workflow import

* Constants add DATASOURCE_PASSWORD_REGEX

* #747 sql task password Log desensitization

* deal with import project have sub process

* modify export process addTaskNodeParam method name

* add testAddTaskNodeSpecialParam UT

* add ProcessDefinitionServiceTest-ut to pom

* add testImportSubProcess in ProcessDefinitionServiceTest

* add testImportSubProcess in ProcessDefinitionServiceTest

* add testImportProcessDefinition

* fix sonar bug: not enough arguments
2020-01-03 11:24:01 +08:00
Tboy
95dedd94f3
Merge pull request #1686 from clay4444/fixSignOutBug
Fix SessionService signOut method exception handle bug
2020-01-03 10:06:03 +08:00
loushang
65bb0e03b7 fix SessionServiceTest testSignOut bug (add mock) 2020-01-02 20:08:14 +08:00
Yelli
26ed786c4d #1544 fix bug: workflow import (#1676)
* modify FileUtils.readFile2Str

* #1300 Add right alignment function in sql email content

* cancel formatted for alert_mail_template.ftl

* #747 sql task password Log desensitization

* cancel mail_temple

* edit ExcelUtils

* modify test method name

* #747 sql task password Log desensitization

* #1544 workflow import

* Constants add DATASOURCE_PASSWORD_REGEX

* #747 sql task password Log desensitization

* deal with import project have sub process

* modify export process addTaskNodeParam method name

* add testAddTaskNodeSpecialParam UT

* add ProcessDefinitionServiceTest-ut to pom

* add testImportSubProcess in ProcessDefinitionServiceTest

* add testImportSubProcess in ProcessDefinitionServiceTest

* add testImportProcessDefinition
2020-01-02 19:45:05 +08:00
loushang
5949c8d6fb fix SessionService signOut method exception bug 2020-01-02 17:20:57 +08:00
samz406
dc6c18902a modify UdfFuncServiceTest UT (#1644) 2019-12-31 23:21:09 +08:00
loushang
b549ee3901 fix zk monitor bug 2019-12-31 10:30:20 +08:00
loushang
dc4e46db41 fix Monitor bug 2019-12-30 18:58:38 +08:00
Tboy
4032c4e64b
Merge pull request #1618 from samz406/1227-2
before delete Udf check exist
2019-12-30 18:30:49 +08:00
lilin
96d49a0950 remove delete check exist 2019-12-30 18:26:10 +08:00
bao liang
d16500e8be [Feature] merge some configurations #1635 (#1636)
* merge hadoop.properties into common.properties

* merge hadoop,zookeeper.properties into common.properties
remove combined.properties/master.properties/worker.properties

* change db user/pwd to test/test
2019-12-30 18:14:31 +08:00