Commit Graph

228 Commits

Author SHA1 Message Date
lenboo
572b142a53 [maven-release-plugin] prepare for next development iteration 2020-07-04 16:10:37 +08:00
lenboo
a4272efa55 [maven-release-plugin] prepare release 1.3.1 2020-07-04 16:04:36 +08:00
lenboo
fd3485ad0a revert version to 1.3.1 2020-07-04 15:54:37 +08:00
lgcareer
071437ffa4
add PropertyPlaceholderHelper on license (#3129)
* update license header location

* add PropertyPlaceholderHelper on license
2020-07-04 15:50:50 +08:00
lgcareer
03dfba474e
update license header location (#3126) 2020-07-04 15:31:55 +08:00
lenboo
3d4d816349 [maven-release-plugin] prepare for next development iteration 2020-07-03 15:56:45 +08:00
lenboo
2ff94a4362 [maven-release-plugin] prepare release 1.3.1 2020-07-03 15:55:56 +08:00
gaojun2048
58897aa2c5 merge 1.3.0-release branch code 2020-06-29 18:42:26 +08:00
gaojun2048
3cf9afc195 merge 1.3.0-release branch code 2020-06-29 18:42:18 +08:00
gabry.wu
5222483471 Merge pull request #2898 from 597365581/patch-6
merge method updateuser if and fix bug#2892
2020-06-29 17:39:55 +08:00
lgcareer
9a5a818b13 fix download resource file that under the directory error (#3007)
* add mybatis-3 notice in the top-level notice

* fix download resource file that under the directory error
2020-06-29 17:39:53 +08:00
qiaozhanwei
8ec4298c95 No master don't create command #2571 (#2602)
* dispatch task fail will set task status failed

* 1,no worker condition , master will while ture wait for worker startup
2,worker response task status sync wait for result

* 1,no worker condition , master will while ture wait for worker startup
2,worker response task status sync wait for result

* 1,no worker condition , master will while ture wait for worker startup
2,worker response task status sync wait for result

* 1,no worker condition , master will while ture wait for worker startup
2,worker response task status sync wait for result

* 1,no worker condition , master will while ture wait for worker startup
2,worker response task status sync wait for result

* 1,no worker condition , master will while ture wait for worker startup
2,worker response task status sync wait for result

* 1,no worker condition , master will while ture wait for worker startup
2,worker response task status sync wait for result

* 1,no worker condition , master will while ture wait for worker startup
2,worker response task status sync wait for result

* 1,no worker condition , master will while ture wait for worker startup
2,worker response task status sync wait for result

* 1,no worker condition , master will while ture wait for worker startup
2,worker response task status sync wait for result

* 1,no worker condition , master will while ture wait for worker startup
2,worker response task status sync wait for result

* 1,task status statistics and process status statistics bug fix (#2357)
2,worker group bug fix

* 1,task status statistics and process status statistics bug fix (#2357)
2,worker group bug fix

* 1,task status statistics and process status statistics bug fix (#2357)
2,worker group bug fix

* 1,task status statistics and process status statistics bug fix (#2357)
2,worker group bug fix

* send mail error, #2466 bug fix

* send mail error, #2466 bug fix

* send mail error, #2466 bug fix

* send mail error, #2466 bug fix

* #2486 bug fix

* host and workergroup compatible

* EnterpriseWeChatUtils modify

* EnterpriseWeChatUtils modify

* EnterpriseWeChatUtils modify

* #2499 bug fix

* add comment

* revert comment

* revert comment

* #2499 buf fix

* #2499 bug fix

* #2499 bug fix

* #2499 bug fix

* #2499 bug fix

* #2499 bug fix

* #2499 bug fix

* no valid worker group,master can kill task directly

* no valid worker group,master can kill task directly

* no valid worker group,master can kill task directly

* no valid worker group,master can kill task directly

* no valid worker group,master can kill task directly

* no valid worker group,master can kill task directly

* no valid worker group,master can kill task directly

* no valid worker group,master can kill task directly

* no valid worker group,master can kill task directly

* No master don't create command #2571

* No master don't create command #2571

* No master don't create command #2571

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
2020-06-29 17:39:07 +08:00
tlhhup
eafd640163 Zookeeper management interface in monitoring center displays abnormal #2547 (#2595)
Co-authored-by: dailidong <dailidong66@gmail.com>
2020-06-29 17:39:07 +08:00
qiaozhanwei
043a20f2c7 No master don't create command #2571 (#2575)
* dispatch task fail will set task status failed

* 1,no worker condition , master will while ture wait for worker startup
2,worker response task status sync wait for result

* 1,no worker condition , master will while ture wait for worker startup
2,worker response task status sync wait for result

* 1,no worker condition , master will while ture wait for worker startup
2,worker response task status sync wait for result

* 1,no worker condition , master will while ture wait for worker startup
2,worker response task status sync wait for result

* 1,no worker condition , master will while ture wait for worker startup
2,worker response task status sync wait for result

* 1,no worker condition , master will while ture wait for worker startup
2,worker response task status sync wait for result

* 1,no worker condition , master will while ture wait for worker startup
2,worker response task status sync wait for result

* 1,no worker condition , master will while ture wait for worker startup
2,worker response task status sync wait for result

* 1,no worker condition , master will while ture wait for worker startup
2,worker response task status sync wait for result

* 1,no worker condition , master will while ture wait for worker startup
2,worker response task status sync wait for result

* 1,no worker condition , master will while ture wait for worker startup
2,worker response task status sync wait for result

* 1,task status statistics and process status statistics bug fix (#2357)
2,worker group bug fix

* 1,task status statistics and process status statistics bug fix (#2357)
2,worker group bug fix

* 1,task status statistics and process status statistics bug fix (#2357)
2,worker group bug fix

* 1,task status statistics and process status statistics bug fix (#2357)
2,worker group bug fix

* send mail error, #2466 bug fix

* send mail error, #2466 bug fix

* send mail error, #2466 bug fix

* send mail error, #2466 bug fix

* #2486 bug fix

* host and workergroup compatible

* EnterpriseWeChatUtils modify

* EnterpriseWeChatUtils modify

* EnterpriseWeChatUtils modify

* #2499 bug fix

* add comment

* revert comment

* revert comment

* #2499 buf fix

* #2499 bug fix

* #2499 bug fix

* #2499 bug fix

* #2499 bug fix

* #2499 bug fix

* #2499 bug fix

* no valid worker group,master can kill task directly

* no valid worker group,master can kill task directly

* no valid worker group,master can kill task directly

* no valid worker group,master can kill task directly

* no valid worker group,master can kill task directly

* no valid worker group,master can kill task directly

* no valid worker group,master can kill task directly

* no valid worker group,master can kill task directly

* no valid worker group,master can kill task directly

* No master don't create command #2571

* No master don't create command #2571

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
2020-06-29 17:39:00 +08:00
Simon
d8a165ac6f fix #2450 (#2493)
* fix #1441

* support custom datax config

* support datax custom config

* support datax custom config

* support datax custom config

* fix #2450

Co-authored-by: 张世鸣 <zhangshiming@cvte.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
2020-06-29 17:38:53 +08:00
songgg
b06c118af6 [optimization]:prevent repeated database updates (#2396)
* sqlTask failed to run

* prevent repeated database updates

* prevent repeated database updates

* prevent repeated database updates

Co-authored-by: songqh <songquanhe@foxmail.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
2020-06-29 17:38:37 +08:00
qiaozhanwei
eecf992432 update UT in pom.xml 2020-06-29 17:38:29 +08:00
BoYiZhang
bcdf58969c Fix document errors (#2382)
* Remove invalid code

* fix document errors

Co-authored-by: zhanglong <zhanglong@ysstech.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
2020-06-29 17:38:20 +08:00
songgg
0ad610a0f0 [optimization]:remove duplicate code (#2377)
* sqlTask failed to run

* remove duplicate code

* remove duplicate code

* remove duplicate code

* remove duplicate code

* remove duplicate code

Co-authored-by: songqh <songquanhe@foxmail.com>
2020-06-29 17:36:42 +08:00
itbasketplayer
d03f9daa0e .gitignore remove taskInstance/index.vue(#2300) and no need to findProcessDefineById 2 times(#2373) (#2374)
* some optimization

* revert

* ProcessDefinitionService#updateProcessDefinition findProcessDefineById 1 time
2020-06-29 17:36:42 +08:00
BoYiZhang
7fa0c0f79a Remove invalid code (#2342)
Co-authored-by: zhanglong <zhanglong@ysstech.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
2020-06-29 17:36:41 +08:00
songgg
d8c7a44e82 some annotation optimizations (#2333)
Co-authored-by: songqh <songquanhe@foxmail.com>
2020-06-29 17:36:38 +08:00
Rubik-W
668b86f12b fix:Optimize content returned by interface (#2294) 2020-06-29 17:36:37 +08:00
lgcareer
82d6dc9e52 new feature for #404 add resource tree function (#2323)
* add create resource directory

* add create resource directory

* update the resource test

* add upgrade sql in version 1.2.2

* Adding request parameter id to update queryResourceListPaging

* set isDirectory value is false default

* add full name to update updateResource

* remove request parameter isDirectory to update createResource method

* update queryResourceListPaging with change get to post

* update updateResource method with remove fullName

* File management list modification (#1976)

* add resource component

* add resource tree visitor

* return json string

* update queryResourceList

* upload file need fullName

* add method rootNode

* Shell task resources and authorization resources (#1989)

* File management list modification

* Shell task resources and authorization resources

* download resource when execute task

* download resource when execute task

* update authorization type

* download resource when execute task

* Spark task resource changes (#1990)

* File management list modification

* Shell task resources and authorization resources

* Spark task resource changes

* download resource when execute task

* update udf function service

* add resource type in ResourceComponent

* UDF resource tree and change DAG style (#2019)

* File management list modification

* Shell task resources and authorization resources

* Spark task resource changes

* UDF resource tree and change DAG style

* add deleteIds method in ResourceMapper and ResourceMapperTest

* Add comments on class and method

* add queryResourceByName method in controller

* update verify-name with change name to full name

* update queryResource with add parameter pid

* update queryResource with add parameter pid

* add resource ids in process definition and delete resource need judge whether it is used by any process definition

* Breadcrumb development (#2033)

* File management list modification

* Shell task resources and authorization resources

* Spark task resource changes

* UDF resource tree and change DAG style

* Breadcrumb development

* Breadcrumb development

* Resource tree bug fix (#2040)

* File management list modification

* Shell task resources and authorization resources

* Spark task resource changes

* UDF resource tree and change DAG style

* Breadcrumb development

* Breadcrumb development

* Resource tree bug fix

* update resource service test

* Fix github action rerun failed

* add status of PARENT_RESOURCE_NOT_EXIST

* Fix github action rerun failed (#2067)

* update resource service test

* Fix github action rerun failed

* add status of PARENT_RESOURCE_NOT_EXIST

* Change crumb position

* Change crumb position (#2068)

* build resource process definition map

* UDF changed to multiple choice

* UDF changed to multiple choice (#2077)

* Change crumb position

* UDF changed to multiple choice

* build resource process definition map (#2076)

* update resource service test

* Fix github action rerun failed

* add status of PARENT_RESOURCE_NOT_EXIST

* build resource process definition map

* update resource name also need update all the children full name

* need add queryResource

* update resource name also need update all the children full name (#2096)

* update resource service test

* Fix github action rerun failed

* add status of PARENT_RESOURCE_NOT_EXIST

* build resource process definition map

* update resource name also need update all the children full name

* need add queryResource

* Limit customization file content to no more than 3000 lines

* Limit customization file content to no more than 3000 lines(#2128)

* Limit customization file content to no more than 3000 lines(#2128) (#2140)

* Change crumb position

* UDF changed to multiple choice

* Limit customization file content to no more than 3000 lines

* Limit customization file content to no more than 3000 lines(#2128)

* add queryResourceJarList

* add queryResourceJarList

* add queryResourceJarList

* add queryResourceJarList (#2192)

* update resource service test

* Fix github action rerun failed

* add status of PARENT_RESOURCE_NOT_EXIST

* build resource process definition map

* update resource name also need update all the children full name

* need add queryResource

* add queryResourceJarList

* add queryResourceJarList

* add queryResourceJarList

* Modify the main jar package

* Modify the main jar package (#2200)

* Change crumb position

* UDF changed to multiple choice

* Limit customization file content to no more than 3000 lines

* Limit customization file content to no more than 3000 lines(#2128)

* Modify the main jar package

* add resource filter in order to get filtered resource

* add comments of resource filter

* update list children by resource

* choose main jar with resource tree (#2220)

* update resource service test

* Fix github action rerun failed

* add status of PARENT_RESOURCE_NOT_EXIST

* build resource process definition map

* update resource name also need update all the children full name

* need add queryResource

* add queryResourceJarList

* add queryResourceJarList

* add queryResourceJarList

* add resource filter in order to get filtered resource

* add comments of resource filter

* update list children by resource

* Return null if query resource list is empty

* update queryResource method change parameter pid to id

* getResouDelete checksum and modify parameter namerceId

* revert .env

* remove parameter pid

* Delete request interface

* go back to the last page

* jar interface call

* Fix issue #2234 and #2228

* change resource name with full name

* Fix issue #2234 and #2228 (#2246)

* update resource service test

* Fix github action rerun failed

* add status of PARENT_RESOURCE_NOT_EXIST

* build resource process definition map

* update resource name also need update all the children full name

* need add queryResource

* add queryResourceJarList

* add queryResourceJarList

* add queryResourceJarList

* add resource filter in order to get filtered resource

* add comments of resource filter

* update list children by resource

* Return null if query resource list is empty

* update queryResource method change parameter pid to id

* revert .env

* remove parameter pid

* Fix issue #2234 and #2228

* change resource name with full name

* Fix list query value error

* remove unauth-file with authorize-resource-tree

* Repair data cannot be echoed

* Repair data cannot be echoed

* execute mr and spark task need query resource name before

* Authorized resource interface replacement

* Authorized resource interface replacement

* Filter UDF resources

* Change parameters

* need query all authorized directory children when create task

* Change normalize.scss import method and animation.scss license modification

* Delete file list update processing

* It's fixed that resource not deleted in hdfs when delete it.

* add tooltips

* add tooltips (#2310)

* Echo workflow name and modify udf management name

* [new feature]add resource tree function

* revert front code in order to be same as dev branch

* revert front code in order to be same as dev branch

* revert common.properties and application.properties

* add super method

* update flink parameter test

* update flink parameter and unit test

* update resource service test

* If resource list is empty,need init it

* update flink parameter test

Co-authored-by: break60 <790061044@qq.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>
2020-06-29 17:36:36 +08:00
gabry.wu
f9c91b8cbe Adapting partial code(file name start with S #2) to the sonar cloud rule (#2270) 2020-06-29 17:36:36 +08:00
gabry.wu
30b98b89a2 Adapting partial code(file name start with T) to the sonar cloud rule (#2271) 2020-06-29 17:36:33 +08:00
gabry.wu
a1f7614476 Adapting partial code(file name start with W) to the sonar cloud rule (#2287) 2020-06-29 17:36:33 +08:00
WeiYe
6cca69369b Update test code logger class name (#2303) 2020-06-29 17:36:33 +08:00
gabry.wu
afa9a6467a Adapting partial code(file name start with Q) to the sonar cloud rule (#2242) 2020-06-29 17:36:29 +08:00
gabry.wu
e1771de45a Adapting partial code(file name start with R) to the sonar cloud rule (#2243) 2020-06-29 17:36:29 +08:00
jin
24fa31939b feat:optimizing code (#2245)
* feat:optimizing code

* add licence and  format code
2020-06-29 17:36:29 +08:00
Simon
5c30205574 support custom datax configuration (#2224)
* fix #1441

* support custom datax config

* support datax custom config

* support datax custom config

* support datax custom config

Co-authored-by: 张世鸣 <zhangshiming@cvte.com>
2020-06-29 17:36:29 +08:00
Rubik-W
9fe04524ae fix: Api server startup fail (#2198) 2020-06-29 17:36:28 +08:00
t1mon
55cc1c7c5b Replace Charset.forName("UTF-8") with StandardCharsets.UTF_8. (#2193) 2020-06-29 17:36:28 +08:00
gabry.wu
7f7f50a828 Adapting partial code(file name start with L) to the sonar cloud rule (#2171) 2020-06-29 17:36:28 +08:00
gabry.wu
610c95a470 use static method in static class JSON (#2156) 2020-06-29 17:36:27 +08:00
tswstarplanet
89c0a796f9 fix field access modifier (#2150) 2020-06-29 17:36:22 +08:00
gabry.wu
6a8cef08e4 Adapting partial code(file name start with E) to the sonar cloud rule (#1999)
* Adapting partial code(file name start with E) to the sonar cloud rule

* remove isEmpty invoke

* resolve conflicts
2020-06-29 17:36:13 +08:00
Tboy
7ed28af4fe
Upgrade curator version (#2083)
* upgrade curator version
issue: https://github.com/apache/incubator-dolphinscheduler/issues/2020
curator issue: https://github.com/apache/curator/pull/297

* add DefaultEnsembleProvider override

* add some unit test

* add License
2020-03-05 19:00:06 +08:00
qiaozhanwei
d5c795bd45
fix ugly logger usage (#2063)
* fix ugly logger usage

* remove space
2020-03-05 11:10:48 +08:00
qiaozhanwei
3e5a939913
delete unused import (#2066) 2020-03-05 11:07:44 +08:00
qiaozhanwei
b8014d91cf
add workflow id, name for email content (#2071)
* add tenantService UT

* fix spell

* #464 email content

* modify mysql1.2.2 ddl
2020-03-05 11:04:00 +08:00
Rubik
1ea48375f4
Fixbug i18n #2008 (#2035)
* fix:add chinese message

* fix:add chinese message

* fix:'getLocale()' can return not null

* fix:optimize message content
2020-03-02 10:29:42 +08:00
Yelli
84409b57ee
fix taskinstance.isSubprocess misjudgment (#2007)
* add processInstanceService UT

* fix taskInstance.isSubProcess
add processInstanceService UT

* add taskInstance.isSubProcess UT

* modify taskInstance.isSubProcess UT

* add license

* remove author&date
2020-02-28 16:15:21 +08:00
gabry.wu
5ca4862ea9
remove String.format for org.slf4j.Logger (#2001) 2020-02-25 14:19:26 +08:00
Yelli
619fb30c50
#1310 add executor user to the process instance & task instance pages (#1973)
* add executor name in processDefinitionInstance & taskInstance

* modify processInstanceTest

* modify processInstanceTest

* #1310 add executor user to the process instance & task instance pages

* add processInstanceUT & taskInstanceUT

* add processInstanceUT & taskInstanceUT

* modify processInstanceTest & taskInstanceTest

* Remove duplicate code

* add userServiceUT
2020-02-20 13:25:41 +08:00
dailidong
24f22974a2
solve jar conflict, delete servlet-api 2.5 jar (#1966)
* update README about DolphinScheduler

* Update issue templates

* update

* regularize api pom xml
update rpc maven compile to 1.8

* regularize api pom xml

* change commons.lang3.StringUtils to common.utils.StringUtils

* update pom.xml

* update

* correct equals method

* jasper-runtime is needed when api server start

* jasper-runtime jar is needed when api server start

* combine logback config of master/worker/alert/api server to one logback.xml

* remove tomcat runtime jar

* add UT

* add license

* remove jasper-runtime jar, not need anymore

* sovle jar conflict, remove servlet-api 2.5 jar

* remove servlet 2.5

Co-authored-by: DS <escheduler@outlook.com>
2020-02-17 10:16:40 +08:00
qiaozhanwei
9b75bebf1b
1,add sendAsync method 2,refactor LoggerClient (#1965)
* 1,remove dolphinscheduler-rpc module  2,add dolphinscheduler-remote module 3,add dolphinscheduler-service module 4,refactor LoggerServer module (#1925)

* 1,remove dolphinscheduler-rpc module
2,add dolphinscheduler-remote module
3,add dolphinscheduler-service module
4,refactor LoggerServer module

* ProcessUtils modify

* Refactor architecture (#1926)

* move version to parent pom

* move version properties to parent pom for easy management

* remove freemarker dependency

* delete CombinedApplicationServer

* #1871 correct spelling

* #1873 some updates for TaskQueueZkImpl

* #1875 remove unused properties in pom

* #1878
1. remove tomcat dependency
2. remove combined_logback.xml in api module
3. format pom.xml for not aligning

* #1885 fix api server startup failure
1. add jsp-2.1 dependency
2. remove jasper-runtime dependency

* add stringutils ut (#1921)

* add stringutils ut

* Newfeature for #1675. (#1908)

Continue to finish the rest works, add the cache feature for dependence,mr,python,sub_process,procedure and shell.

* Add modify user name for process definition (#1919)

* class overrides equals() and should therefore also override hashCode()

* #1862 add modify user in process difinition list

* #1862 add pg-1.2.2 ddl.sql

* modify ScriptRunnerTest

* add updateProessDifinition UT

* modify updateProcessDifinition UT

* modify updateProcessDifinition UT

* modify mysql 1.2.2 ddl.sql&dml.sql

* add scope test to mysql in pom

* modify pg-1.2.2 ddl.sql

* refactor module

* updates

Co-authored-by: khadgarmage <khadgar.mage@outlook.com>
Co-authored-by: zhukai <boness@qq.com>
Co-authored-by: Yelli <amarantine@my.com>

* dolphinscheduler-common remove spring (#1931)

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* SpringApplicationContext class title add license (#1932)

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* add license (#1934)

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* move datasource classes to dao module

* fix send4LetterWord bug

* Refactor architecture (#1936)

* move datasource classes to dao module

* fix send4LetterWord bug

* exclude jasper-compiler in case of runtime conflict

* LoggerServiceTest remove ProcessDao (#1944)

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* LoggerServiceTest remove ProcessDao

* exclude jasper-compiler in case of runtime conflict (#1938)

* move datasource classes to dao module

* fix send4LetterWord bug

* exclude jasper-compiler in case of runtime conflict

* DataAnaylysisServiceTest and ProcessDefinitionService modify

* remote module add comment

* OSUtilsTest modify

* add finally block to close channel

* add finally block to close channel (#1951)

* move datasource classes to dao module

* fix send4LetterWord bug

* exclude jasper-compiler in case of runtime conflict

* add finally block to close channel

* refactor log client service

* add sendSync method

* 1,quartz.properties add conf category
2,dolphinscheduler-daemon.sh modify

* dolphinscheduler-binary.xml modify

* add sendAsync method (#1962)

* add sendAsync method

* add sendAsync method

* 1,add sendAsync method
2,refactor LoggerClient

* LogClientService modify

Co-authored-by: Tboy <technoboy@yeah.net>
Co-authored-by: khadgarmage <khadgar.mage@outlook.com>
Co-authored-by: zhukai <boness@qq.com>
Co-authored-by: Yelli <amarantine@my.com>
2020-02-16 19:12:01 +08:00
gabry.wu
99ac739851
rename vaild to valid (#1961) 2020-02-15 18:55:16 +08:00
Yeleights
87cb33b233 deal with conflict 2020-02-14 22:46:45 +08:00