* [Bug-3140]fix the deadlock between start and stop of ZKServer
* use Log framework to print information
* fix code smells; add path prefix of embedded zk server
* Update dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/zk/ZKServer.java
Co-authored-by: Yichao Yang <1048262223@qq.com>
* optimize the code
Co-authored-by: Yichao Yang <1048262223@qq.com>
* merge from 1.3.1-release
* [bugfix] remove Conflicting configuration sudo and user (#3038)
Co-authored-by: zhangchunyang1024 <zhangchunyang0123@gamil.com>
* remove some standard notices in order to cut down the NOTICE file. (#3101)
* #3084 remove some standard notices in order to cut down the NOTICE file.
* update exclude element sql/soft_version of rat plugin
* update soft_version
* merge from 1.3.1-release
* fix 3112:download udf resource need find the tenant code of resource owner (#3114)
* fix 3112:download udf resource need find the tenant code of resource owner
* fix 2975:download resource need find the tenant code of resource owner
* remove useless code
* merge from 1.3.1-release
* remove some notes (#3127)
Co-authored-by: lenboo <baoliang@analysys.com.cn>
* fix Permission denied if not found dolphinscheder-env.sh on 1.3.1 (#3128)
* fix Permission denied if not found dolphinscheder-env.sh
* update some comments on method
* add PropertyPlaceholderHelper on license (#3129)
* update license header location
* add PropertyPlaceholderHelper on license
* merge from 1.3.1-release
* change from 1.3.0 to 1.3.2
* add Set import
Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: zhangchunyang <zhangchunyang0123@gmail.com>
Co-authored-by: zhangchunyang1024 <zhangchunyang0123@gamil.com>
Co-authored-by: lgcareer <18610854716@163.com>
Co-authored-by: lenboo <baoliang@analysys.com.cn>
* add maxWaitTime config when build zookeeper client
* throw exception when connect zk timeout
* use dedicated exception instead of a generic one
Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>
2. remove getMapper method,replace with toList()
3. rewrite toList method,support constructCollectionType.
4. parseObject(resp, Map.class) replace to toMap() method
5. rename JacksonSerialize to JsonSerializer
rewrite createArrayNode,createObjectNode method will Affect too many lines ，i will fix it in the next pr.
* fix worker group config no effect
* remove codehaus janino jar
the license about janino maybe not compatiable with Apache v2
* Merge remote-tracking branch 'upstream/dev-1.3.0' into dev-1.3.0
# Conflicts:
# dolphinscheduler-server/src/main/resources/config/install_config.conf
* datasource config
* Update datasource.properties
* fix RunConfig bug
* remove param monitor server state
* fix table T_DS_ALERT
* update h2 database
* fix#2910 master server will show exception for some time when it restart
* fix oom when no master is active
* fix worker oom when master server restart
* fix oom
* fix
* add UT
* fix worker group config no effect
* fix bug: send task result to master until success.
* fix bug: send task result to master until success.
* add sleep
* set null if send result failed
* set null if send result failed
* change max length of username to 39
* test for gc
* change version to 1.3.0
* check and add some license files
* check and add some license files
* revert
Co-authored-by: baoliang <baoliang@analysys.com.cn>
* copy resources need get top directory first
* copy resources need get top directory first
* fix #2860:copy files and create empty directory
* Fix bug:If user didn't set the value of tenant,release process definition online fail
* feature: add number configuration for master dispatch tasks
* fix bug(#2762) the master would be blocked when worker group not exists
* fix bug(#2762) the master would be blocked when worker group not exists
* fix ut
* fix ut
* fix bug(2781): cannot pause work flow when task state is "submit success"
* fix code smell
* add mysql other param blank judge
* test
* update comments
* update comments
* add ut
* fix bug: Restart the worker service again, the previously submitted successful tasks are not executed
* update comments
* add sleep
* add null point check
* fix bug:After the master is fault-tolerant, it cannot resume operation
* fix bug: do not failover the host is 'NULL' process
* fix bug:worker failover error.
Co-authored-by: baoliang <baoliang@analysys.com.cn>
* feature: add number configuration for master dispatch tasks
* fix bug(#2762) the master would be blocked when worker group not exists
* fix bug(#2762) the master would be blocked when worker group not exists
* fix ut
* fix ut
* fix bug(2781): cannot pause work flow when task state is "submit success"
* fix code smell
* add mysql other param blank judge
* test
* update comments
* update comments
* add ut
* fix bug: Restart the worker service again, the previously submitted successful tasks are not executed
* update comments
* add sleep
* add null point check
* fix bug:After the master is fault-tolerant, it cannot resume operation
* fix bug: do not failover the host is 'NULL' process
Co-authored-by: baoliang <baoliang@analysys.com.cn>
* add LoggerServerTest UT
* add LoggerServerTest UT
* add LoggerServerTest UT
add RemoveTaskLogRequestCommandTest UT
add RemoveTaskLogResponseCommandTest
* master select worker filter high load worker #2704
* master select worker filter high load worker #2704
* master select worker filter high load worker #2704
* master select worker filter high load worker #2704
* master select worker filter high load worker #2704
* master select worker filter high load worker #2704
* add not worker log and remove worker invalid property
* process definition json worker group convert #2794
* process definition json worker group convert #2794
* process definition json worker group convert #2794
* process definition json worker group convert #2794
* process definition json worker group convert #2794
* process definition json worker group convert #2794
* process definition json worker group convert #2794
* process definition json worker group convert #2794
* compatible old version host not port delete log file
Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
* add LoggerServerTest UT
* add LoggerServerTest UT
* add LoggerServerTest UT
add RemoveTaskLogRequestCommandTest UT
add RemoveTaskLogResponseCommandTest
Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
* script variable has "processDefinitionId" is error #2664
* blank in eamil and left font align #2648
* delete log files while deleting process instances #2463
* delete log files while deleting process instances #2463
* delete log files while deleting process instances #2463
* delete log files while deleting process instances #2463
Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
* fix bug: some bugs about worker group from 1.2.0 to 1.3.0
* fix bug: scheduler from 1.2.0 to 1.3.0 do not work
* fix bug: scheduler from 1.2.0 to 1.3.0 do not work
* fix ut bug
Co-authored-by: baoliang <baoliang@analysys.com.cn>
* Fix some code specification issues
* Pre-compile regular expressions to improve efficiency and solve concurrency problems
* Solve the nullpoint problem that Object.equals may bring
* Modify if else structure, add {} to improve code readability
Co-Authored-By: daili <daili@users.noreply.github.com>
* Update ResourceTreeVisitor.java
re-run test process
Co-authored-by: daili <daili@users.noreply.github.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
* dispatch task fail will set task status failed
* 1，no worker condition , master will while ture wait for worker startup
2，worker response task status sync wait for result
* 1，no worker condition , master will while ture wait for worker startup
2，worker response task status sync wait for result
* 1，no worker condition , master will while ture wait for worker startup
2，worker response task status sync wait for result
* 1，no worker condition , master will while ture wait for worker startup
2，worker response task status sync wait for result
* 1，no worker condition , master will while ture wait for worker startup
2，worker response task status sync wait for result
* 1，no worker condition , master will while ture wait for worker startup
2，worker response task status sync wait for result
* 1，no worker condition , master will while ture wait for worker startup
2，worker response task status sync wait for result
* 1，no worker condition , master will while ture wait for worker startup
2，worker response task status sync wait for result
* 1，no worker condition , master will while ture wait for worker startup
2，worker response task status sync wait for result
* 1，no worker condition , master will while ture wait for worker startup
2，worker response task status sync wait for result
* 1，no worker condition , master will while ture wait for worker startup
2，worker response task status sync wait for result
* 1，task status statistics and process status statistics bug fix (#2357)
2，worker group bug fix
* 1，task status statistics and process status statistics bug fix (#2357)
2，worker group bug fix
* 1，task status statistics and process status statistics bug fix (#2357)
2，worker group bug fix
* 1，task status statistics and process status statistics bug fix (#2357)
2，worker group bug fix
* send mail error, #2466 bug fix
* send mail error, #2466 bug fix
* send mail error, #2466 bug fix
* send mail error, #2466 bug fix
* #2486 bug fix
* host and workergroup compatible
* EnterpriseWeChatUtils modify
* EnterpriseWeChatUtils modify
* EnterpriseWeChatUtils modify
* #2499 bug fix
* add comment
* revert comment
* revert comment
* #2499 buf fix
* #2499 bug fix
* #2499 bug fix
* #2499 bug fix
* #2499 bug fix
* #2499 bug fix
* #2499 bug fix
* no valid worker group，master can kill task directly
* no valid worker group，master can kill task directly
* no valid worker group，master can kill task directly
* no valid worker group，master can kill task directly
* no valid worker group，master can kill task directly
* no valid worker group，master can kill task directly
* no valid worker group，master can kill task directly
* no valid worker group，master can kill task directly
* no valid worker group，master can kill task directly
* No master don't create command #2571
* No master don't create command #2571
* No master don't create command #2571
* Worker Group display #2627
* Worker Group display #2627
* Worker Group display #2627
* Worker Group display #2627
* Worker Group display #2627
* Worker Group display #2627
Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
* dispatch task fail will set task status failed
* 1，no worker condition , master will while ture wait for worker startup
2，worker response task status sync wait for result
* 1，no worker condition , master will while ture wait for worker startup
2，worker response task status sync wait for result
* 1，no worker condition , master will while ture wait for worker startup
2，worker response task status sync wait for result
* 1，no worker condition , master will while ture wait for worker startup
2，worker response task status sync wait for result
* 1，no worker condition , master will while ture wait for worker startup
2，worker response task status sync wait for result
* 1，no worker condition , master will while ture wait for worker startup
2，worker response task status sync wait for result
* 1，no worker condition , master will while ture wait for worker startup
2，worker response task status sync wait for result
* 1，no worker condition , master will while ture wait for worker startup
2，worker response task status sync wait for result
* 1，no worker condition , master will while ture wait for worker startup
2，worker response task status sync wait for result
* 1，no worker condition , master will while ture wait for worker startup
2，worker response task status sync wait for result
* 1，no worker condition , master will while ture wait for worker startup
2，worker response task status sync wait for result
* 1，task status statistics and process status statistics bug fix (#2357)
2，worker group bug fix
* 1，task status statistics and process status statistics bug fix (#2357)
2，worker group bug fix
* 1，task status statistics and process status statistics bug fix (#2357)
2，worker group bug fix
* 1，task status statistics and process status statistics bug fix (#2357)
2，worker group bug fix
* send mail error, #2466 bug fix
* send mail error, #2466 bug fix
* send mail error, #2466 bug fix
* send mail error, #2466 bug fix
* #2486 bug fix
* host and workergroup compatible
* EnterpriseWeChatUtils modify
* EnterpriseWeChatUtils modify
* EnterpriseWeChatUtils modify
* #2499 bug fix
* add comment
* revert comment
* revert comment
* #2499 buf fix
* #2499 bug fix
* #2499 bug fix
* #2499 bug fix
* #2499 bug fix
* #2499 bug fix
* #2499 bug fix
Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
* add create resource directory
* add create resource directory
* update the resource test
* add upgrade sql in version 1.2.2
* Adding request parameter id to update queryResourceListPaging
* set isDirectory value is false default
* add full name to update updateResource
* remove request parameter isDirectory to update createResource method
* update queryResourceListPaging with change get to post
* update updateResource method with remove fullName
* File management list modification (#1976)
* add resource component
* add resource tree visitor
* return json string
* update queryResourceList
* upload file need fullName
* add method rootNode
* Shell task resources and authorization resources (#1989)
* File management list modification
* Shell task resources and authorization resources
* download resource when execute task
* download resource when execute task
* update authorization type
* download resource when execute task
* Spark task resource changes (#1990)
* File management list modification
* Shell task resources and authorization resources
* Spark task resource changes
* download resource when execute task
* update udf function service
* add resource type in ResourceComponent
* UDF resource tree and change DAG style (#2019)
* File management list modification
* Shell task resources and authorization resources
* Spark task resource changes
* UDF resource tree and change DAG style
* add deleteIds method in ResourceMapper and ResourceMapperTest
* Add comments on class and method
* add queryResourceByName method in controller
* update verify-name with change name to full name
* update queryResource with add parameter pid
* update queryResource with add parameter pid
* add resource ids in process definition and delete resource need judge whether it is used by any process definition
* Breadcrumb development (#2033)
* File management list modification
* Shell task resources and authorization resources
* Spark task resource changes
* UDF resource tree and change DAG style
* Breadcrumb development
* Breadcrumb development
* Resource tree bug fix (#2040)
* File management list modification
* Shell task resources and authorization resources
* Spark task resource changes
* UDF resource tree and change DAG style
* Breadcrumb development
* Breadcrumb development
* Resource tree bug fix
* update resource service test
* Fix github action rerun failed
* add status of PARENT_RESOURCE_NOT_EXIST
* Fix github action rerun failed (#2067)
* update resource service test
* Fix github action rerun failed
* add status of PARENT_RESOURCE_NOT_EXIST
* Change crumb position
* Change crumb position (#2068)
* build resource process definition map
* UDF changed to multiple choice
* UDF changed to multiple choice (#2077)
* Change crumb position
* UDF changed to multiple choice
* build resource process definition map (#2076)
* update resource service test
* Fix github action rerun failed
* add status of PARENT_RESOURCE_NOT_EXIST
* build resource process definition map
* update resource name also need update all the children full name
* need add queryResource
* update resource name also need update all the children full name (#2096)
* update resource service test
* Fix github action rerun failed
* add status of PARENT_RESOURCE_NOT_EXIST
* build resource process definition map
* update resource name also need update all the children full name
* need add queryResource
* Limit customization file content to no more than 3000 lines
* Limit customization file content to no more than 3000 lines(#2128)
* Limit customization file content to no more than 3000 lines(#2128) (#2140)
* Change crumb position
* UDF changed to multiple choice
* Limit customization file content to no more than 3000 lines
* Limit customization file content to no more than 3000 lines(#2128)
* add queryResourceJarList
* add queryResourceJarList
* add queryResourceJarList
* add queryResourceJarList (#2192)
* update resource service test
* Fix github action rerun failed
* add status of PARENT_RESOURCE_NOT_EXIST
* build resource process definition map
* update resource name also need update all the children full name
* need add queryResource
* add queryResourceJarList
* add queryResourceJarList
* add queryResourceJarList
* Modify the main jar package
* Modify the main jar package (#2200)
* Change crumb position
* UDF changed to multiple choice
* Limit customization file content to no more than 3000 lines
* Limit customization file content to no more than 3000 lines(#2128)
* Modify the main jar package
* add resource filter in order to get filtered resource
* add comments of resource filter
* update list children by resource
* choose main jar with resource tree (#2220)
* update resource service test
* Fix github action rerun failed
* add status of PARENT_RESOURCE_NOT_EXIST
* build resource process definition map
* update resource name also need update all the children full name
* need add queryResource
* add queryResourceJarList
* add queryResourceJarList
* add queryResourceJarList
* add resource filter in order to get filtered resource
* add comments of resource filter
* update list children by resource
* Return null if query resource list is empty
* update queryResource method change parameter pid to id
* getResouDelete checksum and modify parameter namerceId
* revert .env
* remove parameter pid
* Delete request interface
* go back to the last page
* jar interface call
* Fix issue #2234 and #2228
* change resource name with full name
* Fix issue #2234 and #2228 (#2246)
* update resource service test
* Fix github action rerun failed
* add status of PARENT_RESOURCE_NOT_EXIST
* build resource process definition map
* update resource name also need update all the children full name
* need add queryResource
* add queryResourceJarList
* add queryResourceJarList
* add queryResourceJarList
* add resource filter in order to get filtered resource
* add comments of resource filter
* update list children by resource
* Return null if query resource list is empty
* update queryResource method change parameter pid to id
* revert .env
* remove parameter pid
* Fix issue #2234 and #2228
* change resource name with full name
* Fix list query value error
* remove unauth-file with authorize-resource-tree
* Repair data cannot be echoed
* Repair data cannot be echoed
* execute mr and spark task need query resource name before
* Authorized resource interface replacement
* Authorized resource interface replacement
* Filter UDF resources
* Change parameters
* need query all authorized directory children when create task
* Change normalize.scss import method and animation.scss license modification
* Delete file list update processing
* It's fixed that resource not deleted in hdfs when delete it.
* add tooltips
* add tooltips (#2310)
* Echo workflow name and modify udf management name
* [new feature]add resource tree function
* revert front code in order to be same as dev branch
* revert front code in order to be same as dev branch
* revert common.properties and application.properties
* add super method
* update flink parameter test
* update flink parameter and unit test
* update resource service test
* If resource list is empty,need init it
* update flink parameter test
Co-authored-by: break60 <790061044@qq.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>
* 1，remove dolphinscheduler-rpc module 2，add dolphinscheduler-remote module 3，add dolphinscheduler-service module 4，refactor LoggerServer module (#1925)
* 1，remove dolphinscheduler-rpc module
2，add dolphinscheduler-remote module
3，add dolphinscheduler-service module
4，refactor LoggerServer module
* ProcessUtils modify
* Refactor architecture (#1926)
* move version to parent pom
* move version properties to parent pom for easy management
* remove freemarker dependency
* delete CombinedApplicationServer
* #1871 correct spelling
* #1873 some updates for TaskQueueZkImpl
* #1875 remove unused properties in pom
* #1878
1. remove tomcat dependency
2. remove combined_logback.xml in api module
3. format pom.xml for not aligning
* #1885 fix api server startup failure
1. add jsp-2.1 dependency
2. remove jasper-runtime dependency
* add stringutils ut (#1921)
* add stringutils ut
* Newfeature for #1675. (#1908)
Continue to finish the rest works, add the cache feature for dependence,mr,python,sub_process,procedure and shell.
* Add modify user name for process definition (#1919)
* class overrides equals() and should therefore also override hashCode()
* #1862 add modify user in process difinition list
* #1862 add pg-1.2.2 ddl.sql
* modify ScriptRunnerTest
* add updateProessDifinition UT
* modify updateProcessDifinition UT
* modify updateProcessDifinition UT
* modify mysql 1.2.2 ddl.sql&dml.sql
* add scope test to mysql in pom
* modify pg-1.2.2 ddl.sql
* refactor module
* updates
Co-authored-by: khadgarmage <khadgar.mage@outlook.com>
Co-authored-by: zhukai <boness@qq.com>
Co-authored-by: Yelli <amarantine@my.com>
* dolphinscheduler-common remove spring (#1931)
* dolphinscheduler-common remove spring
* dolphinscheduler-common remove spring
* dolphinscheduler-common remove spring
* dolphinscheduler-common remove spring
* dolphinscheduler-common remove spring
* SpringApplicationContext class title add license (#1932)
* dolphinscheduler-common remove spring
* dolphinscheduler-common remove spring
* dolphinscheduler-common remove spring
* dolphinscheduler-common remove spring
* dolphinscheduler-common remove spring
* dolphinscheduler-common remove spring
* add license (#1934)
* dolphinscheduler-common remove spring
* dolphinscheduler-common remove spring
* dolphinscheduler-common remove spring
* dolphinscheduler-common remove spring
* dolphinscheduler-common remove spring
* dolphinscheduler-common remove spring
* Refactor architecture (#1936)
* move datasource classes to dao module
* fix send4LetterWord bug
* LoggerServiceTest remove ProcessDao (#1944)
* dolphinscheduler-common remove spring
* dolphinscheduler-common remove spring
* dolphinscheduler-common remove spring
* dolphinscheduler-common remove spring
* dolphinscheduler-common remove spring
* dolphinscheduler-common remove spring
* LoggerServiceTest remove ProcessDao
* exclude jasper-compiler in case of runtime conflict (#1938)
* move datasource classes to dao module
* fix send4LetterWord bug
* exclude jasper-compiler in case of runtime conflict
* DataAnaylysisServiceTest and ProcessDefinitionService modify
* remote module add comment
* OSUtilsTest modify
* add finally block to close channel (#1951)
* move datasource classes to dao module
* fix send4LetterWord bug
* exclude jasper-compiler in case of runtime conflict
* add finally block to close channel
Co-authored-by: Tboy <technoboy@yeah.net>
Co-authored-by: khadgarmage <khadgar.mage@outlook.com>
Co-authored-by: zhukai <boness@qq.com>
Co-authored-by: Yelli <amarantine@my.com>
