Commit Graph

305 Commits

Author SHA1 Message Date
Kirs
e2243d63be
[Feature-#3961][Registry]Registry-SPI (#5562)
[Feature#3961]Registry SPI

All the logical structure of the registry must be converted into a tree structure within the system, so some plug-ins must be converted internally, such as ETCD
The registry supports distributed locks.
todo: The specific information about the registration center of the API module needs to be adjusted.
2021-06-09 15:21:02 +08:00
Chouc
75be09735a
[DS-5559][fix][Master Server] Master Server was shutdown but the process still in system (#5588)
* Close spring context to destory beans that has running thread
2021-06-04 13:27:18 +08:00
Shiwen Cheng
a5a0c7c5f8
[Fix-5570][Worker] Fix worker.groups in worker.properties is still commented and improve install scripts and configurations (#5571) 2021-06-02 11:21:01 +08:00
ruanwenjun
842c5400e6
[Improvement][Server] Must restart master if Zk reconnect (#5210) (#5211) 2021-05-25 05:03:29 +08:00
zhuangchong
1b7dcea610
[Improvement-5440][Maven] Project module maven dependency optimization (#5441)
* Optimize dependencies between modules.

* update module-alert dependency.

* add repository in the pom xml.

* update Central Repository in the pom.xml.
2021-05-19 10:05:00 +08:00
ruanwenjun
a925f64571
[Feature][api] schedule add timezone support #5259 (#5465)
* [Feature][api] schedule add timezone support #5259

* import moment-timezone

* fix code smell
2021-05-17 21:08:20 +08:00
baoliang
156dadac69 Merge remote-tracking branch 'upstream/dev' into json_split
# Conflicts:
#	dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/master/runner/MasterExecThread.java
#	dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/utils/DependentExecute.java
2021-05-11 10:08:28 +08:00
JinyLeeChina
515fa4912e
[Feature][JsonSplit] Fix ProcessService ut (#5435)
* Fix ProcessService ut

* codeStyle

* Fix AlertPlugin ut

* Fix AlertPlugin ut

* Fix server module ut

* Fix server module ut

* Fix api ut

* codeStyle

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-05-10 23:17:56 +08:00
zhuangchong
9c77faa8ac
[Improvement-5058][Master] The master-server creates the DB data source twice (#5139)
* reduce the number of data sources created twice by master server.

* update ProcessAlertManagerTest test class to increase coverage.

* add DolphinSchedulerPluginLoaderTest test ignore.

Co-authored-by: zhuang chong
2021-05-10 09:56:17 +08:00
JinyLeeChina
847865fba6
[Feature][JsonSplit] Fix dao ut (#5433)
* update taskParams/add task delayTime/fix conditionType bug

* update codeStyle for merge to dev

* fix process lineage bug

* fix codeStyle for json_split

* Fix dao ut

* Fix dao ut

* codeStyle

* modify postgreSQL

* fix TaskInstanceMapper ut

* fix ProjectMapper ut

* fix ProcessDefinitionDao ut

* datasource

* fix postgreSQL

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-05-09 13:53:14 +08:00
JinyLeeChina
2a44cb9de6
[Feature][JsonSplit] update codeStyle for merge to dev (#5392)
* update taskParams/add task delayTime/fix conditionType bug

* update codeStyle for merge to dev

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-04-26 22:30:35 +08:00
baoliang
746d9eca1d Merge remote-tracking branch 'upstream/dev' into json_split
# Conflicts:
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/ProcessDefinitionController.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProcessDefinitionVersionServiceImpl.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProjectServiceImpl.java
#	dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/controller/ProcessDefinitionControllerTest.java
#	dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/ProcessDefinitionVersionServiceTest.java
#	dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/model/TaskNode.java
#	dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/master/runner/MasterBaseTaskExecThread.java
#	dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/task/processdure/ProcedureTask.java
#	dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/process/ProcessService.java
2021-04-26 14:14:05 +08:00
JinyLeeChina
c3558965c1
update taskParams/add task delayTime/fix conditionType bug (#5385)
Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-04-26 11:28:35 +08:00
JinyLeeChina
db96bf2dfe
[Feature][JsonSplit] Fix batchMove of processDefinition bug (#5371)
* update SnowFlake

* update processDefinite from processInstance

* update processDefinite from processInstance

* Fix task logger path

* Fix dependTask bug

* Fix batchMove of processDefinition bug

* codeStyle

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-04-24 15:33:23 +08:00
JinyLeeChina
862565a7c7
[Feature][JsonSplit] Fix dependTask bug (#5360)
* update SnowFlake

* update processDefinite from processInstance

* update processDefinite from processInstance

* Fix task logger path

* Fix dependTask bug

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-04-23 14:23:04 +08:00
wanghong1314
e12a26289b
【BUG-5328】 modify process contain depend task is always running (#5344)
Co-authored-by: wanghong1314 <wanghong1@onething.net>
2021-04-21 22:29:53 +08:00
JinyLeeChina
d60f31bcf3
[Feature][JsonSplit] update processDefinite from processInstance (#5325)
* update SnowFlake

* update processDefinite from processInstance

* update processDefinite from processInstance

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-04-20 15:33:37 +08:00
JinyLeeChina
49b234609a
update SnowFlake (#5321)
Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-04-19 23:12:09 +08:00
luoyuan
513eb76919
[Improvement-5147][Service]This judgment always returns true (#5156)
* this expression which always evaluates "true"

* delete unused variables

* LogClientService add AutoCloseable

* reformat code

* reformat code

* reformat code

* add log in ProcessService

* reformat code

* Optimize the code

* Optimize the code

* [improvement]modify all methods that refer to LogClientService in the code.

* delete unused code
2021-04-19 17:48:09 +08:00
QuakeWang
5ef1f731b7
Update PeerTaskInstancePriorityQueueTest.java (#5314)
change `TaskInstance taskInstanceMediumPriority = createTaskInstance("high", Priority.MEDIUM);` to `TaskInstance taskInstanceMediumPriority = createTaskInstance("medium", Priority.MEDIUM);`
2021-04-19 09:48:43 +08:00
JinyLeeChina
f7a06a033d
[Feature][JsonSplit] Fix master/worker about shellTask (#5300)
* Fix quoted bug about processDefineId of processInstance

* Fix master/worker about shellTask

* codeStyle

* codeStyle

* update taskType

* codeStyle

* update timeoutNotifyStrategy

* codeStyle

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-04-18 12:45:30 +08:00
wangxj3
6dbbf6d55d
fix bug of If start a process instance form fail node or stop node ,the param will be init. (#5267)
Co-authored-by: wangxj <wangxj31>
2021-04-16 23:14:22 +08:00
wenjun
f3dfaacca2
[Improvement][Service] Optimize query log (#5205)
* [Improvement][Service] Optimize query log

* remove IPUtils
2021-04-15 20:25:01 +08:00
Shiwen Cheng
1c1f6663a6
[Improvement][WorkerGroup] Optimize and reduce database and zookeeper queries and replace worker address input with multiple-select (#5182)
* [Improvement][Server] Optimize and reduce database and ZooKeeper query

* [Improvement][WorkerGroup] Replace worker address input with multiple-select

* [Improvement][WorkerGroup] Fix unit test

* [Improvement][API&UI] Optimize long host to short host and long host display in k8s
2021-04-14 21:39:39 +08:00
baoliang
0166aebf71 Merge remote-tracking branch 'upstream/dev' into json_split
# Conflicts:
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/enums/Status.java
#	dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/utils/StringUtils.java
#	dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/ProjectMapper.java
#	dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/ProjectMapper.xml
#	dolphinscheduler-service/src/test/java/org/apache/dolphinscheduler/service/process/ProcessServiceTest.java
2021-04-13 23:10:19 +08:00
JinyLeeChina
645847c096
Fix quoted bug about processDefineId of processInstance (#5263)
Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-04-13 22:47:36 +08:00
Bruce Lau
3e64f43138
[Test-4124][service] Add the testCreateCommand test of ProcessService Class (#5257)
* [uint test] Improve code test coverage of ProcessService class

* [Test-4124][service] Add the testCreateCommand test of ProcessService Class

Co-authored-by: liuziying <liuziying@longfor.com>
2021-04-12 18:55:00 +08:00
JinyLeeChina
31188ffca0
[Feature][JsonSplit] Fix taskInstance (#5246)
* Fix dependency failure

* Fix taskInstance

* task instance list page

* code review

* fix logger path

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-04-11 16:52:19 +08:00
wen-hemin
cc82a98f50
[FixBug][JsonSplit] dag show bugs (#5226)
* fix: dag show bugs

* fix: code format

* fix: code format

* fix: code format

Co-authored-by: wen-hemin <wenhemin@apache.com>
2021-04-08 14:18:01 +08:00
JinyLeeChina
a8f2894b79
[Feature][JsonSplit] Fix master/processInstance bug (#5206)
* transform taskCode from long to string

* fix process bug

* code review

* code review

* code review

* Fix master/processInstance bug

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-04-04 11:26:42 +08:00
bao liang
86ce8f133a
[JsonSplit-4417][master/worker/api]json split- remove json in task instance. (#5160)
* [json split] refactor json in task instance

* add task_pramas in task_instance

* feature #4417 refactor json split in task instance

* code style

* code style

* code style

* code style

* code style

* update

* update

* update unit test

* update
2021-04-03 17:58:04 +08:00
JinyLeeChina
dd36da8d1f
[Feature][JsonSplit] fix process bug (#5193)
* transform taskCode from long to string

* fix process bug

* code review

* code review

* code review

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-04-01 17:45:44 +08:00
JinyLeeChina
689106d114
transform taskCode from long to string (#5174)
Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-03-30 11:07:28 +08:00
JinyLeeChina
a467fffd7b
[Feature][JsonSplit] fix taskId in json (#5165)
* modify checkDAGRing and ProcessService method

* merge

* modify dagRing

* modify process instance for project home page

* fix save process bug

* codeStyle

* Fix logical bug in saving process definition

* codeSytle

* Fix bug in interface of  queryProcessDefinitionList

* codeSytle

* Fix api bug"

* fix taskId in processDefinitionJson

* fix json taskId

* codeStyle

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-03-29 14:22:40 +08:00
lenboo
0914699fc5 Merge branch 'json_split' of https://github.com/apache/incubator-dolphinscheduler into spilit 2021-03-26 00:06:26 +08:00
lenboo
f05ba84968 Merge remote-tracking branch 'upstream/dev' into spilit
# Conflicts:
#	dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/ProcessInstanceMapper.java
#	dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/TaskInstanceMapperTest.java
2021-03-25 23:54:01 +08:00
Simon
cd19507271
[feature][jsonsplit]fix copy/export/import processDefinition bugs (#5146)
* Modify Project and ProjectUser Mapper

* Modify Project and ProjectUser Mapper

* project_code is bigint(20)

* modify ERROR name

* modify saveProcessDefine, remove the duplicate code with createTaskAndRelation

* modify import/export processdefinition, add genProcessData

* fix ut and bug

* code style

* repalce project_id with code

* conflicts solve

* conflicts solve

* conflicts solve

* bugfix

* modify listResources mothod and remove getResourceIds mothod

* 1

* conflicts solve

* modify listResources mothod and remove getResourceIds mothod

* modify listResources mothod and remove getResourceIds mothod

* replace processDefinitionVersion with processDefinitionLog

* codestyle

* codestyle

* add mapper module ut

* codestyle

* fix ProcessInstanceMapperTest

* codestyle

* conflicts solve

* conflicts solve

* conflicts solve

* conflicts solve

* conflicts solve

* fix ProcessInstanceMapperTest

* fix ProjectMapperTest/ProjectUserMapperTest/ScheduleMapperTest

* fix ProjectMapperTest/ProjectUserMapperTest/ScheduleMapperTest

* fix TaskInstanceMapperTest

* add TaskDefinitionLogMapperTest/TaskDefinitionMapperTest and bugfix

* codestyle

* codestyle

* Refactor process lineage

* Refactor process lineage

* codestyle

* codestyle

* Refactor process lineage

* Refactor process lineage

* add WorkFlowLineageMapperTest/WorkFlowLineageServiceTest

* add WorkFlowLineageMapperTest/WorkFlowLineageServiceTest

* add ProcessServiceTest

* 1

* 1

* fix copy/export/import processDefinition bugs

* 1

* codestyle
2021-03-25 18:27:47 +08:00
Simon
3c9b214da7
[Feature][jsonsplit]add ProcessServiceTest (#5097)
* Modify Project and ProjectUser Mapper

* Modify Project and ProjectUser Mapper

* project_code is bigint(20)

* modify ERROR name

* modify saveProcessDefine, remove the duplicate code with createTaskAndRelation

* modify import/export processdefinition, add genProcessData

* fix ut and bug

* code style

* repalce project_id with code

* conflicts solve

* conflicts solve

* conflicts solve

* bugfix

* modify listResources mothod and remove getResourceIds mothod

* 1

* conflicts solve

* modify listResources mothod and remove getResourceIds mothod

* modify listResources mothod and remove getResourceIds mothod

* replace processDefinitionVersion with processDefinitionLog

* codestyle

* codestyle

* add mapper module ut

* codestyle

* fix ProcessInstanceMapperTest

* codestyle

* conflicts solve

* conflicts solve

* conflicts solve

* conflicts solve

* conflicts solve

* fix ProcessInstanceMapperTest

* fix ProjectMapperTest/ProjectUserMapperTest/ScheduleMapperTest

* fix ProjectMapperTest/ProjectUserMapperTest/ScheduleMapperTest

* fix TaskInstanceMapperTest

* add TaskDefinitionLogMapperTest/TaskDefinitionMapperTest and bugfix

* codestyle

* codestyle

* Refactor process lineage

* Refactor process lineage

* codestyle

* codestyle

* Refactor process lineage

* Refactor process lineage

* add WorkFlowLineageMapperTest/WorkFlowLineageServiceTest

* add WorkFlowLineageMapperTest/WorkFlowLineageServiceTest

* add ProcessServiceTest

* 1

* 1
2021-03-24 15:14:22 +08:00
wen-hemin
e53367bdcc
[Feature][JsonSplit] task page, rename field "description" to "desc" (#5137)
* fix: task page, rename "description" to "desc"

* fix: The default value of field dependence is null.

* fix: frontend compile check

Co-authored-by: wen-hemin <wenhemin@apache.com>
2021-03-24 15:05:54 +08:00
JinyLeeChina
1eaf1f6234
[Feature][JsonSplit] fix taskId in processDefinitionJson (#5136)
* modify checkDAGRing and ProcessService method

* merge

* modify dagRing

* modify process instance for project home page

* fix save process bug

* codeStyle

* Fix logical bug in saving process definition

* codeSytle

* Fix bug in interface of  queryProcessDefinitionList

* codeSytle

* Fix api bug"

* fix taskId in processDefinitionJson

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-03-24 11:44:14 +08:00
Shiwen Cheng
0d065c0712
[Feature-5062][*] Create/Edit/Delete Worker Group in Web UI (#5113)
* [Improvement][Server] Rename MASTER_PREFIX to MASTER_TYPE and WORKER_PREFIX to WORKER_TYPE

* [Feature][Api&Dao] Support api and dao of worker group

* [Feature][UI] Support to create, edit and delete worker group

* [Feature][API&UI] Support work groups from database and zookeeper to display together in worker group management

* [Feature][API&Dao] Support long ip list and check request parameters when saving worker group

* [Feature][*] Refactor ipList to addrList

* [Feature][Server] Support host manager to select worker host by worker group in database

* [Improvement][Server] Improve the config of master and worker

* [Improvement][*] Rename zkRegistered to systemDefault

* [Improvement][Test] Fix unit test and improve ui

* [Improvement][Server] Improve getServerMaps

Co-authored-by: dailidong <dailidong66@gmail.com>
2021-03-23 23:47:41 +08:00
JinyLeeChina
d99934d0da
[Feature][JsonSplit] Fix api bug (#5102)
* modify checkDAGRing and ProcessService method

* merge

* modify dagRing

* modify process instance for project home page

* fix save process bug

* codeStyle

* Fix logical bug in saving process definition

* codeSytle

* Fix bug in interface of  queryProcessDefinitionList

* codeSytle

* Fix api bug"

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-03-23 10:44:25 +08:00
lenboo
010b49c933 merge from dev 2021-03-22 15:18:16 +08:00
zhuangchong
ba039dc252
[Improvement][dao] Resource features SQL slow query optimization. (#4947)
* Resourse features SQL slow query optimization.

* update resource mapper queryResourcePaging method.

* split resource and resource_uesr sql statement.

* determine the size of the collection.

* update process service class code style.

* update ResourceUserMapper java class code style.

* add resourceUserMapper in the test config file.

* solve code smell

* split paging query join statement.

* update ResourceUserMapper class.

* remove sql join statement.

* update resource test class code style.

* update ResourcesServiceTest test class.

* rerun ut.

* update user service test class.

* solve query resource list method code smell.
2021-03-20 12:50:26 +08:00
wangxj3
29d42fd92d
[Feature-#3805][server-master] Task parameter transfer (#5077)
* fix out param format bug

Co-authored-by: wangxj <wangxj31>
2021-03-18 11:50:10 +08:00
Shiwen Cheng
5856a12855
[Improvement-4984][Worker] Refactor and Improve worker load balance (#4996)
* [Improvement][Remote] Improve unit tests

* [Improvement][Worker] Improve worker load balance

* [Improvement][Worker] Fix code smells

* [Improvement][Worker] Rename weight to hostWeight
2021-03-18 10:03:58 +08:00
Ts686
6216817861
[Improvement-4971][alert|dao|service]Remove redundant PropertyUtils.class (#5020)
* [Improvement][alert|dao|service]
1. Remove redundant PropertyUtils.class (#4971)
2. Remove useless import package.

* [Improvement][alert|dao|service]
1. Remove redundant PropertyUtils.class (#4971)
2. Remove useless import package.

* [Improvement][alert|dao|service]
1. Remove redundant PropertyUtils.class (#4971)
2. Remove useless import package.

* [Improvement][alert|dao|service]
1. Remove redundant PropertyUtils.class (#4971)
2. Remove useless import package.
2021-03-17 15:28:07 +08:00
JinyLeeChina
d98c2c5cfe
[Feature][JsonSplit] Fix bug in interface of queryProcessDefinitionList (#5043)
* modify checkDAGRing and ProcessService method

* merge

* modify dagRing

* modify process instance for project home page

* fix save process bug

* codeStyle

* Fix logical bug in saving process definition

* codeSytle

* Fix bug in interface of  queryProcessDefinitionList

* codeSytle

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-03-16 15:44:03 +08:00
lenboo
a574d2e38c Merge remote-tracking branch 'upstream/dev' into spilit 2021-03-12 10:44:43 +08:00
Shiwen Cheng
ae608e024a
[Fix-5037][Server] Fix that both the master and the worker is hanging after restarting and stopping (#5038)
* [Fix-5037][Server] Fix that both the master and the worker is hanging after restarting and stopping

* [Improvement][*] Replace commons.lang.StringUtils with common.utils.StringUtils
2021-03-12 09:33:30 +08:00
wenjun
ea6b1de120
[Improvement][worker] Optimize KillYarnJob (4939) (#4943) 2021-03-11 22:13:44 +08:00
JinyLeeChina
c010ac1f1d
[Feature][JsonSplit] Fix logical bug in saving process definition (#5009)
* modify checkDAGRing and ProcessService method

* modify process instance for project home page

* fix save process bug

* Fix logical bug in saving process definition

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-03-09 12:54:25 +08:00
JinyLeeChina
7f0678d3be
[Feature][JsonSplit] Fix save process bug (#5001)
* modify checkDAGRing and ProcessService method

* merge

* modify dagRing

* modify process instance for project home page

* fix save process bug

* codeStyle

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-03-08 17:01:02 +08:00
Simon
6111f05250
[Feature][jsonspilt]fix ProcessInstanceMapperTest (#4989)
* Modify Project and ProjectUser Mapper

* Modify Project and ProjectUser Mapper

* project_code is bigint(20)

* modify ERROR name

* modify saveProcessDefine, remove the duplicate code with createTaskAndRelation

* modify import/export processdefinition, add genProcessData

* fix ut and bug

* code style

* repalce project_id with code

* conflicts solve

* conflicts solve

* conflicts solve

* bugfix

* modify listResources mothod and remove getResourceIds mothod

* 1

* conflicts solve

* modify listResources mothod and remove getResourceIds mothod

* modify listResources mothod and remove getResourceIds mothod

* replace processDefinitionVersion with processDefinitionLog

* codestyle

* codestyle

* add mapper module ut

* codestyle

* fix ProcessInstanceMapperTest

* codestyle

* conflicts solve

* conflicts solve

* conflicts solve

* conflicts solve

* conflicts solve

* fix ProcessInstanceMapperTest
2021-03-08 10:22:36 +08:00
bao liang
5e2b97b573
[feature-4417][JsonSplit]refactor json in process instance and process definition (#4993)
* remove the use of json

* remove json in process definition and process instance
2021-03-08 08:44:34 +08:00
JinyLeeChina
39a9c0c392
[Feature][jsonspilt]modify process instance for project home page (#4991)
* modify checkDAGRing and ProcessService method

* merge

* modify dagRing

* modify process instance for project home page

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-03-07 18:12:01 +08:00
Simon
f24ecff840
[Feature][jsonsplit]add mapper module ut and some bugfix (#4957)
* Modify Project and ProjectUser Mapper

* Modify Project and ProjectUser Mapper

* project_code is bigint(20)

* modify ERROR name

* modify saveProcessDefine, remove the duplicate code with createTaskAndRelation

* modify import/export processdefinition, add genProcessData

* fix ut and bug

* code style

* repalce project_id with code

* conflicts solve

* conflicts solve

* conflicts solve

* bugfix

* modify listResources mothod and remove getResourceIds mothod

* 1

* conflicts solve

* modify listResources mothod and remove getResourceIds mothod

* modify listResources mothod and remove getResourceIds mothod

* replace processDefinitionVersion with processDefinitionLog

* codestyle

* codestyle

* add mapper module ut

* codestyle
2021-03-06 14:25:00 +08:00
lenboo
5bda7c344c Merge remote-tracking branch 'upstream/dev' into spilit 2021-03-05 22:11:55 +08:00
Shiwen Cheng
8c68e2c60b
[Fix][Worker] Fix worker cannot start and alertClientService NullPointerException (#4956)
* [Fix][Worker] Fix worker cannot start

* [Fix][Worker] Fix alertClientService NullPointerException

* [Fix][Worker] Improve ut coverage

* [Improvement][CI] Improve workflow ci
2021-03-05 09:50:49 +08:00
JinyLeeChina
779596cf05
[Feature][JsonSplit] modify checkDAGRing and ProcessService method (#4931)
* modify checkDAGRing and ProcessService method

* merge

* modify dagRing

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-03-04 10:22:14 +08:00
lenboo
dcf97920aa merge from dev 2021-03-03 2021-03-03 10:44:12 +08:00
Simon
c342980032
[Feature][jsonsplit] repalce project_id with code (#4904)
* Modify Project and ProjectUser Mapper

* Modify Project and ProjectUser Mapper

* project_code is bigint(20)

* modify ERROR name

* modify saveProcessDefine, remove the duplicate code with createTaskAndRelation

* modify import/export processdefinition, add genProcessData

* fix ut and bug

* code style

* repalce project_id with code

* conflicts solve

* conflicts solve

* conflicts solve

* bugfix
2021-03-02 16:43:19 +08:00
lgcareer
43ba29a2dd
[Improvement-4624] When the server exist in the dead server list of zk,need stop service byself (#4626)
* [Improvement-4624] When the server exist in the dead server list of zk,need stop service byself

* [Improvement-4624]fix check style and add MaterRegistryTest

* [Improvement-4624]fix check style and add ZookeeperRegistryCenterTest

* [Improvement-4624]fix check style and add ZookeeperRegistryCenterTest

* [Improvement-4624]add RegisterOperatorTest

* [Improvement-4624]update RegisterOperatorTest

* [Improvement-4624]resolve code smell

* [Improvement-4624]revert LICENSE-@form-create-element-ui
2021-03-02 14:05:00 +08:00
lenboo
2603d52044 Merge remote-tracking branch 'upstream/dev' into spilit 2021-03-01 15:20:34 +08:00
Simon
6d087ccce0
[Feature][JsonSplit] fix processDefinitionService ut and bug (#4894)
* Modify Project and ProjectUser Mapper

* Modify Project and ProjectUser Mapper

* project_code is bigint(20)

* modify ERROR name

* modify saveProcessDefine, remove the duplicate code with createTaskAndRelation

* modify import/export processdefinition, add genProcessData

* fix ut and bug

* code style
2021-03-01 15:01:17 +08:00
zhuangchong
a388b6853d
Optimize HashMap Initial Capacity. (#4896) 2021-02-27 21:36:03 +08:00
lenboo
929655fdbe refactor task node 2021-02-26 15:05:32 +08:00
lenboo
0571681ac4 Merge remote-tracking branch 'upstream/json_split' into spilit 2021-02-25 18:09:48 +08:00
lenboo
ccc5aa91d8 merge from dev 2021-02-25 18:09:32 +08:00
bao liang
fd7aa53fd3
[Feature-4417][JsonSplit] definition log extends definition (#4870)
* task definition log extends task definition

* process definition log extends process definition
process task relation log extends process task relation

* code style

* code style

* code style

* update log extends definition

* code style

* code style
2021-02-25 18:03:19 +08:00
WilliamChen-luckbob
e47e4a70ab
[Fix] while repeat running, the global parameters will not set to the user specified start parameters. (#4779)
* [fix]when repeat running, there are no startparams which was set in the first start

* [fix] cannot set global param while rerun process instance

* fix code format

* delete unrelated test

* Update ExecutorControllerTest.java

* fix code smell

* Update ExecutorService2Test.java

don't know why e2e failed

* replace duplicate code
2021-02-24 23:48:23 +08:00
Simon
2fdeba0988
[Feature][JsonSplit]modify import/export processdefinition, add genProcessData (#4864)
* Modify Project and ProjectUser Mapper

* Modify Project and ProjectUser Mapper

* project_code is bigint(20)

* modify ERROR name

* modify saveProcessDefine, remove the duplicate code with createTaskAndRelation

* modify import/export processdefinition, add genProcessData
2021-02-24 18:55:45 +08:00
lenboo
22729669f9 Merge remote-tracking branch 'upstream/dev' into spilit 2021-02-24 17:50:26 +08:00
JinyLeeChina
a32b40ee95
[Feature][JsonSplit] add gen taskNodeList method (#4861)
* add gen taskNodeList method

* modify code style

* modify code style

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-02-24 17:36:01 +08:00
wuxiaofei
b2a871b4c5 [BUG][PeerTaskInstancePriorityQueue]string formatter 2021-02-24 12:18:29 +08:00
lenboo
2afc9f3bfc merge from dev 2021-02-24 12:10:07 +08:00
wuxiaofei
8986c845a9 [BUG][PeerTaskInstancePriorityQueue]poll with timeout is not currently supported 2021-02-24 11:47:19 +08:00
wuxiaofei
488726139f [Performance][PeerTaskInstancePriorityQueue]code style formatter 2021-02-24 10:52:48 +08:00
wuxiaofei
911cf061e0 Merge branch 'dev' of https://github.com/apache/incubator-dolphinscheduler into dev_wuxiaofei_server 2021-02-24 10:49:41 +08:00
wuxiaofei
3a215cb952 [Performance][PeerTaskInstancePriorityQueue]Nearly realization of poll with timeout 2021-02-24 10:49:07 +08:00
dailidong
551759965d
Merge pull request #4743 from apache/params-trans
[Feature-3805][mater-worker-ui] The implement of passing variables between tasks
2021-02-23 19:06:13 +08:00
Shiwen Cheng
149b0dab4d
[Improvement][Docker] Sync docker conf templates to the latest conf properties (#4772) 2021-02-23 14:47:10 +08:00
wuxiaofei
6e0093c9ca [Test][service]Adjust test of queue 2021-02-22 15:48:59 +08:00
wuxiaofei
9d8b35a4b4 [Test][TaskPQImplTest]license header appended 2021-02-22 14:14:37 +08:00
wuxiaofei
8d00386d27 [CodeStyle][QueueTest]import style 2021-02-22 13:25:57 +08:00
wuxiaofei
450c84baf1 [CodeStyle][PeerQueueTest]TimeUnit seperated from previous import 2021-02-22 13:24:15 +08:00
wuxiaofei
28975d7397 [CodeStyle][QueueTest]empty line more than one line of impor style 2021-02-22 12:25:02 +08:00
wuxiaofei
d1189674af [CodeStyle][QueueTest]code style 2021-02-22 12:21:14 +08:00
wuxiaofei
1838c817a1 [CodeStyle][QueueTest]import style 2021-02-22 12:18:23 +08:00
wuxiaofei
9da30dd175 [Test][TaskPriorityQueueImplTest]Test with coverity 2021-02-22 12:14:25 +08:00
wuxiaofei
bc2a8fb149 [Test][PeerTaskInstancePriorityQueueTest]testPoll added 2021-02-22 10:52:38 +08:00
wuxiaofei
5faa5d1703 [BUG][PeerTaskInstancePriorityQueue]poll implemented in PriorityQueue 2021-02-22 10:20:22 +08:00
bao liang
62e961e3f1
[Feature][JsonSplit]refactor remove the json in process instance and definition (#4828)
* #4417 [JsonSplit] refactor process definition json

* [#4177] refactor json split. remove json in process definition and process instance

* [#4177] refactor json split. copy process definition need set task code 0L.

* code style

* code style

* code style

* code style

* update

* update
2021-02-22 10:13:32 +08:00
wuxiaofei
b352c52f33 [Improvement][master] Replace sleep&task with poll in TaskPriorityQueueConsumer#4137 2021-02-21 23:47:48 +08:00
lenboo
4c9c41933d Merge remote-tracking branch 'upstream/dev' into spilit
# Conflicts:
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/ProcessInstanceService.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProcessDefinitionServiceImpl.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProcessDefinitionVersionServiceImpl.java
2021-02-21 14:54:18 +08:00
Simon
abf6723d38
[Feature][JsonSplit] modify saveProcessDefine remove the duplicate code (#4810)
* Modify Project and ProjectUser Mapper

* Modify Project and ProjectUser Mapper

* project_code is bigint(20)

* modify ERROR name

* modify saveProcessDefine, remove the duplicate code with createTaskAndRelation
2021-02-20 15:11:19 +08:00
wangxj3
874ca07e6e
[Feature-3805][mater-worker-ui]fix function of passing variables between tasks bug (#4811)
* global initParam and set Param

* fix dataFormat error

* fix deal outParams bug

* fix code style

* fix code style

* fix code style

* fix code style

* fix code style

* fix code style

* fix code style

* add test

* fix code style (variable name)

* fix reset globalParams bug

* fix format bug

* fix SonarCloud Code Analysis

* add test

* fix code style

* fix code style

* modify test

* fix test Coverage

* fix code style

* 4 test

* add test pom

* add header

Co-authored-by: wangxj <wangxj31>
2021-02-20 14:32:46 +08:00
Shiwen Cheng
8879d8ff60
[Improvement][*] Define version with revision globally and update version to 1.3.6-SNAPSHOT (#4773)
* [Improvement][*] Update version to 1.4.0-SNAPSHOT

* [Improvement][*] Update tab to space under checkstyle

* [Improvement][*] Define version with revision globally

* [Improvement][*] Update version to 1.3.6-SNAPSHOT
2021-02-20 14:22:49 +08:00
dddyszy
ce007813fb
[Improvement][Api] add @Override on the Override method (#4808)
Co-authored-by: 施洲轶 <shizhy@yoozoo.com>
2021-02-19 18:15:43 +08:00
lenboo
59811d0efb Merge remote-tracking branch 'upstream/dev' into params-trans 2021-02-19 16:02:55 +08:00
Shiwen Cheng
15a5b05883
[Improvement-3369][api] Introduce executor and datasource service interface for clear code (#4759) 2021-02-18 23:27:37 +08:00
yimaixinchen
938621ab34
initStateLister rename to initStateListener (#4795)
Co-authored-by: v_taoouyang <v_taoouyang@tencent.com>
2021-02-18 18:48:09 +08:00
bao liang
67eae43154
[Feature-4417][JsonSplit] refactor process dag generate (#4790)
* #4417 [JsonSplit] refactor dag generate

* #4417 [JsonSplit] refactor dag generate

* #4417 [JsonSplit] add postNodeVersion and preNodeVersion

* #4417 [JsonSplit] code style

* #4417 [JsonSplit] code style

* #4417 [JsonSplit] code style
2021-02-18 17:19:33 +08:00
JinyLeeChina
069e9f980e
[Feature][JsonSplit] modify ProcessService createTaskAndRelation (#4770)
* add task query

* modify codestyle

* add task delete/update/swich method

* add task delete/update/swich method

* codestyle

* use updateById save task definition

* modify method name

* code style

* code style

* modify ProcessService createTaskAndRelation

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-02-12 08:43:44 +08:00
JinyLeeChina
763a50d0cf
[Feature][JsonSplit] add relation method and refactor task method (#4737)
* add task query

* modify codestyle

* add task delete/update/swich method

* add task delete/update/swich method

* codestyle

* use updateById save task definition

* modify method name

* code style

* code style

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-02-09 09:52:29 +08:00
wangxj3
87ff13fc86
[Feature-#3805][server-master] global params of master (#4678)
* global initParam and set Param

* fix dataFormat error

* fix deal outParams bug

* fix code style

* fix code style

* fix code style

* fix code style

* fix code style

* fix code style

* fix code style

* add test

* fix code style (variable name)

* fix reset globalParams bug

Co-authored-by: wangxj <wangxj31>
2021-02-07 17:10:33 +08:00
bao liang
f31eee4341
[Feature][JsonSplit] refactor process definition update (#4708)
* add code in task_instance and process instance

* delete process_definition_id in t_ds_task_instance

* add task_code task_definition_version process_definition_code in task instance

* add task_code task_definition_version process_definition_code in task instance

* refactor process instance

* refactor process instance update

* refactor json-split for process definition and task definition
refactor process instance update

* refactor json-split for process definition and task definition
refactor process instance update

* code style

* code style

* code style

* code style

* refactor code
2021-02-07 16:15:35 +08:00
zt-1997
14d49bc584
[FIX-2101]It solves the problem that the name of the branch will not change after the name of the branch is changed in the workflow page and instance page. (#4555)
Co-authored-by: zt-1997 <“18841012545@163.com”>
2021-02-02 16:07:37 +08:00
Segun Ogundipe
e083e28720
[Improvement][service] Refactor service module to fix code smell (#4513)
* chore: Refactor code to fix code smell in service module

* chore: Add licence header to new files

* chore: Fix comment from code review

* chore[service]: Reduce the number of custom runtime exception to one
2021-01-29 19:44:33 +08:00
Kirs
73b0eaa999
merge dev to alert_plugin_design (#4530)
* merge from 1.3.3-release

* refactor code style

* refactor code style

* refactor code style

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* refactor ut test

* refactor ut test

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* fix #3900 kill multi yarn app in one job

* refactor ut

* merge from 1.3.3-release

* refactor ut

* refactor ut

* refactor

* refactor

* refactor code style

* refactor code style

* refactor code style

* refactor code style

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* add ProcessUtils UT

* refactor code style

* refactor code style

* refactor code style

* refactor code style

* [Draft][Merge][133-dev]133 merge dev (#4031)

* [Feture-3327][ui]Add the function of re-uploading files in the resource center

* [Feture-3327][ui]Add the function of re-uploading files in the resource center (#3394)

* Before creating a workflow, clear the canvas

* [Fix-3256][ui] herry pick commit from dev for Fix admin user info update error (#3306)

* [Feture-3327][ui]Add the function of re-uploading files in the resource center

Co-authored-by: wuchunfu <319355703@qq.com>

* [Improvement-3327][api]support re-upload the resource file (#3395)

* [Fix-3390][server]Running hive sql task need find the hdfs path correctly (#3396)

* [Fix-3390][api]Running hive sql task need find the hdfs path correctly

* [Fix-3390][api]Running hive sql task need find the hdfs path correctly

* update soft version

* hive UDF function to modify the background color

* fix

* fix bug: Fix master task dependency check bug

* cancel spark task version check (#3406)

Co-authored-by: Eights-Li <yelli.hl@gmail.com>

* [Bug][ui]Fix front-end bug #3413

* [Feature][ambari_plugin]support one worker can belongs different worker groups when execute install script (#3410)

* Optimize dag

* Update actions.js (#3401)

* [Fix-3256][ui] Fix admin user info update error (#3425) (#3428)

* [PROPOSAL-3139] Datasource selection changes from radio to select

* [PROPOSAL-3139] Datasource selection changes from radio to select

* [BUG FIX] issues #3256

* [BUG FIX] issues #3256

* [BUG FIX] issues #3256

* [Fix-3256][ui] Fix admin user info update error

* [Fix-3256][ui] Fix admin user info update error

* [Fix-3256][ui] Fix admin user info update error

* [Fix-3256][ui] Fix admin user info update error

* reset createUser.vue

* [Fix-3256][ui] Fix admin user info update error

Co-authored-by: dailidong <dailidong66@gmail.com>

Co-authored-by: wuchunfu <319355703@qq.com>
Co-authored-by: dailidong <dailidong66@gmail.com>

* [Fix-3433][api]Fixed that release the imported process definition which version is below 1.3.0 will be failure

* dag connection add check

* fix

* [Fix-3423][dao][sql]Fixed that the resource file of the task node can't be found when upgrade from 1.2.0 to 1.3.x (#3454)

* Remove node deep monitoring

* If worker group id is null,don't need to set the value of the worker group (#3460)

* [Fix-3423][dao][sql]Fixed that the resource file of the task node can't be found when upgrade from 1.2.0 to 1.3.x

* [Fix-3423][dao]If worker group id is null,don't need to set the value of the worker group

* [ui]Code optimization

* fix

* fix

* [fix-3058][ui]Move rtTargetArr to jsPlumbHandle.js

* [optimization][ui]Prevent the shell script input box from being empty

* [Fix-3462][api]If login user is admin,need list all udfs (#3465)

* [Fix-3462][api]If login user is admin,need list all udfs

* [Fix-3462][api]add the test on the method of QueryUdfFuncList

* [Fix-3462][api]fix the code smell

* [Fix-3462][api]fix the code smell

* [Fix-3462][api]fix the code smell

* [Fix-3463][api]Fixed that run the sql task will be failure after rename the udf resource (#3482)

* [fixBug-3058][ui]Fix connection abnormalities in historical workflow instance data

* [Feture-3327][ui]Add the function of re-uploading files in the udf subdirectory

* fix bug: Fix master task dependency check bug (#3473)

Co-authored-by: lenboo <baoliang@analysys.com.cn>

* [maven-release-plugin] prepare release 1.3.2

* [maven-release-plugin] prepare for next development iteration

* fix ci_e2e fail (#3497)

* [Fix-3469][api]Should filter the resource by the different program type (#3498)

* [Fix-3463][api]Fixed that run the sql task will be failure after rename the udf resource

* [Fix-3469][api]Should list python file and jar file

* [Fix-3469][api]Should filter the resource by the different program type

* [Fix-3469][api]fix the code smell

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback (#3499)

* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* [maven-release-plugin] prepare release 1.3.2

* [maven-release-plugin] prepare for next development iteration

* [Fix-3469][ui]The value of maintenance resources and the filtering of resources according to different program types

* fix

* Revert "fix ci_e2e fail (#3497)"

This reverts commit e367f90bb7.

* test

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback (#3503)

* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test

* test release 1.3.2 version rollback

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* [maven-release-plugin] prepare release 1.3.2

* [maven-release-plugin] prepare for next development iteration

* test release 1.3.2 version rollback (#3504)

* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test

* test release 1.3.2 version rollback

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* [maven-release-plugin] prepare release 1.3.2

* [maven-release-plugin] prepare for next development iteration

* fix ds muti-level directory in zk, which lead to fail to assign work

* add login user check some actions in api

* [Hotfix][ci] Fix e2e ci docker image build error

* modify tag 1.3.0 to HEAD

* modify tag 1.3.0 to HEAD (#3525)

* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test

* test release 1.3.2 version rollback

* modify tag 1.3.0 to HEAD

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* remove OGNL part of the mybaits notice (#3526)

* [maven-release-plugin] prepare release 1.3.2

* [maven-release-plugin] prepare for next development iteration

* release 1.3.2 version rollback (#3527)

* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test

* test release 1.3.2 version rollback

* modify tag 1.3.0 to HEAD

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* [ui]Script input box to modify the delay loading time

* fix

* fix

* fix

* fix

* modify general user can't create token

* [ui]It is forbidden to select non-existent resources and modify the tree display data format

* modify general user can't create token (#3533)

* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test

* test release 1.3.2 version rollback

* modify tag 1.3.0 to HEAD

* modify general user can't create token

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* if task is null , set task type is null instead of "null"

* [Fix-3536][api]If user didn't have tenant,create resource directory will NPE (#3537)

* [Fix-3536][api]If user didn't have tenant,create resource will NPE

* [Fix-3536][api]If user didn't have tenant,create resource directory will NPE

* modify general user can't create,delete,update token (#3538)

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* [ui]Resource delete OK button to increase loading, change the number of homepage display cursor

* fix

* [Fix-3616][Server] when worker akc/response master exception , async retry (#3748)

* [fixbug][ui]Repair the master and worker management instrument display

* [Fix-3238][docker]Fix that can not create folder in docker with standalone mode (#3741)

* [fixbug][ui]Remove non-existent or deleted resources disabled

* [fixBug-3621][ui]If the workflow instance status is executing status, it is forbidden to select

* [fix-3553][ui]Repair click workflow connection, select the entire path

* fix

* fix

* [Fix-3238][docker]Fix that can not create folder in docker with standalone mode

* [Fix-3616][Server] when worker akc/response master exception , async retry (#3776)

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* The batch delete function in the workflow definition and workflow instance pages cannot be canceled if selected.

* [Improvement-3720][ui] js mailbox verification fix

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect (#3784)

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* The batch delete function in the workflow definition and workflow instance pages cannot be canceled if selected.

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
Co-authored-by: zhuangchong <zhuangchong8@163.com>
Co-authored-by: JinyLeeChina <42576980+JinyLeeChina@users.noreply.github.com>

* [Fix-3124][docker]Fix that can not build a docker image on windows (#3765)

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect (#3786)

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* The batch delete function in the workflow definition and workflow instance pages cannot be canceled if selected.

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
Co-authored-by: zhuangchong <zhuangchong8@163.com>
Co-authored-by: JinyLeeChina <42576980+JinyLeeChina@users.noreply.github.com>

* [Fix-3258][Security][Worker group manage] Connot get create time and update time,report DateTimeParseException (#3787)

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* The batch delete function in the workflow definition and workflow instance pages cannot be canceled if selected.

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

* [BugFixed] issue #3258 (#3265)

* 'ExecutionStatus'

* '3258'

* Update WorkerGroupServiceTest.java

* Delete UserState.java

* Delete ResourceSyncService.java

* Delete core-site.xml

* Delete hdfs-site.xml

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
Co-authored-by: zhuangchong <zhuangchong8@163.com>
Co-authored-by: JinyLeeChina <42576980+JinyLeeChina@users.noreply.github.com>
Co-authored-by: dailidong <dailidong66@gmail.com>

* [fixBug-3792][ui]Click on the sidebar to adapt the width of the pie chart on the project homepage

* [Bug-3713][HadoopUtils] catfile method Stream not closed  (#3715)

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* dag  add close button

* reset last version

* reset last version

* dag add close buttion

dag add close buttion

* update  CLICK_SAVE_WORKFLOW_BUTTON  xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* Update CreateWorkflowLocator.java

modify submit workflow button

* Update CreateWorkflowLocator.java

* Update CreateWorkflowLocator.java

modify CLICK_ADD_BUTTON

* Update CreateWorkflowLocator.java

delete print

* Update CreateWorkflowLocator.java

1

* Update CreateWorkflowLocator.java

1

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dag.vue

* Update dag.vue

* Update dag.vue

* Update CreateWorkflowLocator.java

* Revert "Setting '-XX:+DisableExplicitGC ' causes netty memory leaks"

This reverts commit 3a2cba7a

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dolphinscheduler-daemon.sh

* catfile method Stream not closed

* catfile method Stream not closed

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>

* [Fix-#3713][common]Fix that catfile method Stream not closed

* [Fix-#3713][common]Fix that catfile method Stream not closed (#3810)

* [Bug-3713][HadoopUtils] catfile method Stream not closed  (#3715)

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* dag  add close button

* reset last version

* reset last version

* dag add close buttion

dag add close buttion

* update  CLICK_SAVE_WORKFLOW_BUTTON  xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* Update CreateWorkflowLocator.java

modify submit workflow button

* Update CreateWorkflowLocator.java

* Update CreateWorkflowLocator.java

modify CLICK_ADD_BUTTON

* Update CreateWorkflowLocator.java

delete print

* Update CreateWorkflowLocator.java

1

* Update CreateWorkflowLocator.java

1

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dag.vue

* Update dag.vue

* Update dag.vue

* Update CreateWorkflowLocator.java

* Revert "Setting '-XX:+DisableExplicitGC ' causes netty memory leaks"

This reverts commit 3a2cba7a

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dolphinscheduler-daemon.sh

* catfile method Stream not closed

* catfile method Stream not closed

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>

* [Fix-#3713][common]Fix that catfile method Stream not closed

Co-authored-by: BoYiZhang <39816903+BoYiZhang@users.noreply.github.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>

* [Fix-#3487][api、dao] cherry pick from dev to fix that create folder duplicate name under multithreading

* [Hotfix-3131][api] Fix the new tenant already exists prompt (#3132)

* Bugfix: Fix the new tenant already exists prompt

* Feature: Add test cases

* Update TenantServiceTest.java

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>

* Set up JDK 11 for SonarCloud in github action. (#3052)

* Set up JDK 11 for SonarCloud in github action.

* Fix javadoc error with JDK 11.

* Prevent Javadoc from stopping if it finds any html errors.

* [fixBug-3621][ui]Select the batch checkbox to unfilter the instances in the executing state

* add verify tenant name cannot contain special characters.

* [fixBug-3840][ui]The tenant code only allows letters or a combination of letters and numbers

* fix

* fix

* fix

* [Fix-#3702][api] When re-upload the resource file but don't change the name or desc,it need replace the origin resource file. (#3862)

* [Fix-#3702][api] When re-upload the resource file but don't change the name or desc,it will not replace the origin resource file.

* [Fix-#3702][api] When re-upload the resource file but don't change the name or desc,it will not replace the origin resource file.

* [fixbug-3621][ui]Workflow instance ready to stop and ready to suspend state prohibits checking

* [fixbug-3887][ui]Fix missing English translation of re-upload files

* add process define name verify. (#3879)

* Revert "[1.3.3-release][fix-3835][ui] When the tenantName contains "<", the tenant drop-down list is blankadd verify tenant name cannot contain special characters."

* revert pr 3872

* [FIX-3617][Service]after subtask fault tolerance, 2 task instances are generated (#3830)

* fix bug(#3617): after subtask fault tolerance, 2 task instances are generated.

* delete unused code

* update code smell

* refactor sub work command process

* add process service ut

* add license header

* fix some code smell

* chang ut java8 to java11

* update sonar to java11

* copy ut config from dev

* remove checkstyle

* revert to 1.3.3

* change proess service test to executor service

* add process service test

* add process service test

* revert

* revert

* add comments

* change dev to 1.3.3-release

* revert

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [Fix-#3487][sql] add dolphinscheduler_dml.sql under 1.3.3_schema (#3907)

* [FIX-3836][1.3.3-release-API] process definition validation name interface prompt information error  (#3899)

* fix bug : error message

* fix code smell

* fix code smell

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat (#3913)

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat

Co-authored-by: Kirs <acm_master@163.com>

* Repair check box cannot be canceled

* [fix-3843][api] When update workflow definition,if name already exists, the prompt is not friendly

* [fix-3843][api] When update workflow definition,if name already exists, the prompt is not friendly

* [fix-#3843][api]When update workflow definition,if name already exists, the prompt is not friendly (#3918)

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat

* [fix-3843][api] When update workflow definition,if name already exists, the prompt is not friendly

* [fix-3843][api] When update workflow definition,if name already exists, the prompt is not friendly

Co-authored-by: Kirs <acm_master@163.com>

* [Fix-#3487][sql] update uc_dolphin_T_t_ds_resources_un

* Workflow definition name re-modified and added check

* [fix-#3843][api]When update workflow definition,if name already exists, the prompt is not friendly.

* update code.

* [#3931][ui]Field name optimization for spark, flink, and mr

* change version from 1.3.2-SNAPSHOT to 1.3.3-SNAPSHOT (#3934)

* [maven-release-plugin] prepare release 1.3.3

* [maven-release-plugin] prepare for next development iteration

* [ambari-plugin]change version 1.3.2 to 1.3.3 (#3935)

* fix bug:3615 After the task is executed successfully, but the next task has not been submitted, stop the master

* [fixBug-3964][ui]Switch back and forth over timeout alarm, the selected value is empty

* solve too many files, close logClientService (#3971)

* fix #3966 sub process doesnot send alert mail after process instance ending. (#3972)

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [Fix-#3618][server] resolve task executed finished but not release the file handle (#3975)

* [Fix-#3618][server] resolve task executed finished but not release the file handle

* [Fix-#3618][server] resolve task executed finished but not release the file handle

* [Fix-#3958][api] files should not be created successfully in the directory of the authorized file

* [FIX-3966] The timeout warning does not take effect in sub_process (#3982)

* fix #3966 sub process doesnot send alert mail after process instance ending.

* fix bug 3964: sub_process The timeout warning does not take effect
add timeout warning for sub_process/dependent task.

* fix code smell

* fix code smell

* fix code smell

* update worker group inherit from parent

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* fix import dolphinscheduler_mysql.sql insert admin user data

* [FIX-3929] condition task would post wrong tasks when failover. (#3999)

* fix #3966 sub process doesnot send alert mail after process instance ending.

* fix bug 3964: sub_process The timeout warning does not take effect
add timeout warning for sub_process/dependent task.

* fix code smell

* fix code smell

* fix code smell

* update worker group inherit from parent

* remove stdout in logback configuration

* fix bug #3929 condition task would post error when failover.

* remove unused test

* add comments

* add skip node judge

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [FIX-3929]  because of no lock, start up failover would dispatch two same tasks. (#4004)

* fix #3966 sub process doesnot send alert mail after process instance ending.

* fix bug 3964: sub_process The timeout warning does not take effect
add timeout warning for sub_process/dependent task.

* fix code smell

* fix code smell

* fix code smell

* update worker group inherit from parent

* remove stdout in logback configuration

* fix bug #3929 condition task would post error when failover.

* remove unused test

* add comments

* add skip node judge

* fix bug 3929: because of no lock, start up failover would dispatch two same tasks.

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* revert pom version to 1.3.3-release

* [maven-release-plugin] prepare release 1.3.3

* [maven-release-plugin] prepare for next development iteration

* [release]revert pom version to 1.3.3-release

* fix bug 4010: remove failed condition tasks from error-task-list. (#4011)

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [maven-release-plugin] prepare release 1.3.3

* [maven-release-plugin] prepare for next development iteration

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* refactor code style

* refactor code style

* refactor code style

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* refactor ut test

* refactor ut test

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* refactor ut

* merge from 1.3.3-release

* refactor ut

* refactor ut

* refactor

* refactor

* refactor code style

* refactor code style

* refactor code style

* refactor code style

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* refactor code style

Co-authored-by: break60 <790061044@qq.com>
Co-authored-by: wuchunfu <319355703@qq.com>
Co-authored-by: lgcareer <18610854716@163.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>
Co-authored-by: lenboo <baoliang@analysys.com.cn>
Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
Co-authored-by: Yelli <amarantine@my.com>
Co-authored-by: Eights-Li <yelli.hl@gmail.com>
Co-authored-by: JinyLeeChina <42576980+JinyLeeChina@users.noreply.github.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>
Co-authored-by: XiaotaoYi <v-xiayi@hotmail.com>
Co-authored-by: Yichao Yang <1048262223@qq.com>
Co-authored-by: zhuangchong <zhuangchong8@163.com>
Co-authored-by: BoYiZhang <39816903+BoYiZhang@users.noreply.github.com>
Co-authored-by: muzhongjiang <mu_zhongjiang@163.com>
Co-authored-by: Jave-Chen <baicai.chen@gmail.com>
Co-authored-by: zhuangchong <zhuangchong6@163.com>
Co-authored-by: zhuangchong <37063904+zhuangchong@users.noreply.github.com>
Co-authored-by: Kirs <acm_master@163.com>
Co-authored-by: lgcareer <lgcareer@apache.org>
Co-authored-by: wulingqi <wulingqi@baijiahulian.com>

* Revert "[Draft][Merge][133-dev]133 merge dev (#4031)" (#4057)

This reverts commit ad2d9f99d0.

* [Fix][API] Condition task null pointer exception (#4056)

* add ProcessUtils UT

* modify ProcessUtils & ProcessUtilsTest

* Enhance user experience, add close button, return to the previous page (#4006)

Co-authored-by: zhanglong <zhanglong@ysstech.com>

* [FIX_BUG][server-test] dismiss of server warm-up time in RoundRobinSelectorTest (#4067)

* [Fix][api] Fix build parameter error of sqlserver when create. (#4015)

* [fix-#3962][api] Avoid ClassCastException for LoggerService.queryLog().

* [Fix][api] Fix build parameter error of sqlserver when create.

* [Feature-3985][Datax] Datax supports setting up running memory (#3986)

* Datax supports setting up running memory

* Datax supports setting up running memory

* Datax supports setting up running memory

* When running a task, the resource file is lost, which results in an error

* add unit test

* add unit test

* add unit test

* add test unit

* add test unit

* add test unit

* fix code smell

* add test unit

* add test unit

* [Improvement-3767][api] Task instance supports query by process instance name (#3825)

* Task instance supports query by process instance name.

* add test code checkstyle.

* add test param.

* resolve the sonar duplication check.

* solve logger single-line string exceeds 200 characters.

* resolve the sonar check.

* Resolve code conflicts.

Co-authored-by: zhuangchong <zhuangchong8@163.com>

* deal with magic value

* [Feature-4050][server] Spark task support udv inject (#4061)

* #4050 spark task support udv inject

* modify spark task UT

* modify sparkTaskExecutionCtx

* add exp for spark task get main jar method

* deal with magic value

* [Improvement-3471][common] JSONUtils.toMap It is not necessary to check whether the JSON method is empty again. #3471 (#3481)

* JSONUtils.toMap call improvement.

* [Fix-4054][Api] Fix The last week of the month for adding/editing timing, preview and save timing will report an error

* 解决单独执行子节点空指针的问题

* using OSUtils.execCmd when kill yarn app

* 解决单独执行子节点空指针的问题(增加checkstyle)

* 解决单独执行子节点空指针的问题(增加checkstyle)

* [FIX-#4083][server]fix taskInstance state change error
Concurrent processing of ack message and result message causes the execution sequence to be wrong

# this close # 4083

* code style

* fix replaceNRTtoUnderline NullPointerException #4098 (#4100)

* fix replaceNRTtoUnderline NullPointerException

* add  unit Test

* add taskResponseTest

* add taskResponseTest

* code smell

* Time is too small and the task is not finished

* Time is too small and the task is not finished

* [FIX-4034][server] fix sqoop import fail (#4036)

* fix #4043, sqoop import query fail

* fix #4043, sqoop task hard code & code style

* add license for SqoopConstants

* add private constructor for SqoopConstants

* fixed sqoop mysql pwd have special character

* fix checkstyle

* fix sqoop task log

* remove unused constants

* Time is too small and the task is not finished

* Time is too small and the task is not finished

* test

* remove assert

* test

* test

* fix task instance status judgment error

* fix: security page disappear delay problem when force refresh under GENERAL_USER

* improvement: resovle download url with resolveURL to prevent change of apiPrefix

* fix sqoop task jdbc string contains special char (#4105)

* [Bug][Common] read file garbled (#3479)

* fix bug : Random code problem

Co-authored-by: zhanglong <zhanglong@ysstech.com>

* [Improvement-3933][db operation] Improve the performance of sql query (#3940)

* optimize select * case

* emove redundancy

* bug fixed

* Update en_US.js

* Update startup.sh

* optimize

* optimize code

* optimize

* bug fixed

* add ut

* bug fixed

* bug fixed

* bug fixed

* bug fixed

* Delete WorkFlowLineageMapper.xml

* Delete createTenement.vue

* recove wrongly deleted file

* Update WorkFlowLineageMapper.xml

* Update createTenement.vue

* [Feature][API]enable response resources gzip compression (#4121)

* enable response compression

* add server.compression.mime-types with default value explicitly

* [Improvement][API] ignore noNodeException when get worker groups (#4120)

* ignore noNodeException when get worker groups

* add ut

* [Improvement][Code style] FIX SPELL WAITTING TO WAITING , etc. (#4118)

* FIX SPELL

* FIX SPELL AND  Optimizing code conventions

* add ut  cannot construct process instance, return null;

* add ut testExportProcessMetaData

* add ut testExportProcessMetaData

* add ut testImportProcessSchedule

* add ut MasterExecThreadTest

* add ut MasterExecThreadTest

* add ut testSubProcessViewTree

* add ut testComplementWithStartNodeList

* add ut testRecurseFindSubProcessId

* add ut testRecurseFindSubProcessId

* add ut testRecurseFindSubProcessId

* [FIX#4033] $[] conflicts with mysql keywords (#4111)

* [FIX#4033] $[] conflicts with mysql keywords
We currently only use this symbol for dates, so I filtered out the number type.
this close #4033

* test

* fix error

* split sqoop import hive database and table (#4141)

* upgrade quartz version to 2.3.0

* add HikariCP-java6,c3p0,mchange-commons-java license

* upgrade jackson version to 2.9.10

* start from force success

* correct param name

* correct names

* remove c3p0,mchange-commons-java license

* update Check code style

* update  code style

* Update pom.xml

* Update pom.xml

* Update LICENSE

not need to add HikariCP-java6

* Delete LICENSE-HikariCP-java6.txt

* Update known-dependencies.txt

* Update TaskInstanceMapper.xml

optimize page of [TaskInstance] load data slow

* [FIX-PR-4097][server-master]task ack miss (#4189)

When the message of successful execution arrives earlier than
the message of ack,
the message of ack will be discarded,
resulting in some information missing

* fix bug #4125 (#4127)

* fix bug #4125

* code style!

* code style.

Co-authored-by: chengp <chengp@chengp.net>

* [FIX-4190][DAO] When the amount of json data is large, the process list page display slowly. (#4201)

* fix 4190: When the amount of json data is large, process list page display slowly

* fix 4190: When the amount of json data is large, process list page display slowly

* fix 4190: When the amount of json data is large, process list page display slowly

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [Fix-3457][flink] fix flink args build problem (#4166)

* [Fix][Flink] fix flink args build problem

* [Fix][Flink] fix FlinkArgsUtilsTest

* [Improvement][UI] hide version and cluster input when deployMode is local

* [common]del windows file( not support windows ) (#4204)

* del support win

* del support win

* [Feature-4138][Master] dispatch workgroup error add sleep time (#4139)

* When there are tasks with assignment failure and the number of tasks in the current task queue is less than 10, sleep for 1 second

* When there are tasks with assignment failure and the number of tasks in the current task queue is less than 10, sleep for 1 second

* fix code smell & code style

* fix code smell & code style

Co-authored-by: zhanglong <zhanglong@ysstech.com>

* [FIX-#4172][server-worker] kill task NPE (#4182)

* [FIX-#4172][server-worker] kill task NPE

The cache task will be sent when the Process is generated. Before that, if a kill task appears, then NPE will appear
Modification method: write into the cache when the task is received, and mark it as preData
If the task is killed before the Process is generated, delete the cache directly at this time
It will be judged before the process is generated. If the task has been killed, it will not be executed.
After the new process is created, write it into the cache, and judge again, if kill, then kill the process.

this closes #4172

* Delete the commented out code
Add spring beans

* code smell

* add test

* add test

* fix error

* test

* test

* revert

* fix error

* [Feature-3878]Replace the page with element-ui (#4065)

* [Feature-3878]Introduce elment-ui and replace the security center module page with elment-ui

* Change node version

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* Replace the page with element-ui

* Replace user management with elment-ui

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* Repair the last Sunday of each month

* support auto eslint for .js, .vue file on save

* update .eslintrc.yml and license check exclude .eslintignore

* fix eslint: fix syntax by npm run lint automatically

* fix eslint: fatal syntax errors

* fix eslint: expected '!==' but instead saw '!=', expected '===' but instead saw '=='

* fix eslint: Unexpected side effect in 'cacheParams' computed property

* fix eslint: assigned a value but never used

* fix eslint: component has been registered but not used

* fix eslint: unexpected mutation of prop

* fix bug: start from the setting nodes with NODE_PRE would be NPE. (#4219)

* deleted invalid code as assigned in issue 4215 (#4221)

deleted the code at following two places:
org.apache.dolphinscheduler.service.process.ProcessService#checkTaskExistsInTaskQueue
org.apache.dolphinscheduler.service.process.ProcessService#taskZkInfo

* [FIX-#3177]Task time parameter parsing error (#4224)

* [FIX-#3177]Task time parameter parsing error

rerun schedule time error

this closes #3177

* [FIX-#3177]Task time parameter parsing error

rerun schedule time error

this closes #3177

* fix sql error (#4227)

* [FIX-3177]Task time parameter parsing error (#4228)

* [FIX-3177]Task time parameter parsing error
when system.datetime !=null $[datetime] = system.datetime
else $[datetime] = current time

* remove unused import

* fix date_convert null

* fix time cover

* fix time cover

* reformat

* add ut

* [Fix][UI]: fix re-login problem in new tab and state synchronization problem in multiple tabs (#4162)

* fix(ui): re-login problem in new tab

* refresh page in new tab automatically

* support to reload router view in lightweight

* optimize visibility code

* [Improvement][UI] Add no-var rule for eslint and add 'npm run lint:fix' command (#4225)

Add no-var rule for eslint, and fix related files
Add npm run lint:fix command, which distinguished from the default command npm run lint
The behavior of command npm run lint and command npm run lint:fix should be distinguished
The first command is used to lint only, and the second one is used to lint and fix problems automatically

* [FIX][UI ]  security user state  tenantCode User Type Display abnormal (#4255)

* fix user state error

* fix userType  error

* fix tenantCode  error

* [Fix-4222][Master]Add the priority queue to ensure that tasks are submitted according to priority. (#4250)

* [Fix-4222][Master]Add the priority queue to ensure that tasks are submitted according to priority.

* [Fix-4222][Master]Add the priority queue to ensure that tasks are submitted according to priority.

* [Fix-4222][Master]Remove useless import

* [Fix-4222][Master]Reformat code style

* [Fix-4222][Master]Reformat code style

* [Fix-4222][Master]Reformat code style

* [Fix-4222][Master]add PeerTaskInstancePriorityQueueTest

* [Fix-4222][Master]Fix code smell

* [Fix-4222][Master]Reformat code style

* [Fix-4222][Master]Fix code smell

Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>

* update bug template (#4260)

* add pasue icon button and adjust operation button style

* Align each item one by one in locale

* match frontend and backend for force success

* [Improvement-4069][server] When the tenant does not exist, the task execution should throw an exception (#4108)

* when  the tenant does not exist, the task execution should throw an exception

* remote method createWorkDirAndUserIfAbsent

* set the task status failed when the tenant code does not exist.

* add taskLog.

* update check os user exists

* update TaskExecuteThreadTest test method.

* solving sonar fail.

* remove AbstractTask.getCurTaskParamsClass()and replace with TaskParametersUtils.getParameters() (#4262)

* remove getCurTaskParamsClass() in AbstractTask.java and replace it with TaskParametersUtils.getParameters()

* remove unused imports in AbstractTask.java

* reformat

* fix code style and test

* [Fix-4271][server] Fix IOException or NoSuchFileException in logger server (#4272)

* fix version funcation delete. (#4265)

* [improvement][config] Update datasource.properties,add mysql meta data template (#4266)

* Update datasource.properties

[新增]元数据mysql连接模版

* Update datasource.properties

Co-authored-by: dailidong <dailidong66@gmail.com>

* fix taskInstance submitTime is empty. (#4274)

* [Improvement-3878]Tenant list delete user name (#4278)

* [Fix-4268] Fix NumberFormatException when visiting a doc.html or swagger-ui.html (#4269)

* [Fix-4268] Fix NumberFormatException when visiting a doc.html or swagger-ui.html

* Fix checkstyle error

* Update swagger-models in known-dependencies.txt

* [Improvement-3878][ui]Fix the list style (#4280)

* [Improvement-3878]Tenant list delete user name

* [Improvement-3878][ui]Fix the list style

* [FIX] [UI] fix create project cancel button invalid (#4282)

* fix create project cancel button invalid

* [Improvement] Refactor code to support distributed tracing (#4270)

* Refactor code to support tracing

* Extension network protocol, support context and version
* Extension master asynchronous queue support context
* Extract scan task method from MasterSchedulerService for tracing

* fix

* fix

* add test case

* fix

* fix

Co-authored-by: hailin0 <hailin0@yeah.net>

* [Improvement] Add Flink job name (#4285)

* [Improvement] Add Flink job name

* fix typo

* [Improvement][ui]List vacancy optimization and icon icon repair (#4286)

* [Improvement][ui] List vacancy optimization and icon icon repair

* adjust locale for new pr

* [FEATURE-736] integrate ldap authentication (#3743)

* [FEATURE-736] integrate ldap authentication

* add ldap authentication type
* refactor authentication with ldap and password
* add  createUser for ldap in user service
* remove duplicate password authenticator

* [Fix-4289][*] Flink name with disappeared and unescaped problem (#4290)

* fix flink name not display in process definition editor

* fix flink name not escape problem

* simplify escape method

* fix database is the mysql DB keyword. (#4295)

* fix the token management list does not display the user name. (#4302)

* Dev imp server process utils (#4263)


* Adds the comment on the WINDOWSATTERN

Co-authored-by: Kirs <acm_master@163.com>
Co-authored-by: 0002939 <licongyang@mininglamp.com>

* fix version close function does not take effect. (#4307)

* fix user management authorization operation exceptions do not prompt exception information. (#4292)

* [Fix][UI] Fix ui style problem and refactor form style (#4329)

* rename from-model to form-model

* [UI] remove duplicated css style code

* [UI] refactor css style of form model

* fix form model overflow problem in ie

* Align each item one by one in locale

* fix el-dialog width is too wide

* fix locale

* [Improvement][UI] Improve script box and dialog css style (#4331)

* [Fix-4335][UI] Fix IE 9-11 not supported

* [FIx-4338][UI] Fix invalid date problem in IE

* [Improvement] Use environment variable $DOLPHINSCHEDULE_OPTS as daemon startup arguments (#4341)

Co-authored-by: hailin0 <hailin0@yeah.net>

* [Fix][common] only two yarns can be selected(#4314) (#4344)

* modify_Status (#4352)

Co-authored-by: zt-1997 <“18841012545@163.com”>

* fix the page does not refresh after clicking the delete function.. (#4360)

* [FIX-4326][UI] Add description field to data source list (#4362)

* [Feature-4318][API] Rename tenant code (#4320)

Rename tenant code
Rename tenant code (English version)
Rname tenant name in ui label
Rname TENANT_CODE_EXIST to OS_TENANT_CODE_EXIST
Rname enum  TENANT_CODE_* to OS_TENANT_CODE_*

* [Fix-3233][api-server] Fix the bug when compile the code in Java9 env (#4312)

* [Fix-3233][api-server] Fix the bug when compile the code in Java9 env
 delete release-docs/licenses/LICENSE-jsp-api-2.1-6.1.14.txt
 delete ant-1.6.5.jar core-3.1.1.jar jsp-api-2.1-6.1.14.jar
 delete LICENSE-ant-1.6.5.txt,LICENSE-core-3.1.1.txt
 delete maven repository address of ant-1.6.5.jar ,core-3.1.1.jar ,jsp-api-2-1-6.1.14.jar

Co-authored-by: 李长福 <changfu.li@dmall.com>

* [Improvement][ui] Introduce remixicon, delete ans-ui license and font-awesome (#4313)

* [Improvement-3878]Tenant list delete user name

* [Improvement-3878][ui]Fix the list style

* [Improvement][ui] List vacancy optimization and icon icon repair

* Add font-awesome license and delete ans-ui license

* Introduce remixicon

* [Improvement-4257][api]  Increase "queryProcessDefinitionByName" API (#4390)

* 1. Increase "queryProcessDefinitionByName" API.
2. Will queryProcessDefinitionById api "PROCESS_INSTANCE_EXIST" state is modified to "PROCESS_DEFINE_NOT_EXIST".

* Correct spelling mistakes in words

* Logging should not be vulnerable to injection attacks

Co-authored-by: yinyong <yinyong@netposa.com>

* remove the operation resume_from_forced_success

* refactor code styles

* readme file fixed (#4374)

* readme fix

* readme fix

* readme fix

* readme fix

* readme fix

Co-authored-by: TinaPang <pangtianyang@analysys.com.cn>

* [Fix-4293][UI] Fix data loss in switched tab (#4415)

* [Fix-4293][UI] Fix data loss in switched tab

* The disclaimer should be same with the website

* [Improvement][UI] Rename disabled to enabled in menu (#4414)

* [Feature-#130] pass global param values when starting new process instance (#4372)

* [DS-130][feat] pass global param values when starting new process instance
    add optional param for start-process-instance api
    reuse command_param in command table for persistence
    overload curingGlobalParams function in ParameterUtils
    not adapt the UI code yet

* change import order

* support datetime expression

* print start params

* (fix) avoid npe when cmdParam is null

Change-Id: I3b4c4b5fa1df316ff221e27146e45d7d4d3a404e

* fix alert-spi error
todo sql script

* [DS-4396][feat] Add project information to email alert message (#4413)

* [DS-4396][feat] Add project information to email alert message
- build find project and user sql to ProjectMapper
- add project information to alert db.

* [DS-4396][style] change style

* add page query plugin instance
add plugin instance check name

* add page query plugin instance
add plugin instance check name

* add props field

* ding talk some filed is not required

* fix bug

* [Improvement][UI] Correct spelling mistakes (#4440)

* fix error

* revert

* [FEATURE-4451][API]After the workflow editing page is submitted, the user chooses whether to go online or not. 0 means not to go online, 1 means online. (#4437)

* vvvv

* vvvvv

* vvvvv

* testReleaseProcessDefinition

Co-authored-by: zt-1997 <“18841012545@163.com”>

* [Feature-4428][dao、UI] It is recommended to add an owner in the data source center and the Resource Center to make it easier to manage (#4442)

* add owner to datasource list and resource list

* add owner to datasource list and resource list

Co-authored-by: bobqiu <xiaoqiu369@gmail.com>

* [Fix-4455][common] fix parse shell output (#4456)

* add name field provide to ui

* [FIX][UT]testVerifyTenantCode assert error (#4460)

* [FIX][UT]testVerifyTenantCode assert error

* [FIX][UT]testVerifyTenantCode assert error

* code smell

* fix the exception of update null point of workflow instance due to no… (#4459)

* fix the exception of update null point of workflow instance due to not add global parameter when create workflow definition

* optimization of the leading package

Co-authored-by: yaoshui <huangqitao@accesscorporate.com.cn>

* chore: set up license-eye to check license headers (#4453)

* [Improvement][alert] Refactor alert module to fix code smell (#4434)

* chore: Refactore dolphinscheduler-alert to fix code smell

* chore: Refactor code to fix codestyle error

* fix: use shellformat(shfmt) and shellcheck fix mvnw (#3692)

use shellformat(shfmt) and shellcheck fix mvnw

* [Feature-4451][UI]: add an option to the UI page for saving the workflow definition: whether to go online? online by default (#4474)

Co-authored-by: linyanbin <linyanbin@lizhi.fm>

* [FIX-4247][Api]Fixed the problem that the global parameters are not updated after the workflow instance click rerun

Co-authored-by: zt-1997 <“18841012545@163.com”>

* [Improvement-4435][datasource] the datasource tests the connection and returns details when the connection is wrong (#4436)

* add datasource test connection return result message.

* add code checkstyle.

* spark/hive datasoure test connection add loadKerberosConf.

* solve sonarcloud coverage.

* [Improvement-4480][API] change method checkAdmin to isNotAdmin (#4483)

* change method 'checkAdmin' to 'isNotAdmin', for better understand.

Co-authored-by: 蒙强 <mengqiang@zhongan.com>

* fix send error

* add ui

* delete useless constant definitions
code style

* code style

* fix license head error

* [Improvement-#4481][API] ProcessDefinitionController save API optimization 

Co-authored-by: lei.zou <lei.zou@rootcloud.com>

* fix ut error

* fix ut error

* del unused jar

* [Feature-4423][UI] When the process definition is running, the pop-up box can manually set the global parameters. (#4433)

* [DS-130][feat] pass global param values when starting new process instance
    add optional param for start-process-instance api
    reuse command_param in command table for persistence
    overload curingGlobalParams function in ParameterUtils
    not adapt the UI code yet

* change import order

* support datetime expression

* print start params

* (fix) avoid npe when cmdParam is null

Change-Id: I3b4c4b5fa1df316ff221e27146e45d7d4d3a404e

* [Feature-4423][UI] When the process definition is running, the pop-up box can manually set the global parameters
(frontend) adapt the UI code
(backend-fix) add empty string check for param

Change-Id: I710db55f5059f8bd324c79f4494f2798d58e7b19

* add Startup parameters label

Change-Id: I5ac82031ea1b64abec330ee8cf2991477a28fcaa

* reuse i18n label

Change-Id: I5f322cb1dd8e2cade0c679bd025fc984e31bf3ae

* import alert-plugin

* upgrade jboss-logging

* upgrade jboss-logging

* fix ut error

* fix Vulnerabilities

* fix Vulnerabilities

* fix Vulnerabilities

* code smell

* [fix-4472][DataSource] Hive JDBC partial permission parameter concatenation JDBC URL error (#4473)

* Solve the problem of parameter splicing of permission part in Hive JDBC URL.

* update code style.

* add test check in pom.xml.

* [FIX-#4468] [UI] Workflow relationships Name filtering invalidated #4508

Co-authored-by: 花花 <huahua@thinkingdata.cn>

* fix Vulnerabilities

* add  public constructors

* fix  Vulnerabilities

* [NOTICE Update]Update year to 2021 (#4521)

* [feat-4496][server]  Add to! {} is used to mark the custom parameters to be output as-is in sql (#4497)

* feat([server]): Add to! {} is used to mark the custom parameters to be output as-is in sql

Before pre-compiling sql, replace the custom parameters marked with !{}
to prevent the parameters in the hive plus partition path from being
replaced with single quotes

Closes This closes #4496

* feat([server]): Add to! {} is used to mark the custom parameters to be output as-is in sql

Before pre-compiling sql, replace the custom parameters marked with !{}
to prevent the parameters in the hive plus partition path from being
replaced with single quotes

Closes This closes #4496

* feat([server]): Add to! {} is used to mark the custom parameters to be output as-is in sql

Before pre-compiling sql, replace the custom parameters marked with !{}
to prevent the parameters in the hive plus partition path from being
replaced with single quotes

Closes This closes #4496

* resolve conflicts

* add eg

* add eg

* code style

* revert mysql config

Co-authored-by: baoliang <baoliang@analysys.com.cn>
Co-authored-by: Eights-LI <yelli.eights@gmail.com>
Co-authored-by: bao liang <29528966+lenboo@users.noreply.github.com>
Co-authored-by: break60 <790061044@qq.com>
Co-authored-by: wuchunfu <319355703@qq.com>
Co-authored-by: lgcareer <18610854716@163.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
Co-authored-by: Yelli <amarantine@my.com>
Co-authored-by: Eights-Li <yelli.hl@gmail.com>
Co-authored-by: JinyLeeChina <42576980+JinyLeeChina@users.noreply.github.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>
Co-authored-by: XiaotaoYi <v-xiayi@hotmail.com>
Co-authored-by: Yichao Yang <1048262223@qq.com>
Co-authored-by: zhuangchong <zhuangchong8@163.com>
Co-authored-by: BoYiZhang <39816903+BoYiZhang@users.noreply.github.com>
Co-authored-by: muzhongjiang <mu_zhongjiang@163.com>
Co-authored-by: Jave-Chen <baicai.chen@gmail.com>
Co-authored-by: zhuangchong <zhuangchong6@163.com>
Co-authored-by: zhuangchong <37063904+zhuangchong@users.noreply.github.com>
Co-authored-by: lgcareer <lgcareer@apache.org>
Co-authored-by: wulingqi <wulingqi@baijiahulian.com>
Co-authored-by: hailin0 <hailin0@yeah.net>
Co-authored-by: zhanglong <zhanglong@ysstech.com>
Co-authored-by: t1mon <178317391@qq.com>
Co-authored-by: yangquan <iyeeku@qq.com>
Co-authored-by: liliang1991 <liliang_68@126.com>
Co-authored-by: liang.li.c <liang.li.c@17zuoye.com>
Co-authored-by: felix.wang <59079269+felix-thinkingdata@users.noreply.github.com>
Co-authored-by: chengshiwen <chengshiwen0103@gmail.com>
Co-authored-by: yinancx <mockibird@live.cn>
Co-authored-by: Yarlung <turingdojo@163.com>
Co-authored-by: karlsun <karlsun@tencent.com>
Co-authored-by: 曹聚阳 <yinancx@live.com>
Co-authored-by: wangxj3 <857234426@qq.com>
Co-authored-by: chengp <5058557@qq.com>
Co-authored-by: chengp <chengp@chengp.net>
Co-authored-by: Saksham Gupta <saksham1319@gmail.com>
Co-authored-by: Tq <36755957+Tianqi-Dotes@users.noreply.github.com>
Co-authored-by: Jatham <59549328+Jatham922@users.noreply.github.com>
Co-authored-by: hailin0 <hailin0@foxmail.com>
Co-authored-by: zh0122 <zh0122@gmail.com>
Co-authored-by: geosmart <geosmart@hotmail.com>
Co-authored-by: 李丛阳 <liby-1987@163.com>
Co-authored-by: 0002939 <licongyang@mininglamp.com>
Co-authored-by: kamisamak <1057372918@qq.com>
Co-authored-by: zt-1997 <41983395+zt-1997@users.noreply.github.com>
Co-authored-by: zt-1997 <“18841012545@163.com”>
Co-authored-by: xiaojingXU <68894048+xxjingcd@users.noreply.github.com>
Co-authored-by: guohaozhang <zhishengqianjun@gmail.com>
Co-authored-by: cooper <1322849632@qq.com>
Co-authored-by: 李长福 <changfu.li@dmall.com>
Co-authored-by: yy0812 <32183971+yy0812@users.noreply.github.com>
Co-authored-by: yinyong <yinyong@netposa.com>
Co-authored-by: ptyp <30622951+ptyp@users.noreply.github.com>
Co-authored-by: TinaPang <pangtianyang@analysys.com.cn>
Co-authored-by: Dean Wong <wangding85@gmail.com>
Co-authored-by: K.O <okoahin@gmail.com>
Co-authored-by: qiubo <38316980+qiubo369@users.noreply.github.com>
Co-authored-by: bobqiu <xiaoqiu369@gmail.com>
Co-authored-by: Hxssssss <huangqt1992@163.com>
Co-authored-by: yaoshui <huangqitao@accesscorporate.com.cn>
Co-authored-by: Zhenxu Ke <kezhenxu94@apache.org>
Co-authored-by: Segun Ogundipe <davephenoms@gmail.com>
Co-authored-by: xiagw <fxiaxiaoyu@gmail.com>
Co-authored-by: linyanbin666 <35388422+linyanbin666@users.noreply.github.com>
Co-authored-by: linyanbin <linyanbin@lizhi.fm>
Co-authored-by: qmengss <qmengss@users.noreply.github.com>
Co-authored-by: 蒙强 <mengqiang@zhongan.com>
Co-authored-by: 青年 <1043706593@qq.com>
Co-authored-by: lei.zou <lei.zou@rootcloud.com>
Co-authored-by: 小二黑 <784445949@qq.com>
Co-authored-by: 花花 <huahua@thinkingdata.cn>
Co-authored-by: liuxuedongcn <46193861+liuxuedongcn@users.noreply.github.com>
2021-01-22 16:26:52 +08:00
Kirs
4d553603ee
Merge dev to alert_plugin_design (#4351)
* fixed queryProcessInstanceListPaging query slow (#3893)

because queryProcessInstanceListPaging has contain longtext type field by instance.*,so queryProcessInstanceListPaging query slow , but longtext field not used.

* fixed  queryTaskInstanceListPaging query slow (#3892)

because queryTaskInstanceListPaging  has contain longtext  type field  by instance.*,so  queryTaskInstanceListPaging query slow , but longtext  field not used.

* add verify tenant name cannot contain special characters. (#3865)

* [Fix][Docker] fix startup.sh variables cannot reference in single quotes. (#3895)

* [Fix][Docker] fix startup.sh variables cannot reference in single quotes.(#3894)

Signed-off-by: YoungMan <bushengquan@gmail.com>

* Update Dockerfile

update the  comment
and delete the maridb client

Co-authored-by: dailidong <dailidong66@gmail.com>

* [ambari-plugin]change version 1.3.2 to 1.3.3 (#3935)

* [CodeClean][DAO]Remove redundant comments (#3939)

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* dag  add close button

* reset last version

* reset last version

* dag add close buttion

dag add close buttion

* update  CLICK_SAVE_WORKFLOW_BUTTON  xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* Update CreateWorkflowLocator.java

modify submit workflow button

* Update CreateWorkflowLocator.java

* Update CreateWorkflowLocator.java

modify CLICK_ADD_BUTTON

* Update CreateWorkflowLocator.java

delete print

* Update CreateWorkflowLocator.java

1

* Update CreateWorkflowLocator.java

1

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dag.vue

* Update dag.vue

* Update dag.vue

* Update CreateWorkflowLocator.java

* Revert "Setting '-XX:+DisableExplicitGC ' causes netty memory leaks"

This reverts commit 3a2cba7a

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dolphinscheduler-daemon.sh

* Remove redundant comments
ProcessInstanceMapper# queryProcessInstanceListPaging

* Remove redundant comments
ProcessInstanceMapper# queryProcessInstanceListPaging

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>

* fix bug:3615 After the task is executed successfully, but the next task has not been submitted, stop the master

* When running a task, the resource file is lost, which results in an error

* When running a task, the resource file is lost, which results in an error

* When running a task, the resource file is lost, which results in an error

* [Fix][api] Add queryProjectCreatedAndAuthorizedByUser function (#3658)

* fix getLoginUserCreated bug

* fix getLoginUserCreated bug 1

* fix checkstyle

* modify function name

* modify comment

* fix unit test error

* rollback sqoop task hive target file

Co-authored-by: sunchaohe <sunzhaohe@linklogis.com>
Co-authored-by: dailidong <dailidong66@gmail.com>

* When running a task, the resource file is lost, which results in an error

* When running a task, the resource file is lost, which results in an error

* [fixBug-3964][ui]Switch back and forth over timeout alarm, the selected value is empty

* solve too many files, close logClientService (#3971)

* fix #3966 sub process doesnot send alert mail after process instance ending. (#3972)

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [Fix-#3618][server] resolve task executed finished but not release the file handle (#3975)

* [Fix-#3618][server] resolve task executed finished but not release the file handle

* [Fix-#3618][server] resolve task executed finished but not release the file handle

* [Fix-#3958][api] files should not be created successfully in the directory of the authorized file

* [FIX-3966] The timeout warning does not take effect in sub_process (#3982)

* fix #3966 sub process doesnot send alert mail after process instance ending.

* fix bug 3964: sub_process The timeout warning does not take effect
add timeout warning for sub_process/dependent task.

* fix code smell

* fix code smell

* fix code smell

* update worker group inherit from parent

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [fix-#3962][api] Avoid ClassCastException for LoggerService.queryLog(). (#3963)

* Enhance user experience, add close button to file details page

* [Test-3557][API] full cover DataAnalysisServiceImpl (#3605)

* generate equals and hashCode method

* make up for the state of oblivion

* simplify countQueueState method

* delete try catch, because this never throw exception

* full cover DataAnalysisServiceImpl

* improve test coverage

* adjust code style

* simplify countCommandState method

* reduce duplication

* simplify countTaskDtos method

* Update DataAnalysisServiceImpl.java

Co-authored-by: dailidong <dailidong66@gmail.com>

* fix import dolphinscheduler_mysql.sql insert admin user data

* [FIX-3929] condition task would post wrong tasks when failover. (#3999)

* fix #3966 sub process doesnot send alert mail after process instance ending.

* fix bug 3964: sub_process The timeout warning does not take effect
add timeout warning for sub_process/dependent task.

* fix code smell

* fix code smell

* fix code smell

* update worker group inherit from parent

* remove stdout in logback configuration

* fix bug #3929 condition task would post error when failover.

* remove unused test

* add comments

* add skip node judge

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [FIX-3929]  because of no lock, start up failover would dispatch two same tasks. (#4004)

* fix #3966 sub process doesnot send alert mail after process instance ending.

* fix bug 3964: sub_process The timeout warning does not take effect
add timeout warning for sub_process/dependent task.

* fix code smell

* fix code smell

* fix code smell

* update worker group inherit from parent

* remove stdout in logback configuration

* fix bug #3929 condition task would post error when failover.

* remove unused test

* add comments

* add skip node judge

* fix bug 3929: because of no lock, start up failover would dispatch two same tasks.

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* revert pom version to 1.3.3-release

* [maven-release-plugin] prepare release 1.3.3

* [maven-release-plugin] prepare for next development iteration

* the resources API document does not match the interface parameters

* [release]revert pom version to 1.3.3-release

* fix bug 4010: remove failed condition tasks from error-task-list. (#4011)

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [maven-release-plugin] prepare release 1.3.3

* [maven-release-plugin] prepare for next development iteration

* fix bug

* reset code

* Update udfs.vue

* [Feature-3941][ui] Remove tenant name  (#3949)

* FeatureUI-3941: Removed Tenant

* FeatureUI-3941: Removed tenantName in the suggested pages

* FeatureUI-3941: Updated the suggested files

* FeatureUI-3941: Updated the TenantController

* FeatureUI-3941: Updated the TenantController and the pom.xml

* FeatureUI-3941: Updated the TenantControllerTest

* [Improvement][script] daemon script usage (#3968)

* Improvement daemon script usage

* don't remove status command

don't remove status command

Co-authored-by: dailidong <dailidong66@gmail.com>

* fix #3688, Optimization of table building statements (#3689)

* merge from 1.3.3-release

* merge from 1.3.3-release

* [Improvement-4012][common/remote] Json util code integration,remove the remote module json util (#4013)

* Json util code integration,remove the remote module json util.

* update code checkstyle.

* update code checkstyle.

* add code checkstyle.

* add test class.

* update JSONUtils class.

* update JSONUtils class.

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* refactor code style

* refactor code style

* refactor code style

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* refactor ut test

* refactor ut test

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* fix #3900 kill multi yarn app in one job

* refactor ut

* merge from 1.3.3-release

* refactor ut

* refactor ut

* refactor

* refactor

* refactor code style

* refactor code style

* refactor code style

* refactor code style

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* add ProcessUtils UT

* refactor code style

* refactor code style

* refactor code style

* refactor code style

* [Draft][Merge][133-dev]133 merge dev (#4031)

* [Feture-3327][ui]Add the function of re-uploading files in the resource center

* [Feture-3327][ui]Add the function of re-uploading files in the resource center (#3394)

* Before creating a workflow, clear the canvas

* [Fix-3256][ui] herry pick commit from dev for Fix admin user info update error (#3306)

* [Feture-3327][ui]Add the function of re-uploading files in the resource center

Co-authored-by: wuchunfu <319355703@qq.com>

* [Improvement-3327][api]support re-upload the resource file (#3395)

* [Fix-3390][server]Running hive sql task need find the hdfs path correctly (#3396)

* [Fix-3390][api]Running hive sql task need find the hdfs path correctly

* [Fix-3390][api]Running hive sql task need find the hdfs path correctly

* update soft version

* hive UDF function to modify the background color

* fix

* fix bug: Fix master task dependency check bug

* cancel spark task version check (#3406)

Co-authored-by: Eights-Li <yelli.hl@gmail.com>

* [Bug][ui]Fix front-end bug #3413

* [Feature][ambari_plugin]support one worker can belongs different worker groups when execute install script (#3410)

* Optimize dag

* Update actions.js (#3401)

* [Fix-3256][ui] Fix admin user info update error (#3425) (#3428)

* [PROPOSAL-3139] Datasource selection changes from radio to select

* [PROPOSAL-3139] Datasource selection changes from radio to select

* [BUG FIX] issues #3256

* [BUG FIX] issues #3256

* [BUG FIX] issues #3256

* [Fix-3256][ui] Fix admin user info update error

* [Fix-3256][ui] Fix admin user info update error

* [Fix-3256][ui] Fix admin user info update error

* [Fix-3256][ui] Fix admin user info update error

* reset createUser.vue

* [Fix-3256][ui] Fix admin user info update error

Co-authored-by: dailidong <dailidong66@gmail.com>

Co-authored-by: wuchunfu <319355703@qq.com>
Co-authored-by: dailidong <dailidong66@gmail.com>

* [Fix-3433][api]Fixed that release the imported process definition which version is below 1.3.0 will be failure

* dag connection add check

* fix

* [Fix-3423][dao][sql]Fixed that the resource file of the task node can't be found when upgrade from 1.2.0 to 1.3.x (#3454)

* Remove node deep monitoring

* If worker group id is null,don't need to set the value of the worker group (#3460)

* [Fix-3423][dao][sql]Fixed that the resource file of the task node can't be found when upgrade from 1.2.0 to 1.3.x

* [Fix-3423][dao]If worker group id is null,don't need to set the value of the worker group

* [ui]Code optimization

* fix

* fix

* [fix-3058][ui]Move rtTargetArr to jsPlumbHandle.js

* [optimization][ui]Prevent the shell script input box from being empty

* [Fix-3462][api]If login user is admin,need list all udfs (#3465)

* [Fix-3462][api]If login user is admin,need list all udfs

* [Fix-3462][api]add the test on the method of QueryUdfFuncList

* [Fix-3462][api]fix the code smell

* [Fix-3462][api]fix the code smell

* [Fix-3462][api]fix the code smell

* [Fix-3463][api]Fixed that run the sql task will be failure after rename the udf resource (#3482)

* [fixBug-3058][ui]Fix connection abnormalities in historical workflow instance data

* [Feture-3327][ui]Add the function of re-uploading files in the udf subdirectory

* fix bug: Fix master task dependency check bug (#3473)

Co-authored-by: lenboo <baoliang@analysys.com.cn>

* [maven-release-plugin] prepare release 1.3.2

* [maven-release-plugin] prepare for next development iteration

* fix ci_e2e fail (#3497)

* [Fix-3469][api]Should filter the resource by the different program type (#3498)

* [Fix-3463][api]Fixed that run the sql task will be failure after rename the udf resource

* [Fix-3469][api]Should list python file and jar file

* [Fix-3469][api]Should filter the resource by the different program type

* [Fix-3469][api]fix the code smell

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback (#3499)

* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* [maven-release-plugin] prepare release 1.3.2

* [maven-release-plugin] prepare for next development iteration

* [Fix-3469][ui]The value of maintenance resources and the filtering of resources according to different program types

* fix

* Revert "fix ci_e2e fail (#3497)"

This reverts commit e367f90bb7.

* test

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback (#3503)

* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test

* test release 1.3.2 version rollback

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* [maven-release-plugin] prepare release 1.3.2

* [maven-release-plugin] prepare for next development iteration

* test release 1.3.2 version rollback (#3504)

* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test

* test release 1.3.2 version rollback

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* [maven-release-plugin] prepare release 1.3.2

* [maven-release-plugin] prepare for next development iteration

* fix ds muti-level directory in zk, which lead to fail to assign work

* add login user check some actions in api

* [Hotfix][ci] Fix e2e ci docker image build error

* modify tag 1.3.0 to HEAD

* modify tag 1.3.0 to HEAD (#3525)

* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test

* test release 1.3.2 version rollback

* modify tag 1.3.0 to HEAD

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* remove OGNL part of the mybaits notice (#3526)

* [maven-release-plugin] prepare release 1.3.2

* [maven-release-plugin] prepare for next development iteration

* release 1.3.2 version rollback (#3527)

* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test

* test release 1.3.2 version rollback

* modify tag 1.3.0 to HEAD

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* [ui]Script input box to modify the delay loading time

* fix

* fix

* fix

* fix

* modify general user can't create token

* [ui]It is forbidden to select non-existent resources and modify the tree display data format

* modify general user can't create token (#3533)

* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test

* test release 1.3.2 version rollback

* modify tag 1.3.0 to HEAD

* modify general user can't create token

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* if task is null , set task type is null instead of "null"

* [Fix-3536][api]If user didn't have tenant,create resource directory will NPE (#3537)

* [Fix-3536][api]If user didn't have tenant,create resource will NPE

* [Fix-3536][api]If user didn't have tenant,create resource directory will NPE

* modify general user can't create,delete,update token (#3538)

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* [ui]Resource delete OK button to increase loading, change the number of homepage display cursor

* fix

* [Fix-3616][Server] when worker akc/response master exception , async retry (#3748)

* [fixbug][ui]Repair the master and worker management instrument display

* [Fix-3238][docker]Fix that can not create folder in docker with standalone mode (#3741)

* [fixbug][ui]Remove non-existent or deleted resources disabled

* [fixBug-3621][ui]If the workflow instance status is executing status, it is forbidden to select

* [fix-3553][ui]Repair click workflow connection, select the entire path

* fix

* fix

* [Fix-3238][docker]Fix that can not create folder in docker with standalone mode

* [Fix-3616][Server] when worker akc/response master exception , async retry (#3776)

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* The batch delete function in the workflow definition and workflow instance pages cannot be canceled if selected.

* [Improvement-3720][ui] js mailbox verification fix

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect (#3784)

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* The batch delete function in the workflow definition and workflow instance pages cannot be canceled if selected.

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
Co-authored-by: zhuangchong <zhuangchong8@163.com>
Co-authored-by: JinyLeeChina <42576980+JinyLeeChina@users.noreply.github.com>

* [Fix-3124][docker]Fix that can not build a docker image on windows (#3765)

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect (#3786)

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* The batch delete function in the workflow definition and workflow instance pages cannot be canceled if selected.

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
Co-authored-by: zhuangchong <zhuangchong8@163.com>
Co-authored-by: JinyLeeChina <42576980+JinyLeeChina@users.noreply.github.com>

* [Fix-3258][Security][Worker group manage] Connot get create time and update time,report DateTimeParseException (#3787)

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* The batch delete function in the workflow definition and workflow instance pages cannot be canceled if selected.

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

* [BugFixed] issue #3258 (#3265)

* 'ExecutionStatus'

* '3258'

* Update WorkerGroupServiceTest.java

* Delete UserState.java

* Delete ResourceSyncService.java

* Delete core-site.xml

* Delete hdfs-site.xml

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
Co-authored-by: zhuangchong <zhuangchong8@163.com>
Co-authored-by: JinyLeeChina <42576980+JinyLeeChina@users.noreply.github.com>
Co-authored-by: dailidong <dailidong66@gmail.com>

* [fixBug-3792][ui]Click on the sidebar to adapt the width of the pie chart on the project homepage

* [Bug-3713][HadoopUtils] catfile method Stream not closed  (#3715)

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* dag  add close button

* reset last version

* reset last version

* dag add close buttion

dag add close buttion

* update  CLICK_SAVE_WORKFLOW_BUTTON  xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* Update CreateWorkflowLocator.java

modify submit workflow button

* Update CreateWorkflowLocator.java

* Update CreateWorkflowLocator.java

modify CLICK_ADD_BUTTON

* Update CreateWorkflowLocator.java

delete print

* Update CreateWorkflowLocator.java

1

* Update CreateWorkflowLocator.java

1

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dag.vue

* Update dag.vue

* Update dag.vue

* Update CreateWorkflowLocator.java

* Revert "Setting '-XX:+DisableExplicitGC ' causes netty memory leaks"

This reverts commit 3a2cba7a

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dolphinscheduler-daemon.sh

* catfile method Stream not closed

* catfile method Stream not closed

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>

* [Fix-#3713][common]Fix that catfile method Stream not closed

* [Fix-#3713][common]Fix that catfile method Stream not closed (#3810)

* [Bug-3713][HadoopUtils] catfile method Stream not closed  (#3715)

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* dag  add close button

* reset last version

* reset last version

* dag add close buttion

dag add close buttion

* update  CLICK_SAVE_WORKFLOW_BUTTON  xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* Update CreateWorkflowLocator.java

modify submit workflow button

* Update CreateWorkflowLocator.java

* Update CreateWorkflowLocator.java

modify CLICK_ADD_BUTTON

* Update CreateWorkflowLocator.java

delete print

* Update CreateWorkflowLocator.java

1

* Update CreateWorkflowLocator.java

1

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dag.vue

* Update dag.vue

* Update dag.vue

* Update CreateWorkflowLocator.java

* Revert "Setting '-XX:+DisableExplicitGC ' causes netty memory leaks"

This reverts commit 3a2cba7a

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dolphinscheduler-daemon.sh

* catfile method Stream not closed

* catfile method Stream not closed

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>

* [Fix-#3713][common]Fix that catfile method Stream not closed

Co-authored-by: BoYiZhang <39816903+BoYiZhang@users.noreply.github.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>

* [Fix-#3487][api、dao] cherry pick from dev to fix that create folder duplicate name under multithreading

* [Hotfix-3131][api] Fix the new tenant already exists prompt (#3132)

* Bugfix: Fix the new tenant already exists prompt

* Feature: Add test cases

* Update TenantServiceTest.java

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>

* Set up JDK 11 for SonarCloud in github action. (#3052)

* Set up JDK 11 for SonarCloud in github action.

* Fix javadoc error with JDK 11.

* Prevent Javadoc from stopping if it finds any html errors.

* [fixBug-3621][ui]Select the batch checkbox to unfilter the instances in the executing state

* add verify tenant name cannot contain special characters.

* [fixBug-3840][ui]The tenant code only allows letters or a combination of letters and numbers

* fix

* fix

* fix

* [Fix-#3702][api] When re-upload the resource file but don't change the name or desc,it need replace the origin resource file. (#3862)

* [Fix-#3702][api] When re-upload the resource file but don't change the name or desc,it will not replace the origin resource file.

* [Fix-#3702][api] When re-upload the resource file but don't change the name or desc,it will not replace the origin resource file.

* [fixbug-3621][ui]Workflow instance ready to stop and ready to suspend state prohibits checking

* [fixbug-3887][ui]Fix missing English translation of re-upload files

* add process define name verify. (#3879)

* Revert "[1.3.3-release][fix-3835][ui] When the tenantName contains "<", the tenant drop-down list is blankadd verify tenant name cannot contain special characters."

* revert pr 3872

* [FIX-3617][Service]after subtask fault tolerance, 2 task instances are generated (#3830)

* fix bug(#3617): after subtask fault tolerance, 2 task instances are generated.

* delete unused code

* update code smell

* refactor sub work command process

* add process service ut

* add license header

* fix some code smell

* chang ut java8 to java11

* update sonar to java11

* copy ut config from dev

* remove checkstyle

* revert to 1.3.3

* change proess service test to executor service

* add process service test

* add process service test

* revert

* revert

* add comments

* change dev to 1.3.3-release

* revert

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [Fix-#3487][sql] add dolphinscheduler_dml.sql under 1.3.3_schema (#3907)

* [FIX-3836][1.3.3-release-API] process definition validation name interface prompt information error  (#3899)

* fix bug : error message

* fix code smell

* fix code smell

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat (#3913)

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat

Co-authored-by: Kirs <acm_master@163.com>

* Repair check box cannot be canceled

* [fix-3843][api] When update workflow definition,if name already exists, the prompt is not friendly

* [fix-3843][api] When update workflow definition,if name already exists, the prompt is not friendly

* [fix-#3843][api]When update workflow definition,if name already exists, the prompt is not friendly (#3918)

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat

* [fix-3843][api] When update workflow definition,if name already exists, the prompt is not friendly

* [fix-3843][api] When update workflow definition,if name already exists, the prompt is not friendly

Co-authored-by: Kirs <acm_master@163.com>

* [Fix-#3487][sql] update uc_dolphin_T_t_ds_resources_un

* Workflow definition name re-modified and added check

* [fix-#3843][api]When update workflow definition,if name already exists, the prompt is not friendly.

* update code.

* [#3931][ui]Field name optimization for spark, flink, and mr

* change version from 1.3.2-SNAPSHOT to 1.3.3-SNAPSHOT (#3934)

* [maven-release-plugin] prepare release 1.3.3

* [maven-release-plugin] prepare for next development iteration

* [ambari-plugin]change version 1.3.2 to 1.3.3 (#3935)

* fix bug:3615 After the task is executed successfully, but the next task has not been submitted, stop the master

* [fixBug-3964][ui]Switch back and forth over timeout alarm, the selected value is empty

* solve too many files, close logClientService (#3971)

* fix #3966 sub process doesnot send alert mail after process instance ending. (#3972)

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [Fix-#3618][server] resolve task executed finished but not release the file handle (#3975)

* [Fix-#3618][server] resolve task executed finished but not release the file handle

* [Fix-#3618][server] resolve task executed finished but not release the file handle

* [Fix-#3958][api] files should not be created successfully in the directory of the authorized file

* [FIX-3966] The timeout warning does not take effect in sub_process (#3982)

* fix #3966 sub process doesnot send alert mail after process instance ending.

* fix bug 3964: sub_process The timeout warning does not take effect
add timeout warning for sub_process/dependent task.

* fix code smell

* fix code smell

* fix code smell

* update worker group inherit from parent

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* fix import dolphinscheduler_mysql.sql insert admin user data

* [FIX-3929] condition task would post wrong tasks when failover. (#3999)

* fix #3966 sub process doesnot send alert mail after process instance ending.

* fix bug 3964: sub_process The timeout warning does not take effect
add timeout warning for sub_process/dependent task.

* fix code smell

* fix code smell

* fix code smell

* update worker group inherit from parent

* remove stdout in logback configuration

* fix bug #3929 condition task would post error when failover.

* remove unused test

* add comments

* add skip node judge

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [FIX-3929]  because of no lock, start up failover would dispatch two same tasks. (#4004)

* fix #3966 sub process doesnot send alert mail after process instance ending.

* fix bug 3964: sub_process The timeout warning does not take effect
add timeout warning for sub_process/dependent task.

* fix code smell

* fix code smell

* fix code smell

* update worker group inherit from parent

* remove stdout in logback configuration

* fix bug #3929 condition task would post error when failover.

* remove unused test

* add comments

* add skip node judge

* fix bug 3929: because of no lock, start up failover would dispatch two same tasks.

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* revert pom version to 1.3.3-release

* [maven-release-plugin] prepare release 1.3.3

* [maven-release-plugin] prepare for next development iteration

* [release]revert pom version to 1.3.3-release

* fix bug 4010: remove failed condition tasks from error-task-list. (#4011)

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [maven-release-plugin] prepare release 1.3.3

* [maven-release-plugin] prepare for next development iteration

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* refactor code style

* refactor code style

* refactor code style

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* refactor ut test

* refactor ut test

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* refactor ut

* merge from 1.3.3-release

* refactor ut

* refactor ut

* refactor

* refactor

* refactor code style

* refactor code style

* refactor code style

* refactor code style

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* refactor code style

Co-authored-by: break60 <790061044@qq.com>
Co-authored-by: wuchunfu <319355703@qq.com>
Co-authored-by: lgcareer <18610854716@163.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>
Co-authored-by: lenboo <baoliang@analysys.com.cn>
Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
Co-authored-by: Yelli <amarantine@my.com>
Co-authored-by: Eights-Li <yelli.hl@gmail.com>
Co-authored-by: JinyLeeChina <42576980+JinyLeeChina@users.noreply.github.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>
Co-authored-by: XiaotaoYi <v-xiayi@hotmail.com>
Co-authored-by: Yichao Yang <1048262223@qq.com>
Co-authored-by: zhuangchong <zhuangchong8@163.com>
Co-authored-by: BoYiZhang <39816903+BoYiZhang@users.noreply.github.com>
Co-authored-by: muzhongjiang <mu_zhongjiang@163.com>
Co-authored-by: Jave-Chen <baicai.chen@gmail.com>
Co-authored-by: zhuangchong <zhuangchong6@163.com>
Co-authored-by: zhuangchong <37063904+zhuangchong@users.noreply.github.com>
Co-authored-by: Kirs <acm_master@163.com>
Co-authored-by: lgcareer <lgcareer@apache.org>
Co-authored-by: wulingqi <wulingqi@baijiahulian.com>

* Revert "[Draft][Merge][133-dev]133 merge dev (#4031)" (#4057)

This reverts commit ad2d9f99d0.

* [Fix][API] Condition task null pointer exception (#4056)

* add ProcessUtils UT

* modify ProcessUtils & ProcessUtilsTest

* Enhance user experience, add close button, return to the previous page (#4006)

Co-authored-by: zhanglong <zhanglong@ysstech.com>

* [FIX_BUG][server-test] dismiss of server warm-up time in RoundRobinSelectorTest (#4067)

* [Fix][api] Fix build parameter error of sqlserver when create. (#4015)

* [fix-#3962][api] Avoid ClassCastException for LoggerService.queryLog().

* [Fix][api] Fix build parameter error of sqlserver when create.

* [Feature-3985][Datax] Datax supports setting up running memory (#3986)

* Datax supports setting up running memory

* Datax supports setting up running memory

* Datax supports setting up running memory

* When running a task, the resource file is lost, which results in an error

* add unit test

* add unit test

* add unit test

* add test unit

* add test unit

* add test unit

* fix code smell

* add test unit

* add test unit

* [Improvement-3767][api] Task instance supports query by process instance name (#3825)

* Task instance supports query by process instance name.

* add test code checkstyle.

* add test param.

* resolve the sonar duplication check.

* solve logger single-line string exceeds 200 characters.

* resolve the sonar check.

* Resolve code conflicts.

Co-authored-by: zhuangchong <zhuangchong8@163.com>

* deal with magic value

* [Feature-4050][server] Spark task support udv inject (#4061)

* #4050 spark task support udv inject

* modify spark task UT

* modify sparkTaskExecutionCtx

* add exp for spark task get main jar method

* deal with magic value

* [Improvement-3471][common] JSONUtils.toMap It is not necessary to check whether the JSON method is empty again. #3471 (#3481)

* JSONUtils.toMap call improvement.

* [Fix-4054][Api] Fix The last week of the month for adding/editing timing, preview and save timing will report an error

* 解决单独执行子节点空指针的问题

* using OSUtils.execCmd when kill yarn app

* 解决单独执行子节点空指针的问题(增加checkstyle)

* 解决单独执行子节点空指针的问题(增加checkstyle)

* [FIX-#4083][server]fix taskInstance state change error
Concurrent processing of ack message and result message causes the execution sequence to be wrong

# this close # 4083

* code style

* fix replaceNRTtoUnderline NullPointerException #4098 (#4100)

* fix replaceNRTtoUnderline NullPointerException

* add  unit Test

* add taskResponseTest

* add taskResponseTest

* code smell

* Time is too small and the task is not finished

* Time is too small and the task is not finished

* [FIX-4034][server] fix sqoop import fail (#4036)

* fix #4043, sqoop import query fail

* fix #4043, sqoop task hard code & code style

* add license for SqoopConstants

* add private constructor for SqoopConstants

* fixed sqoop mysql pwd have special character

* fix checkstyle

* fix sqoop task log

* remove unused constants

* Time is too small and the task is not finished

* Time is too small and the task is not finished

* test

* remove assert

* test

* test

* fix task instance status judgment error

* fix: security page disappear delay problem when force refresh under GENERAL_USER

* improvement: resovle download url with resolveURL to prevent change of apiPrefix

* fix sqoop task jdbc string contains special char (#4105)

* [Bug][Common] read file garbled (#3479)

* fix bug : Random code problem

Co-authored-by: zhanglong <zhanglong@ysstech.com>

* [Improvement-3933][db operation] Improve the performance of sql query (#3940)

* optimize select * case

* emove redundancy

* bug fixed

* Update en_US.js

* Update startup.sh

* optimize

* optimize code

* optimize

* bug fixed

* add ut

* bug fixed

* bug fixed

* bug fixed

* bug fixed

* Delete WorkFlowLineageMapper.xml

* Delete createTenement.vue

* recove wrongly deleted file

* Update WorkFlowLineageMapper.xml

* Update createTenement.vue

* [Feature][API]enable response resources gzip compression (#4121)

* enable response compression

* add server.compression.mime-types with default value explicitly

* [Improvement][API] ignore noNodeException when get worker groups (#4120)

* ignore noNodeException when get worker groups

* add ut

* [Improvement][Code style] FIX SPELL WAITTING TO WAITING , etc. (#4118)

* FIX SPELL

* FIX SPELL AND  Optimizing code conventions

* add ut  cannot construct process instance, return null;

* add ut testExportProcessMetaData

* add ut testExportProcessMetaData

* add ut testImportProcessSchedule

* add ut MasterExecThreadTest

* add ut MasterExecThreadTest

* add ut testSubProcessViewTree

* add ut testComplementWithStartNodeList

* add ut testRecurseFindSubProcessId

* add ut testRecurseFindSubProcessId

* add ut testRecurseFindSubProcessId

* [FIX#4033] $[] conflicts with mysql keywords (#4111)

* [FIX#4033] $[] conflicts with mysql keywords
We currently only use this symbol for dates, so I filtered out the number type.
this close #4033

* test

* fix error

* split sqoop import hive database and table (#4141)

* upgrade quartz version to 2.3.0

* add HikariCP-java6,c3p0,mchange-commons-java license

* upgrade jackson version to 2.9.10

* remove c3p0,mchange-commons-java license

* update Check code style

* update  code style

* Update pom.xml

* Update pom.xml

* Update LICENSE

not need to add HikariCP-java6

* Delete LICENSE-HikariCP-java6.txt

* Update known-dependencies.txt

* Update TaskInstanceMapper.xml

optimize page of [TaskInstance] load data slow

* [FIX-PR-4097][server-master]task ack miss (#4189)

When the message of successful execution arrives earlier than
the message of ack,
the message of ack will be discarded,
resulting in some information missing

* fix bug #4125 (#4127)

* fix bug #4125

* code style!

* code style.

Co-authored-by: chengp <chengp@chengp.net>

* [FIX-4190][DAO] When the amount of json data is large, the process list page display slowly. (#4201)

* fix 4190: When the amount of json data is large, process list page display slowly

* fix 4190: When the amount of json data is large, process list page display slowly

* fix 4190: When the amount of json data is large, process list page display slowly

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [Fix-3457][flink] fix flink args build problem (#4166)

* [Fix][Flink] fix flink args build problem

* [Fix][Flink] fix FlinkArgsUtilsTest

* [Improvement][UI] hide version and cluster input when deployMode is local

* [common]del windows file( not support windows ) (#4204)

* del support win

* del support win

* [Feature-4138][Master] dispatch workgroup error add sleep time (#4139)

* When there are tasks with assignment failure and the number of tasks in the current task queue is less than 10, sleep for 1 second

* When there are tasks with assignment failure and the number of tasks in the current task queue is less than 10, sleep for 1 second

* fix code smell & code style

* fix code smell & code style

Co-authored-by: zhanglong <zhanglong@ysstech.com>

* [FIX-#4172][server-worker] kill task NPE (#4182)

* [FIX-#4172][server-worker] kill task NPE

The cache task will be sent when the Process is generated. Before that, if a kill task appears, then NPE will appear
Modification method: write into the cache when the task is received, and mark it as preData
If the task is killed before the Process is generated, delete the cache directly at this time
It will be judged before the process is generated. If the task has been killed, it will not be executed.
After the new process is created, write it into the cache, and judge again, if kill, then kill the process.

this closes #4172

* Delete the commented out code
Add spring beans

* code smell

* add test

* add test

* fix error

* test

* test

* revert

* fix error

* [Feature-3878]Replace the page with element-ui (#4065)

* [Feature-3878]Introduce elment-ui and replace the security center module page with elment-ui

* Change node version

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* Replace the page with element-ui

* Replace user management with elment-ui

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* Repair the last Sunday of each month

* support auto eslint for .js, .vue file on save

* update .eslintrc.yml and license check exclude .eslintignore

* fix eslint: fix syntax by npm run lint automatically

* fix eslint: fatal syntax errors

* fix eslint: expected '!==' but instead saw '!=', expected '===' but instead saw '=='

* fix eslint: Unexpected side effect in 'cacheParams' computed property

* fix eslint: assigned a value but never used

* fix eslint: component has been registered but not used

* fix eslint: unexpected mutation of prop

* fix bug: start from the setting nodes with NODE_PRE would be NPE. (#4219)

* deleted invalid code as assigned in issue 4215 (#4221)

deleted the code at following two places:
org.apache.dolphinscheduler.service.process.ProcessService#checkTaskExistsInTaskQueue
org.apache.dolphinscheduler.service.process.ProcessService#taskZkInfo

* [FIX-#3177]Task time parameter parsing error (#4224)

* [FIX-#3177]Task time parameter parsing error

rerun schedule time error

this closes #3177

* [FIX-#3177]Task time parameter parsing error

rerun schedule time error

this closes #3177

* fix sql error (#4227)

* [FIX-3177]Task time parameter parsing error (#4228)

* [FIX-3177]Task time parameter parsing error
when system.datetime !=null $[datetime] = system.datetime
else $[datetime] = current time

* remove unused import

* fix date_convert null

* fix time cover

* fix time cover

* reformat

* add ut

* [Fix][UI]: fix re-login problem in new tab and state synchronization problem in multiple tabs (#4162)

* fix(ui): re-login problem in new tab

* refresh page in new tab automatically

* support to reload router view in lightweight

* optimize visibility code

* [Improvement][UI] Add no-var rule for eslint and add 'npm run lint:fix' command (#4225)

Add no-var rule for eslint, and fix related files
Add npm run lint:fix command, which distinguished from the default command npm run lint
The behavior of command npm run lint and command npm run lint:fix should be distinguished
The first command is used to lint only, and the second one is used to lint and fix problems automatically

* [FIX][UI ]  security user state  tenantCode User Type Display abnormal (#4255)

* fix user state error

* fix userType  error

* fix tenantCode  error

* [Fix-4222][Master]Add the priority queue to ensure that tasks are submitted according to priority. (#4250)

* [Fix-4222][Master]Add the priority queue to ensure that tasks are submitted according to priority.

* [Fix-4222][Master]Add the priority queue to ensure that tasks are submitted according to priority.

* [Fix-4222][Master]Remove useless import

* [Fix-4222][Master]Reformat code style

* [Fix-4222][Master]Reformat code style

* [Fix-4222][Master]Reformat code style

* [Fix-4222][Master]add PeerTaskInstancePriorityQueueTest

* [Fix-4222][Master]Fix code smell

* [Fix-4222][Master]Reformat code style

* [Fix-4222][Master]Fix code smell

Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>

* update bug template (#4260)

* [Improvement-4069][server] When the tenant does not exist, the task execution should throw an exception (#4108)

* when  the tenant does not exist, the task execution should throw an exception

* remote method createWorkDirAndUserIfAbsent

* set the task status failed when the tenant code does not exist.

* add taskLog.

* update check os user exists

* update TaskExecuteThreadTest test method.

* solving sonar fail.

* remove AbstractTask.getCurTaskParamsClass()and replace with TaskParametersUtils.getParameters() (#4262)

* remove getCurTaskParamsClass() in AbstractTask.java and replace it with TaskParametersUtils.getParameters()

* remove unused imports in AbstractTask.java

* reformat

* [Fix-4271][server] Fix IOException or NoSuchFileException in logger server (#4272)

* fix version funcation delete. (#4265)

* [improvement][config] Update datasource.properties,add mysql meta data template (#4266)

* Update datasource.properties

[新增]元数据mysql连接模版

* Update datasource.properties

Co-authored-by: dailidong <dailidong66@gmail.com>

* fix taskInstance submitTime is empty. (#4274)

* [Improvement-3878]Tenant list delete user name (#4278)

* [Fix-4268] Fix NumberFormatException when visiting a doc.html or swagger-ui.html (#4269)

* [Fix-4268] Fix NumberFormatException when visiting a doc.html or swagger-ui.html

* Fix checkstyle error

* Update swagger-models in known-dependencies.txt

* [Improvement-3878][ui]Fix the list style (#4280)

* [Improvement-3878]Tenant list delete user name

* [Improvement-3878][ui]Fix the list style

* [FIX] [UI] fix create project cancel button invalid (#4282)

* fix create project cancel button invalid

* [Improvement] Refactor code to support distributed tracing (#4270)

* Refactor code to support tracing

* Extension network protocol, support context and version
* Extension master asynchronous queue support context
* Extract scan task method from MasterSchedulerService for tracing

* fix

* fix

* add test case

* fix

* fix

Co-authored-by: hailin0 <hailin0@yeah.net>

* [Improvement] Add Flink job name (#4285)

* [Improvement] Add Flink job name

* fix typo

* [Improvement][ui]List vacancy optimization and icon icon repair (#4286)

* [Improvement][ui] List vacancy optimization and icon icon repair

* [FEATURE-736] integrate ldap authentication (#3743)

* [FEATURE-736] integrate ldap authentication

* add ldap authentication type
* refactor authentication with ldap and password
* add  createUser for ldap in user service
* remove duplicate password authenticator

* [Fix-4289][*] Flink name with disappeared and unescaped problem (#4290)

* fix flink name not display in process definition editor

* fix flink name not escape problem

* simplify escape method

* fix database is the mysql DB keyword. (#4295)

* fix the token management list does not display the user name. (#4302)

* Dev imp server process utils (#4263)


* Adds the comment on the WINDOWSATTERN

Co-authored-by: Kirs <acm_master@163.com>
Co-authored-by: 0002939 <licongyang@mininglamp.com>

* fix version close function does not take effect. (#4307)

* fix user management authorization operation exceptions do not prompt exception information. (#4292)

* [Fix][UI] Fix ui style problem and refactor form style (#4329)

* rename from-model to form-model

* [UI] remove duplicated css style code

* [UI] refactor css style of form model

* fix form model overflow problem in ie

* Align each item one by one in locale

* fix el-dialog width is too wide

* fix locale

* [Improvement][UI] Improve script box and dialog css style (#4331)

* [Fix-4335][UI] Fix IE 9-11 not supported

* [FIx-4338][UI] Fix invalid date problem in IE

* [Improvement] Use environment variable $DOLPHINSCHEDULE_OPTS as daemon startup arguments (#4341)

Co-authored-by: hailin0 <hailin0@yeah.net>

* [Fix][common] only two yarns can be selected(#4314) (#4344)

* Handling conflicts manually

* code style

* ut skip ui

* fix ut error

* fix ut error

* delete SchedulerController Unused field

* delete SchedulerController Unused field

* delete SchedulerController Unused field

* ci skip ui

* ignore AlertPluginManagerTest.java and DolphinPluginLoaderTest.java

* fix ut error

* fix ut error

* test

* code style

* replace ui

* replace ui

* replace ui

* replace ui

* fix license

* fix license

Co-authored-by: 小清 <62982788+597365581@users.noreply.github.com>
Co-authored-by: zhuangchong <37063904+zhuangchong@users.noreply.github.com>
Co-authored-by: spring-bu <37682426+spring-bu@users.noreply.github.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: lgcareer <18610854716@163.com>
Co-authored-by: BoYiZhang <39816903+BoYiZhang@users.noreply.github.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>
Co-authored-by: baoliang <baoliang@analysys.com.cn>
Co-authored-by: BoYiZhang <zhangboyi_mx@163.com>
Co-authored-by: zixi0825 <649790970@qq.com>
Co-authored-by: sunchaohe <sunzhaohe@linklogis.com>
Co-authored-by: break60 <790061044@qq.com>
Co-authored-by: bao liang <29528966+lenboo@users.noreply.github.com>
Co-authored-by: t1mon <178317391@qq.com>
Co-authored-by: zhanglong <zhanglong@ysstech.com>
Co-authored-by: 孙继峰 <sun.jifeng@outlook.com>
Co-authored-by: wulingqi <wulingqi@baijiahulian.com>
Co-authored-by: lgcareer <lgcareer@apache.org>
Co-authored-by: IamMujuziMoses <49279842+IamMujuziMoses@users.noreply.github.com>
Co-authored-by: huzekang <1040080742@qq.com>
Co-authored-by: JinyLeeChina <42576980+JinyLeeChina@users.noreply.github.com>
Co-authored-by: Eights-LI <yelli.eights@gmail.com>
Co-authored-by: wuchunfu <319355703@qq.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
Co-authored-by: Yelli <amarantine@my.com>
Co-authored-by: Eights-Li <yelli.hl@gmail.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>
Co-authored-by: XiaotaoYi <v-xiayi@hotmail.com>
Co-authored-by: Yichao Yang <1048262223@qq.com>
Co-authored-by: zhuangchong <zhuangchong8@163.com>
Co-authored-by: muzhongjiang <mu_zhongjiang@163.com>
Co-authored-by: Jave-Chen <baicai.chen@gmail.com>
Co-authored-by: zhuangchong <zhuangchong6@163.com>
Co-authored-by: hailin0 <hailin0@yeah.net>
Co-authored-by: yangquan <iyeeku@qq.com>
Co-authored-by: liliang1991 <liliang_68@126.com>
Co-authored-by: liang.li.c <liang.li.c@17zuoye.com>
Co-authored-by: felix.wang <59079269+felix-thinkingdata@users.noreply.github.com>
Co-authored-by: chengshiwen <chengshiwen0103@gmail.com>
Co-authored-by: Yarlung <turingdojo@163.com>
Co-authored-by: karlsun <karlsun@tencent.com>
Co-authored-by: wangxj3 <857234426@qq.com>
Co-authored-by: chengp <5058557@qq.com>
Co-authored-by: chengp <chengp@chengp.net>
Co-authored-by: Saksham Gupta <saksham1319@gmail.com>
Co-authored-by: Tq <36755957+Tianqi-Dotes@users.noreply.github.com>
Co-authored-by: Jatham <59549328+Jatham922@users.noreply.github.com>
Co-authored-by: hailin0 <hailin0@foxmail.com>
Co-authored-by: zh0122 <zh0122@gmail.com>
Co-authored-by: geosmart <geosmart@hotmail.com>
Co-authored-by: 李丛阳 <liby-1987@163.com>
Co-authored-by: 0002939 <licongyang@mininglamp.com>
Co-authored-by: kamisamak <1057372918@qq.com>
2021-01-08 15:56:19 +08:00
Kirs
14785f4e39
[Feature-#3828][server] Delete recipients and CCs in the process defi… (#4303)
* [Feature-#3828][server] Delete recipients and CCs in the process definition and select the associated alert group instead

* The specific alarm instance associated with the alert group
* The process definition only associates the alert group, and does not care about the specific alarm instance.

this closes #3828

* remove test cc

* remove test cc

* fix test error

* add sql upgrade script
delete alert column(show_type、alert_type、re、cc)

* code style

* update test
Note the statement to delete the column, and provide related instructions

* delete AlertGroup field(alert_type)

* fix ut error

* fix ut error

* fix ut error

* code style

* delete api-doc receiverCC

* delete UserAlertGroupMapper

* delete UserAlertGroup

* delete UserAlertGroupApi

* alert group add coulmn user_id

* alter user_id to create_user_id

* userinfo remove alert group info
2020-12-31 10:18:53 +08:00
zhuangchong
169a4ace77
[Feature-3749][Alert-SPI] SqlTask should send notifications by alert server api (#4080)
* add sqltask send sync alert server.

* update alert-sms license.

* update AlertServer test.

* remote EmailAlertPluginTest.

* update sqltask.

* update test class.
2020-12-10 22:41:11 +08:00
zhuangchong
54802bb531 add test class. 2020-10-28 17:56:28 +08:00
zhuangchong
500d481e72 add test. 2020-10-27 13:32:54 +08:00
zhuangchong
a263455d6a add code checkstyle. 2020-10-23 13:08:07 +08:00
zhuangchong
2df6ecb2fc the alert module support service. 2020-10-23 11:58:08 +08:00
bao liang
39411ce03b
[FIX-3617][Service]fix 2 tasks instance are generated when fault tolerance (#3873)
* fix 2 tasks instance are generated when fault tolerance

* update code style

* update code style

* update code style

* update code style

* update code style

* update code style

* update code style

* update code style

* update code style

* update code style

* update code style

* update ut

* update code style

Co-authored-by: baoliang <baoliang@analysys.com.cn>
2020-10-10 15:05:56 +08:00
guyinyou
3b581455fc
[Feature]Add Python task "task variable / result transfer" implementation (#3659)
* 增加Python Task的“任务变量/结果传递”实现
Signed-off-by: 古崟佑

* add two files license
Signed-off-by: 古崟佑

* fix 'server/worker/task/AbstractCommandExecutor.java' code style
Signed-off-by: 1941815847Cy4 <1941815847cy4@kuaishou.com>

* update DB
Signed-off-by: 古崟佑

* update DB -- 2
Signed-off-by: 古崟佑

* fix codeStyle
Signed-off-by: 古崟佑

* fix codestyle
Signed-off-by: 古崟佑

* fix codeStyle
Signed-off-by: 古崟佑

* fix codeStyle
Signed-off-by: 古崟佑

* fix codeStyle
Signed-off-by: 古崟佑

* add VarPoolUtils Test
Signed-off-by: 古崟佑

* fix VarPoolUtilsTest codeStyle
Signed-off-by: 古崟佑

* fix VarPoolUtilsTest codeStyle
Signed-off-by: 古崟佑

* fix VarPoolUtilsTest codeStyle
Signed-off-by: 古崟佑

* fix VarPoolUtilsTest codeStyle
Signed-off-by: 古崟佑

* fix VarPoolUtilsTest codeStyle
Signed-off-by: 古崟佑

* add test config for VarPoolUtilsTest
Signed-off-by: 古崟佑

* fix unit test
Signed-off-by: 古崟佑

* fix codeStyle
Signed-off-by: 古崟佑

* fix VarPoolUtilsTest.java
Signed-off-by: 古崟佑

* fix
Signed-off-by: 古崟佑

* change the test class path
Signed-off-by: 古崟佑

* fix
Signed-off-by: 古崟佑

* fix "print the error message"
Signed-off-by: 古崟佑

* fix bug
Signed-off-by: 古崟佑

* fix
Signed-off-by: 古崟佑

Co-authored-by: 1941815847Cy4 <1941815847cy4@kuaishou.com>
2020-09-25 13:20:42 +08:00
CalvinKirs
a1ce54b8e6
[Improvement][server] Fix some sonar analyzed bugs (#3415)
* [FIX_BUG][*]fix sonar bugs
fix assert error(assertions always pass)

* fix assert error(assertions always pass)
insert data should judging the success status

* fix assert error(assertions always pass)

* use AtomicReference<Object> to replace unsafe volatile Object

* use AtomicReference<Object> to replace unsafe volatile Object

* lazy initialization holder class Singleton...

* reformat code
judge database operations

* reformat code
judge database operations

* revert

* use ThreadLocalRandom to build random numbers

* reform code

* fix dao test error

* judge database operations
2020-08-19 15:15:28 +08:00
vanilla111
a4ee351a3a
delay execution of tasks and improve some designs (#3427) 2020-08-19 11:36:30 +08:00
Yichao Yang
dce403d062
[Improvement-3369][api] Introduce logger service interface for clear code (#3437)
* [Improvement][api] Introduce logger service interface for clear code

* Remove the code smell
2020-08-10 14:47:16 +08:00
CalvinKirs
6ecc95a338
[FIX_BUG][*] fix some bugs from sonar analysis (#3322)
* [FIX_BUG][*] fix some bugs from sonar analysis

* remove invalid single test

* code smell

* remove exception catches that never happen

* delete invalid exception catch

* remove excess locks

* fix e2e test error

* recovery related exception removal
2020-08-04 02:23:50 +00:00
dailidong
6d903e266f
[Feature][server] add embeded zookeeper server in dolphinscheduler-daemon.sh (#3293)
* update README about DolphinScheduler

* Update issue templates

* update

* regularize api pom xml
update rpc maven compile to 1.8

* regularize api pom xml

* change commons.lang3.StringUtils to common.utils.StringUtils

* update pom.xml

* update

* correct equals method

* jasper-runtime is needed when api server start

* jasper-runtime jar is needed when api server start

* combine logback config of master/worker/alert/api server to one logback.xml

* remove tomcat runtime jar

* add UT

* add license

* remove jasper-runtime jar, not need anymore

* sovle jar conflict, remove servlet-api 2.5 jar

* remove servlet 2.5

* add embedded database h2 and update embedded zookeeper

* delete logs

* add license

* change log level to INFO

* simplify and optimize config

* add development state

* simplify common config

* add worker group config

* add worker group config

* add worker group config

* this command just for getting a quick experience,not recommended for production. this operation will start a standalone zookeeper server

* Merge remote-tracking branch 'upstream/dev' into dev

* add logback for zookeeper

* Update logback-zookeeper.xml

Co-authored-by: lidongdai <escheduler@outlook.com>
Co-authored-by: dailidong <dolphinschedule@gmail.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>
2020-07-31 17:37:02 +08:00
zixi0825
5fe31546ba
[Fix][all] fix spell error (#3289)
* fix spell error

* test unit test

Co-authored-by: sunchaohe <sunzhaohe@linklogis.com>
2020-07-30 18:33:16 +08:00
Yichao Yang
d006bd3ba5
[Improvement][api] Refactor the transaction rollbackfor runtimeexception (#3313)
* [Improvement][api] Refactor the transaction rollbackfor runtimeexception

* Remove the unused import
2020-07-30 15:12:14 +08:00
tswstarplanet
5b7efd2d33
[Feature-3134]Refactor to use a shared singleton Curator Zookeeper Client (#3244)
* [Feature-3134]Refactor to use a shared singleton Curator Zookeeper Client

* autowire CuratorZookeeperClient to ZookeeperOperator

* [Improvement] Add the issue specifications reference (#3221)

* log zookeeper address when the connect state change

* resume the operation of add connect state listener in MasterRegistry

Co-authored-by: Yichao Yang <1048262223@qq.com>
2020-07-23 11:02:30 +08:00
tswstarplanet
753ed58fb9
[Bug-3140]fix the deadlock between start and stop of ZKServer (#3141)
* [Bug-3140]fix the deadlock between start and stop of ZKServer

* use Log framework to print information

* fix code smells; add path prefix of embedded zk server

* Update dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/zk/ZKServer.java

Co-authored-by: Yichao Yang <1048262223@qq.com>

* optimize the code

Co-authored-by: Yichao Yang <1048262223@qq.com>
2020-07-13 18:52:33 +08:00
bao liang
657ea4752f
merge from 1.3.1-release (#3161)
* merge from 1.3.1-release

* [bugfix] remove Conflicting configuration sudo and user (#3038)

Co-authored-by: zhangchunyang1024 <zhangchunyang0123@gamil.com>

* remove some standard notices in order to cut down the NOTICE file. (#3101)

* #3084 remove some standard notices in order to cut down the NOTICE file.

* update exclude element sql/soft_version of rat plugin

* update soft_version

* merge from 1.3.1-release

* fix 3112:download udf resource need find the tenant code of resource owner (#3114)

* fix 3112:download udf resource need find the tenant code of resource owner

* fix 2975:download resource need find the tenant code of resource owner

* remove useless code

* merge from 1.3.1-release

* remove some notes (#3127)

Co-authored-by: lenboo <baoliang@analysys.com.cn>

* fix Permission denied if not found dolphinscheder-env.sh on 1.3.1 (#3128)

* fix Permission denied if not found dolphinscheder-env.sh

* update some comments on method

* add PropertyPlaceholderHelper on license (#3129)

* update license header location

* add PropertyPlaceholderHelper on license

* merge from 1.3.1-release

* change from 1.3.0  to 1.3.2

* add Set import

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: zhangchunyang <zhangchunyang0123@gmail.com>
Co-authored-by: zhangchunyang1024 <zhangchunyang0123@gamil.com>
Co-authored-by: lgcareer <18610854716@163.com>
Co-authored-by: lenboo <baoliang@analysys.com.cn>
2020-07-13 16:35:42 +08:00
tswstarplanet
d87d2d80dd
add maxWaitTime config when build zookeeper client (#3133)
* add maxWaitTime config when build zookeeper client

* throw exception when connect zk timeout

* use dedicated exception instead of a generic one

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>
2020-07-09 20:27:41 +08:00
lenboo
b81872cacd merge from dev-1.3.0 2020-06-19 18:52:30 +08:00
lenboo
714ae33e40 merge from dev 2020-06-19 14:20:24 +08:00
simon824
24120a4641 fix the bad code smell 2020-06-12 11:27:42 +08:00
simon824
f643981f48 1. change windows new line 'CR' to linux new line 'LF'
2. Format code
2020-06-11 10:14:15 +08:00
lenboo
1904a7022a change 1.3.0 to 1.3.0-snapshot 2020-06-10 11:42:15 +08:00
simon824
6e907f150c 1. specified jackson Date format.
2. remove getMapper method,replace with toList()
3. rewrite toList method,support constructCollectionType.
4. parseObject(resp, Map.class) replace to toMap() method
5. rename JacksonSerialize to JsonSerializer

rewrite createArrayNode,createObjectNode method will Affect too many lines ,i will fix it in the next pr.
2020-06-09 13:02:50 +08:00
dailidong
186ba3baab
fix oom when no master is active in dev-1.3.0 (#2918)
* fix worker group config no effect

* remove codehaus janino jar
the license about janino maybe not compatiable with Apache v2

* Merge remote-tracking branch 'upstream/dev-1.3.0' into dev-1.3.0

# Conflicts:
#	dolphinscheduler-server/src/main/resources/config/install_config.conf

* datasource config

* Update datasource.properties

* fix RunConfig bug

* remove param monitor server state

* fix table T_DS_ALERT

* update h2 database

* fix #2910 master server will show exception for some time when it restart

* fix oom when no master is active

* fix worker oom when master server restart

* fix oom

* fix

* add UT

* fix worker group config no effect
2020-06-08 15:55:39 +08:00
bao liang
422fe5136d
[license] add license and notice (#2916)
* fix bug: send task result to master until success.

* fix bug: send task result to master until success.

* add sleep

* set null if send result failed

* set null if send result failed

* change max length of username to 39

* test for gc

* change version to 1.3.0

* check and add some license files

* check and add some license files

* revert

Co-authored-by: baoliang <baoliang@analysys.com.cn>
2020-06-06 22:21:37 +08:00
t1mon
8681036a49
Fix tasks not execute in sub-process when rerun scheduled process. #2903 (#2908) 2020-06-05 18:42:10 +08:00
simon824
572ee0fc64 Using Jackson instead of Fastjson 2020-06-04 16:23:59 +08:00
lgcareer
4441d91dcf
Fix bug:If user didn't set the value of tenant,release process definition fail (#2885)
* copy resources need get top directory first

* copy resources need get top directory first

* fix #2860:copy files and create empty directory

* Fix bug:If user didn't set the value of tenant,release process definition online fail
2020-06-03 13:50:06 +08:00
simon
33bd6586dd Using Jackson instead of Fastjson 2020-06-03 09:19:59 +08:00
simon
f26e246706 Using Jackson instead of Fastjson 2020-06-02 20:00:49 +08:00
张世鸣
e386af700d Using Jackson instead of Fastjson 2020-06-02 17:22:39 +08:00
张世鸣
a9f72c2081 Using Jackson instead of Fastjson 2020-06-02 15:54:34 +08:00
bao liang
d67436ffad
[bug fix] fix: The workflow is fault-tolerant and 2 task instances are generated (#2833)
* feature: add number configuration for master dispatch tasks

* fix bug(#2762) the master would be blocked when worker group not exists

* fix bug(#2762) the master would be blocked when worker group not exists

* fix ut

* fix ut

* fix bug(2781): cannot pause work flow when task state is "submit success"

* fix code smell

* add mysql other param blank judge

* test

* update comments

* update comments

* add ut

* fix bug: Restart the worker service again, the previously submitted successful tasks are not executed

* update comments

* add sleep

* add null point check

* fix bug:After the master is fault-tolerant, it cannot resume operation

* fix bug: do not failover the host is 'NULL' process

* fix bug:worker failover error.

Co-authored-by: baoliang <baoliang@analysys.com.cn>
2020-05-28 12:31:12 +08:00
bao liang
1caac70215
[bug fix]fix bug: After the master is fault-tolerant, it cannot resume operation (#2813)
* feature: add number configuration for master dispatch tasks

* fix bug(#2762) the master would be blocked when worker group not exists

* fix bug(#2762) the master would be blocked when worker group not exists

* fix ut

* fix ut

* fix bug(2781): cannot pause work flow when task state is "submit success"

* fix code smell

* add mysql other param blank judge

* test

* update comments

* update comments

* add ut

* fix bug: Restart the worker service again, the previously submitted successful tasks are not executed

* update comments

* add sleep

* add null point check

* fix bug:After the master is fault-tolerant, it cannot resume operation

* fix bug: do not failover the host is 'NULL' process

Co-authored-by: baoliang <baoliang@analysys.com.cn>
2020-05-27 11:33:10 +08:00
qiaozhanwei
7eb7efbd34
compatible old version host not port delete log file (#2811)
* add LoggerServerTest UT

* add LoggerServerTest UT

* add LoggerServerTest UT
add RemoveTaskLogRequestCommandTest UT
add RemoveTaskLogResponseCommandTest

* master select worker filter high load worker #2704

* master select worker filter high load worker #2704

* master select worker filter high load worker #2704

* master select worker filter high load worker #2704

* master select worker filter high load worker #2704

* master select worker filter high load worker #2704

* add not worker log and remove worker invalid property

* process definition json worker group convert #2794

* process definition json worker group convert #2794

* process definition json worker group convert #2794

* process definition json worker group convert #2794

* process definition json worker group convert #2794

* process definition json worker group convert #2794

* process definition json worker group convert #2794

* process definition json worker group convert #2794

* compatible old version host not port delete log file

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
2020-05-25 17:59:02 +08:00
tlhhup
396b1716e4
fix bug:#2735 (#2770)
1. store process id in zk.
2. resolve host and port through zk path.
3. modify ui key.
2020-05-22 11:20:54 +08:00
bao liang
184d64e852
[BUG FIX] fix bug: dependent task failed when conditions task exists (#2768)
* fix bug 2464: change dependent task for process.

* remove unused code

* add ut

* add ut

* update comments

Co-authored-by: baoliang <baoliang@analysys.com.cn>
2020-05-21 10:34:28 +08:00