JinyLeeChina
c386b64500
[Feature][JsonSplit-api] copy of processDefinition ( #5740 )
...
* select-by-code of processDefinition
* fix ut
* prcessDefinition copy
Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-07-03 14:10:16 +08:00
JinyLeeChina
57414c4df7
[Feature][JsonSplit-api] merging from dev to json_split_two ( #5712 )
...
* [BUG-#5678][Registry]fix registry init node miss (#5686 )
* [Improvement][UI] Update the update time after the user information is successfully modified (#5684 )
* improve
edit the userinfo success, but the updatetime is not the latest.
* Improved shell task execution result log information, adding process.waitFor() and process.exitValue() information to the original log (#5691 )
Co-authored-by: shenglm <shenglm840722@126.com>
* [Feature-#5565][Master Worker-Server] Global Param passed by sense dependencies (#5603 )
* add globalParams new plan with varPool
* add unit test
* add python task varPoolParams
Co-authored-by: wangxj <wangxj31>
* Issue robot translation judgment changed to Chinese (#5694 )
Co-authored-by: chenxingchun <438044805@qq.com>
* the update function should use post instead of get (#5703 )
* enhance form verify (#5696 )
* checkState only supports %s not {} (#5711 )
* [Fix-5701]When deleting a user, the accessToken associated with the user should also be deleted (#5697 )
* update
* fix the codestyle error
* fix the compile error
* support rollback
Co-authored-by: Kirs <acm_master@163.com>
Co-authored-by: kyoty <echohlne@gmail.com>
Co-authored-by: ji04xiaogang <ji04xiaogang@163.com>
Co-authored-by: shenglm <shenglm840722@126.com>
Co-authored-by: wangxj3 <857234426@qq.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>
Co-authored-by: chenxingchun <438044805@qq.com>
Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-06-29 22:55:01 +08:00
JinyLeeChina
5b6c9b7d43
[Feature][JsonSplit-api] select-by-code of processDefinition ( #5687 )
...
* select-by-code of processDefinition
* fix ut
Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-06-25 17:54:07 +08:00
CalvinKirs
fc203fb41c
Merge remote-tracking branch 'origin/dev' into json_split_two
...
# Conflicts:
# .github/actions/translate-on-issue
# docker/build/conf/dolphinscheduler/registry.properties.tpl
# docker/build/startup-init-conf.sh
# docker/docker-swarm/config.env.sh
# docker/kubernetes/dolphinscheduler/templates/_helpers.tpl
# dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/controller/ProcessDefinitionControllerTest.java
# dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/registry/RegistryCenter.java
# dolphinscheduler-service/src/main/resources/registry.properties
2021-06-24 10:48:11 +08:00
Shiwen Cheng
93f1d4df7d
[Improvement-5405][Docker/K8s] Adapt to registry SPI ( #5681 )
2021-06-23 11:30:15 +08:00
Squidyu
664206cb81
wrong word: 'WAITTING' ->'WAITING' ( #5682 )
...
* Add:Name verification remove the first and last spaces.
* Update: wrong word: 'WAITTING' ->'WAITING'
Co-authored-by: Squid <2824638304@qq.com>
2021-06-22 21:56:21 +08:00
kyoty
75813e7866
[bug fix] Some password attributes can be viewd directly in the log ( #5638 )
...
* fix the password display incorrectly
* update
* update
* fic code style
fic code style
* Update SensitiveDataConverterTest.java
* Update SensitiveDataConverter.java
* Update SensitiveDataConverter.java
* Update SensitiveDataConverterTest.java
* Update pom.xml
2021-06-18 19:10:57 +08:00
zhuangchong
e2d6265e26
[Improvement][registry-plugin] Optimize registry plugin loading and initial installation #5635
2021-06-15 20:51:52 +08:00
Kirs
30c88032d3
[Feature-#3961][Registry]Registry-SPI ( #5562 )
...
[Feature#3961]Registry SPI
All the logical structure of the registry must be converted into a tree structure within the system, so some plug-ins must be converted internally, such as ETCD
The registry supports distributed locks.
todo: The specific information about the registration center of the API module needs to be adjusted.
2021-06-15 10:30:26 +08:00
Chouc
3193837f5a
[DS-5559][fix][Master Server] Master Server was shutdown but the process still in system ( #5588 )
...
* Close spring context to destory beans that has running thread
2021-06-15 10:30:26 +08:00
JinyLeeChina
a561a618af
[Feature][JsonSplit-api] api of processDefinition create/update ( #5602 )
...
* processDefinition create/update
* fix codeStyle
* fix codeStyle
* fix ut
Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-06-11 09:43:05 +08:00
Kirs
e2243d63be
[Feature-#3961][Registry]Registry-SPI ( #5562 )
...
[Feature#3961]Registry SPI
All the logical structure of the registry must be converted into a tree structure within the system, so some plug-ins must be converted internally, such as ETCD
The registry supports distributed locks.
todo: The specific information about the registration center of the API module needs to be adjusted.
2021-06-09 15:21:02 +08:00
Chouc
75be09735a
[DS-5559][fix][Master Server] Master Server was shutdown but the process still in system ( #5588 )
...
* Close spring context to destory beans that has running thread
2021-06-04 13:27:18 +08:00
Shiwen Cheng
a5a0c7c5f8
[Fix-5570][Worker] Fix worker.groups in worker.properties is still commented and improve install scripts and configurations ( #5571 )
2021-06-02 11:21:01 +08:00
ruanwenjun
842c5400e6
[Improvement][Server] Must restart master if Zk reconnect ( #5210 ) ( #5211 )
2021-05-25 05:03:29 +08:00
zhuangchong
1b7dcea610
[Improvement-5440][Maven] Project module maven dependency optimization ( #5441 )
...
* Optimize dependencies between modules.
* update module-alert dependency.
* add repository in the pom xml.
* update Central Repository in the pom.xml.
2021-05-19 10:05:00 +08:00
ruanwenjun
a925f64571
[Feature][api] schedule add timezone support #5259 ( #5465 )
...
* [Feature][api] schedule add timezone support #5259
* import moment-timezone
* fix code smell
2021-05-17 21:08:20 +08:00
baoliang
156dadac69
Merge remote-tracking branch 'upstream/dev' into json_split
...
# Conflicts:
# dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/master/runner/MasterExecThread.java
# dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/utils/DependentExecute.java
2021-05-11 10:08:28 +08:00
JinyLeeChina
515fa4912e
[Feature][JsonSplit] Fix ProcessService ut ( #5435 )
...
* Fix ProcessService ut
* codeStyle
* Fix AlertPlugin ut
* Fix AlertPlugin ut
* Fix server module ut
* Fix server module ut
* Fix api ut
* codeStyle
Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-05-10 23:17:56 +08:00
zhuangchong
9c77faa8ac
[Improvement-5058][Master] The master-server creates the DB data source twice ( #5139 )
...
* reduce the number of data sources created twice by master server.
* update ProcessAlertManagerTest test class to increase coverage.
* add DolphinSchedulerPluginLoaderTest test ignore.
Co-authored-by: zhuang chong
2021-05-10 09:56:17 +08:00
JinyLeeChina
847865fba6
[Feature][JsonSplit] Fix dao ut ( #5433 )
...
* update taskParams/add task delayTime/fix conditionType bug
* update codeStyle for merge to dev
* fix process lineage bug
* fix codeStyle for json_split
* Fix dao ut
* Fix dao ut
* codeStyle
* modify postgreSQL
* fix TaskInstanceMapper ut
* fix ProjectMapper ut
* fix ProcessDefinitionDao ut
* datasource
* fix postgreSQL
Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-05-09 13:53:14 +08:00
JinyLeeChina
2a44cb9de6
[Feature][JsonSplit] update codeStyle for merge to dev ( #5392 )
...
* update taskParams/add task delayTime/fix conditionType bug
* update codeStyle for merge to dev
Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-04-26 22:30:35 +08:00
baoliang
746d9eca1d
Merge remote-tracking branch 'upstream/dev' into json_split
...
# Conflicts:
# dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/ProcessDefinitionController.java
# dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProcessDefinitionVersionServiceImpl.java
# dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProjectServiceImpl.java
# dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/controller/ProcessDefinitionControllerTest.java
# dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/ProcessDefinitionVersionServiceTest.java
# dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/model/TaskNode.java
# dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/master/runner/MasterBaseTaskExecThread.java
# dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/task/processdure/ProcedureTask.java
# dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/process/ProcessService.java
2021-04-26 14:14:05 +08:00
JinyLeeChina
c3558965c1
update taskParams/add task delayTime/fix conditionType bug ( #5385 )
...
Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-04-26 11:28:35 +08:00
JinyLeeChina
db96bf2dfe
[Feature][JsonSplit] Fix batchMove of processDefinition bug ( #5371 )
...
* update SnowFlake
* update processDefinite from processInstance
* update processDefinite from processInstance
* Fix task logger path
* Fix dependTask bug
* Fix batchMove of processDefinition bug
* codeStyle
Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-04-24 15:33:23 +08:00
JinyLeeChina
862565a7c7
[Feature][JsonSplit] Fix dependTask bug ( #5360 )
...
* update SnowFlake
* update processDefinite from processInstance
* update processDefinite from processInstance
* Fix task logger path
* Fix dependTask bug
Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-04-23 14:23:04 +08:00
wanghong1314
e12a26289b
【BUG-5328】 modify process contain depend task is always running ( #5344 )
...
Co-authored-by: wanghong1314 <wanghong1@onething.net>
2021-04-21 22:29:53 +08:00
JinyLeeChina
d60f31bcf3
[Feature][JsonSplit] update processDefinite from processInstance ( #5325 )
...
* update SnowFlake
* update processDefinite from processInstance
* update processDefinite from processInstance
Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-04-20 15:33:37 +08:00
JinyLeeChina
49b234609a
update SnowFlake ( #5321 )
...
Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-04-19 23:12:09 +08:00
luoyuan
513eb76919
[Improvement-5147][Service]This judgment always returns true ( #5156 )
...
* this expression which always evaluates "true"
* delete unused variables
* LogClientService add AutoCloseable
* reformat code
* reformat code
* reformat code
* add log in ProcessService
* reformat code
* Optimize the code
* Optimize the code
* [improvement]modify all methods that refer to LogClientService in the code.
* delete unused code
2021-04-19 17:48:09 +08:00
QuakeWang
5ef1f731b7
Update PeerTaskInstancePriorityQueueTest.java ( #5314 )
...
change `TaskInstance taskInstanceMediumPriority = createTaskInstance("high", Priority.MEDIUM);` to `TaskInstance taskInstanceMediumPriority = createTaskInstance("medium", Priority.MEDIUM);`
2021-04-19 09:48:43 +08:00
JinyLeeChina
f7a06a033d
[Feature][JsonSplit] Fix master/worker about shellTask ( #5300 )
...
* Fix quoted bug about processDefineId of processInstance
* Fix master/worker about shellTask
* codeStyle
* codeStyle
* update taskType
* codeStyle
* update timeoutNotifyStrategy
* codeStyle
Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-04-18 12:45:30 +08:00
wangxj3
6dbbf6d55d
fix bug of If start a process instance form fail node or stop node ,the param will be init. ( #5267 )
...
Co-authored-by: wangxj <wangxj31>
2021-04-16 23:14:22 +08:00
wenjun
f3dfaacca2
[Improvement][Service] Optimize query log ( #5205 )
...
* [Improvement][Service] Optimize query log
* remove IPUtils
2021-04-15 20:25:01 +08:00
Shiwen Cheng
1c1f6663a6
[Improvement][WorkerGroup] Optimize and reduce database and zookeeper queries and replace worker address input with multiple-select ( #5182 )
...
* [Improvement][Server] Optimize and reduce database and ZooKeeper query
* [Improvement][WorkerGroup] Replace worker address input with multiple-select
* [Improvement][WorkerGroup] Fix unit test
* [Improvement][API&UI] Optimize long host to short host and long host display in k8s
2021-04-14 21:39:39 +08:00
baoliang
0166aebf71
Merge remote-tracking branch 'upstream/dev' into json_split
...
# Conflicts:
# dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/enums/Status.java
# dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/utils/StringUtils.java
# dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/ProjectMapper.java
# dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/ProjectMapper.xml
# dolphinscheduler-service/src/test/java/org/apache/dolphinscheduler/service/process/ProcessServiceTest.java
2021-04-13 23:10:19 +08:00
JinyLeeChina
645847c096
Fix quoted bug about processDefineId of processInstance ( #5263 )
...
Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-04-13 22:47:36 +08:00
Bruce Lau
3e64f43138
[Test-4124][service] Add the testCreateCommand test of ProcessService Class ( #5257 )
...
* [uint test] Improve code test coverage of ProcessService class
* [Test-4124][service] Add the testCreateCommand test of ProcessService Class
Co-authored-by: liuziying <liuziying@longfor.com>
2021-04-12 18:55:00 +08:00
JinyLeeChina
31188ffca0
[Feature][JsonSplit] Fix taskInstance ( #5246 )
...
* Fix dependency failure
* Fix taskInstance
* task instance list page
* code review
* fix logger path
Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-04-11 16:52:19 +08:00
wen-hemin
cc82a98f50
[FixBug][JsonSplit] dag show bugs ( #5226 )
...
* fix: dag show bugs
* fix: code format
* fix: code format
* fix: code format
Co-authored-by: wen-hemin <wenhemin@apache.com>
2021-04-08 14:18:01 +08:00
JinyLeeChina
a8f2894b79
[Feature][JsonSplit] Fix master/processInstance bug ( #5206 )
...
* transform taskCode from long to string
* fix process bug
* code review
* code review
* code review
* Fix master/processInstance bug
Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-04-04 11:26:42 +08:00
bao liang
86ce8f133a
[JsonSplit-4417][master/worker/api]json split- remove json in task instance. ( #5160 )
...
* [json split] refactor json in task instance
* add task_pramas in task_instance
* feature #4417 refactor json split in task instance
* code style
* code style
* code style
* code style
* code style
* update
* update
* update unit test
* update
2021-04-03 17:58:04 +08:00
JinyLeeChina
dd36da8d1f
[Feature][JsonSplit] fix process bug ( #5193 )
...
* transform taskCode from long to string
* fix process bug
* code review
* code review
* code review
Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-04-01 17:45:44 +08:00
JinyLeeChina
689106d114
transform taskCode from long to string ( #5174 )
...
Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-03-30 11:07:28 +08:00
JinyLeeChina
a467fffd7b
[Feature][JsonSplit] fix taskId in json ( #5165 )
...
* modify checkDAGRing and ProcessService method
* merge
* modify dagRing
* modify process instance for project home page
* fix save process bug
* codeStyle
* Fix logical bug in saving process definition
* codeSytle
* Fix bug in interface of queryProcessDefinitionList
* codeSytle
* Fix api bug"
* fix taskId in processDefinitionJson
* fix json taskId
* codeStyle
Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-03-29 14:22:40 +08:00
lenboo
0914699fc5
Merge branch 'json_split' of https://github.com/apache/incubator-dolphinscheduler into spilit
2021-03-26 00:06:26 +08:00
lenboo
f05ba84968
Merge remote-tracking branch 'upstream/dev' into spilit
...
# Conflicts:
# dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/ProcessInstanceMapper.java
# dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/TaskInstanceMapperTest.java
2021-03-25 23:54:01 +08:00
Simon
cd19507271
[feature][jsonsplit]fix copy/export/import processDefinition bugs ( #5146 )
...
* Modify Project and ProjectUser Mapper
* Modify Project and ProjectUser Mapper
* project_code is bigint(20)
* modify ERROR name
* modify saveProcessDefine, remove the duplicate code with createTaskAndRelation
* modify import/export processdefinition, add genProcessData
* fix ut and bug
* code style
* repalce project_id with code
* conflicts solve
* conflicts solve
* conflicts solve
* bugfix
* modify listResources mothod and remove getResourceIds mothod
* 1
* conflicts solve
* modify listResources mothod and remove getResourceIds mothod
* modify listResources mothod and remove getResourceIds mothod
* replace processDefinitionVersion with processDefinitionLog
* codestyle
* codestyle
* add mapper module ut
* codestyle
* fix ProcessInstanceMapperTest
* codestyle
* conflicts solve
* conflicts solve
* conflicts solve
* conflicts solve
* conflicts solve
* fix ProcessInstanceMapperTest
* fix ProjectMapperTest/ProjectUserMapperTest/ScheduleMapperTest
* fix ProjectMapperTest/ProjectUserMapperTest/ScheduleMapperTest
* fix TaskInstanceMapperTest
* add TaskDefinitionLogMapperTest/TaskDefinitionMapperTest and bugfix
* codestyle
* codestyle
* Refactor process lineage
* Refactor process lineage
* codestyle
* codestyle
* Refactor process lineage
* Refactor process lineage
* add WorkFlowLineageMapperTest/WorkFlowLineageServiceTest
* add WorkFlowLineageMapperTest/WorkFlowLineageServiceTest
* add ProcessServiceTest
* 1
* 1
* fix copy/export/import processDefinition bugs
* 1
* codestyle
2021-03-25 18:27:47 +08:00
Simon
3c9b214da7
[Feature][jsonsplit]add ProcessServiceTest ( #5097 )
...
* Modify Project and ProjectUser Mapper
* Modify Project and ProjectUser Mapper
* project_code is bigint(20)
* modify ERROR name
* modify saveProcessDefine, remove the duplicate code with createTaskAndRelation
* modify import/export processdefinition, add genProcessData
* fix ut and bug
* code style
* repalce project_id with code
* conflicts solve
* conflicts solve
* conflicts solve
* bugfix
* modify listResources mothod and remove getResourceIds mothod
* 1
* conflicts solve
* modify listResources mothod and remove getResourceIds mothod
* modify listResources mothod and remove getResourceIds mothod
* replace processDefinitionVersion with processDefinitionLog
* codestyle
* codestyle
* add mapper module ut
* codestyle
* fix ProcessInstanceMapperTest
* codestyle
* conflicts solve
* conflicts solve
* conflicts solve
* conflicts solve
* conflicts solve
* fix ProcessInstanceMapperTest
* fix ProjectMapperTest/ProjectUserMapperTest/ScheduleMapperTest
* fix ProjectMapperTest/ProjectUserMapperTest/ScheduleMapperTest
* fix TaskInstanceMapperTest
* add TaskDefinitionLogMapperTest/TaskDefinitionMapperTest and bugfix
* codestyle
* codestyle
* Refactor process lineage
* Refactor process lineage
* codestyle
* codestyle
* Refactor process lineage
* Refactor process lineage
* add WorkFlowLineageMapperTest/WorkFlowLineageServiceTest
* add WorkFlowLineageMapperTest/WorkFlowLineageServiceTest
* add ProcessServiceTest
* 1
* 1
2021-03-24 15:14:22 +08:00
wen-hemin
e53367bdcc
[Feature][JsonSplit] task page, rename field "description" to "desc" ( #5137 )
...
* fix: task page, rename "description" to "desc"
* fix: The default value of field dependence is null.
* fix: frontend compile check
Co-authored-by: wen-hemin <wenhemin@apache.com>
2021-03-24 15:05:54 +08:00
