Commit Graph

712 Commits

Author SHA1 Message Date
wen-hemin
d382a7ba8c
[Fix-5509]: createSchedule interface, process definition id -> process definition code (#5756)
* fix: createSchedule interface, process definition id -> process definition code

* fix: add junit

* fix junit

* fix: projectName -> projectCode

* fix UT

* Optimize variable type

Co-authored-by: wen-hemin <wenhemin@apache.com>
2021-07-07 14:01:02 +08:00
JinyLeeChina
654fa39789
[Feature][JsonSplit-api]merge code from dev to json_split_two (#5750)
* [BUG-#5678][Registry]fix registry init node miss (#5686)

* [Improvement][UI] Update the update time after the user information is successfully modified (#5684)

* improve

edit the userinfo success, but the updatetime is not the latest.

* Improved shell task execution result log information, adding process.waitFor() and process.exitValue() information to the original log (#5691)

Co-authored-by: shenglm <shenglm840722@126.com>

* [Feature-#5565][Master Worker-Server] Global Param passed by sense dependencies (#5603)

* add globalParams new plan with varPool

* add unit test

* add python task varPoolParams


Co-authored-by: wangxj <wangxj31>

* Issue robot translation judgment changed to Chinese (#5694)



Co-authored-by: chenxingchun <438044805@qq.com>

* the update function should use post instead of get (#5703)

* enhance form verify (#5696)

* checkState only supports %s not {} (#5711)

* [Fix-5701]When deleting a user, the accessToken associated with the user should also be deleted (#5697)

* update

* fix the codestyle error

* fix the compile error

* support rollback

* [Fix-5699][UI] Fix update user error in user information (#5700)

* [Improvement] the automatically generated spi service name in alert-plugin is wrong (#5676)

* bug fix

the auto generated spi service can't be recongized



* include a  new method

* [Improvement-5622][project management] Modify the title (#5723)

Co-authored-by: Kirs <acm_master@163.com>
Co-authored-by: kyoty <echohlne@gmail.com>
Co-authored-by: ji04xiaogang <ji04xiaogang@163.com>
Co-authored-by: shenglm <shenglm840722@126.com>
Co-authored-by: wangxj3 <857234426@qq.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>
Co-authored-by: chenxingchun <438044805@qq.com>
Co-authored-by: Shiwen Cheng <chengshiwen0103@gmail.com>
Co-authored-by: Jianchao Wang <akingchao@qq.com>
Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-07-05 23:21:57 +08:00
JinyLeeChina
57414c4df7
[Feature][JsonSplit-api] merging from dev to json_split_two (#5712)
* [BUG-#5678][Registry]fix registry init node miss (#5686)

* [Improvement][UI] Update the update time after the user information is successfully modified (#5684)

* improve

edit the userinfo success, but the updatetime is not the latest.

* Improved shell task execution result log information, adding process.waitFor() and process.exitValue() information to the original log (#5691)

Co-authored-by: shenglm <shenglm840722@126.com>

* [Feature-#5565][Master Worker-Server] Global Param passed by sense dependencies (#5603)

* add globalParams new plan with varPool

* add unit test

* add python task varPoolParams


Co-authored-by: wangxj <wangxj31>

* Issue robot translation judgment changed to Chinese (#5694)



Co-authored-by: chenxingchun <438044805@qq.com>

* the update function should use post instead of get (#5703)

* enhance form verify (#5696)

* checkState only supports %s not {} (#5711)

* [Fix-5701]When deleting a user, the accessToken associated with the user should also be deleted (#5697)

* update

* fix the codestyle error

* fix the compile error

* support rollback

Co-authored-by: Kirs <acm_master@163.com>
Co-authored-by: kyoty <echohlne@gmail.com>
Co-authored-by: ji04xiaogang <ji04xiaogang@163.com>
Co-authored-by: shenglm <shenglm840722@126.com>
Co-authored-by: wangxj3 <857234426@qq.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>
Co-authored-by: chenxingchun <438044805@qq.com>
Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-06-29 22:55:01 +08:00
Shiwen Cheng
43be965b34 [Improvement-5567][UI] Add project id in web ui url for sharing and project name in project page (#5568)
* [Improvement-5567][UI] Add project id in web ui url for sharing

* [Improvement-5577][UI] Add Project Name in Project Page
2021-06-15 10:30:26 +08:00
Kirs
30c88032d3 [Feature-#3961][Registry]Registry-SPI (#5562)
[Feature#3961]Registry SPI

All the logical structure of the registry must be converted into a tree structure within the system, so some plug-ins must be converted internally, such as ETCD
The registry supports distributed locks.
todo: The specific information about the registration center of the API module needs to be adjusted.
2021-06-15 10:30:26 +08:00
ruanwenjun
a925f64571
[Feature][api] schedule add timezone support #5259 (#5465)
* [Feature][api] schedule add timezone support #5259

* import moment-timezone

* fix code smell
2021-05-17 21:08:20 +08:00
baoliang
b594afc94c Merge remote-tracking branch 'upstream/dev' into json_split
# Conflicts:
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/ProcessDefinitionController.java
#	dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/task/procedure/ProcedureTask.java
2021-05-07 01:38:31 +08:00
ruanwenjun
97f08429f4
[Improvement] Change the status box text (#5421) 2021-05-06 09:22:51 +08:00
zhuangchong
88a07f7b55
[Feature-4093][server] Support for stored procedures and stored function calls and data source supports DB2 (#4094)
* Support for stored procedures and stored function calls and data source supports DB2.


Co-authored-by: zhuangchong <zhuangchong8@163.com>
2021-04-29 18:29:44 +08:00
BoYiZhang
2a59ed092c
[Improvement-#3476][UI]Resource files support copy path operations (#5348)
* Improvement-5346 :  Resource files support copy path operations

* Improvement-5346 :  Resource files support copy path operations

* this close #5346

The current resource file does not support copying paths

When using the path is very easy to output errors. Add copy button. Support copy operation

* fix code smell
2021-04-29 16:29:29 +08:00
ruanwenjun
cbbe9c333c
[Improvement][api-server] Update the project owner (#5394)
* [Bug][api-server] delete the user ,when query the project by join the table of t_ds_user will return null. (#5326)

* when delete a user, do project ownership check

* fix ut

* fix vulnerability
2021-04-29 14:50:05 +08:00
baoliang
746d9eca1d Merge remote-tracking branch 'upstream/dev' into json_split
# Conflicts:
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/ProcessDefinitionController.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProcessDefinitionVersionServiceImpl.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProjectServiceImpl.java
#	dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/controller/ProcessDefinitionControllerTest.java
#	dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/ProcessDefinitionVersionServiceTest.java
#	dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/model/TaskNode.java
#	dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/master/runner/MasterBaseTaskExecThread.java
#	dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/task/processdure/ProcedureTask.java
#	dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/process/ProcessService.java
2021-04-26 14:14:05 +08:00
wen-hemin
7b2f33426b
fix: process definition version delete bug (#5380)
Co-authored-by: wen-hemin <wenhemin@apache.com>
2021-04-25 21:11:09 +08:00
wen-hemin
fd54dc0ce2
[FixBug][JsonSplit] depend task, the field "process definition id" modify to "process definition code" (#5377)
* fix: depend task, the field "process definition id" modify to "process definition code"

* fix: depend task, the field "process definition id" modify to "process definition code"

Co-authored-by: wen-hemin <wenhemin@apache.com>
2021-04-25 19:36:36 +08:00
ruanwenjun
e6d8da484f
[Improvement][API] Create Datasource #5083 (#5116)
* [Improvement][API] Create Datasource #5083

  * Refact create/ update datasource API

* fix Code Smells

* fix code smell

* resolve code smell

* use ConnectionParam to replace BaseDatasource

* solve license

* add ut

* code check

* add ut

* fix ut coverage

* fix ut
2021-04-25 13:51:59 +08:00
Shiwen Cheng
2e1d6975fd
[Improvement][Build] Optimize long release name and remove unused nginx release and upgrade spring boot version (#5308)
* [Build] Optimize long release name and remove unused nginx release

* [Build] Upgrade spring boot version

* [Improvement] Improve code style
2021-04-25 09:55:38 +08:00
wen-hemin
2a81b29b8f
fix: process instance version page bug (#5319)
Co-authored-by: wen-hemin <wenhemin@apache.com>
2021-04-19 16:22:17 +08:00
Kirs
ac15f45b0b
Update the main repository link (#5286) 2021-04-16 10:56:48 +08:00
Shiwen Cheng
1c1f6663a6
[Improvement][WorkerGroup] Optimize and reduce database and zookeeper queries and replace worker address input with multiple-select (#5182)
* [Improvement][Server] Optimize and reduce database and ZooKeeper query

* [Improvement][WorkerGroup] Replace worker address input with multiple-select

* [Improvement][WorkerGroup] Fix unit test

* [Improvement][API&UI] Optimize long host to short host and long host display in k8s
2021-04-14 21:39:39 +08:00
wenjun
cc7a4446f5
[Improvement][Task] DataX Node doesn't Support Clickhouse as datasource (#5180) (#5243)
* [Improvement][Module Name] DataX Node doesn't Support Clickhouse as the datasource (#5180)

* add ut

* add ut

* resolve code smell
2021-04-14 11:31:40 +08:00
baoliang
0166aebf71 Merge remote-tracking branch 'upstream/dev' into json_split
# Conflicts:
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/enums/Status.java
#	dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/utils/StringUtils.java
#	dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/ProjectMapper.java
#	dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/ProjectMapper.xml
#	dolphinscheduler-service/src/test/java/org/apache/dolphinscheduler/service/process/ProcessServiceTest.java
2021-04-13 23:10:19 +08:00
wen-hemin
305288caea
fix: When modify the node name, circular dependence bug (#5220)
Co-authored-by: wen-hemin <wenhemin@apache.com>
2021-04-07 11:14:33 +08:00
wen-hemin
4e97a0c4fd
fix: When process definition state is release, process version to set enable (#5216)
Co-authored-by: wen-hemin <wenhemin@apache.com>
2021-04-06 16:46:58 +08:00
JuFeng Li
e8c9c33d7e
[Fix-5055][UI] Fix can not upload file to correct dir (#5056)
* fix udf resource can not upload to the correct dir,and when delete udf resource can not refresh page

Co-authored-by: 李巨丰 <lijf@2345.com>
2021-04-01 20:57:46 +08:00
wanghong1314
817328ee8f
[Improvement-5172][UI]process list add Timing state add online/offline marking color (#5190)
Co-authored-by: wanghong <wanghong1@xunlei.com>
2021-03-31 20:50:12 +08:00
wen-hemin
15b2b1c0ec
[FixBug][Jsonsplit] When to save process definition, upload task code field #4417 (#5178)
* fix: task page, rename "description" to "desc"

* fix: The default value of field dependence is null.

* fix: frontend compile check

* fix: When create process, the version button is set to unavailable

* fix: sql condition bug

* fix: When to save process definition, upload task code field

Co-authored-by: wen-hemin <wenhemin@apache.com>
2021-03-30 14:28:08 +08:00
lenboo
c8ac6f99fe Merge remote-tracking branch 'upstream/dev' into spilit
# Conflicts:
#	dolphinscheduler-common/src/main/resources/common.properties
2021-03-30 11:31:42 +08:00
wen-hemin
0edd783373
[FIxBug][JsonSplit] When create process, the version button is set to unavailable #5137 (#5168)
* fix: task page, rename "description" to "desc"

* fix: The default value of field dependence is null.

* fix: frontend compile check

* fix: When create process, the version button is set to unavailable

Co-authored-by: wen-hemin <wenhemin@apache.com>
2021-03-29 17:24:01 +08:00
JinyLeeChina
a467fffd7b
[Feature][JsonSplit] fix taskId in json (#5165)
* modify checkDAGRing and ProcessService method

* merge

* modify dagRing

* modify process instance for project home page

* fix save process bug

* codeStyle

* Fix logical bug in saving process definition

* codeSytle

* Fix bug in interface of  queryProcessDefinitionList

* codeSytle

* Fix api bug"

* fix taskId in processDefinitionJson

* fix json taskId

* codeStyle

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-03-29 14:22:40 +08:00
zhuangchong
f6d62a4924
[Improvement-5144][UI] error v-show directives cannot be put on template tags (#5145)
* fix error v-show directives cannot be put on template tags.

* rerun e2e.
2021-03-26 16:42:13 +08:00
lenboo
0914699fc5 Merge branch 'json_split' of https://github.com/apache/incubator-dolphinscheduler into spilit 2021-03-26 00:06:26 +08:00
lenboo
f05ba84968 Merge remote-tracking branch 'upstream/dev' into spilit
# Conflicts:
#	dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/ProcessInstanceMapper.java
#	dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/TaskInstanceMapperTest.java
2021-03-25 23:54:01 +08:00
wen-hemin
e53367bdcc
[Feature][JsonSplit] task page, rename field "description" to "desc" (#5137)
* fix: task page, rename "description" to "desc"

* fix: The default value of field dependence is null.

* fix: frontend compile check

Co-authored-by: wen-hemin <wenhemin@apache.com>
2021-03-24 15:05:54 +08:00
Shiwen Cheng
0d065c0712
[Feature-5062][*] Create/Edit/Delete Worker Group in Web UI (#5113)
* [Improvement][Server] Rename MASTER_PREFIX to MASTER_TYPE and WORKER_PREFIX to WORKER_TYPE

* [Feature][Api&Dao] Support api and dao of worker group

* [Feature][UI] Support to create, edit and delete worker group

* [Feature][API&UI] Support work groups from database and zookeeper to display together in worker group management

* [Feature][API&Dao] Support long ip list and check request parameters when saving worker group

* [Feature][*] Refactor ipList to addrList

* [Feature][Server] Support host manager to select worker host by worker group in database

* [Improvement][Server] Improve the config of master and worker

* [Improvement][*] Rename zkRegistered to systemDefault

* [Improvement][Test] Fix unit test and improve ui

* [Improvement][Server] Improve getServerMaps

Co-authored-by: dailidong <dailidong66@gmail.com>
2021-03-23 23:47:41 +08:00
lenboo
010b49c933 merge from dev 2021-03-22 15:18:16 +08:00
Shiwen Cheng
8ac72e80e6
[Feature-5087][SqlTask] Add a switch to send mail and print head logs in SqlTask (#5088)
* [Feature-5087][SqlTask] Add the switch to send mail and print head logs in SqlTask

* [Feature-5087][SqlTask] Add SqlParametersTest
2021-03-19 15:07:39 +08:00
Shiwen Cheng
91c29e6ca3
[Improvement-5094][UI] Improve to add a main switch for refresh_in_switched_tab (#5095) 2021-03-19 11:58:33 +08:00
Shiwen Cheng
9008fa4b0c
[Fix-5089][UI] Fix code mirror cannot be displayed normally in sqoop task (#5090)
* [Improvement][UI] Rename from-mirror to form-mirror

* [Fix-5089][UI] Fix code mirror cannot be displayed normally in sqoop task
2021-03-19 11:57:46 +08:00
zhuangchong
705715c313
[Improvement-5059][API] Verify that the new workflow definition version is the current version before removing it #5059 (#5060)
* this process definition version is used, not allowed to delete.

* update ProcessDefinitionVersionServiceTest test class.

* update check has associated process definition sql.
2021-03-18 09:37:17 +08:00
lenboo
c33cc87af4 Merge remote-tracking branch 'upstream/dev' into spilit 2021-03-14 21:47:30 +08:00
Chenshuai
cc2a94a973
[Improvement-4988][docs] Uppercase the first letter the state of Process Instance (#4990)
* [DS-4988][docs] Uppercase the first letter the state of Process Instance

* [DS-4988][docs] Fix uppercase the first letter the state of Process Instance

* Fix uppercase the first letter the state of Process Instance

* Fix uppercase the first letter the state of Process Instance

* delete duplicate key name and delete

Co-authored-by: jim <ccit_cs@qq.com>
2021-03-12 16:48:33 +08:00
lenboo
6ef54ebe9b Merge remote-tracking branch 'upstream/dev' into spilit 2021-03-11 17:01:29 +08:00
Shiwen Cheng
beb5adb985
[Feature][MR] Support mapreduce name (#5029) 2021-03-11 12:35:08 +08:00
lenboo
81dcd6873d Merge remote-tracking branch 'upstream/dev' into spilit 2021-03-09 18:19:08 +08:00
Shiwen Cheng
e74932cfc2
[Feature-4960][Spark] Add name input for Spark and improve Spark & MR args (#4959)
* [Improvement][Spark] Improve spark args and add name input

* [Improvement][MR] Improve map reduce args

* [Improvement][Spark] Fix check style

* [Improvement][Spark] Fix check style for spark, flink and mr
2021-03-09 12:51:35 +08:00
lenboo
bb7106d100 Merge remote-tracking branch 'upstream/dev' into spilit 2021-03-08 10:23:16 +08:00
Shiwen Cheng
71bd231c95
[Improvement][UI] Unify form model by m-list-box (#4982) 2021-03-08 08:58:38 +08:00
lenboo
5bda7c344c Merge remote-tracking branch 'upstream/dev' into spilit 2021-03-05 22:11:55 +08:00
chengshiwen
fb2b55c150 [Improvement][Docker] Improve readme, notes, and long host display 2021-03-05 15:15:23 +08:00
Shiwen Cheng
24fda3fbb1
[Feature][Flink] Support parallelism input (#4937)
* [Feature][Flink] Support parallelism input

* [Feature][Flink] Fix check style

* [Feature][Flink] Remove FLINK_DETACH
2021-03-03 21:15:20 +08:00