Commit Graph

5060 Commits

Author SHA1 Message Date
wen-hemin
5f5aec93c4
[Fix-5518]: The data analysis state count interface, projectId change to projectCode (#5837)
* fix: the data analysis state count interface, projectId change to projectCode

* fix: the data analysis state count interface, projectId change to projectCode

* fix checkstyle

* fix checkstyle

* fix: the process state count page use "projectCode"

* fix: English comments

Co-authored-by: wen-hemin <wenhemin@apache.com>
2021-07-19 16:02:05 +08:00
wen-hemin
901bc9a43c
[Fix-5517]: The workflow lineage interface, projectId change to projectCode (#5834)
* fix: The workflow lineage api, projectId change to projectCode

* fix checkstyle

Co-authored-by: wen-hemin <wenhemin@apache.com>
2021-07-19 09:49:40 +08:00
sky
741d757dcb
[Feature][JsonSplit-api]taskDefinition save interface (#5828)
* create task definition api

create task definition api

create task definition api

* fix code smell

* use taskdefinitionlogs not taskdefinition

* fix code smell

* trigger GitHub actions

* fix unit test question

* fix unit test question

* fix unit test question
2021-07-17 15:35:13 +08:00
JinyLeeChina
13c607eeb9
[Feature][JsonSplit-api]merge code from dev to json_2 (#5827)
* [BUG-#5678][Registry]fix registry init node miss (#5686)

* [Improvement][UI] Update the update time after the user information is successfully modified (#5684)

* improve

edit the userinfo success, but the updatetime is not the latest.

* Improved shell task execution result log information, adding process.waitFor() and process.exitValue() information to the original log (#5691)

Co-authored-by: shenglm <shenglm840722@126.com>

* [Feature-#5565][Master Worker-Server] Global Param passed by sense dependencies (#5603)

* add globalParams new plan with varPool

* add unit test

* add python task varPoolParams


Co-authored-by: wangxj <wangxj31>

* Issue robot translation judgment changed to Chinese (#5694)



Co-authored-by: chenxingchun <438044805@qq.com>

* the update function should use post instead of get (#5703)

* enhance form verify (#5696)

* checkState only supports %s not {} (#5711)

* [Fix-5701]When deleting a user, the accessToken associated with the user should also be deleted (#5697)

* update

* fix the codestyle error

* fix the compile error

* support rollback

* [Fix-5699][UI] Fix update user error in user information (#5700)

* [Improvement] the automatically generated spi service name in alert-plugin is wrong (#5676)

* bug fix

the auto generated spi service can't be recongized



* include a  new method

* [Improvement-5622][project management] Modify the title (#5723)

* [Fix-5714] When updating the existing alarm instance, the creation time should't be updated (#5715)



* add a new init method.

* [Fix#5758] There are some problems in the api documentation that need to be improved (#5759)

* add the necessary parameters

* openapi improve

* fix code style error

* [FIX-#5721][master-server] Global params parameter missing (#5757)



Co-authored-by: wangxj <wangxj31>

* [Fix-5738][UI] The cancel button in the pop-up dialog of `batch copy` and `batch move`  doesn't work. (#5739)

* Update relatedItems.vue

* Update relatedItems.vue

* [Improvement#5741][Worker] Improve task process status log  (#5776)

* [Improvement-5773][server] need to support two parameters related to task (#5774)

* add some new parameter for task

* restore official properties

* improve imports

* modify a variable's name

Co-authored-by: jiang hua <jiang.hua@zhaopin.com.cn>

* [FIX-5786][Improvement][Server] When the Worker turns down, the MasterServer cannot handle the Remove event correctly and throws NPE

* [Improvement][Worker] Task log may be lost #5775 (#5783)

* [Imporvement #5725][CheckStyle] upgrade checkstyle file (#5789)

* [Imporvement #5725][CheckStyle] upgrade checkstyle file
  Upgrade checkstyle.xml to support checkstyle version 8.24+

* change ci checkstyle version

* [Fix-5795][Improvement][Server] The starttime field in the HttpTask log is not displayed as expected.  (#5796)

* improve timestamp format

make the startime in the log of httptask to be easier to read.


* fix bad code smell and update the note.

* [Imporvement #5621][job instance] start-time and end-time (#5621) (#5797)

·the list of workflow instances is sorted by start time and end time
·This closes #5621

* fix (#5803)

Co-authored-by: shuangbofu <fusb@tuya.com>

* fix: Remove duplicate "registryClient.close" method calls (#5805)

Co-authored-by: wen-hemin <wenhemin@apache.com>

* [Improvement][SPI] support load single plugin (#5794)

change load operation of 'registry.plugin.dir'

* [Improvement][Api Module] refactor registry client, remove spring annotation (#5814)

* fix: refactor registry client, remove spring annotation

* fix UT

* fix UT

* fix checkstyle

* fix UT

* fix UT

* fix UT

* fix: Rename RegistryCenterUtils method name

Co-authored-by: wen-hemin <wenhemin@apache.com>

* [Fix-5699][UI] Fix update user error in user information introduced by #5700 (#5735)

* [Fix-5726] When we used the UI page, we found some problems such as parameter validation, parameter update shows success but actually work (#5727)

* enhance the validation in UI

* enchance form verifaction

* simplify disable condition

* fix conflicts

Co-authored-by: Kirs <acm_master@163.com>
Co-authored-by: kyoty <echohlne@gmail.com>
Co-authored-by: ji04xiaogang <ji04xiaogang@163.com>
Co-authored-by: shenglm <shenglm840722@126.com>
Co-authored-by: wangxj3 <857234426@qq.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>
Co-authored-by: chenxingchun <438044805@qq.com>
Co-authored-by: Shiwen Cheng <chengshiwen0103@gmail.com>
Co-authored-by: Jianchao Wang <akingchao@qq.com>
Co-authored-by: Tanvi Moharir <74228962+tanvimoharir@users.noreply.github.com>
Co-authored-by: Hua Jiang <jianghuachinacom@163.com>
Co-authored-by: jiang hua <jiang.hua@zhaopin.com.cn>
Co-authored-by: Wenjun Ruan <861923274@qq.com>
Co-authored-by: Tandoy <56899730+Tandoy@users.noreply.github.com>
Co-authored-by: 傅双波 <786183073@qq.com>
Co-authored-by: shuangbofu <fusb@tuya.com>
Co-authored-by: wen-hemin <39549317+wen-hemin@users.noreply.github.com>
Co-authored-by: wen-hemin <wenhemin@apache.com>
Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-07-16 14:55:08 +08:00
JinyLeeChina
47de7cae7d
[Feature][JsonSplit-api] replace projectName with projectCode for taskDefinition (#5815)
* fix processDefinitonController projectCode

* queryAllByProjectCode of processDefinition

* query taskNodeList of processDefinition

* replace projectName with projectCode for taskDefinition

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-07-14 10:12:31 +08:00
JinyLeeChina
f66a16676a
[Feature][JsonSplit-api]query taskNodeList of processDefinition (#5802)
* fix processDefinitonController projectCode

* queryAllByProjectCode of processDefinition

* query taskNodeList of processDefinition

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-07-13 09:45:37 +08:00
JinyLeeChina
006ef97125
[Feature][JsonSplit-api]queryAllByProjectCode of processDefinition (#5801)
* fix processDefinitonController projectCode

* queryAllByProjectCode of processDefinition

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-07-12 11:39:39 +08:00
JinyLeeChina
7da7bb0890
fix processDefinitonController projectCode (#5800)
Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-07-12 10:40:46 +08:00
wen-hemin
d2a9e05a66
[Fix-5778]: The scheduler list、preview、delete interface, projectName -> projectCode (#5779)
* fix: createSchedule interface, process definition id -> process definition code

* fix: add junit

* fix junit

* fix: projectName -> projectCode

* fix UT

* Optimize variable type

* fix: the updateSchedule interface, projectName -> projectCode

* fix comment

* fix: The scheduler online/offline interface, projectName -> projectCode

* fix UT

* fix: the scheduler list-page interface, projectName -> projectCode, processDefinitionId -> processDefinitionCode

* add UT

* fix: The scheduler list、preview、delete interface, projectName -> projectCode

Co-authored-by: wen-hemin <wenhemin@apache.com>
2021-07-12 10:21:17 +08:00
wen-hemin
72535a47e3
[Fix-5510]: the scheduler list-page interface, projectName -> projectCode, processDefinitionId -> processDefinitionCode (#5771)
* fix: createSchedule interface, process definition id -> process definition code

* fix: add junit

* fix junit

* fix: projectName -> projectCode

* fix UT

* Optimize variable type

* fix: the updateSchedule interface, projectName -> projectCode

* fix comment

* fix: The scheduler online/offline interface, projectName -> projectCode

* fix UT

* fix: the scheduler list-page interface, projectName -> projectCode, processDefinitionId -> processDefinitionCode

* add UT

Co-authored-by: wen-hemin <wenhemin@apache.com>
2021-07-09 10:13:00 +08:00
wen-hemin
e4f427a8d8
[Fix-5763]: The scheduler online/offline interface, projectName -> projectCode (#5764)
* fix: createSchedule interface, process definition id -> process definition code

* fix: add junit

* fix junit

* fix: projectName -> projectCode

* fix UT

* Optimize variable type

* fix: the updateSchedule interface, projectName -> projectCode

* fix comment

* fix: The scheduler online/offline interface, projectName -> projectCode

* fix UT

Co-authored-by: wen-hemin <wenhemin@apache.com>
2021-07-08 13:59:40 +08:00
wen-hemin
cfa22d7c89
[Fix-5511]: the updateSchedule interface, projectName -> projectCode (#5761)
* fix: createSchedule interface, process definition id -> process definition code

* fix: add junit

* fix junit

* fix: projectName -> projectCode

* fix UT

* Optimize variable type

* fix: the updateSchedule interface, projectName -> projectCode

* fix comment

Co-authored-by: wen-hemin <wenhemin@apache.com>
2021-07-07 18:15:19 +08:00
wen-hemin
d382a7ba8c
[Fix-5509]: createSchedule interface, process definition id -> process definition code (#5756)
* fix: createSchedule interface, process definition id -> process definition code

* fix: add junit

* fix junit

* fix: projectName -> projectCode

* fix UT

* Optimize variable type

Co-authored-by: wen-hemin <wenhemin@apache.com>
2021-07-07 14:01:02 +08:00
JinyLeeChina
654fa39789
[Feature][JsonSplit-api]merge code from dev to json_split_two (#5750)
* [BUG-#5678][Registry]fix registry init node miss (#5686)

* [Improvement][UI] Update the update time after the user information is successfully modified (#5684)

* improve

edit the userinfo success, but the updatetime is not the latest.

* Improved shell task execution result log information, adding process.waitFor() and process.exitValue() information to the original log (#5691)

Co-authored-by: shenglm <shenglm840722@126.com>

* [Feature-#5565][Master Worker-Server] Global Param passed by sense dependencies (#5603)

* add globalParams new plan with varPool

* add unit test

* add python task varPoolParams


Co-authored-by: wangxj <wangxj31>

* Issue robot translation judgment changed to Chinese (#5694)



Co-authored-by: chenxingchun <438044805@qq.com>

* the update function should use post instead of get (#5703)

* enhance form verify (#5696)

* checkState only supports %s not {} (#5711)

* [Fix-5701]When deleting a user, the accessToken associated with the user should also be deleted (#5697)

* update

* fix the codestyle error

* fix the compile error

* support rollback

* [Fix-5699][UI] Fix update user error in user information (#5700)

* [Improvement] the automatically generated spi service name in alert-plugin is wrong (#5676)

* bug fix

the auto generated spi service can't be recongized



* include a  new method

* [Improvement-5622][project management] Modify the title (#5723)

Co-authored-by: Kirs <acm_master@163.com>
Co-authored-by: kyoty <echohlne@gmail.com>
Co-authored-by: ji04xiaogang <ji04xiaogang@163.com>
Co-authored-by: shenglm <shenglm840722@126.com>
Co-authored-by: wangxj3 <857234426@qq.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>
Co-authored-by: chenxingchun <438044805@qq.com>
Co-authored-by: Shiwen Cheng <chengshiwen0103@gmail.com>
Co-authored-by: Jianchao Wang <akingchao@qq.com>
Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-07-05 23:21:57 +08:00
JinyLeeChina
14035c03e2
[Feature][JsonSplit-api]move of processDefinition, the projectCode instead of projectName (#5744)
* select-by-code of processDefinition

* fix ut

* prcessDefinition copy

* move of processDefinition api and the projectCode instead of projectName

* fix ut

* fix ut

* fix review

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-07-05 18:04:17 +08:00
JinyLeeChina
c386b64500
[Feature][JsonSplit-api] copy of processDefinition (#5740)
* select-by-code of processDefinition

* fix ut

* prcessDefinition copy

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-07-03 14:10:16 +08:00
JinyLeeChina
57414c4df7
[Feature][JsonSplit-api] merging from dev to json_split_two (#5712)
* [BUG-#5678][Registry]fix registry init node miss (#5686)

* [Improvement][UI] Update the update time after the user information is successfully modified (#5684)

* improve

edit the userinfo success, but the updatetime is not the latest.

* Improved shell task execution result log information, adding process.waitFor() and process.exitValue() information to the original log (#5691)

Co-authored-by: shenglm <shenglm840722@126.com>

* [Feature-#5565][Master Worker-Server] Global Param passed by sense dependencies (#5603)

* add globalParams new plan with varPool

* add unit test

* add python task varPoolParams


Co-authored-by: wangxj <wangxj31>

* Issue robot translation judgment changed to Chinese (#5694)



Co-authored-by: chenxingchun <438044805@qq.com>

* the update function should use post instead of get (#5703)

* enhance form verify (#5696)

* checkState only supports %s not {} (#5711)

* [Fix-5701]When deleting a user, the accessToken associated with the user should also be deleted (#5697)

* update

* fix the codestyle error

* fix the compile error

* support rollback

Co-authored-by: Kirs <acm_master@163.com>
Co-authored-by: kyoty <echohlne@gmail.com>
Co-authored-by: ji04xiaogang <ji04xiaogang@163.com>
Co-authored-by: shenglm <shenglm840722@126.com>
Co-authored-by: wangxj3 <857234426@qq.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>
Co-authored-by: chenxingchun <438044805@qq.com>
Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-06-29 22:55:01 +08:00
JinyLeeChina
5b6c9b7d43
[Feature][JsonSplit-api] select-by-code of processDefinition (#5687)
* select-by-code of processDefinition

* fix ut

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-06-25 17:54:07 +08:00
CalvinKirs
fc203fb41c Merge remote-tracking branch 'origin/dev' into json_split_two
# Conflicts:
#	.github/actions/translate-on-issue
#	docker/build/conf/dolphinscheduler/registry.properties.tpl
#	docker/build/startup-init-conf.sh
#	docker/docker-swarm/config.env.sh
#	docker/kubernetes/dolphinscheduler/templates/_helpers.tpl
#	dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/controller/ProcessDefinitionControllerTest.java
#	dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/registry/RegistryCenter.java
#	dolphinscheduler-service/src/main/resources/registry.properties
2021-06-24 10:48:11 +08:00
Shiwen Cheng
93f1d4df7d
[Improvement-5405][Docker/K8s] Adapt to registry SPI (#5681) 2021-06-23 11:30:15 +08:00
Squidyu
664206cb81
wrong word: 'WAITTING' ->'WAITING' (#5682)
* Add:Name verification remove the first and last spaces.

* Update: wrong word: 'WAITTING' ->'WAITING'

Co-authored-by: Squid <2824638304@qq.com>
2021-06-22 21:56:21 +08:00
xingchun-chen
8ebd6469e6
update submodule translate-on-issue (#5677)
* Add issue robot automatic reply and Translation

Co-authored-by: chenxingchun <438044805@qq.com>
2021-06-22 15:12:41 +08:00
kyoty
60f93d16eb
update maven repository url (#5674) 2021-06-22 14:03:04 +08:00
kyoty
d64e3cb658
add the pageSize parameter (#5663) 2021-06-22 10:28:54 +08:00
kyoty
e628e4e8fd
[Improvement][API] Simplify the Check of Result by introducing several new methods. (#5666) 2021-06-22 01:47:24 +08:00
kyoty
e2eed1f24f
[Improvement] Fix a potential problem when editing an existing mysql data source 2021-06-21 18:52:25 +08:00
YulongZ
f7e551282a
docker-compose wrong env (#5664)
* docker-compose wrong env
2021-06-21 11:51:49 +08:00
myangle1120
a7682143ac
[Feature-#5565][UI] Global Param passed by sense dependencies (#5628)
* modify-globalParams

* del comment

Co-authored-by: liangfengjie <liangfengjie@hetao101.com>
2021-06-18 19:19:07 +08:00
kyoty
75813e7866
[bug fix] Some password attributes can be viewd directly in the log (#5638)
* fix the password display incorrectly

* update

* update

* fic code style

fic code style

* Update SensitiveDataConverterTest.java

* Update SensitiveDataConverter.java

* Update SensitiveDataConverter.java

* Update SensitiveDataConverterTest.java

* Update pom.xml
2021-06-18 19:10:57 +08:00
zhuangchong
b21e821845
fix PSQLException: ERROR: operator does not exist. (#5654) 2021-06-17 21:23:39 +08:00
justoy
0d7c32a1e8
Update TaskInstanceCacheManagerImpl.java (#5656) 2021-06-17 18:45:34 +08:00
zhuangchong
3026f04d85
[Improvement-5452][Task] ds flink task support submit a PyFlink job via the CLI (#5453)
* flink task support submit a PyFlink job via the CLI.

* optimize attribute name.

* Modify pyflink parameter judgment logic
2021-06-17 15:19:25 +08:00
kyoty
b05957db41
[Improvement] The newly introduced spi pugin support work in windows os
The newly introduced spi can't work in windows os.
2021-06-17 14:54:55 +08:00
JinyLeeChina
85f3ab43a5
[Feature][JsonSplit-api] define task api (#5647)
* processDefinition create/update

* fix codeStyle

* fix codeStyle

* fix ut

* api of project

* fix ut

* project update method

* define task api

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-06-17 10:57:30 +08:00
LOUKHNATI Mohamed Khalil
86ffc1f3f2
[Bug][WorkerServer] SqlTask NullPointerException (#5556)
* [Bug][WorkerServer] add null checks to sqlParamsMap and sqlParamsMap.get(i), Add test to verify it

* Change Test Name

* Fix Code style issues, Modify checking null for Sql params in sqlParamsMap with clearer logging.
2021-06-16 18:21:41 +08:00
kyoty
015ed7ffa6
[Bug Fix] Edit the note of existed datasource does't work. (#5642)
* Update DataSourceServiceImpl.java

* Update DataSourceServiceImpl.java
2021-06-16 17:29:17 +08:00
kyoty
0a5670f195
Fix the bug Edit User Info will always shows failed. (#5627)
* Update createUser.vue

* Update createUser.vue
2021-06-16 13:03:01 +08:00
kyoty
67711442d5
[Improvement][SQL] Query return number In SQL should be configurable (#5632)
* [Improvement][SQL] Query return number should be configurable #5580

* Update SqlTaskTest.java

* Update sql.vue
2021-06-16 09:40:21 +08:00
zhuangchong
e2d6265e26
[Improvement][registry-plugin] Optimize registry plugin loading and initial installation #5635 2021-06-15 20:51:52 +08:00
kyoty
9d70c7e534
Update ProcessDefinitionMapperTest.java (#5633) 2021-06-15 19:29:09 +08:00
kyoty
c5bea3c774
[Fix-5540][JSON Split] Fix some new tables and fields are missing in the sql upgrade script (#5611)
* Update dolphinscheduler_ddl.sql

* Update dolphinscheduler_ddl.sql

* Update dolphinscheduler_ddl.sql

* Update dolphinscheduler_ddl.sql

* Update dolphinscheduler_ddl.sql

* Update dolphinscheduler_ddl.sql

* Update dolphinscheduler_ddl.sql

* Update dolphinscheduler_ddl.sql

* Create dolphinscheduler_ddl.sql

* Update dolphinscheduler_postgre.sql

* Update dolphinscheduler_ddl.sql

* Update dolphinscheduler_ddl.sql

* Update dolphinscheduler_ddl.sql

* Create dolphinscheduler_ddl.sql

* Update dolphinscheduler_mysql.sql

* Update dolphinscheduler_postgre.sql

* Update dolphinscheduler_ddl.sql

* Update dolphinscheduler_ddl.sql

* Update dolphinscheduler_postgre.sql
2021-06-15 14:45:39 +08:00
JinyLeeChina
728bd31f73
[Feature][JsonSplit-api] api of project (#5630)
* processDefinition create/update

* fix codeStyle

* fix codeStyle

* fix ut

* api of project

* fix ut

* project update method

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-06-15 10:52:03 +08:00
kyoty
71a44f8a57 [Fix-5483] [Bug][API] Can't view variables in the page of Process Instance (#5631) 2021-06-15 10:30:26 +08:00
Shiwen Cheng
3fb01e1b4c [Fix-5596][Python] Fix conflict between python_home and datax_home configuration in dolphinscheduler_env.sh (#5612) 2021-06-15 10:30:26 +08:00
xingchun-chen
109e6543b9 modify issues translation robot (#5624)
* Add issue robot automatic reply and Translation


Co-authored-by: chenxingchun <438044805@qq.com>
2021-06-15 10:30:26 +08:00
Shiwen Cheng
43be965b34 [Improvement-5567][UI] Add project id in web ui url for sharing and project name in project page (#5568)
* [Improvement-5567][UI] Add project id in web ui url for sharing

* [Improvement-5577][UI] Add Project Name in Project Page
2021-06-15 10:30:26 +08:00
xingchun-chen
6e818032b9 modify issues translation robot (#5614)
* Add issue robot automatic reply and Translation
2021-06-15 10:30:26 +08:00
Shiwen Cheng
47812d2691 [Fix][Docker] Fix docker image build error (#5613) 2021-06-15 10:30:26 +08:00
blackberrier
053e548bf3 [Improvement-5539][Master] Check status of taskInstance from cache (#5572)
* improvement:check status of taskInstance from cache

* issue5572 use timer instead of while&sleep; consider concurrent modification

* use computeifpresent instead of lock

* simplify getByTaskInstanceId function

* add ut for TaskInstanceCacheManagerImpl; fix bug in TaskInstanceCacheManagerImpl

* add Apache license header;add test class in root pom
2021-06-15 10:30:26 +08:00
Kirs
30c88032d3 [Feature-#3961][Registry]Registry-SPI (#5562)
[Feature#3961]Registry SPI

All the logical structure of the registry must be converted into a tree structure within the system, so some plug-ins must be converted internally, such as ETCD
The registry supports distributed locks.
todo: The specific information about the registration center of the API module needs to be adjusted.
2021-06-15 10:30:26 +08:00