Commit Graph

1085 Commits

Author SHA1 Message Date
insist777
ebdf903dcc
[Improvement-11468] Support re running historical version workflow (#11489)
* Support re running historical version workflow

 Support re running historical version workflow

* Update ExecutorServiceImpl.java

* Update ExecutorServiceImpl.java
2022-08-18 09:46:54 +08:00
insist777
006d8e21a6
[Improvement] File deletion logic optimization (#11519)
* [Improvement] File deletion logic optimization
2022-08-18 09:38:06 +08:00
kezhenxu94
f6ef628030
Support reading application properties from kubernetes config map (#11402) 2022-08-16 20:25:17 +08:00
Wenjun Ruan
7ff34c3947
[Feature-7024] Add waiting strategy to support master/worker can recover from registry lost (#11368)
* Add waiting strategy to support master/worker can recover from registry lost

* throw exception when zookeeper registry start failed due to interrupted
2022-08-13 09:52:03 +08:00
insist777
8217784044
[Fix-1360]: Fix Description length check (#11371)
* [Fix-1360]: Fix Description length check

* Update dolphinscheduler_h2.sql

h2 Increase the length of some fields

* Update dolphinscheduler_ddl.sql
2022-08-11 13:26:50 +08:00
fuchanghai
6e2bed4d6b
[fix-11378] fix API-Server release response message typo (#11400)
* [fix-11295] remove duplicate classes

* [fix-11378] fix response message error

Co-authored-by: fuchanghai <changhai.fu@marketingforce.com>
2022-08-11 09:21:32 +08:00
caishunfeng
0464123c2b
[Feature-11223] support stream task (#11350)
* add task execute type

* update task definition list paging

* update task instance list paging

* stream task start

* [Feature][UI] Some changes to execute task.
    * Set the connection edge to dashed line.
    * Add FLINK_STREAM task.

* add stream task

* flink savepoint and cancel

* fix query bug

* add stream task definition

* add task instance for stream task

* delete stream task definition state

* update api for stream task definition edit

* modify search for stream task instance

* add language

* delete task type search for stream task definition

* change task type search for stream task instance

* add jump button

* add savepoint

* add down log for stream task instance

* ui test

* stream task start

* run DAG

* [Fix][UI] Fix the stream task edgs not to be dashed when filling back.

* [Feature][UI] Remove some fields for FLINK_STREAM.

* add start modal

* add dryRun column for stream task instance

* fix duration

* fix pon

* fix build error

* Add success tip

* add auto sync for stream task instance

* remove forgien key for task instance

* license header

* UT fix

* modify locales

* recover common config

* fix UT

* add doc

Co-authored-by: Amy <amywang0104@163.com>
Co-authored-by: devosend <devosend@gmail.com>
2022-08-10 21:44:43 +08:00
xuhhui
a50f110d3e
[Feature][Api] Refactor org.apache.dolphinscheduler.api.controller.QueueController (#11154)
* [Feature][Api] Refactor org.apache.dolphinscheduler.api.controller.QueueController
2022-08-10 11:34:37 +08:00
Wenjun Ruan
8774415197
Split ExecutionStatus to WorkflowExecutionStatus and TaskExecutionStatus (#11340) 2022-08-10 11:00:23 +08:00
fuchanghai
8ded072f79
[Improvement-11293][dolphinscheduler-api] remove some variables that redundant in subclasses of AbstractAuthenticator (#11305)
* [fix-10729]The request's url given by the test case looks different from the controller layer. Some usages and parameter format errors

* [fix-10729]The request's url given by the test case looks different from the controller layer. Some usages and parameter format errors

* [fix-10729]The request's url given by the test case looks different from the controller layer. Some usages and parameter format errors

* [fix-10729]The request's url given by the test case looks different from the controller layer. Some usages and parameter format errors

* [fix-10729]fix the code smell

* [fix-10729]decorate DataSourceControllerTest with public

* [fix-10729]replace import *

* [fix-10729]

* [fix-10729]

* [fix-10729]

* [fix-10729]checkstyle

* [Improvement][ExecutorServiceImpl] remove some unnecessary

* [fix-11181]text check and word spell

* Update dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ExecutorServiceImpl.java

Co-authored-by: Eric Gao <ericgao.apache@gmail.com>

* Update dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ExecutorServiceImpl.java

Co-authored-by: Eric Gao <ericgao.apache@gmail.com>

* Update ExecutorServiceImpl.java

remove the error logging and update the comment

* Update dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ExecutorServiceImpl.java

Co-authored-by: Eric Gao <ericgao.apache@gmail.com>

* [Improvement-10760][dolphinscheduler-api] Some variables are redundant

* [Improvement-10760][dolphinscheduler-api] Some variables are redundant and fix checkstyle

* [Improvement-10760][dolphinscheduler-api] Some variables are redundant and fix checkstyle

* [Improvement-10760][dolphinscheduler-api] add import

* [Improvement-10760][dolphinscheduler-api] remove not use

* [Improvement-11293][dolphinscheduler-api] remove some variables that redundant in subclasses of AbstractAuthenticator

* [Improvement-11293][dolphinscheduler-api] remove some variables that redundant in subclasses of AbstractAuthenticator

Co-authored-by: fuchanghai <changhai.fu@marketingforce.com>
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2022-08-05 14:11:12 +08:00
JiPeng Wang
c9cf8c7774
no-admin user query alert group fix (#11301) 2022-08-04 23:01:15 +08:00
zhuxt2015
c34503e3f6
[fix] Process definition length too long when copy or import (#10621) 2022-08-04 15:04:14 +08:00
陈家名
aef2fbf36c
[Python] Migrate exists method call in PythonGateway to corresponding service (#11188)
Migrate all exists method call in PythonGateway to corresponding service
2022-08-04 12:50:21 +08:00
fuchanghai
cb001b581a
[Improvement-10760][dolphinscheduler-api] Some variables are redundant and fix checkstyle (#11213)
* [fix-10729]The request's url given by the test case looks different from the controller layer. Some usages and parameter format errors

* [fix-10729]The request's url given by the test case looks different from the controller layer. Some usages and parameter format errors

* [fix-10729]The request's url given by the test case looks different from the controller layer. Some usages and parameter format errors

* [fix-10729]The request's url given by the test case looks different from the controller layer. Some usages and parameter format errors

* [fix-10729]fix the code smell

* [fix-10729]decorate DataSourceControllerTest with public

* [fix-10729]replace import *

* [fix-10729]checkstyle

* [Improvement][ExecutorServiceImpl] remove some unnecessary

* [fix-11181]text check and word spell

* Update dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ExecutorServiceImpl.java

Co-authored-by: Eric Gao <ericgao.apache@gmail.com>

* Update dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ExecutorServiceImpl.java

Co-authored-by: Eric Gao <ericgao.apache@gmail.com>

* Update ExecutorServiceImpl.java

remove the error logging and update the comment

* Update dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ExecutorServiceImpl.java

Co-authored-by: Eric Gao <ericgao.apache@gmail.com>

* [Improvement-10760][dolphinscheduler-api] Some variables are redundant

* [Improvement-10760][dolphinscheduler-api] Some variables are redundant and fix checkstyle

* [Improvement-10760][dolphinscheduler-api] Some variables are redundant and fix checkstyle

* [Improvement-10760][dolphinscheduler-api] add import

* [Improvement-10760][dolphinscheduler-api] remove not use

Co-authored-by: fuchanghai <changhai.fu@marketingforce.com>
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2022-08-01 09:24:02 +08:00
JinYong Li
e5cca0e79b
[Fix-11007] [Master] fix forced_success bug (#11088)
* fix forced_success bug

* add comments

* add transactional

* refactor code

Co-authored-by: JinyLeeChina <jiny.li@foxmail.com>
2022-07-30 23:28:31 +08:00
fuchanghai
d73ad2e7d5
[BUG-11181] [Document] text check (#11183) 2022-07-29 23:49:46 +08:00
JiPeng Wang
cd0c50f3f2
[Fix-11140] Added resource parameters for function verification (#11141)
* Added resource parameters for function verification

* checkstyle fixed

* checkstyle fixed

* check style fixed for env test

* rerun

* rerun
2022-07-28 14:03:12 +08:00
fuchanghai
81fa09678c
[Improvement][ExecutorServiceImpl] remove some unnecessary (#11122) 2022-07-28 09:57:19 +08:00
Lyle Shaw
181cd2ff79
[Feature] Add CURD to the project/tenant/user section of the API and add its tests. (#10913) 2022-07-27 14:33:33 +08:00
zhuxt2015
cc2f17ddcc
[Enhancement][API] Copy workflow with scheduler info (#10677)
* support batch copy workflow with scheduler
2022-07-27 13:45:45 +08:00
xuhhui
8004e97b4b
[Feature][Api] Refactor org.apache.dolphinscheduler.api.controller.ProjectController (#10832)
* [Feature][Api] Refactor org.apache.dolphinscheduler.api.controller.ProjectController
2022-07-26 15:24:48 +08:00
zhuxt2015
5ebd53893c
[Feature-10604] Task name in the copied and imported workflow without copy suffix (#10607)
* fix

* remove imported task suffix
2022-07-26 15:21:33 +08:00
zhuxt2015
4d9db3451e
[Bug] [API] If the single log length is long, it will causes the View Log page to hang (#10612) 2022-07-26 10:14:45 +08:00
jackfanwan
ccb7641468
修改数据质量查询的权限判断 (#10977)
Co-authored-by: fanwanlong <fanwanlong@kezaihui.com>
2022-07-25 09:35:53 +08:00
旺阳
28b40da1b1
[Improve] Hide Change Pwd Page When Login With LDAP (#11041)
* hide-change-pwd-page-when-ldap-login

* add unit test
2022-07-20 18:48:16 +08:00
WangJPLeo
fd59f0bb32
[Fix-11031] AccessToken can only be used by the creator. (#11032)
* AccessToken can only be used by the creator.

* supplement ut

* checkstyle fix
2022-07-20 16:32:03 +08:00
fuchanghai
1a08f3970d
[fix-10729]The request's url given by the test case looks different f… (#10734)
* [fix-10729]The request's url given by the test case looks different from the controller layer. Some usages and parameter format errors
2022-07-20 15:42:45 +08:00
Wenjun Ruan
5e9c7dad23
Add dolphinscheduler-bom to manage the dependency version (#11025) 2022-07-20 10:37:31 +08:00
Wenjun Ruan
083ab2b5c9
Remove dao in worker (#10994) 2022-07-15 20:07:18 +08:00
SongTao Zhuang
dad8981cfe
fix: typo executor service (#10980) 2022-07-15 18:58:39 +08:00
Jiajie Zhong
81930e5420
[python] Fix tasks with multiple upstream and workflow query error (#10941)
* when task with more than one upstreams, mapper
   TaskDefinitionMapper method queryByName will return
   more than one record, and failed the mybatis result
   type, so we have to add `limit 1` to it to
* add multiple runs of example in integrate test
* Change from subprocess.Popen to subprocess.call_check
  in integrating test which will raise an error when failed
2022-07-14 22:06:07 +08:00
Mr.An
ed7ec8ac10
[fix-10954]Querying the task group list returns empty data (#10955)
* fix task group query page retrun is null
2022-07-14 11:16:28 +08:00
WangJPLeo
9f34a837b8
[Fix-10909] Returns executionStatus for each process instance status and access token module support. (#10922)
* Returns executionStatus for each process instance status and access token module support
2022-07-13 20:45:52 +08:00
Jiajie Zhong
2d7a48b0f0
[fix] Can not query exists environments (#10884)
always return empty set
2022-07-13 17:21:29 +08:00
zhuxt2015
c9afc1a1f4
display timing record start time (#10915) 2022-07-13 09:53:48 +08:00
Jiajie Zhong
0db9bbd538
[fix] delete workflow or task used by other tasks (#10873)
currently, users can delete process definitions used
in other sub-process tasks or in other dependent tasks.
but this change will break the dependence of those task
and failed DAG, this patch add validation of those
delete behavior to avoid this error
2022-07-12 21:22:06 +08:00
陈家名
59cd86157f
[Python] Supports creating or editing resources. (#10823) 2022-07-12 20:44:59 +08:00
caishunfeng
aa8b88a875
[Feature-10871] add workflow executing data query (#10875)
* add workflow executing data query
* fix sonar check for interrupted
2022-07-11 16:55:55 +08:00
Jiajie Zhong
a6154220dc
[fix] update queue and tenant with same code error (#10877)
in ae6aa53f I use the same logic test to create or update
tenant and queue which failed create with the same value,
this patch will fix it. and also I use the constructor to reduce
the new object logic here
2022-07-11 11:06:30 +08:00
WangJPLeo
3756a233d8
[Fix-10863] Command execution result data statistics remove userId condition. (#10864)
* Command execution result data statistics remove userId condition.

* e2e rerun
2022-07-10 17:17:35 +08:00
houshitao
cc41485d50
[Bug][Api] Complete Data workflow remove duplicates. (#10868)
Co-authored-by: houshitao <shitaohou@163.com>
2022-07-10 01:31:44 +08:00
pinkhello
030fb89d6e
[Fixed-10833] [Bug] [Quartz] timezone display doesn't match the next_fire_time in ds 3.0.0-beta1 version (#10865)
* closed [10619]  [Improvement][Master] batch remove TaskInstaceId and workflowInstanceId

* fixed # 10833 timezone display error

* checkstyle
2022-07-09 22:28:08 +08:00
houshitao
73f4846c49
[Bug][Api] Complete Data workflow remove duplicates. (#10850)
* [Bug][Api] Complete Data workflow remove duplicates.

Co-authored-by: houshitao <shitaohou@163.com>
Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>
2022-07-09 21:29:11 +08:00
Jiajie Zhong
ae6aa53f96
[python] Fix submit and run error (#10792)
because we add permission, so it failed out
createQueue method. this patch fix it and
do some refactor of our tenant and queue
validator code
2022-07-08 10:54:41 +08:00
qianli2022
1e48bbc5c6
[Feature-10629][Improvement] Support multi cluster environments - namespace use cluster (#10650)
* server code

* namespace ui

* fix dao test

* fix test

* fix

* docs and image update

Co-authored-by: qianl4 <qianl4@cicso.com>
2022-07-07 16:05:52 +08:00
Wenjun Ruan
426567348e
Remove quartz in service (#10748)
* Remove quartz in service
2022-07-06 15:43:55 +08:00
Wenjun Ruan
67d14fb7b3
[Fix-10785] Fix state event handle error will not retry (#10786)
* Fix state event handle error will not retry

* Use state event handler to deal with the event
2022-07-06 14:53:28 +08:00
WangJPLeo
b7fbf6c4ad
[API] Do not specify rollback for ensure in case of any exception or error. (#10793) 2022-07-06 14:22:46 +08:00
WangJPLeo
d2fe16d252
[Optimization] Config module resource relation fix. (#10718)
* Config module resource relation fix.

* Boolean judge fix

* e2e rerun

* e2e rerun

* e2e rerun

* ArrayList fix

* change variable name.

* Add transaction processing on create data resource method
2022-07-05 15:51:23 +08:00
zhuxt2015
4bbf3c8ea1
fix (#10678) 2022-07-01 11:30:01 +08:00