test/DolphinScheduler
1
0
Fork 0
mirror of https://gitee.com/dolphinscheduler/DolphinScheduler.git synced 2024-12-02 12:17:43 +08:00
Code Issues Actions 10 Packages Projects Releases Wiki Activity
7,280 Commits 27 Branches 59 Tags 276 MiB
7bf49a7179
Commit Graph

428 Commits

Author SHA1 Message Date
Yann Ann
c87f6d89a4
[fix#12195] Change date to timestamp to support cross diff time zones (#12239) 
* fix DS support cross time zone, use timestamp replace date
 2022-10-01 12:50:42 +08:00
caishunfeng
40090d459e
[Improvement] remove log-server and server module (#12206) 
* remove log-server module

* remove dolphinscheduler-server module
 2022-09-30 16:41:41 +08:00
Wenjun Ruan
aacabc6ec7
Remove workerGroup in registry (#12217) 2022-09-30 13:58:53 +08:00
fuchanghai
e27c79974d
[fix-#11753] send alert error alert data id (#11774) 
* [fix-#11753] send alert error alert data id

Co-authored-by: fuchanghai <changhai.fu@marketingforce.com>
 2022-09-29 15:34:40 +08:00
Eric Gao
d1e409e9a2
[Improvement][Test] Remove powermock in service and server modules (#12164) 
* Remove powermock and refactor some related code in dolphinscheduler-service and dolphinscheduler-server modules
* Remove redundant comments
* Add null check
 2022-09-29 08:16:23 +08:00
xuhhui
6466cc7c41
[Improvement][Master] Construct processInstance may NPE when master handling command (#12056) 
* [Improvement][Master] Construct processInstance may NPE when master handling command

* use an enpty map

Co-authored-by: xuhaihui <xuhaihui@cmss.chinamobile.com>
 2022-09-27 09:13:48 +08:00
Wenjun Ruan
fba5a8eaa0
Fix insert command error due to the id is not null (#12092) 2022-09-22 09:52:32 +08:00
Wenjun Ruan
26dad5e2a8
Fix update TaskDefinition error (#12060) 2022-09-20 22:23:16 +08:00
insist777
5b384f3fab
[Feature-11473][Task]Support test task (#11670) 
* [improve] support test tasks

* [improve] support test tasks

* [improve] support test tasks

* [improve] support test tasks

* [improve] support test tasks

* Update TaskExecuteThread.java

* try solve e2e q

* try solve e2e q

* try solve e2e q

* try solve e2e q

* try solve e2e q

* try solve e2e q

* try solve e2e q

* try solve e2e q

* try solve e2e q

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* Update DataSource.java

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* Update messages_zh_CN.properties

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* Update messages.properties

* Update messages_en_US.properties

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks
 2022-09-17 13:40:56 +08:00
Wenjun Ruan
d833a28b2e
Fix cannot save processDefinition (#11931) 2022-09-16 13:54:18 +08:00
Wenjun Ruan
d3a77c68e6
Make LogServiceClient Singleton (#11777) 2022-09-14 16:18:08 +08:00
caishunfeng
2e61c76c22
[Improvement] Add remote task model (#11767) 2022-09-08 13:41:36 +08:00
kezhenxu94
efedc9a53c
Bump up dependencies to fix cves (#11765) 2022-09-08 09:05:05 +08:00
caishunfeng
f8d46a26c1
[Feature-11530] add state history for process instance (#11757) 
* [Feature] add state history for process instance (#97)

* add state history for process instance

* upsertProcessInstance

* remove unuse method

* fix UT

Co-authored-by: caishunfeng <534328519@qq.com>
 2022-09-03 10:14:05 +08:00
Wenjun Ruan
67e7f88d8b
Refactor heart beat task, use json to serialize/deserialize (#11702) 
* Refactor heart beat task, use json to serialize/deserialize
 2022-08-31 16:20:23 +08:00
Wenjun Ruan
03e1e6fe45
[Bug] [Worker] Optimize the getAppId method to avoid worker OOM when kill task (#11701) 
* Fix kill job may cause worker oom
 2022-08-31 15:25:01 +08:00
Wenjun Ruan
b96d69701a
[Improvement-#11613] Add spi priority factory (#11614) 
* Add spi priority factory

* Add doc

* Add override log

* Use lombok

* Add comment
 2022-08-24 19:51:47 +08:00
Wenjun Ruan
4d13a5104b
Fix kill task failed will cause the taskGroup cannot release and add taskGroup log (#11469) 2022-08-23 10:05:03 +08:00
kezhenxu94
f6ef628030
Support reading application properties from kubernetes config map (#11402) 2022-08-16 20:25:17 +08:00
JiPeng Wang
13d400ceb6
[Fix-11465] Serial first, the last executing instance will be stopped (#11466) 
* Serial first, the last executing instance will be stopped

* ut fix
 2022-08-13 22:30:56 +08:00
Wenjun Ruan
7ff34c3947
[Feature-7024] Add waiting strategy to support master/worker can recover from registry lost (#11368) 
* Add waiting strategy to support master/worker can recover from registry lost

* throw exception when zookeeper registry start failed due to interrupted
 2022-08-13 09:52:03 +08:00
longtb
be66035dee
[Improvement] Avoid using search in for and start using testSaveTaskDefine (#11383) 
* [Improvement] Avoid using search in for and start using testSaveTaskDefine()

* [Improvement] not import *

Co-authored-by: zhangshunmin <zhangshunmin@kezaihui.com>
 2022-08-12 09:37:24 +08:00
Wenjun Ruan
5fa3e7b1ed
Add pause to task instance status (#11390) 2022-08-11 09:25:53 +08:00
caishunfeng
0464123c2b
[Feature-11223] support stream task (#11350) 
* add task execute type

* update task definition list paging

* update task instance list paging

* stream task start

* [Feature][UI] Some changes to execute task.
    * Set the connection edge to dashed line.
    * Add FLINK_STREAM task.

* add stream task

* flink savepoint and cancel

* fix query bug

* add stream task definition

* add task instance for stream task

* delete stream task definition state

* update api for stream task definition edit

* modify search for stream task instance

* add language

* delete task type search for stream task definition

* change task type search for stream task instance

* add jump button

* add savepoint

* add down log for stream task instance

* ui test

* stream task start

* run DAG

* [Fix][UI] Fix the stream task edgs not to be dashed when filling back.

* [Feature][UI] Remove some fields for FLINK_STREAM.

* add start modal

* add dryRun column for stream task instance

* fix duration

* fix pon

* fix build error

* Add success tip

* add auto sync for stream task instance

* remove forgien key for task instance

* license header

* UT fix

* modify locales

* recover common config

* fix UT

* add doc

Co-authored-by: Amy <amywang0104@163.com>
Co-authored-by: devosend <devosend@gmail.com>
 2022-08-10 21:44:43 +08:00
Wenjun Ruan
8774415197
Split ExecutionStatus to WorkflowExecutionStatus and TaskExecutionStatus (#11340) 2022-08-10 11:00:23 +08:00
Wenjun Ruan
acd3d3fab1
Fix TaskGroup cannot work and will cause master dead loop (#11254) 
* Fix TaskGroup cannot work and will cause master dead loop

* Remove acquireTaskGroupAgain in ProcessServiceImpl
 2022-08-04 14:43:45 +08:00
Wenjun Ruan
8e21c38c00
Write alert result into db (#11221) 2022-08-01 13:04:27 +08:00
JinYong Li
e5cca0e79b
[Fix-11007] [Master] fix forced_success bug (#11088) 
* fix forced_success bug

* add comments

* add transactional

* refactor code

Co-authored-by: JinyLeeChina <jiny.li@foxmail.com>
 2022-07-30 23:28:31 +08:00
Wenjun Ruan
5e9c7dad23
Add dolphinscheduler-bom to manage the dependency version (#11025) 2022-07-20 10:37:31 +08:00
caishunfeng
65d2b8f14e
[Bug] fix task params pass (#11053) 2022-07-20 10:14:51 +08:00
Mr.An
d7df899909
[Fix-11003]Task group queue is not updated to final state (#11004) 
* fix after the task group is forced to start the task. Task group status has not changed
 2022-07-19 17:48:16 +08:00
Wenjun Ruan
083ab2b5c9
Remove dao in worker (#10994) 2022-07-15 20:07:18 +08:00
Wenjun Ruan
2be1d4bf0a
Fix worker cannot shutdown due to resource close failed or heart beat check failed (#10979) 
* Use try-with-resource to close resource, and add heart error threshold to avoid worker cannot close due to heart beat check failed

* Move heartbeat error threshold to applicaiton.yml
 2022-07-15 20:06:53 +08:00
WangJPLeo
2397423eb6
Solve transaction problems under multi-threading (#10917) 2022-07-13 11:23:47 +08:00
Wenjun Ruan
2a67866718
Catch exception when check state in StateWheelExecuteThread (#10908) 
* Catch exception when check state
 2022-07-13 10:51:20 +08:00
Wenjun Ruan
f639a2eed4
[Fix-10854] Fix database restart may lost task instance status (#10866) 
* Fix database update error doesn't rollback the task instance status

* Fix database error may cause workflow dead with running status
 2022-07-11 09:57:00 +08:00
Wenjun Ruan
3f69ec8f28
[Fix-10842] Fix master/worker failover will cause status incorrect (#10839) 
* Fix master failover will not update task instance status
* Add some failover log
* Fix worker failover will rerun task more than once
* Fix workflowInstance failover may rerun already success taskInstance
 2022-07-09 11:54:59 +08:00
Wenjun Ruan
426567348e
Remove quartz in service (#10748) 
* Remove quartz in service
 2022-07-06 15:43:55 +08:00
Wenjun Ruan
67d14fb7b3
[Fix-10785] Fix state event handle error will not retry (#10786) 
* Fix state event handle error will not retry

* Use state event handler to deal with the event
 2022-07-06 14:53:28 +08:00
WangJPLeo
b7fbf6c4ad
[API] Do not specify rollback for ensure in case of any exception or error. (#10793) 2022-07-06 14:22:46 +08:00
zhuxt2015
182b9c9b26
[Feature][Master] Failover workflow instance base on priority (#10754) 
* failover workflow instance base on priority
 2022-07-04 23:35:24 +08:00
Wenjun Ruan
35a10d092f
[Fix-10666] Workflow submit failed will still in memory and never retry (#10667) 
* Workflow submit failed will still in memory and never retry
 2022-07-04 22:08:15 +08:00
houshitao
f8650b7b7c
[Bug] [Api-Service] Workflow and subworkflow instance does not change after clicking rerun。 (#10727) 
* [Bug] [Api-Service] Workflow and subworkflow instance does not change after clicking rerun.

* [Bug] [Api-Service] Workflow and subworkflow instance does not change after clicking rerun.

* [Bug] [Api-Service] Workflow and subworkflow instance does not change after clicking rerun.

Co-authored-by: houshitao <shitaohou@163.com>
 2022-07-04 22:05:23 +08:00
WangJPLeo
8f621ff98b
[Optimization] Calculate global parameter and local parameter at master. (#10704) 
* Global parameter and local parameter calculation external expansion.

* k8s task ut fix.

* TimePlaceholderUtils import DateUtils fix

* follow the review comments to fix.

* follow the review comments to fix.

* e2e rerun
 2022-06-30 22:45:25 +08:00
Wenjun Ruan
247ca4ae8a
[Improvement-10617] Add comment in slot check (#10618) 2022-06-30 15:58:49 +08:00
wjf
75581b7543
[Improvement]Delete the getSession in the Registry interface (#10633) 2022-06-29 11:20:52 +08:00
Wenjun Ruan
35b25da863
Validate master/worker config (#10649) 2022-06-28 20:17:43 +08:00
Wenjun Ruan
66624c5c86
[Bug] [Master] Worker failover will cause task cannot be failover (#10631) 
* fix worker failover may lose event
 2022-06-28 16:08:35 +08:00
WangJPLeo
b5184138fa
[Feature] Time function analysis extension. (#10624) 
* Time function analysis extension.

* param add.

* clear useless logs and update method notes

* permission omission fix.

* extending time functions to optimize static methods

* e2e rerun.
 2022-06-28 11:24:51 +08:00
Wenjun Ruan
db595b3eff
Optimize master log, use MDC to inject workflow instance id and task instance id in log (#10516) 
* Optimize master log, add workflow instance id and task instance id in log

* Use MDC to set the workflow info in log4j

* Add workflowInstanceId and taskInstanceId in MDC
 2022-06-23 11:45:06 +08:00