test/DolphinScheduler
1
0
Fork 0
mirror of https://gitee.com/dolphinscheduler/DolphinScheduler.git synced 2024-12-02 04:08:31 +08:00
Code Issues Actions 10 Packages Projects Releases Wiki Activity
6,771 Commits 27 Branches 59 Tags 276 MiB
fe8525744d
Commit Graph

6771 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Stalary
fe8525744d [doc] Add run the task alone doc. (#10581) 
(cherry picked from commit b2df357814)
 2022-07-27 22:23:23 +08:00
xiangzihao
b88b522827 [doc] Add doc about how to use healthcheck endpoints (#10592) 
(cherry picked from commit 33cc68ed4a)
 2022-07-27 22:20:13 +08:00
xiangzihao
b943c003a6 [doc] Add docs about complement dependent mode (#10587) 
(cherry picked from commit b21e211d2c)
 2022-07-27 22:19:27 +08:00
陈家名
dd4b99298a [python] Support using full-name definitions for resources (#10551) 
(cherry picked from commit 7970573365)
 2022-07-27 22:13:04 +08:00
Devosend
4f37b7c377
fix space validate for create folder bug (#11106) (#11172) 
(cherry picked from commit 2f2de7f90e)
 2022-07-27 17:16:58 +08:00
caishunfeng
9742502ea3 fix k8s master and worker start error (#11156) 2022-07-27 09:39:15 +08:00
Devosend
ebdc34b25f [Fix][UI] Add duplicate of file suffix (#11136) 2022-07-27 09:38:34 +08:00
zhuxt2015
1b0f8c04d3 [Bug] [API] If the single log length is long, it will causes the View Log page to hang (#10612) 2022-07-27 09:38:25 +08:00
Devosend
4622a8036a [Fix][UI] fix bug where The Gantt chart does not show the execution status (#11135) 
* fix bug where The Gantt chart does not show the execution status of all components

* modify round method
 2022-07-27 09:38:02 +08:00
Wenjun Ruan
b791e9158e [Fix-10918] Close datasource when expire from guava cache (#11120) 
* Close datasource when expire from guava cache
* Remove duplicate datasource in HiveDataSourceClient
 2022-07-27 09:37:18 +08:00
songjianet
8581a464db [Fix][UI] Fix the problem of too large spacing in operation. (#11090) 
* [Fix][UI] Fix the problem of too large spacing in operation.

* [Fix][UI] Fix the problem of too large spacing in operation.
 2022-07-27 09:35:04 +08:00
Devosend
9164ec8bf1 fix view variable display bug (#11100) 2022-07-27 09:34:43 +08:00
Amy0104
34d2c513c5 [Fix][UI] Widen the segment separator input box. (#11096) 2022-07-27 09:34:30 +08:00
xiangzihao
dba1d076c1 add oracle dependency (#11095) 2022-07-27 09:34:05 +08:00
Assert
7e1d8c2d86 [Feature-10594][ui] Data Quality Result Display Optimize (#10835) 2022-07-20 14:56:36 +08:00
pinkhello
59d8f222c8 [Fix-10539] [Bug] [UI] Copy task a to B, change the user-defined para… (#10611) 
* fixed 10539: [Bug] [ui]Copy task a to B, change the user-defined parameters of Task B, and change the parameters of task a

* fixed 10539: [Bug] [ui]Copy task a to B, change the user-defined parameters of Task B, and change the parameters of task a

* feat: reactor code

* feat: remove console.info

* fixed: reactor deepclone

* feat: code style
 2022-07-20 11:40:43 +08:00
caishunfeng
b9e3187ad5 [Feature-10871] add workflow executing data query (#10875) 
* add workflow executing data query
* fix sonar check for interrupted
 2022-07-19 16:10:42 +08:00
Amy0104
bb6aaf755e [Fix][UI] Set the next node to not required. (#10932) 
* [Fix][UI] Set the next node to not required.

* [Fix][UI] Remove the unuse i
 2022-07-19 15:37:14 +08:00
Amy0104
47782ba555 [Fix][UI] Add validator to the next node field of the switch task. (#10898) 2022-07-19 15:37:02 +08:00
Wenjun Ruan
5a2ea0b76b Fix compile error 2022-07-19 12:30:26 +08:00
Wenjun Ruan
648d469b62 Return status 2022-07-19 12:27:49 +08:00
Wenjun Ruan
f3250bf5fa Fix worker cannot shutdown due to resource close failed or heart beat check failed (#10979) 
* Use try-with-resource to close resource, and add heart error threshold to avoid worker cannot close due to heart beat check failed

* Move heartbeat error threshold to applicaiton.yml

(cherry picked from commit 2be1d4bf0a)
 2022-07-19 12:23:20 +08:00
Wenjun Ruan
17f0be5bfb Kill task when we do master failover to avoid task lost (#10997) 
(cherry picked from commit 998e4d74dd)
 2022-07-19 12:22:26 +08:00
Wenjun Ruan
44ddb6908e Fix kill yarn job error when failover caused by doesn't set ProcessDefinition (#10948) 
(cherry picked from commit b245e7c973)
 2022-07-19 12:22:13 +08:00
Wenjun Ruan
812d7a8f26 Fix taskInstance's host is not worker nettyServer address (#10926) 
* Fix taskInstance's host is not worker nettyServer address

* Remove unnecessary mock

(cherry picked from commit df0416c193)
 2022-07-19 12:21:59 +08:00
Wenjun Ruan
527ee472fb Catch exception when check state in StateWheelExecuteThread (#10908) 
* Catch exception when check state

(cherry picked from commit 2a67866718)
 2022-07-19 12:21:20 +08:00
Wenjun Ruan
9ee20cffef [Fix-10827] Fix network error cause worker cannot send message to master (#10886) 
* Fix network error cause worker cannot send message to master

(cherry picked from commit cade66a9b6)
 2022-07-19 12:20:18 +08:00
Wenjun Ruan
b259deb196 [Fix-10854] Fix database restart may lost task instance status (#10866) 
* Fix database update error doesn't rollback the task instance status

* Fix database error may cause workflow dead with running status

(cherry picked from commit f639a2eed4)
 2022-07-19 12:12:57 +08:00
Wenjun Ruan
71edaf41a2 [Fix-10842] Fix master/worker failover will cause status incorrect (#10839) 
* Fix master failover will not update task instance status
* Add some failover log
* Fix worker failover will rerun task more than once
* Fix workflowInstance failover may rerun already success taskInstance

(cherry picked from commit 3f69ec8f28)
 2022-07-19 12:08:32 +08:00
Wenjun Ruan
4fc9bce444 [Fix-10785] Fix state event handle error will not retry (#10786) 
* Fix state event handle error will not retry

* Use state event handler to deal with the event

(cherry picked from commit 67d14fb7b3)
 2022-07-19 12:03:15 +08:00
Wenjun Ruan
da03ef32e9 Fix compile error 2022-07-19 12:00:37 +08:00
Wenjun Ruan
04c47034d4 Add task prepare metrics 2022-07-19 11:52:18 +08:00
Wenjun Ruan
7500e99682 [Fix-10785] Fix state event handle error will not retry (#10786) 
* Fix state event handle error will not retry

* Use state event handler to deal with the event

(cherry picked from commit 67d14fb7b3)
 2022-07-19 11:52:06 +08:00
Wenjun Ruan
3b923e5933 [Fix-10666] Workflow submit failed will still in memory and never retry (#10667) 
* Workflow submit failed will still in memory and never retry

(cherry picked from commit 35a10d092f)
 2022-07-19 11:47:04 +08:00
Wenjun Ruan
6c83967ebe [Improvement-10617] Add comment in slot check (#10618) 
(cherry picked from commit 247ca4ae8a)
 2022-07-19 11:42:31 +08:00
Wenjun Ruan
4b224ae2e5 Validate master/worker config (#10649) 
(cherry picked from commit 35b25da863)
 2022-07-19 11:42:17 +08:00
Wenjun Ruan
db31deb54f [Bug] [Master] Worker failover will cause task cannot be failover (#10631) 
* fix worker failover may lose event

(cherry picked from commit 66624c5c86)
 2022-07-19 11:41:25 +08:00
Wenjun Ruan
c488a9f828 Fix compile error 2022-07-19 11:36:34 +08:00
Wenjun Ruan
fc1c1f6ad1 add CMDPARAM_COMPLEMENT_DATA_SCHEDULE_DATE_LIST 2022-07-19 11:30:47 +08:00
Wenjun Ruan
3ab9ee13fc Optimize master log, use MDC to inject workflow instance id and task instance id in log (#10516) 
* Optimize master log, add workflow instance id and task instance id in log

* Use MDC to set the workflow info in log4j

* Add workflowInstanceId and taskInstanceId in MDC

(cherry picked from commit db595b3eff)
 2022-07-19 11:29:41 +08:00
Wenjun Ruan
9a28d32057 Remove the schedule thread in LowerWeightHostManager (#10310) 
(cherry picked from commit b100f6c489)
 2022-07-19 11:23:59 +08:00
Wenjun Ruan
90c87f0121 [Fix-10413] Fix Master startup failure the server still hang (#10500) 
* Fix Master startup failure the server still hang

(cherry picked from commit 117f78ec4b)
 2022-07-19 11:22:53 +08:00
Wenjun Ruan
9a59054655 Fix PeerTaskInstancePriorityQueue cannot contains method use taskInstanceId to check (#10491) 
(cherry picked from commit 0bdfa0cff9)
 2022-07-19 11:22:33 +08:00
Wenjun Ruan
9a4c7f876a Fix TaskProcessorFactory#getTaskProcessor get common processor is not thread safe (#10479) 
* Fix TaskProcessorFactory#getTaskProcessor get common processor is not thread safe

(cherry picked from commit ad2646ff1f)
 2022-07-19 11:22:16 +08:00
Wenjun Ruan
1def1729bc Add lombok 2022-07-19 11:20:37 +08:00
Wenjun Ruan
52815975bc Add some warning log in master (#10383) 
* Add some warn log in master

* fix may skip sleep

(cherry picked from commit b0d9d3f9ab)
 2022-07-19 11:13:39 +08:00
Wenjun Ruan
318a8e3ae0 [Feature][metrics] Add master, worker metrics (#10326) 
* Add mater metrics

* fix UT

* Add url to mysql profile

* Add worker metrics

* Update grafana config

* Add system metrics doc

* Add process failover counter

* Add metrics image

* Change jpg to png

* Add command insert metrics

* Fix UT

* Revert UT

(cherry picked from commit e21d7b1551)
 2022-07-19 11:12:28 +08:00
Wenjun Ruan
81cadd15d2 Optimize MasterServer, add MasterRPCService (#10371) 
* Optimize MasterServer, avoid NPE

(cherry picked from commit 3ecbee3885)
 2022-07-19 11:10:08 +08:00
Wenjun Ruan
4ceb420873 Fix TaskProcessorFactory#getTaskProcessor get common processor is not thread safe (#10479) 
* Fix TaskProcessorFactory#getTaskProcessor get common processor is not thread safe

(cherry picked from commit ad2646ff1f)
 2022-07-19 11:02:57 +08:00
juzimao
ae4c3ab4e1 [Fix] Fix the JSONUtils tool class time zone problem #10282 (#10284) 
* [Fix] Fix the JSONUtils tool class time zone problem #10282

* [Fix] Fix the JSONUtils tool class time zone problem #10282

* Update JSONUtils.java

remove unnessnary log

* Update JSONUtilsTest.java

add timezone import

Co-authored-by: xiangzihao <460888207@qq.com>
 2022-07-19 09:47:58 +08:00