Commit Graph

339 Commits

Author SHA1 Message Date
Wenjun Ruan
dc4dad135c
Fix TaskGroupCoordinator might cause OOM when there is a lot of waiting TaskGroupQueue (#15773) 2024-03-30 09:43:42 +08:00
Wenjun Ruan
dcc9d64ef6
Fix Switch logic task doesn't check the branch exist (#15755) 2024-03-24 20:52:00 +08:00
Wenjun Ruan
2e169076dc
Fix delay execution task will never be dispatched (#15751)
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2024-03-22 16:23:55 +08:00
caishunfeng
99a2461785
[Improvement] change js engine (#15747)
* change js engine

* add dependent jar
2024-03-22 11:31:47 +08:00
caishunfeng
4a255fd523
Fix switch js, add black key (#15680)
Co-authored-by: fuchanghai <changhaifu@apache.org>
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2024-03-18 20:24:54 +08:00
calvin
eab71f1071
[Improvement-15707][Master] Work out the issue that the workflow with a task dependency couldn't work well. (#15712) 2024-03-14 17:57:13 +08:00
ZhongJinHacker
738da1cda3
[Fix][Master] Fix Potential danger in the event of a worker failover (#15689)
* clean unused import

* fix style check

* fix when path is null or empty, it will cause serverhost is null,

* fix UT test (#15684)

* [Fix-15639] parameterPassing is null case NPE (#15678)

Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>

* fix  when path is null or empty, it will cause serverhost is null,
2024-03-11 21:50:21 +08:00
Wenjun Ruan
04a6b0d281
Fix task mighe be dispatched even if it has been killed (#15662) 2024-03-04 17:30:43 +08:00
Wenjun Ruan
2b4c1e06b6
Fix recover pause workflowInstance may generate failover taskInstance (#15611) 2024-02-22 15:57:08 +08:00
Wenjun Ruan
8a35e8b2f8
Fix switch task will throw exception when there is not parameters (#15594) 2024-02-19 11:12:22 +08:00
Wenjun Ruan
3526b42df6
Add Server HealthIndicator (#15274) 2024-02-07 14:07:30 +08:00
Jay Chung
01eb8f834f
fix: start param for wf not work (#15544)
* fix: start param for wf not work

fix: #15280

* fix test
2024-02-05 16:17:30 +08:00
caishunfeng
ef9ed3db55
fix switch js (#15487)
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2024-02-05 10:23:47 +08:00
Jay Chung
8efaa9fa1f
fix: data quality can not use (#15551)
* fix: data quality can not use

fix: #15468, #15249, #14858

---------

Co-authored-by: Rick Cheng <rickchengx@gmail.com>
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2024-02-05 09:54:06 +08:00
Wenjun Ruan
89742332a8
Fix TaskGroupQueue will never be wakeup due to wakeup failed at one time (#15528) 2024-02-01 17:13:58 +08:00
Wenjun Ruan
25810a81dc
Exit JVM when OOM (#15538) 2024-01-31 13:59:30 +08:00
sleo
431414729f
【Master】【Bug】Fix a bug, When the worker service offline, workerNodeInfo cache in master cannot delete the offline worker (#15459)
* remove work from workerNodeInfo when work server down

* modify log msg

Co-authored-by: xiangzihao <zihaoxiang@apache.org>

* add unit test

* add unit test

* modify the code style

* add license header

---------

Co-authored-by: xiangzihao <zihaoxiang@apache.org>
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-01-25 18:10:36 +08:00
Wenjun Ruan
64e1e67581
Use DefaultUncaughtExceptionHandler to log the uncached exception (#15496)
Co-authored-by: fuchanghai <changhaifu@apache.org>
2024-01-18 14:59:17 +08:00
Wenjun Ruan
44c356db4d
TaskGroupPriority only compare When TaskGroup is same (#15486) 2024-01-18 10:59:54 +08:00
Wenjun Ruan
603dfa2e4b
Fix notify failover WorkflowInstance will cause NPE (#15499) 2024-01-17 10:24:41 +08:00
Wenjun Ruan
7b9c9e0eb2
Enable set ServerLoadProtection fot Master/Worker (#15439) 2024-01-14 22:17:58 +08:00
Gallardot
a405abec9e
[Bug][Master]serial_wait strategy workflow unable to wake up (#15270)
* fix: serial_wait strategy workflow unable to wake up

Signed-off-by: Gallardot <gallardot@apache.org>

* fix: serial_wait strategy workflow unable to wake up

Signed-off-by: Gallardot <gallardot@apache.org>

---------

Signed-off-by: Gallardot <gallardot@apache.org>
Co-authored-by: fuchanghai <changhaifu@apache.org>
2024-01-12 17:53:24 +08:00
Gallardot
575b89e2f9
[Bug][Registry] Optimizing waiting strategy (#15223)
* [Improvement][Registry] Optimizing waiting strategy

Signed-off-by: Gallardot <gallardot@apache.org>
2024-01-02 22:41:50 +09:00
Wenjun Ruan
d6dea4633c
Optimize server startup log (#15362) 2023-12-27 20:20:21 +08:00
Wenjun Ruan
5523a62825
Remove taskQueue and looper in worker (#15292) 2023-12-27 19:51:45 +08:00
Wenjun Ruan
5b6b0ceb31
Fix resource file usage(Delete Resource/ResourceUser which is deprecated) (#15355) 2023-12-23 22:47:59 +08:00
Gallardot
f8a44ff719
[Bug][Master] send ACK event timeout (#15346) 2023-12-21 18:54:44 +08:00
ll
0c470ffe66
[Fix #15129] [Dependent] The date rules of the dependent node are ambiguous. (#15289)
* [Fix-15129][Dependent] Fix the ambiguity in date rules for dependent node.

* [fix #15129] Revert ddl

* restore findLastProcessInterval

* update: mvn spotless:apply

---------

Co-authored-by: 李乐 <lile@boimc.com>
Co-authored-by: xiangzihao <460888207@qq.com>
2023-12-08 16:18:33 +08:00
Wenjun Ruan
14272dafab
Fix failover Master might not release taskGroup (#15287) 2023-12-06 13:31:46 +08:00
Wenjun Ruan
2119e41800
[Improvement] Move delay calculation to Master (#15278) 2023-12-06 09:37:14 +08:00
Wenjun Ruan
43f5f24529
Add dolphinscheduler-extract-common module (#15266) 2023-12-05 21:06:39 +08:00
caishunfeng
b5eddc0ce8
fix switch condition (#15228)
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2023-11-27 14:03:37 +08:00
Wenjun Ruan
1d23e4fe5d
Remove spring cache for dao (#15184) 2023-11-24 14:54:05 +08:00
Dyqer
cca1c4a172
fix can't stop bug (#15191)
Co-authored-by: Leoric Yue <leoric.yue@zoom.us>
2023-11-20 21:02:44 +08:00
Wenjun Ruan
d6175f3bbf
Exclude DataSourceAutoConfiguration in worker server (#15169) 2023-11-16 19:17:39 +08:00
旺阳
5afd94136c
[Fix][Data Quality] Change t_ds_dq_rule_input_entry field name fix PostgreSQL not support value issue (#14992)
* change data quality field name

* fix-dao

* fix-dao

* fix-dao

* fix-dao

* update

* test

* update

* update
2023-11-13 11:04:30 +08:00
fuchanghai
2622685784
[Bug] Fix endless loop (#15092)
* [fix-#13045] fix endless loop

---------

Co-authored-by: fuchanghai <‘2875334588@qq.com’>
2023-11-08 11:03:55 +08:00
Eric Gao
d675d32771
Fix confusing constant string for unit convertor (#15126) 2023-11-06 21:32:36 +08:00
Wei Xiaonan
c0ed68121d
[Feature-14832][Listener]Implementation of Listener Mechanism (#14981)
* first commit

* 1. sql: sync ddl
2. front-end: change to ternary expression
3. back-end: correct license header in ListenerEvent.java

* test case

* frontend: remove unnecessary console

* fix unit test

* remove log depends on user-provided value

* fix dolphinscheduler_postgresql.sql

* sync database schema

* fix unit test

* fix unit test

* fix some NIT.

* extract GLOBAL_ALERT_GROUP_ID into variable

* fix ddl bug

* add column task_type in t_ds_fav_task in upgrade/3.2.0_schema

* add unit test
2023-10-29 20:28:05 +08:00
Wenjun Ruan
a5284e4024
Remove mapper usage in tools (#15073) 2023-10-25 22:58:47 +08:00
Orange Summer
688f84452d
[Feature-14678][Master][UI]Dependent task parameter passing (#14702)
* feat: dependent task parameter passing

* style: code format

* refactor: remove inappropriate log

* refactor: Modify unreasonable code structure

* refactor: Modify unreasonable code structure

* refactor: Solve conflicts caused by merge

---------

Co-authored-by: Rick Cheng <rickchengx@gmail.com>
Co-authored-by: BaoLiang <29528966+lenboo@users.noreply.github.com>
2023-10-25 16:24:30 +08:00
caishunfeng
c59b2d5b8c
remove sub workflow finish notify (#15057)
Co-authored-by: xiangzihao <460888207@qq.com>
2023-10-23 10:19:57 +08:00
Wenjun Ruan
33084d281f
Add dolphinscheduler-dao-plugin module (#15019) 2023-10-14 17:58:40 +08:00
dbac
d683553324
[Improvement][Master] Calculate the remainTime then we set the delay execution. (#15012)
* [#15007][fix][worker-server] fix bug task delay execution

* [#15007][fix][worker-server] fix bug task delay execution
    mvn spotless:apply

* [#15007][fix][worker-server] fix bug task delay execution
    worker fix  task delay logic

---------

Co-authored-by: 旺阳 <wang@lqwang.net>
2023-10-11 02:32:07 -05:00
Wenjun Ruan
629fced9ef
Fix task log appender will not be closed immediate after throwing exception (#14989) 2023-10-07 20:19:02 -05:00
Aaron Wang
73e846d03e
[Improvement-14884][Master] Add overload state in master heartbeat to trigger slot change (#14887) 2023-09-14 10:36:42 +08:00
sgw
76bbcbeb30
[Improvement][Master] Split the task dependency mode of the dependent node into workflow dependent and task dependent (#14824)
* [Improvement][Master] Split the task dependency mode of the dependent node into workflow dependent and task dependent. (#11970)

* [Improvement][Master] Split the task dependency mode of the dependent node into workflow dependent and task dependent. (#11970)

- delete useless output

* [Improvement][Master] Split the task dependency mode of the dependent node into workflow dependent and task dependent. (#11970)

- add log
- run spotless check

---------

Co-authored-by: 旺阳 <qingwli@cisco.com>
Co-authored-by: JinYong Li <42576980+JinyLeeChina@users.noreply.github.com>
2023-09-09 17:23:34 +08:00
eye-gu
7ae4fb3787
[Fix-14149][master] task finished so not dispatch (#14161) 2023-08-31 20:08:09 +08:00
eye-gu
33525a17c3
[Fix-14828][master] check task id not null to avoid NPE (#14829)
Co-authored-by: eye <eye.gu@aloudata.com>
Co-authored-by: Aaron Wang <wangweirao16@gmail.com>
Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
2023-08-31 18:34:31 +08:00
Wenjun Ruan
742f0c3e60
Add workflowInstanceHost and taskInstanceHost in ITaskInstanceExecutionEvent (#14827) 2023-08-29 13:32:17 +08:00