Commit Graph

292 Commits

Author SHA1 Message Date
eye-gu
7ae4fb3787
[Fix-14149][master] task finished so not dispatch (#14161) 2023-08-31 20:08:09 +08:00
eye-gu
33525a17c3
[Fix-14828][master] check task id not null to avoid NPE (#14829)
Co-authored-by: eye <eye.gu@aloudata.com>
Co-authored-by: Aaron Wang <wangweirao16@gmail.com>
Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
2023-08-31 18:34:31 +08:00
Wenjun Ruan
742f0c3e60
Add workflowInstanceHost and taskInstanceHost in ITaskInstanceExecutionEvent (#14827) 2023-08-29 13:32:17 +08:00
Wenjun Ruan
d223d654cc
Add rpc benchmark test (#14797) 2023-08-27 20:59:55 +08:00
sgw
9c2838ccc9
fix the status of logical task is not be correctly changed when task is finished. (#14820) (#14822) 2023-08-27 13:20:37 +08:00
sgw
ac04288e2b
fix the full log path of logical task is not be set when task is dispatched (#14819) (#14821) 2023-08-27 12:24:04 +08:00
Wenjun Ruan
55cc77226f
[RPC] Refactor the remote module. (#14776)
* Add dolphinscheduler-extract-base
* Add dolphinscheduler-extract-master
* Add dolphinscheduler-extract-worker
* Add dolphinscheduler-extract-alert
2023-08-23 09:43:17 +08:00
caishunfeng
59998f4c75
[Improvement] add dispatch fail times when compare (#14761)
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2023-08-18 15:26:01 +08:00
JieguangZhou
8e64026e93
fix fault tolerant enviroment missing (#14735) 2023-08-11 14:35:31 +08:00
eye-gu
7c37a7f9fe
[Fix-14714][master] package task instance before put into cache (#14718)
Co-authored-by: eye <eye.gu@aloudata.com>
Co-authored-by: xiangzihao <460888207@qq.com>
2023-08-09 13:41:28 +08:00
Wenjun Ruan
db62ce0e47
Fix serial mode will cause NPE in Workflow bootstrap (#14703) 2023-08-04 23:43:57 +08:00
Tengting Xu
9963bfc612
[Workflow] Print WorkflowInstance detail/ TaskInstance detail when finished (#14677)
* [Workflow] Print WorkflowInstance detail/ TaskInstance detail when finished

* [Workflow] minor fix

* [Workflow] add ut

* [Workflow] add header

---------

Co-authored-by: tengting.xu <xtt@dipeak.com>
2023-08-01 20:50:00 +08:00
KingsleyY
4a80e3a021
[DS-14566][master] fix serial_wait command_param does not replace global_params (#14606)
This closes #14566

Co-authored-by: yangxin3 <yangxin3@newhope.cn>
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
Co-authored-by: xiangzihao <460888207@qq.com>
2023-07-25 13:36:35 +08:00
Aaron Wang
ca760aabd4
[Fix-14512][BOM] kubernetes-client version conflict (#14561) 2023-07-21 18:17:53 +08:00
Wenjun Ruan
cbd354d134
Add host/port in heartbeat (#14591) 2023-07-20 21:26:35 +08:00
Orange Summer
af2986eca1
[Feature-14545][Master][UI]Cross workflow parameter passing (#14552)
* feat: cross-workflow parameter passing

* refactor: remove useless parameters

* refactor: remove useless code

* refactor: code format

* docs: update docs

* docs: docs format

* more effective information

Co-authored-by: xiangzihao <zihaoxiang@apache.org>

* more effective information

Co-authored-by: xiangzihao <zihaoxiang@apache.org>

* fix: wrong writing of log code

* fix: perfect error log

---------

Co-authored-by: xiangzihao <zihaoxiang@apache.org>
Co-authored-by: xiangzihao <460888207@qq.com>
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>
2023-07-19 18:00:44 +08:00
Wenjun Ruan
399d0d0ce9
Add log for NetUtils (#14578) 2023-07-18 21:17:17 +08:00
Wenjun Ruan
e6d94632b9
Add WorkflowExecuteContext (#14544) 2023-07-14 18:40:01 +08:00
旺阳
c812bf9d49
[Bug] [Mater] Not got latest task intance by task code (#14529)
* add task code->instance map

---------

Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2023-07-14 14:51:39 +08:00
Wenjun Ruan
0246327083
Polish config, expose rpc config in application.yml (#14501) 2023-07-11 09:29:31 +08:00
Wenjun Ruan
406aeaf659
Use jvm_args_env.sh to set JVM args (#14487) 2023-07-10 17:39:57 +08:00
Eric Gao
0d2ea3e27b
Fix worker failover service comment (#14494) 2023-07-10 15:54:06 +08:00
Aaron Wang
0b69236b04
[Feature-14251][Task] Support yarn queue definition in yarn task (#14310) 2023-07-07 14:24:47 +08:00
旺阳
d38d504332
[Fix] Fix running task instance throught api gots failed (#14433)
* update logic

* split method
2023-07-06 11:38:26 +08:00
Aaron Wang
64e5741063
[Bug-14378][Master] Trigger task execution in workflow instance page cause workflow instance keep running status (#14389) 2023-06-24 22:10:14 +08:00
kami
5cbe170e43
[Bug] [Master] Fix complementListDate will throw IndexOutOfBoundsException (#14361) 2023-06-19 23:23:05 +08:00
Rick Cheng
b49b91c45d
[Improvement-14331][common] Remove the duplicate ParameterUtils (#14332) 2023-06-14 15:00:54 +08:00
旺阳
1c0dfbb044
[Improvement][Etcd] Support SSL In Etcd And Enhance Etcd In Helm (#13924) 2023-06-14 13:29:50 +08:00
旺阳
465e7ae6ee
[Bug] [Master] Workflow keep running when task has no id (#14315)
* change completeTaskMap to completeTaskSet

* Update dolphinscheduler-master/src/main/java/org/apache/dolphinscheduler/server/master/runner/WorkflowExecuteRunnable.java

Co-authored-by: Wenjun Ruan <wenjun@apache.org>

* change method name

* fix ut

* fix spotless

---------

Co-authored-by: Wenjun Ruan <wenjun@apache.org>
2023-06-14 12:53:16 +08:00
Wenjun Ruan
a55612e4f7
Fix switch task use boolean type error (#14326) 2023-06-13 13:26:05 +08:00
Kerwin
75f4fa2881
Fix the complement function does not generate the next running instances in parallel mode (#14335) 2023-06-12 19:40:26 +08:00
Wenjun Ruan
aa2d9a7f05
Add baseDao (#14316) 2023-06-09 19:02:50 +08:00
JieguangZhou
e7cdc7c136
[Feature][Task Plugin] Add Dynamic task to generate dynamic worklfows with list parameters (#14127)
* add dynamic task plugin

* fix ui

* update code

* add DOC

* ADD UT

* fix doc

* fix sanity check

* add upgrade ddl
* fix sql
2023-06-07 16:01:06 +08:00
HQFGit
d587b5f03d
[Bug][Master][Fixes 14216]workflow state keep READY_STOP when sub_process state … (#14221)
* [Bug] [Master] workflow state keep READY_STOP when sub_process state STOP
Fixes #14216

* [Bug] [Master] workflow state keep READY_STOP when sub_process state STOP
Fixes  mvn spotless:apply#14216

* Update testFile.txt.crc

* Update testFile.txt.crc

* Delete testFile.txt.crc
2023-06-06 11:04:32 +08:00
Wenjun Ruan
5e9f1de075
Remove state check for dependent/subProcess in StateWheelExecuteThread (#14242) 2023-05-31 16:39:12 +08:00
Wenjun Ruan
d810c487fd
Fix taskGroupQueue doesn't removed from inQueue where wakeup failed (#14200)
* Fix taskGroupQueue doesn't removed from inQueue where wakeup failed

* Fix avaliable taskGroup is 0 the TaskGroupQueue doesn't out queue
2023-05-25 14:42:32 +08:00
Wenjun Ruan
deca0a7ea1
Fix master memory leak due to MasterTaskExecuteRunnableDoesn't be removed (#14162)
* Fix master memory leak due to MasterTaskExecuteRunnableDoesn't be removed

* Update dolphinscheduler-master/src/main/java/org/apache/dolphinscheduler/server/master/runner/execute/MasterTaskExecuteRunnable.java

Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
2023-05-24 13:49:32 +08:00
Wenjun Ruan
05df0269f6
Fix task exception might not exist in task instance log (#14085) 2023-05-21 09:55:35 +08:00
Kerwin
8315279620
[Feature-#14148][Task] Added status check interval and dependent failure policy parameters for dependent task nodes (#14150)
* Added a dependent task failure policy in the dependent task node
2023-05-19 14:49:50 +08:00
Kerwin
c6512edc3a
revert [fix-#13956] [Master]taskId is null cause NPE #13980 (#14142) 2023-05-18 19:28:27 +08:00
Kerwin
503d54aff9
Fix dependent task nodes in workflow instances cannot display data (#14141) 2023-05-18 19:24:53 +08:00
Wenjun Ruan
cf91544b99
Refactor logic task (#13948) 2023-04-26 10:01:51 +08:00
fuchanghai
d617620b00
[improve-#13956] taskId is null (#13980)
Co-authored-by: fuchanghai <‘2875334588@qq.com’>
2023-04-21 15:49:43 +08:00
eye-gu
1c836bf011
[Fix-13950][master] fix the key of master node failover (#13952)
Co-authored-by: eye <eye.gu@aloudata.com>
2023-04-19 17:52:38 +08:00
ralphgj
f78ef9ec08
[Fix-13815][master] Fix dependent task incorrect timeout alerts (#13822) 2023-04-19 16:14:13 +08:00
ralphgj
7835ddfdb7
[Fix-13780][api] Fix the problem of process instance DAG can't show dependent task (#13823) 2023-04-19 15:40:51 +08:00
Aaron Wang
45586f71d5
[BUG][Resource Center] Task can not select main JAR (#13536) 2023-04-18 18:38:16 +08:00
Kerwin
79cb707b57
[Improvement-13932][API] Tenant parameters and Worker group parameters are put together in the task startup parameters (#13935)
* [Improvement][API] Tenant parameters and Worker group parameters are put together in the task startup parameters
2023-04-18 14:56:38 +08:00
Wenjun Ruan
61a689aa5a
Use percentage to represent memory/cpu usage (#13896) 2023-04-13 15:48:20 +08:00
Rick Cheng
7ee66f2d02
[Feature-13429][Remote Logging] Add support for writing task logs to Google Cloud Storage (#13777) 2023-04-06 17:49:33 +08:00