Wenjun Ruan
e6d94632b9
Add WorkflowExecuteContext ( #14544 )
2023-07-14 18:40:01 +08:00
Rick Cheng
3de0b5a38c
[Feature-14339][Parameter] Add parameter combining of project-level parameters ( #14452 )
2023-07-10 10:17:46 +08:00
Jay Chung
221df4a278
revert: Datasource need test binding to create prod ( #14381 )
...
We should keep our step as less as possible, for now, we have to
test datasource and binding it to the prod datasource, I think it will
make our users do not thing to use datasource, so I do like to revert
it and keep step as less as possible
related to https://github.com/apache/dolphinscheduler/pull/11670
2023-06-27 10:59:17 +08:00
Rick Cheng
b49b91c45d
[Improvement-14331][common] Remove the duplicate ParameterUtils ( #14332 )
2023-06-14 15:00:54 +08:00
Wenjun Ruan
aa2d9a7f05
Add baseDao ( #14316 )
2023-06-09 19:02:50 +08:00
JieguangZhou
e7cdc7c136
[Feature][Task Plugin] Add Dynamic task to generate dynamic worklfows with list parameters ( #14127 )
...
* add dynamic task plugin
* fix ui
* update code
* add DOC
* ADD UT
* fix doc
* fix sanity check
* add upgrade ddl
* fix sql
2023-06-07 16:01:06 +08:00
Wenjun Ruan
d810c487fd
Fix taskGroupQueue doesn't removed from inQueue where wakeup failed ( #14200 )
...
* Fix taskGroupQueue doesn't removed from inQueue where wakeup failed
* Fix avaliable taskGroup is 0 the TaskGroupQueue doesn't out queue
2023-05-25 14:42:32 +08:00
haibingtown
af7da0630e
[Feature] Add project name/workflow definition name/instance id/task name to the built-in variables ( #14099 )
...
* [Feature] Add project name/workflow definition name/instance id/task name to the built-in variables
2023-05-25 11:02:39 +08:00
Drake Youngkun Min
801216bd35
RECOVER_TOLERANCE_FAULT_PROCESS CommandType needs the start parameters ( #13958 )
2023-05-10 13:02:21 +08:00
Wenjun Ruan
cf91544b99
Refactor logic task ( #13948 )
2023-04-26 10:01:51 +08:00
Aaron Wang
45586f71d5
[BUG][Resource Center] Task can not select main JAR ( #13536 )
2023-04-18 18:38:16 +08:00
Kerwin
79cb707b57
[Improvement-13932][API] Tenant parameters and Worker group parameters are put together in the task startup parameters ( #13935 )
...
* [Improvement][API] Tenant parameters and Worker group parameters are put together in the task startup parameters
2023-04-18 14:56:38 +08:00
旺阳
61637cc0a3
[Bug] [Cron] Parse corn expression error ( #13841 )
...
* fix-cron
* update catch exception
2023-04-09 12:50:42 +08:00
Wenjun Ruan
41a8ba9aab
Support alert server HA ( #13865 )
...
* Support alert server cluster
* Remove WORKER_ALERT_LISTEN_HOST in statefulset-dolphinscheduler-worker.yaml
2023-04-05 17:31:50 +08:00
Rick Cheng
85605bbef9
[Improvement-13653][Remote Logging] Support api-server to get task log from remote target ( #13654 )
2023-03-30 12:07:58 +08:00
caishunfeng
9d9ae9ad54
[improvement] support self-dependent ( #13818 )
2023-03-29 18:18:03 +08:00
Wenjun Ruan
68660ec96b
Refactor remote command ( #13809 )
...
* Refactor remote command
* Rename Command to Message
2023-03-29 17:54:57 +08:00
Wenjun Ruan
d1b6e6f02c
Take over task instance in master failover ( #13798 )
2023-03-28 16:02:57 +08:00
Aaron Wang
8895921d87
[Improvement-13651] Remove useless resources_task ( #13721 )
2023-03-23 22:52:23 +08:00
Wenjun Ruan
c9066e8de9
Use MDC to filter task instance log ( #13673 )
...
* Use MDC to collect task instance log
* Use MDCAutoClosableContext to remove the MDC key
2023-03-06 17:44:23 +08:00
Aaron Wang
047fa2f65e
[Feature-13511] Submit Spark task directly on Kubernetes ( #13550 )
2023-02-21 23:26:21 +08:00
seedscoder
8d12dc0702
[Improvement-13491] Use lombok @Slf4j annotation to generate logger ( #13509 )
2023-02-07 20:32:53 +08:00
Wenjun Ruan
6300a9b8de
Fix serial wait will go into block when there is no task need to submit ( #13486 )
2023-02-02 11:10:37 +08:00
Wenjun Ruan
12dd60fa46
Fix task wake up failed will block the event ( #13466 )
2023-02-01 15:31:04 +08:00
qianli2022
8be32d4145
[Feature][Api] When use api to run a process we want get processInstanceId ( #13184 )
...
* add sql
* add mapper
* add dao
* add excutor
Co-authored-by: qianl4 <qianl4@cicso.com>
2023-01-18 17:58:32 +08:00
Wenjun Ruan
01f4fb26f0
Change remove task instance log to async request ( #13399 )
2023-01-16 15:49:23 +08:00
Wenjun Ruan
98dfd64444
Clear the task group data when delete a project/workflowInstance ( #13383 )
2023-01-12 21:51:51 +08:00
Wenjun Ruan
8a479927f3
Add projectCode in t_ds_process_instance and t_ds_task_instance to remove join ( #13284 )
2023-01-03 09:52:28 +08:00
Wenjun Ruan
52134277a3
Fix task group cannot release when kill task ( #13314 )
2023-01-03 09:52:03 +08:00
Aaron Wang
8503ee0eed
[Improvement][Worker] Startup parameter should have the highest priority ( #13274 )
2022-12-29 11:05:00 +08:00
Wenjun Ruan
fd3afd84ad
Remove alert when delete workflow instance ( #13281 )
2022-12-28 11:40:12 +08:00
Wenjun Ruan
de70421a1e
Remove the log path check ( #13280 )
2022-12-26 22:39:08 +08:00
Wenjun Ruan
14ec4a2398
Remove dao module in worker ( #13242 )
2022-12-22 12:25:29 +08:00
JieguangZhou
66e20271ad
[Feature][Master] Add task caching mechanism to improve the running speed of repetitive tasks ( #13194 )
...
* Supports task instance cache operation
* add task plugin cache
* use SHA-256 to generate key
* Update dolphinscheduler-dao/src/main/resources/sql/dolphinscheduler_mysql.sql
Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>
* Update dolphinscheduler-dao/src/main/resources/sql/dolphinscheduler_postgresql.sql
Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>
* Optimizing database Scripts
* Optimize clear cache operation
Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>
2022-12-18 18:17:09 +08:00
JieguangZhou
e4b9b67255
Allow execute task in workflow instance ( #13103 )
2022-12-13 16:43:44 +08:00
Wenjun Ruan
838a0df783
Remove logic task plugin module ( #13171 )
2022-12-13 16:30:44 +08:00
Wenjun Ruan
f5801a0e1c
Fix task group might be rob by multiple task ( #13115 )
2022-12-11 22:58:21 +08:00
Yann Ann
6ef74073cc
[Improve-13001]migrate commons-collections -> commons-collections4 ( #13002 )
2022-12-10 23:50:19 +08:00
Wenjun Ruan
8a152aebc7
Fix task log file might not be clear ( #13102 )
2022-12-06 11:23:20 +08:00
Wenjun Ruan
fed1e85fb8
Fix the invalidate task instance never be deleted ( #13091 )
2022-12-04 18:38:54 +08:00
rickchengx
38b876733c
[Feature-10498] Mask the password in the log of sqoop task ( #11589 )
2022-11-24 14:54:54 +08:00
Kerwin
c916c60853
fix NPE while retry task ( #12903 )
2022-11-16 10:31:52 +08:00
Wenjun Ruan
d99ba29b66
Fix master cluster may loop command unbalanced ( #12891 )
...
(cherry picked from commit 3b2b86661be76b7c1404a910c865d78b7936313d)
2022-11-16 10:20:22 +08:00
XCL
00b18ed422
[Improve]Remove the code logger.isDebugEnabled() #5918 #10542 ( #12738 )
...
Co-authored-by: chunlai.xu <chunlai.xu@zhaopin.com.cn>
2022-11-07 09:58:19 +08:00
Aaron Wang
08335b1032
[Improvement][Task] Improved way to collect yarn job's appIds ( #12197 )
...
* Provide aop way as an optional way to collect yarn job's applicationId, and import new module `dolphinscheduler-aop` to place the aop code.
* Add user property `appId.collect` for user to decide how to collect applicationId.
* Add new environment configuration for each type of yarn tasks to support aop in `dolphinscheduler_env.sh`
* Update docs to declare how to use aop way.
* Update `LogUtils` to support fetch applicationId in different ways based on the user property.
Co-authored-by: gabrywu <gabrywu@apache.com>
2022-10-31 16:52:53 +08:00
Yann Ann
ed209bdf82
[Refactor] Migrate all workergroup-related interface functions from ProcessServiceImpl ( #12493 )
...
* Migrate all workergroup-related interface functions from ProcessServiceImpl
2022-10-26 19:18:32 +08:00
HanayoZz
489e7fe4e2
[Feature-10495][Resource Center] Resource Center Refactor ( #12076 )
...
* resource center refactor - S3 services connection
Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
2022-10-26 13:53:44 +08:00
Yann Ann
04aa125ba2
[Refactor]migrate some task dao functions from ProcessServiceImpl to dao modules. ( #12505 )
...
* migrate some task DAO to dao modules.
* Update dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/repository/impl/ProcessInstanceMapDaoImpl.java
Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
2022-10-26 11:51:58 +08:00
rickchengx
233bbdb09e
[Fix-12451][k8s] Read the kubeconfig from cluster conf ( #12452 )
2022-10-25 11:25:30 +08:00
Wenjun Ruan
a0d37fc7ae
Set tenantDir permission ( #12486 )
2022-10-23 21:54:01 +08:00
