Eric Gao
f7da8f39db
[Feature-16127] Support emr serverless spark ( #16126 )
...
* support emr serverless spark task
Signed-off-by: EricGao888 <ericgao.apache@gmail.com>
---------
Signed-off-by: EricGao888 <ericgao.apache@gmail.com>
2024-08-02 12:10:32 +08:00
zuo
38d599c745
[Fix-15828] fix expandListParameter ( #15878 )
...
* [Fix-15828] Replace the original question mark before regex replacement
2024-07-20 16:48:43 +08:00
xxsc0529
1199a916b3
[Feature-16132][Task Plugin] support for oceanbase datax task ( #16281 )
...
* Add support for oceanbase's datax task --feat
* fix code style
---------
Co-authored-by: xiangzihao <460888207@qq.com>
2024-07-16 09:27:05 +08:00
zuo
8d1680d2f0
[Fix-16279] Support Java 8 date/time type java.time.OffsetDateTime ( #16304 )
...
* [Fix-16279] Support Java 8 date/time type `java.time.OffsetDateTime`
2024-07-15 21:14:51 +08:00
Wenjun Ruan
af83c14298
Fix sonar cannot work ( #16307 )
2024-07-14 22:52:46 +08:00
xiangzihao
761323f3e1
[DSIP-53] Suggest add process lineage parsing for dependent task ( #16288 )
...
* add dsip 53
2024-07-11 21:25:06 +08:00
wangbowen
82ff24ae99
[Fix-16295][Task] Remote Shell Task plugin run error ( #16296 )
2024-07-10 20:02:12 +08:00
gaoyan
82b501aa8e
[Feature-16269][dinky-task]Supports dinky tasks to accept Prepare Params passed within a task or from a previous task ( #15947 )
...
* fix dinky unresolve params
* Supplement the Dinky documentation
2024-07-04 09:52:07 +08:00
blink
d42ac96c55
[Fix-16211] Remove ExecutePath in buildJarCommand to fix Java Task in Jar Mode ( #16238 )
2024-06-29 14:34:30 +08:00
xiangzihao
225e969d2d
[DSIP-51][Task] Remove unused pigeon task plugin ( #16219 )
2024-06-28 09:51:02 +08:00
xiangzihao
e8566818b0
[DSIP-50][UDF Manage] Remove unused udf manage function ( #16212 )
...
* Remove unused udf manage function
2024-06-26 21:08:48 +08:00
Wenjun Ruan
ec6a3e0727
[DSIP-46] Unify the logic task params data structure between front-end and back-end ( #16193 )
2024-06-25 13:32:13 +08:00
Wenjun Ruan
5df4b1cbc3
Fix condition task will cause workflow instance failed ( #16152 )
2024-06-19 11:28:35 +08:00
Wenjun Ruan
5e4afe6f64
Fix task nodes under switch task will not be skipped ( #16108 )
2024-06-05 21:04:23 +08:00
JohnHuang
a7245189a8
[Improvement][Flink] Change Flink command into FLINK_HOME ( #16004 )
2024-05-29 13:32:31 +08:00
旺阳
e93ed78c0f
[Fix] Fix audit log parma typo ( #16077 )
2024-05-28 13:49:19 +08:00
Wenjun Ruan
e6c57430e3
[DSIP-44] Set a delay time to TaskExecuteRunnable if it dispatched failed ( #16069 )
2024-05-27 21:53:11 +08:00
Wenjun Ruan
f754611509
[DSIP-42] Add dolphinscheduler-aws-authentication module ( #16043 )
2024-05-24 14:15:03 +08:00
yinxiaolog
baa252a0ed
[DS-16046][fix] Set PreparedStatement parameter type ( #16050 )
...
Fix the PreparedStatement parameter is TIME, set it to java.sql.Time
Co-authored-by: Aaron Wang <wangweirao16@gmail.com>
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-05-24 13:12:16 +08:00
小可耐
0fc2fe971a
[DSIP-39][parameter] Improvement startup parameters/global parameters/project parameters data type ( #15967 )
...
* [Improvement][parameter] New data types and type filtering
* [Improvement][parameter] Improvement startup parameters/global parameters data type
* fix api interfaces compatible
* add project parameter data type default value
* [Improvement][parameter] New data types and type filtering
* [Improvement][parameter] Improvement startup parameters/global parameters data type
* fix api interfaces compatible
* add project parameter data type default value
* improvement project code
* remove useless imports
* remove method onClearSearchTaskType
* add parameter doc
* optimisation logic
* code conflict resolution
* code conflict resolution
* [Improvement][Monitor] Show master && worker Busy Or Normal Status and Show Commands table list (#15978 )
* update
* test
* add monitor enhance ui
* update
* update
* update doc
* fix spotless
* update
* update
* Update dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/DataAnalysisController.java
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
* Update dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/DataAnalysisController.java
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
* Update dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/ErrorCommandMapper.java
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
* Update dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/ErrorCommandMapper.xml
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
* Update dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/CommandMapper.java
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
* Update dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/ErrorCommandMapper.xml
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
* update
* fix spotless
* update
---------
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
* [Improvement][Monitor] Add UT for montor (#15998 )
* formatting Code
2024-05-21 15:39:43 +08:00
Wenjun Ruan
61915a2d5d
Fix workflow instance restart failed due to duplicate key in varpool ( #16001 )
2024-05-16 11:32:27 +08:00
Eric Gao
dc306bfa1d
[Improvement] Use safe constructor with snake yaml ( #15758 )
2024-05-14 18:04:04 +08:00
Wenjun Ruan
3446fd8ab1
EMR task support replace params placeholder ( #15975 )
...
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2024-05-10 17:34:26 +08:00
cntiger
60b019b729
[Improvement] Fix the git url command injection in pytorch task( #15873 ) ( #15950 )
...
* fix the git url command injection danger(#15873 )
* [Improvement] Fix the git url command injection in pytorch,format code style task(#15873 )
---------
Co-authored-by: cntigers <Xiaohu4321@>
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-05-09 14:50:27 +08:00
Zzih96
5c569b705c
[fix-15907] Fix get remote shell exit code is incorrect ( #15911 )
...
* ./mvnw spotless:apply
* Update dolphinscheduler-task-plugin/dolphinscheduler-task-remoteshell/src/main/java/org/apache/dolphinscheduler/plugin/task/remoteshell/RemoteExecutor.java
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
---------
Co-authored-by: 詹子恒 <ziheng.zhan@longbridge.sg>
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-05-09 14:15:06 +08:00
Wenjun Ruan
0a11cd21bd
Fix jar path is not correct in java task ( #15906 )
2024-04-29 18:07:05 +08:00
John Huang
6c78c8ec9a
[Improvement][Spark] Support Local Spark Cluster ( #15589 )
...
* [Improvement][Spark] Support Local Spark Cluster
* remote default local from deploy mode
---------
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-04-22 15:05:59 +08:00
Wenjun Ruan
285c5a8eb5
[DSIP-28] Donnot scan whole bean under classpath ( #15874 )
2024-04-19 18:12:40 +08:00
XinXing
3fe9fd45b1
[Fix-15706] Seatunnel improvement ( #15852 )
...
* fix_seatunnel_15706
* CodeFormat
* Change to use JSONUtils
* Constants moved to constant code
* Delete empty lines
* Delete empty lines
2024-04-18 10:36:18 +08:00
Wenjun Ruan
9437d276e7
Change ssh heartbeat type to IGNORE ( #15858 )
2024-04-16 22:49:11 +08:00
sleo
f93b27e121
[Worker] Fix will not kill the subprocess in remote when stop a remote-shell task #15570 ( #15629 )
...
* fix cannot kill the subprocess when stop a remote-shell task
* move parse pid logic into ProcessUtils
* extract common logic
---------
Co-authored-by: 旺阳 <wang@lqwang.net>
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-04-10 14:13:52 +08:00
Wenjun Ruan
3b1de41acb
Remove dolphinscheduler-data-quality from dolphinscheduler-task-dataquality ( #15791 )
2024-04-04 23:44:12 +08:00
Tq
cd63069317
[Improvement-15692][task-datasync] rewrite the mockito test file for DatasyncTaskTest. ( #15693 )
...
* 15692 rewrite the mockito test file for DatasyncTaskTest.
* fix format
2024-03-11 16:54:55 +08:00
lch
e3fdb1a3ff
[Fix-15639] parameterPassing is null case NPE ( #15678 )
...
Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
2024-03-08 17:52:24 +08:00
Gallardot
3fda980006
[Task][Remoteshell] fix maven dependency warn ( #15635 )
...
Signed-off-by: Gallardot <gallardot@apache.org>
2024-02-27 19:19:11 +08:00
王渔
00008515c0
Update SeatunnelTask.java ( #15621 )
...
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
Co-authored-by: xiangzihao <460888207@qq.com>
2024-02-27 15:11:11 +08:00
Wenjun Ruan
24ff70dc88
Fix out parameter may loss ( #15617 )
2024-02-23 11:11:06 +08:00
Jay Chung
91d56f4860
fix: data quality may fail in docker mode ( #15563 )
2024-02-05 17:14:59 +08:00
dada Sun
4d6af516b9
[Bug][Task Api] fix 'MACPATTERN' in ProcessUtils and cover all cases on MacOS in ProcessUtilsTest ( #15480 ) ( #15529 )
...
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-02-05 11:41:21 +08:00
Jay Chung
8efaa9fa1f
fix: data quality can not use ( #15551 )
...
* fix: data quality can not use
fix : #15468 , #15249 , #14858
---------
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2024-02-05 09:54:06 +08:00
Wenjun Ruan
4d15932ede
Fix createFile with permission will not work ( #15556 )
2024-02-03 18:00:33 +08:00
Gallardot
3640a28348
[Improvement][HTTP] support custom rendering of http body ( #15531 )
...
Signed-off-by: Gallardot <gallardot@apache.org>
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2024-01-31 11:02:48 +08:00
zhihuasu
d80a29c48c
[BUG] #15013 Fix retryInterval in RetryPolicy will never be used in RetryUtils ( #15014 )
...
* Update RetryUtils.java
assigned
* UT
* Update RetryUtilsTest.java
---------
Co-authored-by: xiangzihao <460888207@qq.com>
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-01-25 15:07:11 +08:00
Wenjun Ruan
5a7deb28f7
Throw IllegalArgumentException if parse time placeholder error ( #15514 )
...
(cherry picked from commit 5fa31e036d77698f3732a42664bfac3e7edcb8e4)
2024-01-22 10:21:54 +08:00
Wenjun Ruan
64e1e67581
Use DefaultUncaughtExceptionHandler to log the uncached exception ( #15496 )
...
Co-authored-by: fuchanghai <changhaifu@apache.org>
2024-01-18 14:59:17 +08:00
Gallardot
c7a11ce6ed
[Improvement][K8S]Optimize MDC for K8S tasks ( #15390 )
...
Signed-off-by: Gallardot <gallardot@apache.org>
Co-authored-by: fuchanghai <changhaifu@apache.org>
2024-01-17 17:56:11 +08:00
aiwenmo
ace0860d3d
[Feature-15475][DinkyTask] DinkyTask supports Dinky-1.0.0 and common sql ( #15479 )
2024-01-16 22:47:22 +08:00
lizhenglei
bd48c99178
[BUG][Task] fix java task classpath ( #15470 )
...
* fix javatask bug
* cover
---------
Co-authored-by: 80597928 <673421862@qq.com>
Co-authored-by: fuchanghai <changhaifu@apache.org>
2024-01-12 14:39:59 +08:00
Wenjun Ruan
bf2cdc5300
Use Druid to split sql ( #15367 )
2024-01-02 12:00:53 +08:00
Gallardot
93ea5f64c3
[Improvement][K8S] Custom label of a K8S task can be passed to the pod ( #15369 )
...
Signed-off-by: Gallardot <gallardot@apache.org>
2023-12-28 14:49:29 +09:00
