test/DolphinScheduler
1
0
Fork 0
mirror of https://gitee.com/dolphinscheduler/DolphinScheduler.git synced 2024-12-05 05:38:30 +08:00
Code Issues Actions 10 Packages Projects Releases Wiki Activity
7,231 Commits 27 Branches 59 Tags 276 MiB
7ca4438682
Commit Graph

178 Commits

Author SHA1 Message Date
zhuangchong
7ca4438682 Resolve AbstractTask code conflicts 2022-10-27 13:03:33 +08:00
Kerwin
f3277277f0
cherry-pick Refactor LogServiceClient Singleton to avoid repeat creation of NettyClient #11777 (#12542) 
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
 2022-10-26 16:20:13 +08:00
Kerwin
344293102d
[3.1.1-prepare][cherry-pick][Improvement] Add remote task model #11767 (#12541) 2022-10-26 15:44:13 +08:00
insist777
3a14fd07b3 [Fix][Task] Fix dependent task can not predicate the status of the corresponding task correctly (#12253) 
* [Fix][Task] dependent task can not predicate the status of the corresponding task correctly
 2022-10-19 11:44:39 +08:00
Kerwin
d2e56af838
[Bug] [spark-sql] In spark-sql, select both SPARK1 and SPARK2 versions and execute ${SPARK_HOME2}/bin/spark-sql (#11721) (#12420) 
select different versions of spark-sql to execute different versions of spark-sql

Co-authored-by: limaiwang <limaiwang@163.com>
 2022-10-18 18:45:54 +08:00
Kerwin
f0ad67f992
cherry-pick source is not available in sh (#12413) 
Co-authored-by: simsicon <simsicon@gmail.com>
 2022-10-18 13:24:57 +08:00
rickchengx
5cfe6a96b4 [Improvement-12372][k8s] Update the deprecated k8s api (#12373) 2022-10-18 09:21:38 +08:00
FlechazoW
39c1144fab [Improvement][Task Plugin] Modify the comment of 'deployMode'. (#12163) 2022-10-18 09:15:37 +08:00
zhuangchong
b4f59af9d9 Remove the DataxTaskTest class because there is no junit5 package. 2022-10-18 09:15:06 +08:00
Kerwin
b559d033d3
fix datax NPE issue (#12388) (#12407) 2022-10-18 09:08:15 +08:00
Kerwin
771dd67b2e cherry-pick [Improvement-12391][API] Workflow definitions that contain logical task nodes support the copy function 2022-10-17 10:34:04 +08:00
caishunfeng
7e39396a76 [maven-release-plugin] prepare for next development iteration 2022-09-26 21:50:03 +08:00
caishunfeng
ae33ba5947 [maven-release-plugin] prepare release 3.1.0 2022-09-26 21:49:27 +08:00
JieguangZhou
f871a4a41f fix python home conflict (#12112) 
(cherry picked from commit 6eb1eb722a)
 2022-09-23 19:43:56 +08:00
Eric Gao
e7b12bf205
[Cherry-Pick][3.1.0-prepare] Cherry-Pick OssOperator for Resource Center to 3.1.0 (#12094) 
* Cherry OssOperator for Resource Center to 3.1.0

* Fix doc dead link
 2022-09-22 15:00:25 +08:00
JieguangZhou
a2b3659fe9 [improve] Optimize MLFlow task plugin for easy of use (#12071) 
optimize code

(cherry picked from commit cf522e2f)
 2022-09-21 13:50:48 +08:00
rickchengx
806ad1d98c [Improvement-11291][sql] Improvement the log output while executing multiple sql statements (#11292) 2022-09-20 10:47:18 +08:00
W.XING
9ad6a049c5 Add chunjun in dolphinscheduler-task-all/pom.xml (#11375) 2022-09-19 15:34:54 +08:00
kezhenxu94
fcc75ef1c6 Bump up dependencies to fix cves (#11765) 2022-09-19 15:10:56 +08:00
Kengo Seki
52b79b017e [Improvement] Replace commons-lang 2 function invocations with commons-lang3 (#11810) 
* [Improvement] Replace commons-lang 2 function invocations with commons-lang3
 2022-09-19 14:48:17 +08:00
JieguangZhou
dde6f63c31 [Feature][PyDolphinScheduler] Support DVC task in pyds #11922 (#11941) 
* add dvc task in pyds

* add BaseDVC class

(cherry picked from commit bbc228e93f)
 2022-09-18 16:49:34 +08:00
caishunfeng
8d1c2d3eeb
cherry-pick [Bug] Fix procedure task param pass #11919 (#11926) 2022-09-14 15:44:01 +08:00
fuchanghai
f8b9aad239
[feature-#11689] change replaceAll to replace in method of tryExecuteSqlResolveColumnNames (#11696) 2022-09-05 09:54:18 +08:00
caishunfeng
f95cdd350a
reduce task dependencies (#11731) 2022-09-01 17:04:07 +08:00
Wenjun Ruan
03e1e6fe45
[Bug] [Worker] Optimize the getAppId method to avoid worker OOM when kill task (#11701) 
* Fix kill job may cause worker oom
 2022-08-31 15:25:01 +08:00
Eric Gao
4a37345436
[Feature][Task Plugin] Support hive cli task plugin (#11651) 
* Add hive cli task plugin

* Add docs for hive-cli task plugin

* Fix hive-cli task related front-end nits and checkstyle

* Fix exception handling logic
 2022-08-30 10:45:51 +08:00
Wenjun Ruan
1b120e3a59
Refactor worker execute task process (#11540) 
* Refactor worker execute task process
 2022-08-26 13:33:51 +08:00
Wenjun Ruan
b96d69701a
[Improvement-#11613] Add spi priority factory (#11614) 
* Add spi priority factory

* Add doc

* Add override log

* Use lombok

* Add comment
 2022-08-24 19:51:47 +08:00
Wenjun Ruan
a41c6824fc
Add dependent task instance log (#11541) 
* Add dependent task instance log

* Optimize log

* Fix dependent task initialize failed will throw exception
 2022-08-23 09:55:38 +08:00
kezhenxu94
13e4720aa2
Upgrade spring boot and swagger (#11574) 2022-08-22 10:27:40 +08:00
fuchanghai
d8d5d392a7
[feature-11408] add hive option (#11493) 2022-08-20 08:25:53 +08:00
JieguangZhou
a974ba74ab
[Feature] New task plugin Pytorch (#11498) 
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
 2022-08-18 10:20:37 +08:00
FangYin Cheng
11d74fd7e8
[test-11432][HttpTask] Add unit tests to http task (#11453) 
* [test][HttpTask] Add unit tests to http task

* [test][HttpTask] fix http task test code format problem

* [test][HttpTask] fix http task test code replace http code to constants
 2022-08-17 22:02:52 +08:00
lugela
e5d06c40a5
[Feature-10182][task-datax] (#11387) 
In the process of realizing datax synchronization, use custom json for better connection to enable kerberos-authenticated clusters, such as using plug-ins such as hdfs and hbase
 2022-08-17 10:01:20 +08:00
kezhenxu94
f6ef628030
Support reading application properties from kubernetes config map (#11402) 2022-08-16 20:25:17 +08:00
Eric Gao
896fef6c98
[Feature][Style] Enable spotless to manage imports (#11458) 2022-08-13 10:05:03 +08:00
Kerwin
b79bad19e5
[Fix-11051][Task]Fix the process exitValue exception when the process times out and th… (#11099) 2022-08-11 09:28:40 +08:00
Wenjun Ruan
5fa3e7b1ed
Add pause to task instance status (#11390) 2022-08-11 09:25:53 +08:00
Stalary
6345a13cdc
[Fix-10392][SqlTask]: Fix list param error when use sql task (#11285) 
* FIX: fix list param in.

* FIX: code bug

* FIX: fix one value

* FIX: header

* FIX: ci

* FIX: fix ci

* MOD: for review

* MOD: for review
 2022-08-11 09:24:44 +08:00
caishunfeng
0464123c2b
[Feature-11223] support stream task (#11350) 
* add task execute type

* update task definition list paging

* update task instance list paging

* stream task start

* [Feature][UI] Some changes to execute task.
    * Set the connection edge to dashed line.
    * Add FLINK_STREAM task.

* add stream task

* flink savepoint and cancel

* fix query bug

* add stream task definition

* add task instance for stream task

* delete stream task definition state

* update api for stream task definition edit

* modify search for stream task instance

* add language

* delete task type search for stream task definition

* change task type search for stream task instance

* add jump button

* add savepoint

* add down log for stream task instance

* ui test

* stream task start

* run DAG

* [Fix][UI] Fix the stream task edgs not to be dashed when filling back.

* [Feature][UI] Remove some fields for FLINK_STREAM.

* add start modal

* add dryRun column for stream task instance

* fix duration

* fix pon

* fix build error

* Add success tip

* add auto sync for stream task instance

* remove forgien key for task instance

* license header

* UT fix

* modify locales

* recover common config

* fix UT

* add doc

Co-authored-by: Amy <amywang0104@163.com>
Co-authored-by: devosend <devosend@gmail.com>
 2022-08-10 21:44:43 +08:00
Assert
b3919d30e7
[Improvement-11394][Task Plugin] Remove Passed Parameter --queue in Seatunnel Task Plugin (#11395) 2022-08-10 20:17:16 +08:00
Assert
03582f2e70
[Fix-11376][task plugin]Seatunnel Task cannot save successfully when … (#11377) 2022-08-10 12:54:09 +08:00
Wenjun Ruan
8774415197
Split ExecutionStatus to WorkflowExecutionStatus and TaskExecutionStatus (#11340) 2022-08-10 11:00:23 +08:00
Eric Gao
6a02870926
[Feature][style] Add spotless maven plugin for automatic style fix. (#11272) 
* [Feature][style] Add spotless maven plugin for automatic style fix (#10963)

* Fix spotless ratchet configuration

* Remove license-check and decrease line length threshold value

* Update related docs

* Remove checkstyle and add pre-commit hook

* Test updated pre-commit hook

* Replace checkstyle with spotless in CI

* Remove reviewdog
 2022-08-06 19:29:23 +08:00
microeastcowboy
9a53c6ac34
fix work oom when task logs's size is very large (#11224) 2022-08-01 15:28:06 +08:00
xuhhui
bfff3a7c5d
fix error (#11206) 2022-07-30 18:20:20 +08:00
xuhhui
513f336015
[Feature] Add new task type chunjun (#10937) 2022-07-28 19:25:29 +08:00
caishunfeng
818648df7d
[Bug-11071] fix local params (#11184) 
* fix local params

* add CI case
 2022-07-28 15:07:04 +08:00
Wenjun Ruan
052ceaacd0
Add is success in loop task status (#11169) 2022-07-28 12:12:01 +08:00
Assert
cdfe115247
[Feature-10837][Task Plugin] Sqoop Task Add --split-by Parameter (#10960) 
* [Feature-10837][Task Plugin] Sqoop Task Add --split-by Parameter

* checkstyle
 2022-07-26 17:41:13 +08:00