JieguangZhou
df491f78cc
fix sagemaker sync status ( #13264 )
2022-12-22 20:17:17 +08:00
Wenjun Ruan
14ec4a2398
Remove dao module in worker ( #13242 )
2022-12-22 12:25:29 +08:00
Wenjun Ruan
b2336b0ce9
Remove datasource in procedure task ( #13198 )
2022-12-19 17:14:59 +08:00
youzipi
042ec74a24
[refactor] cache compiled regex pattern ( #13208 )
...
* [refactor] cache compiled regex pattern
* [refactor] cache compiled regex pattern
* [refactor] cache compiled regex pattern
2022-12-18 11:24:11 +08:00
HuangWei
43e6ac32dd
[Fix-13187] refactor OpenmldbParameters ( #13190 )
2022-12-14 20:13:45 +08:00
Wenjun Ruan
838a0df783
Remove logic task plugin module ( #13171 )
2022-12-13 16:30:44 +08:00
Wenjun Ruan
70ccffeee2
Format task parameter as pretty json ( #13173 )
2022-12-13 16:30:21 +08:00
Wenjun Ruan
ab96a3d0c0
Fix datasource jdbc url param doesn't work ( #13098 )
2022-12-12 11:46:40 +08:00
Aaron Wang
560d232cbd
[Bug][Worker] Fix duplicated key bug when have out parameters of same name ( #12930 )
2022-12-12 11:07:23 +08:00
Yann Ann
6ef74073cc
[Improve-13001]migrate commons-collections -> commons-collections4 ( #13002 )
2022-12-10 23:50:19 +08:00
Wenjun Ruan
169168ef34
Add plugin-all module ( #13079 )
2022-12-02 23:19:08 +08:00
Eric Gao
c2ae76c0bd
Expand all static imports and add CI check to block wildcard static imports ( #13067 )
2022-12-01 14:15:28 +08:00
Wenjun Ruan
32a8241ef2
Don't source env file in task, this will override tenant's environment ( #13028 )
2022-11-28 21:09:50 +08:00
Kerwin
ae670e6e1c
[Bug-13010] [Task] The Flink SQL task page selects the pre-job deployment mode, but the task executed by the worker is the Flink local mode ( #13011 )
2022-11-28 10:09:49 +08:00
Kerwin
d480c271ec
[Bug-12994] [Worker] Fix kill process does not take effect ( #12995 )
2022-11-27 14:27:48 +08:00
Wenjun Ruan
a2ff140f43
Fix sql task will send alert if we don't choose the send email ( #12984 )
2022-11-24 21:26:52 +08:00
rickchengx
38b876733c
[Feature-10498] Mask the password in the log of sqoop task ( #11589 )
2022-11-24 14:54:54 +08:00
JieguangZhou
6e224d08a2
[fix] Python task can not pass the parameters to downstream task. ( #12961 )
2022-11-22 16:11:45 +08:00
JieguangZhou
7c90bf01bc
[Improvement][Task Plugin] Improvement Kubeflow task plugin ( #12928 )
...
add example
check phase in status:conditions
2022-11-17 15:59:44 +08:00
Assert
2dbc79693e
[feature][task plugin] Add new task plugin for apache linkis ( #12693 )
2022-11-17 10:14:36 +08:00
Wenjun Ruan
92a9b7e243
Use System#lineSeparator to replace /n ( #12889 )
2022-11-15 13:57:45 +08:00
JieguangZhou
229c554912
[feature][task] Add Kubeflow task plugin for MLOps scenario ( #12843 )
2022-11-11 16:08:38 +08:00
simsicon
43d69ae157
Upgrade clickhouse jdbc driver ( #12639 )
2022-11-11 13:38:58 +08:00
Kerwin
c4fb555883
Modify shell env to bash ( #12742 )
2022-11-07 11:21:11 +08:00
Wenjun Ruan
2c3cc31a4c
Fix flink sql cannot run due to missing main jar ( #12705 )
2022-11-04 23:02:00 +08:00
Wenjun Ruan
eaef68856b
Change command file permission to 755 ( #12678 )
...
(cherry picked from commit 95484d930762712f4ab68ece27ca3b65a2073ab2)
2022-11-03 22:51:37 +08:00
JieguangZhou
a47b4d0672
[Feature][Task] Transfer files between tasks ( #12552 )
...
* add data transfer between tasks
* add delete DATA_TRANSFER API
* convert Result to DeleteDataTransferResponse
* add api UT
* fix final
* fix doc
2022-11-03 16:33:39 +08:00
amao
b71d4983dd
[Feature-6586][Server]add some ds process definition demo when init ( #11759 )
...
* [Feature-6586][Server]add some ds process definition demo when init
1.add some ds process definition demo when init, to display what task type can run and make user easy to
use ds.
2.need configure the JVM parameters (-Ddemo=true) to turn on the StandaloneServer service
3.need modify the tenant information in it
2022-10-31 22:35:11 +08:00
rickchengx
3c31ddfd9f
[Improvement-12536][k8s] Support the command for the container in k8s task plugin ( #12538 )
2022-10-31 19:06:41 +08:00
Aaron Wang
08335b1032
[Improvement][Task] Improved way to collect yarn job's appIds ( #12197 )
...
* Provide aop way as an optional way to collect yarn job's applicationId, and import new module `dolphinscheduler-aop` to place the aop code.
* Add user property `appId.collect` for user to decide how to collect applicationId.
* Add new environment configuration for each type of yarn tasks to support aop in `dolphinscheduler_env.sh`
* Update docs to declare how to use aop way.
* Update `LogUtils` to support fetch applicationId in different ways based on the user property.
Co-authored-by: gabrywu <gabrywu@apache.com>
2022-10-31 16:52:53 +08:00
HanayoZz
489e7fe4e2
[Feature-10495][Resource Center] Resource Center Refactor ( #12076 )
...
* resource center refactor - S3 services connection
Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
2022-10-26 13:53:44 +08:00
caishunfeng
dedff70f90
[Improvement] Merge spi.utils into common.utils ( #12502 )
...
* improve JSONUtil
* merge spi.utils into common.utils
* split constants && remove some copy utils method
* remove StringUtils
2022-10-23 21:23:54 +08:00
fuchanghai
f0a34cddec
[improvement-#12416] Use CollectionUtils.isEmpty judge the collection isEmpty to avoid NPE ( #12449 )
2022-10-20 14:26:56 +08:00
Eric Gao
d46e4659d9
[Improvement][Style] Fix remaining style errors and wildcard imports ( #12412 )
...
* Fix remaining style errors and wildcard imports
* Upgrade spotless to the latest stable version, exclude .github folder from spotless check and fix dead link
* Fix spotless matching scope of markdown files
2022-10-19 11:08:04 +08:00
insist777
1e11034049
[Fix][Task] Fix dependent task can not predicate the status of the corresponding task correctly ( #12253 )
...
* [Fix][Task] dependent task can not predicate the status of the corresponding task correctly
2022-10-19 10:47:08 +08:00
simsicon
ba538067f2
[fix] Change the default unix shell executor from sh to bash ( #12180 )
...
Co-authored-by: Kerwin <37063904+zhuangchong@users.noreply.github.com>
2022-10-18 12:57:37 +08:00
Yann Ann
3bef85f546
fix datax NPE issue ( #12388 )
2022-10-18 09:03:16 +08:00
Kerwin
55004bebe0
[Improvement-12391][API] Workflow definitions that contain logical task nodes support the copy function ( #12392 )
...
* Workflow definitions that contain logical task nodes support the copy function.
2022-10-17 10:28:58 +08:00
fuchanghai
b7bd8d7803
[Improvement] add presto in datax ( #12371 )
2022-10-16 17:19:29 +08:00
Yann Ann
7aa8a77fb2
[Migrate][Test] Migrate other UT cases from jUnit 4 to 5, change @Ignore to @Disable
2022-10-16 12:49:11 +08:00
rickchengx
7b44612f28
[Improvement-12372][k8s] Update the deprecated k8s api ( #12373 )
2022-10-14 16:18:35 +08:00
Eric Gao
2f37da0dbc
[Improvement][Test] Block the usage of powermock and move mockito dependencies from sub-modules to root pom ( #12311 )
...
* move mockito dependencies from sub-modules to root pom
* Add check in CI to block the usage of powermock
2022-10-14 12:06:51 +08:00
kezhenxu94
db1d319159
Reorganize some classes in common module, remove duplicate classes ( #12321 )
2022-10-13 10:23:27 +08:00
rickchengx
05099bd8f7
[Improvement-12296][Test] Close static mock ( #12300 )
2022-10-11 16:13:05 +08:00
Eric Gao
875682d267
[Improvement][Test] Migrate all UT cases from jUnit4 to jUnit5 in task-plugin module as an example ( #12299 )
...
* Migrate all UT cases from jUnit4 to jUnit5 in task-plugin module as an example
2022-10-11 11:54:38 +08:00
Eric Gao
a865be09b6
Remove powermock from DatasyncTaskTest ( #12270 )
2022-10-09 12:38:03 +08:00
Tq
c7435a51ac
[Feature][task-plugins] aws datasync task plugins ( #11865 )
...
* submit datasync task
* add aws software package license
2022-10-09 10:47:01 +08:00
FlechazoW
0687697869
[Improvement][Task Plugin] Modify the comment of 'deployMode'. ( #12163 )
2022-10-07 21:21:13 +08:00
Yann Ann
c87f6d89a4
[fix#12195] Change date to timestamp to support cross diff time zones ( #12239 )
...
* fix DS support cross time zone, use timestamp replace date
2022-10-01 12:50:42 +08:00
JieguangZhou
308f632fcc
[Feature][TASK-PLUGIN] AWS Database Migration Service (DMS) ( #11868 )
...
* add aws dms
2022-09-30 10:16:16 +08:00
