Commit Graph

121 Commits

Author SHA1 Message Date
Wenjun Ruan
e9d85914d7
Fix queryByTypeAndJobId might error due to multiple result (#15883) 2024-04-22 17:41:17 +08:00
caishunfeng
99a2461785
[Improvement] change js engine (#15747)
* change js engine

* add dependent jar
2024-03-22 11:31:47 +08:00
Wenjun Ruan
bf2cdc5300
Use Druid to split sql (#15367) 2024-01-02 12:00:53 +08:00
xujiaqiang
d144f48ed7
[Feature-15248][dolphinscheduler-alert-plugins] add alert plugin aliyun-voice (#15248)
* add alert aliyun-voice

* VoiceParam replace these with lombok @Data

* security.ts add alarm_instance params

* optimize alert aliyun vocie code

* Merge Code

* rollback pnpm-lock.yaml to branch :origin/dev version

* add the doc about the new plugin

* add com.aliyun.dyvmsapi20170525 version 2.14

* completed

* run mvn spotless:apply

* Code optimization

* Change to @Data

* Code specification optimization

* add com.aliyund.yvmsapi20170525.jar license

* modify com.aliyund.yvmsapi20170525.jar license

* modify com.aliyund.yvmsapi20170525.jar license

* add aliyun-vocie licenses

* add aliyun-voice link to docsdev.js

* modify vocie module to aliyunVoice

* add lisense

---------

Co-authored-by: xujiaqiang <“xujiaqiangwz@163.com”>
Co-authored-by: 旺阳 <wang@lqwang.net>
Co-authored-by: David Zollo <dailidong66@gmail.com>
2023-12-13 15:56:37 +08:00
Wenjun Ruan
b750db3c37
Add IT for dolphinscheduler-tools module (#15043) 2023-10-20 15:39:42 +08:00
Wenjun Ruan
33084d281f
Add dolphinscheduler-dao-plugin module (#15019) 2023-10-14 17:58:40 +08:00
Wenjun Ruan
55cc77226f
[RPC] Refactor the remote module. (#14776)
* Add dolphinscheduler-extract-base
* Add dolphinscheduler-extract-master
* Add dolphinscheduler-extract-worker
* Add dolphinscheduler-extract-alert
2023-08-23 09:43:17 +08:00
Edison Catto
25480ae9e7
[Feature][Resource Center] Add support for Huawei Cloud OBS (#14643)
* [Feature][Resource Center] Add support for Huawei Cloud OBS as storage of resource center

* add license and add doc

* add 3-party dependency license

* Update LICENSE

* fix

* Update pom.xml

* fix

* fix

---------

Co-authored-by: sunkang <sunkang@hjfruit.com>
Co-authored-by: xiangzihao <460888207@qq.com>
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2023-08-14 13:25:48 +08:00
Aaron Wang
ca760aabd4
[Fix-14512][BOM] kubernetes-client version conflict (#14561) 2023-07-21 18:17:53 +08:00
xiangzihao
758d5af023
[Dependency] Bump h2, snowflake-jdbc, snappy-java version (#14575)
* bump some dependency version
2023-07-19 09:31:50 +08:00
eye-gu
b72077011b
[Fix-13987][bom] set 3.17.2 version protobuf-java (#13998)
Co-authored-by: eye <eye.gu@aloudata.com>
2023-07-11 21:54:24 +08:00
xiangzihao
091d869d66
[CI] [Test] Try to fix jar conficts (#14407)
* try to fix jar conficts
2023-06-26 22:51:51 +08:00
Rick Cheng
4fa0ab4d7e
[feature][Resource] Add support for Azure Blob Storage in the resource center (#14082) 2023-06-12 12:05:49 +08:00
Rick Cheng
ed8ae9c0ee
[Improvement-14286][BOM] Bump clickhouse-jdbc from 0.3.2-patch11 to 0.4.6 (#14287) 2023-06-07 12:08:23 +08:00
Zinway
18d176c5f4
feat: Introduce new datasource vertica (#14232)
Co-authored-by: Zinway Liu <zinway.liu@ucloud.cn>
2023-06-02 13:57:37 +08:00
Rick Cheng
0ab2447d58
[Improvement-14080][script] cleanup dist directory at the beginning of check-LICENSE.sh (#14081) 2023-05-23 10:52:50 +08:00
He Zhao
f1e78627fc
[feature][plugin] Add snowflake datasource in datasource plugin (#13729)
---------

Co-authored-by: weitong <weitong@cisco.com>
2023-05-06 19:30:50 +08:00
Bingchang Chen
77e58ce245
[Feature] Add SSO login by Casdoor (#8706) 2023-05-05 14:43:06 +08:00
Jeremy
db6c061c8b
[Feature][plugin] Add databend datasource in datasource plugin (#13866)
Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>
2023-04-24 21:28:14 +08:00
He Zhao
c0126b7f30
[Feature][plugin] Support Kyuubi datasource (#13642)
---------

Co-authored-by: weitong <weitong@cisco.com>
Co-authored-by: Rick Cheng <38122586+rickchengx@users.noreply.github.com>
2023-04-18 11:12:39 +08:00
Aaron Wang
119f08d2ba
[Bug-13882][Worker] Upgrade kubernetes-client version to 6.0.0 (#13891) 2023-04-17 12:04:36 +08:00
JieguangZhou
273a5e78eb
[FEATURE][Task Plugin]Add remote-shell task plugin (#13801) 2023-03-30 20:17:28 +08:00
Rick Cheng
2dceb6b8ca
[Feature-13428][Resource Center] Add support for Google Cloud Storage in the resource center (#13562) 2023-03-15 19:29:27 +08:00
Tq
78ba71bdbd
[Feature][task]add azure datafactory task type (#13417)
* init data factory

* add task-type-config.yaml

* add data factory ui

* submit doc

* submit license

---------

Co-authored-by: devosend <devosend@gmail.com>
2023-01-30 11:15:20 +08:00
lenian
bcf03ad4d1
[Feature-9467] add DAMENG DataSource (#12860)
* [Feature-9467][DataSource] add DAMENG (达梦) data source

* [Feature-9467] add DM datasource doc (Issue #9467)

* [Feature-9467] add DM IDataBaseOptionKeys

* [Feature-9467] add DM DataSource: DmDataSourceChannelFactoryTest remove testGetDmConnection method

* [Feature-9467] modification package dm to dameng;

* [Feature-9467] add Dameng DmJdbcDriver18 License

* [Feature-9467] add DM DataSource: remove DruidDataSourceClient

Co-authored-by: lenian <yuenian@mananacare.com>
2023-01-04 15:00:24 +08:00
Tq
f0dd4f081f
[Feature][task-plugins-12820] azure azure-sql datasource plugins (#13209)
* submit azure sql code

* change mssql-jdbc version

* merge changes to delete other

* merge changes to delete other

* add dependencies
2022-12-19 10:27:03 +08:00
Yann Ann
6ef74073cc
[Improve-13001]migrate commons-collections -> commons-collections4 (#13002) 2022-12-10 23:50:19 +08:00
insist777
b40c63f02d
[Feature][DataSource]Add trino datasource plugin (#12911) 2022-11-23 10:09:02 +08:00
simsicon
43d69ae157
Upgrade clickhouse jdbc driver (#12639) 2022-11-11 13:38:58 +08:00
seedscoder
1f23d29698
[Improvement-12029][dolphinscheduler-api] Upgrade the snakeyaml version to avoid vulnerabilities (#12726) 2022-11-11 13:33:04 +08:00
JieguangZhou
a47b4d0672
[Feature][Task] Transfer files between tasks (#12552)
* add data transfer between tasks

* add delete DATA_TRANSFER API

* convert Result to DeleteDataTransferResponse

* add api UT

* fix final

* fix doc
2022-11-03 16:33:39 +08:00
Aaron Wang
08335b1032
[Improvement][Task] Improved way to collect yarn job's appIds (#12197)
* Provide aop way as an optional way to collect yarn job's applicationId, and import new module `dolphinscheduler-aop` to place the aop code.
* Add user property `appId.collect` for user to decide how to collect applicationId.
* Add new environment configuration for each type of yarn tasks to support aop in `dolphinscheduler_env.sh`
* Update docs to declare how to use aop way.
* Update `LogUtils` to support fetch applicationId in different ways based on the user property.

Co-authored-by: gabrywu <gabrywu@apache.com>
2022-10-31 16:52:53 +08:00
fuchanghai
77e29b59b2
[Improvement] add springdoc-openapi-ui (#12379)
Co-authored-by: fuchanghai <‘2875334588@qq.com’>
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
2022-10-23 23:17:53 +08:00
Eric Gao
9ab79e064c
[Improvement][Test] Fully remove the usage of powermock from the whole project (#12244)
* Fully remove the usage of powermock from the whole project

* Upgrade org.reflections to 0.10.12
2022-10-10 10:43:38 +08:00
Tq
c7435a51ac
[Feature][task-plugins] aws datasync task plugins (#11865)
* submit datasync task

* add aws software package license
2022-10-09 10:47:01 +08:00
JieguangZhou
308f632fcc
[Feature][TASK-PLUGIN] AWS Database Migration Service (DMS) (#11868)
* add aws dms
2022-09-30 10:16:16 +08:00
Tq
cf5a8894e2
[BUG][BE-12113]fix htrace dependency conflicts (#12126)
* fix htrace dependency conflicts

* replace htrace-core with hbase-noop-htrace
2022-09-26 13:49:08 +08:00
Eric Gao
66958b91d7
[Feature][Resource Center] Add support for Alibaba Cloud OSS as storage of resource center (#11708)
* Add support for Alibaba Cloud OSS as storage of resource center

* Add dependency licenses

* Refactor code, add unit tests and related docs
2022-09-20 22:44:06 +08:00
kezhenxu94
147e0ae8d7
Remove duplicate Spring version management (#11899) 2022-09-11 20:50:07 +08:00
kezhenxu94
105773f54d
Remove htrace dependencies (#11881)
* Remove htrace dependencies

* Update pom.xml
2022-09-10 22:54:38 +08:00
Kengo Seki
3664d85143
[Improve] Upgrade Hadoop to 3.2.4 (#11832) 2022-09-09 10:47:03 +08:00
kezhenxu94
efedc9a53c
Bump up dependencies to fix cves (#11765) 2022-09-08 09:05:05 +08:00
wjf
2eb8b9f7f0
[Feature][Registry] Support etcd as registry (#10981) 2022-09-06 22:25:19 +08:00
fuchanghai
554562bfa9
[fix-#11691] Add lz4-java in clickhouse datasource. (#11697) 2022-09-05 09:15:58 +08:00
kezhenxu94
277f137358
Add Kubernetes configmap reload to all components (#11730) 2022-09-02 12:03:19 +08:00
kezhenxu94
17a9dd25fa
Fix vulnerability in LDAP login (#11586) 2022-08-22 13:49:14 +08:00
kezhenxu94
13e4720aa2
Upgrade spring boot and swagger (#11574) 2022-08-22 10:27:40 +08:00
kezhenxu94
f6ef628030
Support reading application properties from kubernetes config map (#11402) 2022-08-16 20:25:17 +08:00
Wenjun Ruan
c7789bf049
Add base loop task execotor and http template parser (#11137)
* Add base loop task execotor and http template parser

* Add JsonPathUtils
2022-07-26 14:51:50 +08:00
JieguangZhou
af7b359333
[Feature][Task Plugin] Add SageMaker Pipeline task plugin for MLOps scenario (#10826) (#10935)
* add sagemaker pipeline task plugin

[DOC] add Sagemaker task plugin document

change license and fix dependencies

* [fix] Optimize the code

* Update dolphinscheduler-task-plugin/dolphinscheduler-task-sagemaker/src/main/java/org/apache/dolphinscheduler/plugin/task/sagemaker/SagemakerParameters.java

Optimization parameter judgment

Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>

* [fix] nips

Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
2022-07-21 18:16:58 +08:00