Commit Graph

124 Commits

Author SHA1 Message Date
xiangzihao
d67ea805f9
[DSIP-38] Add plugin directory in binary package (#16613) 2024-09-23 09:59:34 +08:00
Wenjun Ruan
9448806af0
[DSIP-61][Master] Refactor thread pool and state event orchestration in master (#16327) 2024-08-26 15:19:39 +08:00
xiangzihao
66548a2ef2
[DSIP-62][Http Alert Plugin] Refactor http alert plugin (#16484)
* add dsip_62

* fix sonar

* fix spotless
2024-08-19 19:09:05 +08:00
Eric Gao
f7da8f39db
[Feature-16127] Support emr serverless spark (#16126)
* support emr serverless spark task

Signed-off-by: EricGao888 <ericgao.apache@gmail.com>

---------

Signed-off-by: EricGao888 <ericgao.apache@gmail.com>
2024-08-02 12:10:32 +08:00
xxsc0529
1199a916b3
[Feature-16132][Task Plugin] support for oceanbase datax task (#16281)
* Add support for oceanbase's datax task --feat

* fix code style

---------

Co-authored-by: xiangzihao <460888207@qq.com>
2024-07-16 09:27:05 +08:00
Wenjun Ruan
46eff34e69
[Fix-16224] Add Shell using file E2E case (#16220)
* Add Shell E2E case

* Add shell using resource file e2e case

* Upgrade checkout from v2 to v4

* Change interval to 500ms

* Upgrade actions/upload-artifact

---------

Co-authored-by: xiangzihao <460888207@qq.com>
2024-06-27 16:04:03 +08:00
xiangzihao
e8566818b0
[DSIP-50][UDF Manage] Remove unused udf manage function (#16212)
* Remove unused udf manage function
2024-06-26 21:08:48 +08:00
Wenjun Ruan
ec6a3e0727
[DSIP-46] Unify the logic task params data structure between front-end and back-end (#16193) 2024-06-25 13:32:13 +08:00
Wenjun Ruan
edeb1b2458
[DSIP-45] Polish the Storage SPI (#16141) 2024-06-17 15:24:56 +08:00
小可耐
7a55adeae9
[Improvement-15919][datasource] Improvement datasource get name (#15920) 2024-04-28 11:02:44 +08:00
Jay Chung
8efaa9fa1f
fix: data quality can not use (#15551)
* fix: data quality can not use

fix: #15468, #15249, #14858

---------

Co-authored-by: Rick Cheng <rickchengx@gmail.com>
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2024-02-05 09:54:06 +08:00
chenrj
c532fea00e
[Improvement] Refactoring K8S task plugin with connections managed in connection center (#14977) 2023-10-31 17:07:24 +08:00
chenrj
5a3827eef4
[DSIP-19] Support sagemaker connections in the connection center, as well as external connections to the connection center in sagemaker tasks (#14976)
* Refactoring `Sagemaker` task plugin with connections managed in connection center.

---------

Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2023-10-31 10:07:05 +08:00
chenrj
e99c5950b9
[DSIP-19] Support zeppelin connections in the connection center, as well as external connections to the connection center in zeppelin tasks (#14434)
* Refactoring zeppelin task plugin with connections managed in connection center

---------

Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2023-10-30 16:44:54 +08:00
Wenjun Ruan
4aab0b234f
Use AdHoc datasource client in sqlTask (#14631)
* Use AdHoc datasource client in sqlTask

* Add method in DataSourceChannel to create PooledDataSourceClient and AdHocDataSourceClient
2023-07-26 08:58:50 +08:00
Xinxing
1d0d85b988
[Feature] doris add datasource (#14391)
* Added doris data source plug-in

* Update DbType.java

* XML configuration

* doc and ui

* Missing catch of NumberFormatException

* code format

---------

Co-authored-by: xiangzihao <460888207@qq.com>
2023-07-20 17:15:22 +08:00
c3Vu
50195c46aa
[Improvement-11913] Mask password when creating/editing alert instances (#14415)
* [Improvement-11913] Mask password when creating/editing alert instances
2023-06-28 14:10:51 +08:00
IT-Kwj
7a24be0672
[Feature-14404][datasource-plugin] add HANA datasource (#14404)
---------

Co-authored-by: WeiJian Kong <>
Co-authored-by: xujiaqiang <Xjq634828243>
2023-06-28 10:56:41 +08:00
Zinway
18d176c5f4
feat: Introduce new datasource vertica (#14232)
Co-authored-by: Zinway Liu <zinway.liu@ucloud.cn>
2023-06-02 13:57:37 +08:00
He Zhao
f1e78627fc
[feature][plugin] Add snowflake datasource in datasource plugin (#13729)
---------

Co-authored-by: weitong <weitong@cisco.com>
2023-05-06 19:30:50 +08:00
Jeremy
db6c061c8b
[Feature][plugin] Add databend datasource in datasource plugin (#13866)
Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>
2023-04-24 21:28:14 +08:00
He Zhao
c0126b7f30
[Feature][plugin] Support Kyuubi datasource (#13642)
---------

Co-authored-by: weitong <weitong@cisco.com>
Co-authored-by: Rick Cheng <38122586+rickchengx@users.noreply.github.com>
2023-04-18 11:12:39 +08:00
Aaron Wang
4bf097ee40
[Improvement-13824][Resource Center] User can only visit resources under tenant defaultPath (#13848)
* [Improvement-13824][Resource Center] User can only visit resources under tenant defaultPath

* remove semicolon

* set null in some specific scenario

* remove useless console.log
2023-04-17 11:56:42 +08:00
JieguangZhou
273a5e78eb
[FEATURE][Task Plugin]Add remote-shell task plugin (#13801) 2023-03-30 20:17:28 +08:00
Jay Chung
6a33d2bec0
fix: Alarm instance port verification (#13827)
close: #13532
2023-03-30 18:57:47 +08:00
He Wang
35d8e8ac17
[Feature][DataSource] add oceanbase datasource (#13675)
Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>
2023-03-07 10:12:58 +08:00
lenian
bcf03ad4d1
[Feature-9467] add DAMENG DataSource (#12860)
* [Feature-9467][DataSource] add DAMENG (达梦) data source

* [Feature-9467] add DM datasource doc (Issue #9467)

* [Feature-9467] add DM IDataBaseOptionKeys

* [Feature-9467] add DM DataSource: DmDataSourceChannelFactoryTest remove testGetDmConnection method

* [Feature-9467] modification package dm to dameng;

* [Feature-9467] add Dameng DmJdbcDriver18 License

* [Feature-9467] add DM DataSource: remove DruidDataSourceClient

Co-authored-by: lenian <yuenian@mananacare.com>
2023-01-04 15:00:24 +08:00
Tq
f0dd4f081f
[Feature][task-plugins-12820] azure azure-sql datasource plugins (#13209)
* submit azure sql code

* change mssql-jdbc version

* merge changes to delete other

* merge changes to delete other

* add dependencies
2022-12-19 10:27:03 +08:00
Wenjun Ruan
5c75d312fa
Add StarRocks datasource (#13204) 2022-12-17 08:44:39 +08:00
Wenjun Ruan
ab96a3d0c0
Fix datasource jdbc url param doesn't work (#13098) 2022-12-12 11:46:40 +08:00
Bharati Kulkarni
2b2adecd85
[Improvement] Fixed Possible JSON ordering permutations problem in Tests (#13118)
* Fixed flaky test

* Fixed imports order
2022-12-08 13:40:08 +08:00
insist777
b40c63f02d
[Feature][DataSource]Add trino datasource plugin (#12911) 2022-11-23 10:09:02 +08:00
caishunfeng
dedff70f90
[Improvement] Merge spi.utils into common.utils (#12502)
* improve JSONUtil

* merge spi.utils into common.utils

* split constants && remove some copy utils method

* remove StringUtils
2022-10-23 21:23:54 +08:00
Eric Gao
d46e4659d9
[Improvement][Style] Fix remaining style errors and wildcard imports (#12412)
* Fix remaining style errors and wildcard imports

* Upgrade spotless to the latest stable version, exclude .github folder from spotless check and fix dead link

* Fix spotless matching scope of markdown files
2022-10-19 11:08:04 +08:00
Yann Ann
b357781df2
[Migrate][Test] Migrate junit4 -> junit5 test cases in common, service, and spi modules (#12363) 2022-10-14 11:06:29 +08:00
kezhenxu94
db1d319159
Reorganize some classes in common module, remove duplicate classes (#12321) 2022-10-13 10:23:27 +08:00
Eric Gao
9ab79e064c
[Improvement][Test] Fully remove the usage of powermock from the whole project (#12244)
* Fully remove the usage of powermock from the whole project

* Upgrade org.reflections to 0.10.12
2022-10-10 10:43:38 +08:00
Yann Ann
c87f6d89a4
[fix#12195] Change date to timestamp to support cross diff time zones (#12239)
* fix DS support cross time zone, use timestamp replace date
2022-10-01 12:50:42 +08:00
zhuxt2015
e6832220c3
[Improvement][SQL Task]use default sql segment separator (#10869) 2022-09-26 11:08:57 +08:00
Eric Gao
66958b91d7
[Feature][Resource Center] Add support for Alibaba Cloud OSS as storage of resource center (#11708)
* Add support for Alibaba Cloud OSS as storage of resource center

* Add dependency licenses

* Refactor code, add unit tests and related docs
2022-09-20 22:44:06 +08:00
ZhaoGuodong
86291bb5e6
[Feature-10273][Datasource] Add dataSource for AWS Athena (#11157) 2022-09-16 09:59:20 +08:00
Wenjun Ruan
b96d69701a
[Improvement-#11613] Add spi priority factory (#11614)
* Add spi priority factory

* Add doc

* Add override log

* Use lombok

* Add comment
2022-08-24 19:51:47 +08:00
Wenjun Ruan
71cf7e1baa
[Fix-10918] Close datasource when expire from guava cache (#11120)
* Close datasource when expire from guava cache
* Remove duplicate datasource in HiveDataSourceClient
2022-07-23 20:43:37 +08:00
Wenjun Ruan
5e9c7dad23
Add dolphinscheduler-bom to manage the dependency version (#11025) 2022-07-20 10:37:31 +08:00
Eric Gao
71f0168510
[Feature] Enable users to create python env from requirements.txt (#10658) 2022-07-06 15:56:39 +08:00
HomminLee
dafed572d1
[Fix][dolphinscheduler-spi] json parse support LocalDateTime (#10440)
* JSONUtils support LocalDateTime

* remove extraneous unit tests

* remove unused classes import.

Co-authored-by: HomminLee <lihongmin@ysstech.com>
2022-06-27 10:19:28 +08:00
kezhenxu94
d80cf21456
Clean up unused dependencies and packaging issues (#9944) 2022-05-31 15:22:41 +08:00
Tq
592e433b1e
[Feature][Datasource]replace DataSourceController API with string JSON and cast to DTO (#10276)
* replace DataSourceController API with string JSON and cast to DTO

* add DataSourceProcessorManager and DataSourceProcessorProvider to datasource processor functions
2022-05-30 14:12:19 +08:00
WangJPLeo
31cd1b5e61
Serial wait for subsequent execution (#9847) 2022-04-29 15:44:05 +08:00
Eric Gao
fd6b43bc81
[Dev] Switch version in pom.xml to dev-SNAPSHOT (#9223) (#9299) 2022-04-07 18:15:19 +08:00