test/DolphinScheduler
1
0
Fork 0
mirror of https://gitee.com/dolphinscheduler/DolphinScheduler.git synced 2024-12-02 04:08:31 +08:00
Code Issues Actions 10 Packages Projects Releases Wiki Activity
8,375 Commits 27 Branches 59 Tags 276 MiB
7acb0f141f
Commit Graph

119 Commits

Author SHA1 Message Date
wangxj3
7acb0f141f [maven-release-plugin] prepare release 3.2.2 2024-06-29 20:13:11 +08:00
wangxj3
0c917dec1e [maven-release-plugin] prepare for next development iteration 2024-06-28 15:46:10 +08:00
wangxj3
463d687119 [maven-release-plugin] prepare release 3.2.2 2024-06-28 15:42:21 +08:00
wangxj3
2b56d3ca44 [maven-release-plugin] prepare for next development iteration 2024-06-27 19:12:54 +08:00
wangxj3
8f578f62ee [maven-release-plugin] prepare release 3.2.2 2024-06-27 18:54:40 +08:00
wangxj3
eecef8a3e9 [maven-release-plugin] prepare for next development iteration 2024-06-26 18:11:14 +08:00
wangxj3
d7ad8437ac [maven-release-plugin] prepare release 3.2.2 2024-06-26 17:39:24 +08:00
xiangzihao
fe96907fbb
[Chore] Revert some incompatible change for 3.2.2 (#16214) 
* revert pr 14977

* fix spotless

* revert pr 14977

* revert pr 14977
 2024-06-26 13:17:51 +08:00
xiangzihao
16bc1e7bfb
Revert "3.2.2 release add task-pure delete some task module to reduce the si…" (#16184) 
This reverts commit 3ad53c5200.
 2024-06-20 13:44:55 +08:00
wangxj3
3ad53c5200
3.2.2 release add task-pure delete some task module to reduce the size of the deployment package (#16094) 
* add task-pure delete some task module to reduce the size of the deployment package

* add version of 3.2.2

* Update Chart.yaml Rollback APPversion

* remove task pure plugin

* fix ci

* add pure datasource plugin

* remove ck，sr from datasource pure

* remove task plugin chunjun ，dms ，datafactory from pure

* add api pom dependency

* spotless:apply

* add dependency license

* [maven-release-plugin] prepare release 3.2.2

* [maven-release-plugin] prepare for next development iteration

* remove datasource-pure：
dolphinscheduler-datasource-sqlserver
dolphinscheduler-datasource-oracle

remove task-pure：
dolphinscheduler-task-dinky
dolphinscheduler-task-pytorch
dolphinscheduler-task-datasync
dolphinscheduler-task-kubeflow
dolphinscheduler-task-emr
dolphinscheduler-task-pigeon
dolphinscheduler-task-flink-stream

* add register pure remove jdbc and etcd

* add dolphinscheduler-storage-pure remove task of
dolphinscheduler-storage-obs
dolphinscheduler-storage-abs
dolphinscheduler-storage-gcs
dolphinscheduler-storage-oss

* mvn spotless:apply

---------

Co-authored-by: wangxj959 <wangxj959@chinaunicom.cn>
Co-authored-by: wangxj <wangxj31>
Co-authored-by: wangxj3 <wangxj@apache.org>
 2024-06-08 17:05:20 +08:00
Wenjun Ruan
e91a272965
Fix SqlServer split merge sql will miss ; (#16118) (#16120) 
(cherry picked from commit d6714bb77a)
 2024-06-06 17:11:51 +08:00
Wenjun Ruan
61915a2d5d
Fix workflow instance restart failed due to duplicate key in varpool (#16001) 2024-05-16 11:32:27 +08:00
小可耐
7a55adeae9
[Improvement-15919][datasource] Improvement datasource get name (#15920) 2024-04-28 11:02:44 +08:00
Wenjun Ruan
9437d276e7
Change ssh heartbeat type to IGNORE (#15858) 2024-04-16 22:49:11 +08:00
XinXing
5ad7b1509f
[Fix-15787] Reuse code and solve the problem of complex SQL parsing exceptions in… (#15833) 
* Reuse code and solve the problem of complex SQL parsing exceptions in druid, corresponding to issue #15787

* Code Format

* Enhanced adaptability to SQL formatting
 2024-04-15 15:01:28 +08:00
songwenyong
5d8808dda4
[Fix-15760][datasource-plugin] fix sql task split error (#15760) (#15794) 
* Fix the bug in SQL splitting by completing the task in two steps: 1. removeComment 2. split

* Add a unit test for Hive SQL splitting.
 2024-04-03 16:31:03 +08:00
zuo
e498116025
Fix Hive jdbc url (#15627) 2024-02-26 14:15:32 +08:00
Jay Chung
91d56f4860
fix: data quality may fail in docker mode (#15563) 2024-02-05 17:14:59 +08:00
Jay Chung
8efaa9fa1f
fix: data quality can not use (#15551) 
* fix: data quality can not use

fix: #15468, #15249, #14858

---------

Co-authored-by: Rick Cheng <rickchengx@gmail.com>
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
 2024-02-05 09:54:06 +08:00
AliceXiaoLu
cb5317f7ca
[bug] Exception when using host in ipv6 format (#14040) 
* DB2 edit again, no username.

* [bug] db2's jdbcUrl should be like this "jdbc:db2://localhost:50000/sample:currentSchema=DB2INST1;characterEncoding=UTF-8;".should be ":"

* Update Db2DataSourceProcessor.java

* Update Db2DataSourceProcessorTest.java

* 【Bug】OracleSource edit again, ConnectionType is null

* when user inputs ipv6,checkhost will return "!IPV4_PATTERN.matcher(host).matches()";
I think we should use isInetAddress to check the host is isValid

* Update AbstractDataSourceProcessor.java

* Update DatabendDataSourceProcessorTest.java

last week other modified this

* Update MySQLDataSourceProcessorTest.java

* Update DataSourceUtilsTest.java

* Update MySQLDataSourceProcessorTest.java

* Update KyuubiDataSourceProcessorTest.java

* Update AbstractDataSourceProcessor.java

if host is ipv6 or ipv4 will OK; if not check other

* Update AbstractDataSourceProcessor.java

* Update AbstractDataSourceProcessor.java

* Update DataSourceUtilsTest.java

增加ipv6数据源验证

* [Bug] [dolphinscheduler-api] queryDataSource'password should be hidden.

---------

Co-authored-by: xiangzihao <460888207@qq.com>
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
 2024-01-15 09:33:08 +08:00
lgcareer
309c8c97a9
[Enhancement][API]Enhance mysql connection properties (#15433) 2024-01-06 19:25:08 +08:00
Wenjun Ruan
bf2cdc5300
Use Druid to split sql (#15367) 2024-01-02 12:00:53 +08:00
Wenjun Ruan
921253109a
[Bug] Close SSH session after remote shell finish (#15348) 
* Close SSH session after remote shell finish

---------

Co-authored-by: David Zollo <davidzollo365@gmail.com>
 2023-12-25 13:04:45 +08:00
Wenjun Ruan
b73194bd35
Fix check value rather than key in AbstractDataSourceProcessor#checkOther (#15351) 2023-12-25 10:00:07 +08:00
xujiaqiang
0feb144180
[Improvement-15260][dolphinscheduler-datasource-hana] add hana related dependencies (#15260) 
* add hana related dependencies

* optimizing HANA data source bugs

* run mvn spotless:apply

* Fix hana datasource getValidationQuery()

* Fix hana datasource testGetJdbcUrl()

---------

Co-authored-by: xujiaqiang <“xujiaqiangwz@163.com”>
Co-authored-by: David Zollo <dailidong66@gmail.com>
 2023-12-12 17:11:27 +08:00
Xinxing
5e135ba041
fix: execute sql error: datasource plugin 'doris' is not found。 (#15123) 
Co-authored-by: xiangzihao <460888207@qq.com>
 2023-11-07 14:37:07 +08:00
izualzhy
8665951981
[fix-#11726] fix error when set connection proerty both in the URL and an argument (#15093) 
Co-authored-by: zhangying14 <zhangying14@zuoyebang.com>
Co-authored-by: xiangzihao <460888207@qq.com>
 2023-11-03 14:44:27 +08:00
chenrj
c532fea00e
[Improvement] Refactoring K8S task plugin with connections managed in connection center (#14977) 2023-10-31 17:07:24 +08:00
chenrj
5a3827eef4
[DSIP-19] Support sagemaker connections in the connection center, as well as external connections to the connection center in sagemaker tasks (#14976) 
* Refactoring `Sagemaker` task plugin with connections managed in connection center.

---------

Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
 2023-10-31 10:07:05 +08:00
chenrj
e99c5950b9
[DSIP-19] Support zeppelin connections in the connection center, as well as external connections to the connection center in zeppelin tasks (#14434) 
* Refactoring zeppelin task plugin with connections managed in connection center

---------

Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
 2023-10-30 16:44:54 +08:00
Wenjun Ruan
bc4326125d
Fix session/thread leak at ssh datasource (#14859) 2023-09-07 11:55:31 +08:00
Wenjun Ruan
9ce8871158
Support get property from env/jvm/common.properties (#14645) 2023-07-28 14:41:58 +08:00
Wenjun Ruan
4aab0b234f
Use AdHoc datasource client in sqlTask (#14631) 
* Use AdHoc datasource client in sqlTask

* Add method in DataSourceChannel to create PooledDataSourceClient and AdHocDataSourceClient
 2023-07-26 08:58:50 +08:00
Wenjun Ruan
6617e3fb72
Revert "datasource test and sql task Remove connection pool issues is #14179 (#14193)" (#14626) 
This reverts commit e4fb5b30a4.
 2023-07-24 17:33:43 +08:00
xuhui1231
e4fb5b30a4
datasource test and sql task Remove connection pool issues is #14179 (#14193) 
* datasource test and sql task Remove connection pool issues is #14179

* datasource test and sql task Remove connection pool issues is #14179 uniform style

* datasource test and sql task Remove connection pool issues is #14179 uniform style by 20230720

* datasource test and sql task Remove connection pool issues is #14179 uniform style by 20230720

---------

Co-authored-by: xuhui <xuhui@3h1china.com>
Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>
 2023-07-21 16:14:21 +08:00
Xinxing
1d0d85b988
[Feature] doris add datasource (#14391) 
* Added doris data source plug-in

* Update DbType.java

* XML configuration

* doc and ui

* Missing catch of NumberFormatException

* code format

---------

Co-authored-by: xiangzihao <460888207@qq.com>
 2023-07-20 17:15:22 +08:00
Rick Cheng
68a3a75211
[Improvement-14457][UT] Improve the expected and actual value of Assertions (#14459) 2023-07-06 13:12:48 +08:00
IT-Kwj
7a24be0672
[Feature-14404][datasource-plugin] add HANA datasource (#14404) 
---------

Co-authored-by: WeiJian Kong <>
Co-authored-by: xujiaqiang <Xjq634828243>
 2023-06-28 10:56:41 +08:00
Jay Chung
221df4a278
revert: Datasource need test binding to create prod (#14381) 
We should keep our step as less as possible, for now, we have to
test datasource and binding it to the prod datasource, I think it will
make our users do not thing to use datasource, so I do like to revert
it and keep step as less as possible

related to https://github.com/apache/dolphinscheduler/pull/11670
 2023-06-27 10:59:17 +08:00
Zinway
18d176c5f4
feat: Introduce new datasource vertica (#14232) 
Co-authored-by: Zinway Liu <zinway.liu@ucloud.cn>
 2023-06-02 13:57:37 +08:00
He Zhao
f1e78627fc
[feature][plugin] Add snowflake datasource in datasource plugin (#13729) 
---------

Co-authored-by: weitong <weitong@cisco.com>
 2023-05-06 19:30:50 +08:00
Jeremy
db6c061c8b
[Feature][plugin] Add databend datasource in datasource plugin (#13866) 
Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>
 2023-04-24 21:28:14 +08:00
AliceXiaoLu
dcf0109db3
【Bug】 OracleSouce edited again, ConnectionType is null (#13881) 2023-04-19 15:57:29 +08:00
He Zhao
c0126b7f30
[Feature][plugin] Support Kyuubi datasource (#13642) 
---------

Co-authored-by: weitong <weitong@cisco.com>
Co-authored-by: Rick Cheng <38122586+rickchengx@users.noreply.github.com>
 2023-04-18 11:12:39 +08:00
JieguangZhou
273a5e78eb
[FEATURE][Task Plugin]Add remote-shell task plugin (#13801) 2023-03-30 20:17:28 +08:00
AliceXiaoLu
ad7ce2db56
fix: DB2 username can not be save (#13764) 
db2's jdbcUrl should be like this "jdbc:db2://localhost:50000/sample:currentSchema=DB2INST1;characterEncoding=UTF-8;".should be ":"
 2023-03-30 18:59:21 +08:00
Jrebel-i
3630b00c9f
Fix Bug [Athena Datasource] datasource other params illegal (#13032) 2023-03-30 16:15:29 +08:00
He Wang
35d8e8ac17
[Feature][DataSource] add oceanbase datasource (#13675) 
Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>
 2023-03-07 10:12:58 +08:00
seedscoder
8d12dc0702
[Improvement-13491] Use lombok @Slf4j annotation to generate logger (#13509) 2023-02-07 20:32:53 +08:00
Wenjun Ruan
b11431db95
Use single thread to refresh kerberos (#13456) 2023-02-01 20:41:05 +08:00