Wenjun Ruan
9448806af0
[DSIP-61][Master] Refactor thread pool and state event orchestration in master ( #16327 )
2024-08-26 15:19:39 +08:00
wangxj3
d2f857ec86
remove duplicate code of AliyunServerlessSparkDataSourceProcessor ( #16422 )
2024-08-06 14:16:48 +08:00
Eric Gao
f7da8f39db
[Feature-16127] Support emr serverless spark ( #16126 )
...
* support emr serverless spark task
Signed-off-by: EricGao888 <ericgao.apache@gmail.com>
---------
Signed-off-by: EricGao888 <ericgao.apache@gmail.com>
2024-08-02 12:10:32 +08:00
Wenjun Ruan
edeb1b2458
[DSIP-45] Polish the Storage SPI ( #16141 )
2024-06-17 15:24:56 +08:00
Wenjun Ruan
d6714bb77a
Fix SqlServer split merge sql will miss ; ( #16118 )
2024-06-06 14:10:13 +08:00
Wenjun Ruan
61915a2d5d
Fix workflow instance restart failed due to duplicate key in varpool ( #16001 )
2024-05-16 11:32:27 +08:00
小可耐
7a55adeae9
[Improvement-15919][datasource] Improvement datasource get name ( #15920 )
2024-04-28 11:02:44 +08:00
Wenjun Ruan
9437d276e7
Change ssh heartbeat type to IGNORE ( #15858 )
2024-04-16 22:49:11 +08:00
XinXing
5ad7b1509f
[Fix-15787] Reuse code and solve the problem of complex SQL parsing exceptions in… ( #15833 )
...
* Reuse code and solve the problem of complex SQL parsing exceptions in druid, corresponding to issue #15787
* Code Format
* Enhanced adaptability to SQL formatting
2024-04-15 15:01:28 +08:00
songwenyong
5d8808dda4
[Fix-15760][datasource-plugin] fix sql task split error ( #15760 ) ( #15794 )
...
* Fix the bug in SQL splitting by completing the task in two steps: 1. removeComment 2. split
* Add a unit test for Hive SQL splitting.
2024-04-03 16:31:03 +08:00
zuo
e498116025
Fix Hive jdbc url ( #15627 )
2024-02-26 14:15:32 +08:00
Jay Chung
91d56f4860
fix: data quality may fail in docker mode ( #15563 )
2024-02-05 17:14:59 +08:00
Jay Chung
8efaa9fa1f
fix: data quality can not use ( #15551 )
...
* fix: data quality can not use
fix : #15468 , #15249 , #14858
---------
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2024-02-05 09:54:06 +08:00
AliceXiaoLu
cb5317f7ca
[bug] Exception when using host in ipv6 format ( #14040 )
...
* DB2 edit again, no username.
* [bug] db2's jdbcUrl should be like this "jdbc:db2://localhost:50000/sample:currentSchema=DB2INST1;characterEncoding=UTF-8;".should be ":"
* Update Db2DataSourceProcessor.java
* Update Db2DataSourceProcessorTest.java
* 【Bug】OracleSource edit again, ConnectionType is null
* when user inputs ipv6,checkhost will return "!IPV4_PATTERN.matcher(host).matches()";
I think we should use isInetAddress to check the host is isValid
* Update AbstractDataSourceProcessor.java
* Update DatabendDataSourceProcessorTest.java
last week other modified this
* Update MySQLDataSourceProcessorTest.java
* Update DataSourceUtilsTest.java
* Update MySQLDataSourceProcessorTest.java
* Update KyuubiDataSourceProcessorTest.java
* Update AbstractDataSourceProcessor.java
if host is ipv6 or ipv4 will OK; if not check other
* Update AbstractDataSourceProcessor.java
* Update AbstractDataSourceProcessor.java
* Update DataSourceUtilsTest.java
增加ipv6数据源验证
* [Bug] [dolphinscheduler-api] queryDataSource'password should be hidden.
---------
Co-authored-by: xiangzihao <460888207@qq.com>
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-01-15 09:33:08 +08:00
lgcareer
309c8c97a9
[Enhancement][API]Enhance mysql connection properties ( #15433 )
2024-01-06 19:25:08 +08:00
Wenjun Ruan
bf2cdc5300
Use Druid to split sql ( #15367 )
2024-01-02 12:00:53 +08:00
Wenjun Ruan
921253109a
[Bug] Close SSH session after remote shell finish ( #15348 )
...
* Close SSH session after remote shell finish
---------
Co-authored-by: David Zollo <davidzollo365@gmail.com>
2023-12-25 13:04:45 +08:00
Wenjun Ruan
b73194bd35
Fix check value rather than key in AbstractDataSourceProcessor#checkOther ( #15351 )
2023-12-25 10:00:07 +08:00
xujiaqiang
0feb144180
[Improvement-15260][dolphinscheduler-datasource-hana] add hana related dependencies ( #15260 )
...
* add hana related dependencies
* optimizing HANA data source bugs
* run mvn spotless:apply
* Fix hana datasource getValidationQuery()
* Fix hana datasource testGetJdbcUrl()
---------
Co-authored-by: xujiaqiang <“xujiaqiangwz@163.com”>
Co-authored-by: David Zollo <dailidong66@gmail.com>
2023-12-12 17:11:27 +08:00
Xinxing
5e135ba041
fix: execute sql error: datasource plugin 'doris' is not found。 ( #15123 )
...
Co-authored-by: xiangzihao <460888207@qq.com>
2023-11-07 14:37:07 +08:00
izualzhy
8665951981
[fix-#11726] fix error when set connection proerty both in the URL and an argument ( #15093 )
...
Co-authored-by: zhangying14 <zhangying14@zuoyebang.com>
Co-authored-by: xiangzihao <460888207@qq.com>
2023-11-03 14:44:27 +08:00
chenrj
c532fea00e
[Improvement] Refactoring K8S task plugin with connections managed in connection center ( #14977 )
2023-10-31 17:07:24 +08:00
chenrj
5a3827eef4
[DSIP-19] Support sagemaker connections in the connection center, as well as external connections to the connection center in sagemaker tasks ( #14976 )
...
* Refactoring `Sagemaker` task plugin with connections managed in connection center.
---------
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2023-10-31 10:07:05 +08:00
chenrj
e99c5950b9
[DSIP-19] Support zeppelin connections in the connection center, as well as external connections to the connection center in zeppelin tasks ( #14434 )
...
* Refactoring zeppelin task plugin with connections managed in connection center
---------
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2023-10-30 16:44:54 +08:00
Wenjun Ruan
bc4326125d
Fix session/thread leak at ssh datasource ( #14859 )
2023-09-07 11:55:31 +08:00
Wenjun Ruan
9ce8871158
Support get property from env/jvm/common.properties ( #14645 )
2023-07-28 14:41:58 +08:00
Wenjun Ruan
4aab0b234f
Use AdHoc datasource client in sqlTask ( #14631 )
...
* Use AdHoc datasource client in sqlTask
* Add method in DataSourceChannel to create PooledDataSourceClient and AdHocDataSourceClient
2023-07-26 08:58:50 +08:00
Wenjun Ruan
6617e3fb72
Revert "datasource test and sql task Remove connection pool issues is #14179 ( #14193 )" ( #14626 )
...
This reverts commit e4fb5b30a4
2023-07-24 17:33:43 +08:00
xuhui1231
e4fb5b30a4
datasource test and sql task Remove connection pool issues is #14179 ( #14193 )
...
* datasource test and sql task Remove connection pool issues is #14179
* datasource test and sql task Remove connection pool issues is #14179 uniform style
* datasource test and sql task Remove connection pool issues is #14179 uniform style by 20230720
* datasource test and sql task Remove connection pool issues is #14179 uniform style by 20230720
---------
Co-authored-by: xuhui <xuhui@3h1china.com>
Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>
2023-07-21 16:14:21 +08:00
Xinxing
1d0d85b988
[Feature] doris add datasource ( #14391 )
...
* Added doris data source plug-in
* Update DbType.java
* XML configuration
* doc and ui
* Missing catch of NumberFormatException
* code format
---------
Co-authored-by: xiangzihao <460888207@qq.com>
2023-07-20 17:15:22 +08:00
Rick Cheng
68a3a75211
[Improvement-14457][UT] Improve the expected and actual value of Assertions ( #14459 )
2023-07-06 13:12:48 +08:00
IT-Kwj
7a24be0672
[Feature-14404][datasource-plugin] add HANA datasource ( #14404 )
...
---------
Co-authored-by: WeiJian Kong <>
Co-authored-by: xujiaqiang <Xjq634828243>
2023-06-28 10:56:41 +08:00
Jay Chung
221df4a278
revert: Datasource need test binding to create prod ( #14381 )
...
We should keep our step as less as possible, for now, we have to
test datasource and binding it to the prod datasource, I think it will
make our users do not thing to use datasource, so I do like to revert
it and keep step as less as possible
related to https://github.com/apache/dolphinscheduler/pull/11670
2023-06-27 10:59:17 +08:00
Zinway
18d176c5f4
feat: Introduce new datasource vertica ( #14232 )
...
Co-authored-by: Zinway Liu <zinway.liu@ucloud.cn>
2023-06-02 13:57:37 +08:00
He Zhao
f1e78627fc
[feature][plugin] Add snowflake datasource in datasource plugin ( #13729 )
...
---------
Co-authored-by: weitong <weitong@cisco.com>
2023-05-06 19:30:50 +08:00
Jeremy
db6c061c8b
[Feature][plugin] Add databend datasource in datasource plugin ( #13866 )
...
Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>
2023-04-24 21:28:14 +08:00
AliceXiaoLu
dcf0109db3
【Bug】 OracleSouce edited again, ConnectionType is null ( #13881 )
2023-04-19 15:57:29 +08:00
He Zhao
c0126b7f30
[Feature][plugin] Support Kyuubi datasource ( #13642 )
...
---------
Co-authored-by: weitong <weitong@cisco.com>
Co-authored-by: Rick Cheng <38122586+rickchengx@users.noreply.github.com>
2023-04-18 11:12:39 +08:00
JieguangZhou
273a5e78eb
[FEATURE][Task Plugin]Add remote-shell task plugin ( #13801 )
2023-03-30 20:17:28 +08:00
AliceXiaoLu
ad7ce2db56
fix: DB2 username can not be save ( #13764 )
...
db2's jdbcUrl should be like this "jdbc:db2://localhost:50000/sample:currentSchema=DB2INST1;characterEncoding=UTF-8;".should be ":"
2023-03-30 18:59:21 +08:00
Jrebel-i
3630b00c9f
Fix Bug [Athena Datasource] datasource other params illegal ( #13032 )
2023-03-30 16:15:29 +08:00
He Wang
35d8e8ac17
[Feature][DataSource] add oceanbase datasource ( #13675 )
...
Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>
2023-03-07 10:12:58 +08:00
seedscoder
8d12dc0702
[Improvement-13491] Use lombok @Slf4j annotation to generate logger ( #13509 )
2023-02-07 20:32:53 +08:00
Wenjun Ruan
b11431db95
Use single thread to refresh kerberos ( #13456 )
2023-02-01 20:41:05 +08:00
Tq
3a6e10397f
[feature][api-datasource]add redshift auth mode to get connection by access key ( #13379 )
...
* submit redshift auth mode
* add front end
* add doc and fix frontend
Co-authored-by: devosend <devosend@gmail.com>
2023-01-16 16:21:26 +08:00
lenian
bcf03ad4d1
[Feature-9467] add DAMENG DataSource ( #12860 )
...
* [Feature-9467][DataSource] add DAMENG （达梦） data source
* [Feature-9467] add DM datasource doc (Issue #9467 )
* [Feature-9467] add DM IDataBaseOptionKeys
* [Feature-9467] add DM DataSource: DmDataSourceChannelFactoryTest remove testGetDmConnection method
* [Feature-9467] modification package dm to dameng;
* [Feature-9467] add Dameng DmJdbcDriver18 License
* [Feature-9467] add DM DataSource: remove DruidDataSourceClient
Co-authored-by: lenian <yuenian@mananacare.com>
2023-01-04 15:00:24 +08:00
Wenjun Ruan
14ec4a2398
Remove dao module in worker ( #13242 )
2022-12-22 12:25:29 +08:00
Wenjun Ruan
b2336b0ce9
Remove datasource in procedure task ( #13198 )
2022-12-19 17:14:59 +08:00
Tq
f0dd4f081f
[Feature][task-plugins-12820] azure azure-sql datasource plugins ( #13209 )
...
* submit azure sql code
* change mssql-jdbc version
* merge changes to delete other
* merge changes to delete other
* add dependencies
2022-12-19 10:27:03 +08:00
Wenjun Ruan
5c75d312fa
Add StarRocks datasource ( #13204 )
2022-12-17 08:44:39 +08:00
