test/DolphinScheduler
1
0
Fork 0
mirror of https://gitee.com/dolphinscheduler/DolphinScheduler.git synced 2024-12-09 07:37:58 +08:00
Code Issues Actions 10 Packages Projects Releases Wiki Activity
5,400 Commits 27 Branches 59 Tags 276 MiB
98f466e09e
Commit Graph

694 Commits

Author SHA1 Message Date
mask
0aea5ae11a
[feature-6071][plugin] add datasource plugin. (#6071) (#6361) 
* [feature-6071][plugin] add datasource plugin. (#6071)

* remove the dependency of datasource in common and task-plugin module.

* the new datasource-api module provides data source connection acquisition.

* add sqlserver data source plug-in.

* add ds license

* fix check style

* update spi pom

* frx conflict resolution

* fix check style

* fix datasource ut

* fix code check style

* fix code check style

* fix datasource ut

* add pg hive mysql oracle clickhouse db2  datasource

* fix datasource package

* update datasource pom

* update datasource pom

* update datasource pom

* update datasource pom

* update pom

* update pom dep

* update pom dep

* fix license

* fix ut

* fix ut

* fix ut

* fix ut

* fix ut

* fix ut

* fix ut

* fix ut

* fix ut

* fix ut

* fix ut

* fix ut

* fix ut

* add oracle package

* modify comments

* conflict resolution

* rename log comment

* rename log comment
 2021-11-09 20:20:07 +08:00
kezhenxu94
05bae0b226
Refactor alert plugins and simplify its usage (#6723) 2021-11-09 18:59:34 +08:00
kezhenxu94
02350ea37b
Fix datasource not work in tests and standalone server, and run all tests by default (#6743) 2021-11-09 16:57:26 +08:00
kezhenxu94
63602bf77c
Clean up the utility codes (#6732) 2021-11-08 23:22:07 +08:00
wind
b6824b4741
[DS-6737][MasterServer] fix event handle twice (#6738) 
Co-authored-by: caishunfeng <534328519@qq.com>
 2021-11-08 16:54:58 +08:00
OS
24bd8a16e1
[Bug][Master] The task of dependent should use taskCode (#6730) 
* fix #6682 The task of dependent should use taskCode

* fix #6682 The task of dependent should use taskCode
 2021-11-07 23:20:35 +08:00
Wenjun Ruan
0afb780718
[Improvement-5807][Dao] Use h2 in UT (#6500) 
* Use H2 in UT

* change scope

* Change profile name to h2

* Add datasource-h2.properties, datasource-postgresql.properties

* Fix UT

* Resolve conflict

* Remove startDatabase
 2021-11-07 23:19:19 +08:00
kezhenxu94
5400abc74f
Refactor registry plugin and simplify its usage (#6712) 2021-11-07 15:18:10 +08:00
Yao WANG
73f20b7553
[Fix-#6641]fix npe when the LowerWeightHostManager can not get the workNodeInfo (#6656) 
* change the return type of getHostWeight

Co-authored-by: ywang46 <ywang46@paypal.com>
 2021-11-07 11:40:54 +08:00
wind
72daba58cd
[DS-6694][Master] Reduce the refresh resource interval of LowerWeight (#6695) 
* [DS-6694][Master] Reduce the refresh resource interval of LowerWeight

* add server node update event handle

Co-authored-by: caishunfeng <534328519@qq.com>
 2021-11-05 19:10:36 +08:00
wangxj3
dc8b87e473
[Feature-#6268][server-master] Serial execte process (#6185) 
* add serial processInstance

* fix bug

* add test

* fix code style

* fix style code

* add sql

* fix sql error

* add api

* add test

* fix code style

* modify api

* delete column , fix mapper

* fix unimport

* fix test

* fix bug of missing param for Python

* fix code style

* fix test

* fix test

Co-authored-by: wangxj <wangxj31>
 2021-11-05 17:25:45 +08:00
xuhhui
2b90d90873
improve install_config.conf description (#6681) 
* improve install_config.conf description

* Update install_config.conf

update accroding review ideas
 2021-11-05 15:13:23 +08:00
wind
ae0b18f3a3
[DS-6638][MasterServer] fix task state no change when failover worker (#6639) 
Co-authored-by: caishunfeng <534328519@qq.com>
 2021-10-31 21:12:55 +08:00
wind
8850baff07
[DS-6640][WorkerServer] support process update host command type (#6642) 
Co-authored-by: caishunfeng <534328519@qq.com>
 2021-10-31 21:10:51 +08:00
wind
06e8e24708
[DS-6616][WorkerServer] fix worker stop fail and fakes death (#6621) 
* [DS-6616][WorkerServer] fix worker stop fail and fakes death

* remove unuse test

* just add try catch to remove zk worker path

Co-authored-by: caishunfeng <534328519@qq.com>
 2021-10-29 17:44:06 +08:00
wind
b642e9ca54
[DS-6582][Master] fix condition task state (#6583) 
Co-authored-by: caishunfeng <534328519@qq.com>
 2021-10-21 19:27:13 +08:00
mask
9b1984c244
[fix-6570][task] add sql task udf resource load (#6570) (#6571) 
* [fix-6570][task] add sql task udf resource load (#6570)

-add sql task udf resource load
-Fix UdfFuncRequest serialization

* fix checkstyle

* add defaultFS in sqlTaskContext

* fix checkstyle
 2021-10-21 15:23:49 +08:00
ououtt
a3982f5025
[Improvement-6540][server] Optimize the code related to the processInstanceExecMaps object (#6556) 
* [DS-6540][refactor-server] Optimize the code related to the processInstanceExecMaps object
 2021-10-20 14:55:23 +08:00
Kirs
de61d65af2
[Task]fix business log using error (#6538) 2021-10-15 20:27:24 +08:00
OS
52a550b6ae
[BUG-6543][Master] process instance state is always running when failure task (#6547) 
* fix-6543: process instance state is always running when failure task exists

* code style

* code style
 2021-10-15 18:04:04 +08:00
Jiajie Zhong
6495a204f3
Fix install.sh for DS 2.0 and add comment to install_config.conf (#6506) 
* Fix install.sh for DS 2.0

* Use #* make it work in macos

* Add comment for properties change

* Add JAVA HOME for only one file change to deploy

* Reduce config change by using default value of: plugin registry, alert, task as

* Change javaHome default value

* Delete config param zkQuorum

* Reorder param

* Add comment for install_config.conf
 2021-10-14 11:13:05 +08:00
wind
50f1766903
[DS-6499][WorkerServer] report busy state when worker is overload (#6512) 
* [DS-6499][WorkerServer] report busy state when worker is overload
 2021-10-13 11:45:56 +08:00
OS
db04a5b04d
[Feature-6471]Cache Process definition in master (#6485) 
* feature-6471 Cache Process definition in master
 2021-10-11 20:22:58 +08:00
wind
e091801e05
[Improvement-6474] [MasterServer] schedule time for process instance optimization (#6477) 
* [DS-6474][MasterServer] change to handle schedule time for process instance in WorkflowExecuteThread

* delete all the valid tasks when complement data if id is not null

* checkstyle

Co-authored-by: caishunfeng <534328519@qq.com>
 2021-10-11 15:13:38 +08:00
OS
a8baa9553f
[Bug-6455][Master]fix bug 6455: cannot stop sub-task (#6458) 
* fix bug: cannot stop the task.

* fix bug: cannot stop the task.

* remove the check thread number
 2021-10-08 14:08:12 +08:00
Hua Jiang
fac6b4afd3
[Fix-5875][API] When I saved the task that had the same name task in another flow ,the service would throw DuplicateKeyException (#6430) 
* using the task code in the dag
 2021-10-08 09:55:54 +08:00
Kirs
ea2a8d26a0
[Improvement#6438][Worker]remove meaningless DB query (#6440) 
org.apache.dolphinscheduler.server.worker.runner.TaskExecuteThread
In the process of task kill, it will query the DB to obtain the task instance object, but this judgment does not make any sense. In fact, the relevant context is already clear, so there is no need to query from DB.
 2021-10-02 13:09:00 +08:00
百岁
d284e44d52
Rename tis to pigeon from backend layer (#6437) 
* add mock getTaskLogName() for UT test pass

* make task-tis rename to task-pigeon

* rename module name 'tis' to pigeon

* rename module name 'tis' to pigeon,make sanity check pass

* remove execute console.log()
 2021-10-01 14:54:04 +08:00
Shukun Zhang
6e453de241
[Feature] workflow level task dry run (#6104) 
* feat workflow level task dry run

* feat workflow level task dry run

* feat workflow level task dry run

* feat workflow level task dry run

* feat workflow level task dry run

* feat workflow level task dry run

* feat workflow level task dry run

* feat workflow level task dry run

* feat workflow level task dry run

* feat workflow level task dry run
 2021-09-30 11:18:06 +08:00
caishunfeng
ea6503b997
[DS-6403][fix] Put DB queries ahead before submit to TaskPriorityQueue (#6406) 
* [DS-6403][fix] Put DB queries ahead before submit to TaskPriorityQueue

* [DS-6403][fix] Put DB queries ahead before submit to TaskPriorityQueue

* [DS-6403][fix] Put DB queries ahead before submit to TaskPriorityQueue

* checkstyle

Co-authored-by: caishunfeng <534328519@qq.com>
 2021-09-29 14:04:35 +08:00
RichardStark
d184ebda1e
issue-6298-added-the-task-plugin (#6299) 2021-09-29 11:36:51 +08:00
OS
c013b49e72
[Fix-6347][Master-API]fix bug #6347 the first schedule_time is error in complement data (#6389) 
* fix bug #6347  complement data errors
 2021-09-28 17:00:46 +08:00
Kirs
67508d8d1b
update project version to 2.0.0-SNAPSHOT (#6383) 2021-09-28 12:00:43 +08:00
caishunfeng
67094d0f7f
[BUG-6353][MasterServer] fix event handle slowly (#6375) 
Co-authored-by: caishunfeng <534328519@qq.com>
 2021-09-27 19:31:22 +08:00
Wenjun Ruan
ebb56b2620
fix dead loop in StateEventResponseWorker (#6281) 2021-09-26 11:57:37 +08:00
OS
20a3741b24
fix bug #6347 the first schedule_time is error in complement data (#6351) 2021-09-26 11:55:57 +08:00
jon-qj
dee191f2bb
provide the default value(task.plugin.dir) (#6340) 2021-09-24 23:40:07 +08:00
Kirs
be158328fb
[Bug #6306][Worker] task log path generated error. (#6338) 2021-09-24 19:55:11 +08:00
caishunfeng
89cc045961
[DS-6323][fix] EventExecuteService run loop without sleep which make cpu high (#6324) 
Co-authored-by: caishunfeng <534328519@qq.com>
 2021-09-24 16:19:02 +08:00
RichardStark
c855a5926c
fix-6316 (#6317) 2021-09-23 22:43:58 +08:00
Kirs
552fd17878
[Plugin][Task]Fix Yarn Task Kill Not valid (#6293) 2021-09-22 21:59:12 +08:00
OS
b71967b017
fix bug-6279 Parallel tasks would be Irregularly submitted twice (#6280) 2021-09-21 12:04:31 +08:00
RichardStark
7098855ee0
restored install.sh (#6257) 
* restored install.sh

* install.sh +x
 2021-09-21 00:53:25 +08:00
Kirs
fab4eae923
[SPI][Task]fix Task parse Err (#6244) 
* [SPI][Task]fix Task parse Err
 2021-09-17 18:17:15 +08:00
Kirs
fced3892ee
[Fix][Task] Wrong complement date (#6009) (#6186) 
* [Fix][Task] Wrong complement date (#6009)
 2021-09-17 17:04:51 +08:00
Kirs
b90152fc2e
[SPI][Task]add SQL Task (#6237) 
Add Sql Task And Fix Task Parse Err
 2021-09-17 13:10:44 +08:00
Kirs
fa0b86ac82
[Fix][Registry]server listener err (#6232) 
* [Fix][Registry]server listener err
 2021-09-16 15:08:00 +08:00
JinyLeeChina
71e2c8808b
[Fix-6194] [Server] fix master buildFlowDag error (#6195) 
* fix bug of view-tree api

* fix bug of view-tree api

* fix ut

* fix ut

* fix master buildFlowDag error

Co-authored-by: JinyLeeChina <297062848@qq.com>
 2021-09-14 11:46:47 +08:00
Kirs
7029062f4c
[Improvement][Task]Check the task plugin configuration when the worke… (#6184) 
* [Improvement][Task]Check the task plugin configuration when the worker starts
 2021-09-14 01:41:54 +08:00
Jiajie Zhong
170280fab2
Simplify constants variables (#6183) 
* Combine `alter` and `registry` properties,
  bese on https://github.com/apache/dolphinscheduler/pull/6180
* Remove duplicate variable `SLASH` and `SINGLE_SLASH`
 2021-09-13 22:03:09 +08:00