test/DolphinScheduler
1
0
Fork 0
mirror of https://gitee.com/dolphinscheduler/DolphinScheduler.git synced 2024-12-01 03:37:46 +08:00
Code Issues Actions 10 Packages Projects Releases Wiki Activity
7,586 Commits 27 Branches 59 Tags 276 MiB
52134277a3
Commit Graph

465 Commits

Author SHA1 Message Date
Wenjun Ruan
52134277a3
Fix task group cannot release when kill task (#13314) 2023-01-03 09:52:03 +08:00
Aaron Wang
8503ee0eed
[Improvement][Worker] Startup parameter should have the highest priority (#13274) 2022-12-29 11:05:00 +08:00
Wenjun Ruan
fd3afd84ad
Remove alert when delete workflow instance (#13281) 2022-12-28 11:40:12 +08:00
Wenjun Ruan
de70421a1e
Remove the log path check (#13280) 2022-12-26 22:39:08 +08:00
Wenjun Ruan
14ec4a2398
Remove dao module in worker (#13242) 2022-12-22 12:25:29 +08:00
JieguangZhou
66e20271ad
[Feature][Master] Add task caching mechanism to improve the running speed of repetitive tasks (#13194) 
* Supports task instance cache operation

* add task plugin cache

* use SHA-256 to generate key

* Update dolphinscheduler-dao/src/main/resources/sql/dolphinscheduler_mysql.sql

Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>

* Update dolphinscheduler-dao/src/main/resources/sql/dolphinscheduler_postgresql.sql

Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>

* Optimizing database Scripts

* Optimize clear cache operation

Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>
 2022-12-18 18:17:09 +08:00
JieguangZhou
e4b9b67255
Allow execute task in workflow instance (#13103) 2022-12-13 16:43:44 +08:00
Wenjun Ruan
838a0df783
Remove logic task plugin module (#13171) 2022-12-13 16:30:44 +08:00
Wenjun Ruan
f5801a0e1c
Fix task group might be rob by multiple task (#13115) 2022-12-11 22:58:21 +08:00
Yann Ann
6ef74073cc
[Improve-13001]migrate commons-collections -> commons-collections4 (#13002) 2022-12-10 23:50:19 +08:00
Wenjun Ruan
8a152aebc7
Fix task log file might not be clear (#13102) 2022-12-06 11:23:20 +08:00
Wenjun Ruan
fed1e85fb8
Fix the invalidate task instance never be deleted (#13091) 2022-12-04 18:38:54 +08:00
rickchengx
38b876733c
[Feature-10498] Mask the password in the log of sqoop task (#11589) 2022-11-24 14:54:54 +08:00
Kerwin
c916c60853
fix NPE while retry task (#12903) 2022-11-16 10:31:52 +08:00
Wenjun Ruan
d99ba29b66
Fix master cluster may loop command unbalanced (#12891) 
(cherry picked from commit 3b2b86661be76b7c1404a910c865d78b7936313d)
 2022-11-16 10:20:22 +08:00
XCL
00b18ed422
[Improve]Remove the code logger.isDebugEnabled() #5918 #10542 (#12738) 
Co-authored-by: chunlai.xu <chunlai.xu@zhaopin.com.cn>
 2022-11-07 09:58:19 +08:00
Aaron Wang
08335b1032
[Improvement][Task] Improved way to collect yarn job's appIds (#12197) 
* Provide aop way as an optional way to collect yarn job's applicationId, and import new module `dolphinscheduler-aop` to place the aop code.
* Add user property `appId.collect` for user to decide how to collect applicationId.
* Add new environment configuration for each type of yarn tasks to support aop in `dolphinscheduler_env.sh`
* Update docs to declare how to use aop way.
* Update `LogUtils` to support fetch applicationId in different ways based on the user property.

Co-authored-by: gabrywu <gabrywu@apache.com>
 2022-10-31 16:52:53 +08:00
Yann Ann
ed209bdf82
[Refactor] Migrate all workergroup-related interface functions from ProcessServiceImpl (#12493) 
* Migrate all workergroup-related interface functions from ProcessServiceImpl
 2022-10-26 19:18:32 +08:00
HanayoZz
489e7fe4e2
[Feature-10495][Resource Center] Resource Center Refactor (#12076) 
* resource center refactor - S3 services connection

Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
 2022-10-26 13:53:44 +08:00
Yann Ann
04aa125ba2
[Refactor]migrate some task dao functions from ProcessServiceImpl to dao modules. (#12505) 
* migrate some task DAO to dao modules.

* Update dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/repository/impl/ProcessInstanceMapDaoImpl.java

Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
 2022-10-26 11:51:58 +08:00
rickchengx
233bbdb09e
[Fix-12451][k8s] Read the kubeconfig from cluster conf (#12452) 2022-10-25 11:25:30 +08:00
Wenjun Ruan
a0d37fc7ae
Set tenantDir permission (#12486) 2022-10-23 21:54:01 +08:00
caishunfeng
dedff70f90
[Improvement] Merge spi.utils into common.utils (#12502) 
* improve JSONUtil

* merge spi.utils into common.utils

* split constants && remove some copy utils method

* remove StringUtils
 2022-10-23 21:23:54 +08:00
Yann Ann
b936b882bb
[Refactor] Migrate all command-related interface functions from ProcessServiceImpl (#12474) 
* migrate all command-related interface functions to CommonService
 2022-10-22 12:39:25 +08:00
fuchanghai
f0a34cddec
[improvement-#12416] Use CollectionUtils.isEmpty judge the collection isEmpty to avoid NPE (#12449) 2022-10-20 14:26:56 +08:00
Yann Ann
0eef2e3e10
[Improvement][Style] Fix code style of ProcessScheduleTask and ProcessServiceImpl (#12447) 2022-10-20 10:31:16 +08:00
Eric Gao
d46e4659d9
[Improvement][Style] Fix remaining style errors and wildcard imports (#12412) 
* Fix remaining style errors and wildcard imports

* Upgrade spotless to the latest stable version, exclude .github folder from spotless check and fix dead link

* Fix spotless matching scope of markdown files
 2022-10-19 11:08:04 +08:00
rickchengx
97edc7d652
[Fix-12356][k8s] fix the null exception when submitting k8s task plugin (#12358) 2022-10-19 09:49:56 +08:00
Kerwin
a8e23008ac
Fix timing scheduling trigger master service report to get command parameter null pointer exception (#12419) 2022-10-19 09:43:36 +08:00
houshitao
934d6d21eb
[Bug] [Service] Use MapUtils to judge globalParamsMap is empty. (#12384) 2022-10-17 14:52:45 +08:00
Eric Gao
2f37da0dbc
[Improvement][Test] Block the usage of powermock and move mockito dependencies from sub-modules to root pom (#12311) 
* move mockito dependencies from sub-modules to root pom

* Add check in CI to block the usage of powermock
 2022-10-14 12:06:51 +08:00
Yann Ann
b357781df2
[Migrate][Test] Migrate junit4 -> junit5 test cases in common, service, and spi modules (#12363) 2022-10-14 11:06:29 +08:00
fuchanghai
22103f975e
[improvement] Migrate all UT cases from jUnit4 to jUnit5 in dao module (#12328) 
* [improvement] Migrate all UT cases from jUnit4 to jUnit5 in dao module

Co-authored-by: Jackie <‘2875334588@qq.com’>
 2022-10-13 15:26:39 +08:00
kezhenxu94
db1d319159
Reorganize some classes in common module, remove duplicate classes (#12321) 2022-10-13 10:23:27 +08:00
Jiajie Zhong
0e1c8d8153
[Feature][API] New restful API task and task relation (#12116) 2022-10-13 09:41:03 +08:00
rickchengx
05099bd8f7
[Improvement-12296][Test] Close static mock (#12300) 2022-10-11 16:13:05 +08:00
Eric Gao
9ab79e064c
[Improvement][Test] Fully remove the usage of powermock from the whole project (#12244) 
* Fully remove the usage of powermock from the whole project

* Upgrade org.reflections to 0.10.12
 2022-10-10 10:43:38 +08:00
Yann Ann
c87f6d89a4
[fix#12195] Change date to timestamp to support cross diff time zones (#12239) 
* fix DS support cross time zone, use timestamp replace date
 2022-10-01 12:50:42 +08:00
caishunfeng
40090d459e
[Improvement] remove log-server and server module (#12206) 
* remove log-server module

* remove dolphinscheduler-server module
 2022-09-30 16:41:41 +08:00
Wenjun Ruan
aacabc6ec7
Remove workerGroup in registry (#12217) 2022-09-30 13:58:53 +08:00
fuchanghai
e27c79974d
[fix-#11753] send alert error alert data id (#11774) 
* [fix-#11753] send alert error alert data id

Co-authored-by: fuchanghai <changhai.fu@marketingforce.com>
 2022-09-29 15:34:40 +08:00
Eric Gao
d1e409e9a2
[Improvement][Test] Remove powermock in service and server modules (#12164) 
* Remove powermock and refactor some related code in dolphinscheduler-service and dolphinscheduler-server modules
* Remove redundant comments
* Add null check
 2022-09-29 08:16:23 +08:00
xuhhui
6466cc7c41
[Improvement][Master] Construct processInstance may NPE when master handling command (#12056) 
* [Improvement][Master] Construct processInstance may NPE when master handling command

* use an enpty map

Co-authored-by: xuhaihui <xuhaihui@cmss.chinamobile.com>
 2022-09-27 09:13:48 +08:00
Wenjun Ruan
fba5a8eaa0
Fix insert command error due to the id is not null (#12092) 2022-09-22 09:52:32 +08:00
Wenjun Ruan
26dad5e2a8
Fix update TaskDefinition error (#12060) 2022-09-20 22:23:16 +08:00
insist777
5b384f3fab
[Feature-11473][Task]Support test task (#11670) 
* [improve] support test tasks

* [improve] support test tasks

* [improve] support test tasks

* [improve] support test tasks

* [improve] support test tasks

* Update TaskExecuteThread.java

* try solve e2e q

* try solve e2e q

* try solve e2e q

* try solve e2e q

* try solve e2e q

* try solve e2e q

* try solve e2e q

* try solve e2e q

* try solve e2e q

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* Update DataSource.java

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* Update messages_zh_CN.properties

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* Update messages.properties

* Update messages_en_US.properties

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks
 2022-09-17 13:40:56 +08:00
Wenjun Ruan
d833a28b2e
Fix cannot save processDefinition (#11931) 2022-09-16 13:54:18 +08:00
Wenjun Ruan
d3a77c68e6
Make LogServiceClient Singleton (#11777) 2022-09-14 16:18:08 +08:00
caishunfeng
2e61c76c22
[Improvement] Add remote task model (#11767) 2022-09-08 13:41:36 +08:00
kezhenxu94
efedc9a53c
Bump up dependencies to fix cves (#11765) 2022-09-08 09:05:05 +08:00