test/DolphinScheduler
1
0
Fork 0
mirror of https://gitee.com/dolphinscheduler/DolphinScheduler.git synced 2024-12-02 12:17:43 +08:00
Code Issues Actions 10 Packages Projects Releases Wiki Activity
6,760 Commits 27 Branches 59 Tags 276 MiB
9164ec8bf1
Commit Graph

233 Commits

Author SHA1 Message Date
caishunfeng
b9e3187ad5 [Feature-10871] add workflow executing data query (#10875) 
* add workflow executing data query
* fix sonar check for interrupted
 2022-07-19 16:10:42 +08:00
Wenjun Ruan
5a2ea0b76b Fix compile error 2022-07-19 12:30:26 +08:00
Wenjun Ruan
17f0be5bfb Kill task when we do master failover to avoid task lost (#10997) 
(cherry picked from commit 998e4d74dd)
 2022-07-19 12:22:26 +08:00
Wenjun Ruan
9ee20cffef [Fix-10827] Fix network error cause worker cannot send message to master (#10886) 
* Fix network error cause worker cannot send message to master

(cherry picked from commit cade66a9b6)
 2022-07-19 12:20:18 +08:00
Wenjun Ruan
b259deb196 [Fix-10854] Fix database restart may lost task instance status (#10866) 
* Fix database update error doesn't rollback the task instance status

* Fix database error may cause workflow dead with running status

(cherry picked from commit f639a2eed4)
 2022-07-19 12:12:57 +08:00
Wenjun Ruan
7500e99682 [Fix-10785] Fix state event handle error will not retry (#10786) 
* Fix state event handle error will not retry

* Use state event handler to deal with the event

(cherry picked from commit 67d14fb7b3)
 2022-07-19 11:52:06 +08:00
Wenjun Ruan
db31deb54f [Bug] [Master] Worker failover will cause task cannot be failover (#10631) 
* fix worker failover may lose event

(cherry picked from commit 66624c5c86)
 2022-07-19 11:41:25 +08:00
Wenjun Ruan
52815975bc Add some warning log in master (#10383) 
* Add some warn log in master

* fix may skip sleep

(cherry picked from commit b0d9d3f9ab)
 2022-07-19 11:13:39 +08:00
Wenjun Ruan
318a8e3ae0 [Feature][metrics] Add master, worker metrics (#10326) 
* Add mater metrics

* fix UT

* Add url to mysql profile

* Add worker metrics

* Update grafana config

* Add system metrics doc

* Add process failover counter

* Add metrics image

* Change jpg to png

* Add command insert metrics

* Fix UT

* Revert UT

(cherry picked from commit e21d7b1551)
 2022-07-19 11:12:28 +08:00
Wenjun Ruan
4ceb420873 Fix TaskProcessorFactory#getTaskProcessor get common processor is not thread safe (#10479) 
* Fix TaskProcessorFactory#getTaskProcessor get common processor is not thread safe

(cherry picked from commit ad2646ff1f)
 2022-07-19 11:02:57 +08:00
devosend
7ddaa2f47d [maven-release-plugin] prepare for next development iteration 2022-06-28 16:41:31 +08:00
devosend
0a1b9bdd52 [maven-release-plugin] prepare release 3.0.0-beta-2 2022-06-28 16:41:30 +08:00
devosend
d68dcda2bb [chore] pre-release change pom.xml 2022-06-28 08:24:21 +08:00
Wenjun Ruan
742944c60d
[Fix] [Worker] Fix worker will hang if fails to start (#10501) 
* [Fix] [Worker] Fix worker will hang if fails to start (#10342)

* Fix worker will hang if fails to start

* Add .run to ignore

Signed-off-by: ruanwenjun <wenjun@apache.org>

* Add import Epoll

* Add cpu_quota in h2 to fix UT

* Remove cpu_quota,memory_max in TaskDefiniitionMapper
 2022-06-22 10:13:55 +08:00
Jiajie Zhong
57ade38939 [maven-release-plugin] prepare release 3.0.0-beta-1 2022-05-26 20:26:32 +08:00
Jiajie Zhong
a9fa6b33a4
[chore] Change release version to 3.0.0-beta-1 (#9957) 2022-05-12 21:49:21 +08:00
guoshupei
ca95d2f928
[Fix-9221] [alert-server] optimization and gracefully close (#9246) 
* [Fix-9221] [alert-server] optimization and gracefully close

This closes #9221

* [Fix-9221] [alert-server] remove unused mock data

This closes #9221

* [Fix-9221] [alert-server] remove unused mock data

This closes #9221

* [Fix-9221] [alert-server] remove unnecessary Mockito stubbings

* [Fix-9221] [alert-server] init AlertPluginManager in AlertServer

* [Fix-9221] [alert-server] AlertServerTest add AlertPluginManager installPlugin

* [Fix-9221] [alert-server] replace @Eventlistener with @PostConstruct

* [Fix-9221] [alert-server] sonar check solution

* [Improvement-9221] [alert] update constructor injection and replace IStoppable with Closeable

Co-authored-by: guoshupei <guoshupei@lixiang.com>
 2022-04-08 10:02:10 +08:00
Eric Gao
fd6b43bc81
[Dev] Switch version in pom.xml to dev-SNAPSHOT (#9223) (#9299) 2022-04-07 18:15:19 +08:00
caishunfeng
ddf1ff98fa
[Improvement-7697][Master/Worker] Change the task ack to runnning callback (#8719) 
* rebase dev

* change task state to dispatch if dispatch success

* update task host when dispatch

* add dispatch task event

* test

* check tenant after enable auto create

* handle dispatch state

Co-authored-by: caishunfeng <534328519@qq.com>
 2022-03-30 17:35:05 +08:00
Kerwin
a1bef43981
[Feature-8579][Task] Task back-end plugin optimization, new plugins only need to modify the plugin's own module (#8584) 
* Task plug-in optimization, task-related classes are migrated to the task plugin module

* remove tasktype enum class.

* fix code style.

* fix code style

* fix code style

* fix ProcessServiceTest.testUpdateResourceInfo test class

* fix code style

* fix test task plugin manager check parameters null

* fix test ut

* update aws license

* change TaskTimeoutStrategy class
 2022-03-15 15:16:36 +08:00
wangyang
0d8079a816
[DS-7992][feat] Alert module judging strategy (#8636) 2022-03-14 14:25:27 +08:00
yimaixinchen
a72742b9de
remove unuse import jsonFormat (#8548) 2022-02-25 22:05:05 +08:00
caishunfeng
84f2cfec39
[Feature][Server] Custom timezone - Add timezone handler unified (#8477) 
* date convert of timezone

* remove @JsonFormat

* add unit test

* fix time preview in scheduler

* optimization & add env config

Co-authored-by: caishunfeng <534328519@qq.com>
 2022-02-24 18:20:34 +08:00
Wenjun Ruan
66dcf2376a
Optimize some code (#8324) 2022-02-10 19:07:53 +08:00
Wenjun Ruan
093815785b
Remove unused class (#8299) 2022-02-08 19:50:19 +08:00
Kerwin
103ccb01da
change version to 2.0.4-SNAPSHOT (#8188) 2022-01-25 15:17:17 +08:00
xuhhui
0cfb2043cf
Improvement: improve Pong.java mistake (#7822) 2022-01-12 01:20:13 +08:00
kezhenxu94
67cc260d52
Split the components into individual package, refactor configurations to use Spring properties (#7160) 
* Split the components into individual package

A follow-up PR will be made to build dedicated Docker images for each
component, so that every component Docker image has minimal jars, which
is easy to maintain and good for security fixes.

* Split the components into individual package

A follow-up PR will be made to build dedicated Docker images for each
component, so that every component Docker image has minimal jars, which
is easy to maintain and good for security fixes.

* Split the components into individual package

A follow-up PR will be made to build dedicated Docker images for each
component, so that every component Docker image has minimal jars, which
is easy to maintain and good for security fixes.
 2021-12-10 11:28:53 +08:00
wind
a80a7b3fcf
[Feature-6963][MasterServer] unified cache manager (#7187) 
* unified cache

* reduce db select

* checkstyle

Co-authored-by: caishunfeng <534328519@qq.com>
 2021-12-07 15:55:22 +08:00
wangxj3
2b519f31e9
[Feature-#6422] task group queue (#6722) 
* add task group

* modify task group

* pull dev

* add license header

* modify code style

* fix code style

* fix sql error

* fix error

* fix test

* fix test

* fix test

* fix test

* fix code style

* fix ut

* code style

* fix unit test

* test ut

* ut

* add unittest

* test ut

* modify back ut

* majorization code

* fix conflict

* fix ut

Co-authored-by: wangxj <wangxj31>
 2021-12-03 14:02:31 +08:00
wind
8e93939121
[DS-6963][MasterServer]add cache manager for tenant and user (#6974) 
* cache manager for tenant and user

* add test

* fix checkstyle

* add dependencies

* fix sonar check

* use spring boot cache configuration

* add license

* check style

* check style

* add license

* change proxy name to processor

* add license header

* check style

* add license file

* add license file

* add license

Co-authored-by: caishunfeng <534328519@qq.com>
 2021-11-30 08:29:25 +08:00
lyq
c78fd24776
bug about dolphinscheduler-remote SendAsync (#6972) 
* bug about SendAsync
 2021-11-24 09:53:22 +08:00
kezhenxu94
4114cb07f6
Add actuator and enable metrics, add Grafana configs (#6840) 2021-11-14 20:22:13 +08:00
kezhenxu94
11844415b0
Refactor datasource and task plugins and simplify its usage (#6767) 2021-11-10 18:29:17 +08:00
wind
daca3baf66
[DS-6653][Server] fix netty create too many channels (#6654) 
Co-authored-by: caishunfeng <534328519@qq.com>
 2021-11-01 12:18:39 +08:00
wind
8850baff07
[DS-6640][WorkerServer] support process update host command type (#6642) 
Co-authored-by: caishunfeng <534328519@qq.com>
 2021-10-31 21:10:51 +08:00
Kirs
67508d8d1b
update project version to 2.0.0-SNAPSHOT (#6383) 2021-09-28 12:00:43 +08:00
OS
e34c65d5a6
[Feature-4355][Master-Worker-API] improvements of master and scheduler module (#6095) 
* [Feature-4355][Master-Worker-API] improvements of master and scheduler module (#6085)

* master refactor:
   1. spi for task submit and other actions(pause, kill)
   2. remove threads for process instance and task instance.
   3. add events for process instance and task instance

* ut npe

* add try catch

* code style

* fix critical bugs

* fix critical bugs

* fix critical bugs

* fix critical bugs
 2021-09-06 16:57:02 +08:00
深刻
4c0993cdf4
[Fix-5781][UT] Fix test coverage in sonar (#5817) 
* build(UT): make jacoco running in offline-instrumentation

issue: #5781

* build(UT): remove the jacoco agent dependency in microbench

issue: #5781
 2021-07-26 22:04:36 +08:00
wangxj3
6df58e8d78
[Feature-#5565][Master Worker-Server] Global Param passed by sense dependencies (#5603) 
* add globalParams new plan with varPool

* add unit test

* add python task varPoolParams


Co-authored-by: wangxj <wangxj31>
 2021-06-25 11:37:23 +08:00
Kirs
e2243d63be
[Feature-#3961][Registry]Registry-SPI (#5562) 
[Feature#3961]Registry SPI

All the logical structure of the registry must be converted into a tree structure within the system, so some plug-ins must be converted internally, such as ETCD
The registry supports distributed locks.
todo: The specific information about the registration center of the API module needs to be adjusted.
 2021-06-09 15:21:02 +08:00
zhuangchong
1b7dcea610
[Improvement-5440][Maven] Project module maven dependency optimization (#5441) 
* Optimize dependencies between modules.

* update module-alert dependency.

* add repository in the pom xml.

* update Central Repository in the pom.xml.
 2021-05-19 10:05:00 +08:00
ruanwenjun
fbef290cc5
[Bug][Service] All tasks that need to send attachment will fail due to chennel idle (#5442) 2021-05-11 03:03:52 +08:00
QuakeWang
f88bbbd637
[Improvement] Create RollViewLogRequestCommandTest.java (#5417) 
* Create RollViewLogRequestCommandTest.java

Add a test case about RollViewLogRequestCommand

* Update pom.xml

add the test files in the parent pom
 2021-05-07 20:58:23 +08:00
QuakeWang
22c62372a6
Create GetLogBytesResponseCommandTest.java (#5413) 
Create a test case
 2021-05-01 23:47:43 +08:00
QuakeWang
d45b27ce2e
[Improvement] Create GetLogBytesRequestCommandTest.java (#5356) 
* Create GetLogBytesRequestCommandTest.java

Add a test case about GetLogBytesRequestCommand

* Update GetLogBytesRequestCommandTest.java

* modify the code format

* Update GetLogBytesRequestCommandTest.java

Submitting the test again

* Update GetLogBytesRequestCommandTest.java

Code changes based on comments
 2021-04-29 15:18:00 +08:00
wenjun
f3dfaacca2
[Improvement][Service] Optimize query log (#5205) 
* [Improvement][Service] Optimize query log

* remove IPUtils
 2021-04-15 20:25:01 +08:00
Shiwen Cheng
5856a12855
[Improvement-4984][Worker] Refactor and Improve worker load balance (#4996) 
* [Improvement][Remote] Improve unit tests

* [Improvement][Worker] Improve worker load balance

* [Improvement][Worker] Fix code smells

* [Improvement][Worker] Rename weight to hostWeight
 2021-03-18 10:03:58 +08:00
CalvinKirs
b4c7e35340 Merge branch 'dev' into rpc 
# Conflicts:
#	tools/dependencies/known-dependencies.txt
 2021-02-26 18:10:49 +08:00
dailidong
551759965d
Merge pull request #4743 from apache/params-trans 
[Feature-3805][mater-worker-ui] The implement of passing variables between tasks
 2021-02-23 19:06:13 +08:00