Commit Graph

206 Commits

Author SHA1 Message Date
kezhenxu94
67cc260d52
Split the components into individual package, refactor configurations to use Spring properties (#7160)
* Split the components into individual package

A follow-up PR will be made to build dedicated Docker images for each
component, so that every component Docker image has minimal jars, which
is easy to maintain and good for security fixes.

* Split the components into individual package

A follow-up PR will be made to build dedicated Docker images for each
component, so that every component Docker image has minimal jars, which
is easy to maintain and good for security fixes.

* Split the components into individual package

A follow-up PR will be made to build dedicated Docker images for each
component, so that every component Docker image has minimal jars, which
is easy to maintain and good for security fixes.
2021-12-10 11:28:53 +08:00
wind
a80a7b3fcf
[Feature-6963][MasterServer] unified cache manager (#7187)
* unified cache

* reduce db select

* checkstyle

Co-authored-by: caishunfeng <534328519@qq.com>
2021-12-07 15:55:22 +08:00
wangxj3
2b519f31e9
[Feature-#6422] task group queue (#6722)
* add task group

* modify task group

* pull dev

* add license header

* modify code style

* fix code style

* fix sql error

* fix error

* fix test

* fix test

* fix test

* fix test

* fix code style

* fix ut

* code style

* fix unit test

* test ut

* ut

* add unittest

* test ut

* modify back ut

* majorization code

* fix conflict

* fix ut

Co-authored-by: wangxj <wangxj31>
2021-12-03 14:02:31 +08:00
wind
8e93939121
[DS-6963][MasterServer]add cache manager for tenant and user (#6974)
* cache manager for tenant and user

* add test

* fix checkstyle

* add dependencies

* fix sonar check

* use spring boot cache configuration

* add license

* check style

* check style

* add license

* change proxy name to processor

* add license header

* check style

* add license file

* add license file

* add license

Co-authored-by: caishunfeng <534328519@qq.com>
2021-11-30 08:29:25 +08:00
lyq
c78fd24776
bug about dolphinscheduler-remote SendAsync (#6972)
* bug about SendAsync
2021-11-24 09:53:22 +08:00
kezhenxu94
4114cb07f6
Add actuator and enable metrics, add Grafana configs (#6840) 2021-11-14 20:22:13 +08:00
kezhenxu94
11844415b0
Refactor datasource and task plugins and simplify its usage (#6767) 2021-11-10 18:29:17 +08:00
wind
daca3baf66
[DS-6653][Server] fix netty create too many channels (#6654)
Co-authored-by: caishunfeng <534328519@qq.com>
2021-11-01 12:18:39 +08:00
wind
8850baff07
[DS-6640][WorkerServer] support process update host command type (#6642)
Co-authored-by: caishunfeng <534328519@qq.com>
2021-10-31 21:10:51 +08:00
Kirs
67508d8d1b
update project version to 2.0.0-SNAPSHOT (#6383) 2021-09-28 12:00:43 +08:00
OS
e34c65d5a6
[Feature-4355][Master-Worker-API] improvements of master and scheduler module (#6095)
* [Feature-4355][Master-Worker-API] improvements of master and scheduler module (#6085)

* master refactor:
   1. spi for task submit and other actions(pause, kill)
   2. remove threads for process instance and task instance.
   3. add events for process instance and task instance

* ut npe

* add try catch

* code style

* fix critical bugs

* fix critical bugs

* fix critical bugs

* fix critical bugs
2021-09-06 16:57:02 +08:00
深刻
4c0993cdf4
[Fix-5781][UT] Fix test coverage in sonar (#5817)
* build(UT): make jacoco running in offline-instrumentation

issue: #5781

* build(UT): remove the jacoco agent dependency in microbench

issue: #5781
2021-07-26 22:04:36 +08:00
wangxj3
6df58e8d78
[Feature-#5565][Master Worker-Server] Global Param passed by sense dependencies (#5603)
* add globalParams new plan with varPool

* add unit test

* add python task varPoolParams


Co-authored-by: wangxj <wangxj31>
2021-06-25 11:37:23 +08:00
Kirs
e2243d63be
[Feature-#3961][Registry]Registry-SPI (#5562)
[Feature#3961]Registry SPI

All the logical structure of the registry must be converted into a tree structure within the system, so some plug-ins must be converted internally, such as ETCD
The registry supports distributed locks.
todo: The specific information about the registration center of the API module needs to be adjusted.
2021-06-09 15:21:02 +08:00
zhuangchong
1b7dcea610
[Improvement-5440][Maven] Project module maven dependency optimization (#5441)
* Optimize dependencies between modules.

* update module-alert dependency.

* add repository in the pom xml.

* update Central Repository in the pom.xml.
2021-05-19 10:05:00 +08:00
ruanwenjun
fbef290cc5
[Bug][Service] All tasks that need to send attachment will fail due to chennel idle (#5442) 2021-05-11 03:03:52 +08:00
QuakeWang
f88bbbd637
[Improvement] Create RollViewLogRequestCommandTest.java (#5417)
* Create RollViewLogRequestCommandTest.java

Add a test case about RollViewLogRequestCommand

* Update pom.xml

add the test files in the parent pom
2021-05-07 20:58:23 +08:00
QuakeWang
22c62372a6
Create GetLogBytesResponseCommandTest.java (#5413)
Create a test case
2021-05-01 23:47:43 +08:00
QuakeWang
d45b27ce2e
[Improvement] Create GetLogBytesRequestCommandTest.java (#5356)
* Create GetLogBytesRequestCommandTest.java

Add a test case about GetLogBytesRequestCommand

* Update GetLogBytesRequestCommandTest.java

* modify the code format

* Update GetLogBytesRequestCommandTest.java

Submitting the test again

* Update GetLogBytesRequestCommandTest.java

Code changes based on comments
2021-04-29 15:18:00 +08:00
wenjun
f3dfaacca2
[Improvement][Service] Optimize query log (#5205)
* [Improvement][Service] Optimize query log

* remove IPUtils
2021-04-15 20:25:01 +08:00
Shiwen Cheng
5856a12855
[Improvement-4984][Worker] Refactor and Improve worker load balance (#4996)
* [Improvement][Remote] Improve unit tests

* [Improvement][Worker] Improve worker load balance

* [Improvement][Worker] Fix code smells

* [Improvement][Worker] Rename weight to hostWeight
2021-03-18 10:03:58 +08:00
CalvinKirs
b4c7e35340 Merge branch 'dev' into rpc
# Conflicts:
#	tools/dependencies/known-dependencies.txt
2021-02-26 18:10:49 +08:00
dailidong
551759965d
Merge pull request #4743 from apache/params-trans
[Feature-3805][mater-worker-ui] The implement of passing variables between tasks
2021-02-23 19:06:13 +08:00
zhuangchong
4aac23481e
[Fix-4757][worker] fix worker failover host error (#4799)
* resolve code conflicts

* resolve code conflicts

* resolve code conflicts

* update WorkerGroupServiceImpl code style.

* update worker group service test host data.

* add ZookeeperNodeHandlerTest class

* change WorkerZkNode to Host class.

* add generate host string method.
2021-02-21 00:34:23 +08:00
wangxj3
874ca07e6e
[Feature-3805][mater-worker-ui]fix function of passing variables between tasks bug (#4811)
* global initParam and set Param

* fix dataFormat error

* fix deal outParams bug

* fix code style

* fix code style

* fix code style

* fix code style

* fix code style

* fix code style

* fix code style

* add test

* fix code style (variable name)

* fix reset globalParams bug

* fix format bug

* fix SonarCloud Code Analysis

* add test

* fix code style

* fix code style

* modify test

* fix test Coverage

* fix code style

* 4 test

* add test pom

* add header

Co-authored-by: wangxj <wangxj31>
2021-02-20 14:32:46 +08:00
Shiwen Cheng
8879d8ff60
[Improvement][*] Define version with revision globally and update version to 1.3.6-SNAPSHOT (#4773)
* [Improvement][*] Update version to 1.4.0-SNAPSHOT

* [Improvement][*] Update tab to space under checkstyle

* [Improvement][*] Define version with revision globally

* [Improvement][*] Update version to 1.3.6-SNAPSHOT
2021-02-20 14:22:49 +08:00
lenboo
59811d0efb Merge remote-tracking branch 'upstream/dev' into params-trans 2021-02-19 16:02:55 +08:00
Shiwen Cheng
8ebddc1f5b
[Fix-3298][K8s] Fix wrong host of task instance in k8s (#4786) 2021-02-18 11:41:58 +08:00
CalvinKirs
12c1347749 rpc 2021-02-09 12:01:30 +08:00
CalvinKirs
dad9436992 Merge branch 'dev' into rpc 2021-02-09 12:00:08 +08:00
wangxj3
87ff13fc86
[Feature-#3805][server-master] global params of master (#4678)
* global initParam and set Param

* fix dataFormat error

* fix deal outParams bug

* fix code style

* fix code style

* fix code style

* fix code style

* fix code style

* fix code style

* fix code style

* add test

* fix code style (variable name)

* fix reset globalParams bug

Co-authored-by: wangxj <wangxj31>
2021-02-07 17:10:33 +08:00
CalvinKirs
98d2406e5a async not needed callback 2021-02-02 10:23:40 +08:00
CalvinKirs
50dbe7d0c9 async not needed callback 2021-02-02 10:06:38 +08:00
CalvinKirs
c9daf6a425 async not needed callback 2021-02-02 00:25:31 +08:00
CalvinKirs
3ba03213fe code style 2021-02-02 00:08:00 +08:00
CalvinKirs
d3a1da47a7 code style 2021-02-01 22:46:35 +08:00
CalvinKirs
4578079368 rpc protocol 2021-02-01 22:18:47 +08:00
CalvinKirs
392136cd24 rpc protocol 2021-02-01 19:44:35 +08:00
CalvinKirs
54ca216abf rpc protocol 2021-02-01 11:16:06 +08:00
CalvinKirs
93fb043be3 code style 2021-01-31 22:35:10 +08:00
CalvinKirs
0a92a53f63 add jmh test 2021-01-31 22:27:51 +08:00
Segun Ogundipe
047b0a312f
[Improvement][remote] Refactor remote module to fix code smell (#4576)
* chore[remote]: Refactor remote module to fix code smell

* chore[remote]: Add licence header

* chore[remote]: Clean code smell

* chore[remote]: Clean code smell

* chore[remote]: Remove unused import
2021-01-30 21:32:13 +08:00
CalvinKirs
0f6228457a add rpc protocol 2021-01-28 13:31:52 +08:00
CalvinKirs
d1658e9118 fix init rpc request error 2021-01-26 10:07:28 +08:00
CalvinKirs
7a976f4c89 fix code smell 2021-01-26 09:50:08 +08:00
CalvinKirs
967ab641dd fix bug and code style 2021-01-26 01:18:47 +08:00
CalvinKirs
246332b076 fix bug 2021-01-26 00:43:54 +08:00
CalvinKirs
a63c2c6cbb fix bug 2021-01-25 23:47:07 +08:00
CalvinKirs
ea97ed1781 fix bug and add license 2021-01-25 23:30:05 +08:00
CalvinKirs
d1f4e6a270 add IUserService Interface 2021-01-25 22:34:25 +08:00