test/DolphinScheduler
1
0
Fork 0
mirror of https://gitee.com/dolphinscheduler/DolphinScheduler.git synced 2024-12-05 05:38:30 +08:00
Code Issues Actions 10 Packages Projects Releases Wiki Activity
5,591 Commits 27 Branches 59 Tags 276 MiB
fc162826d9
Commit Graph

305 Commits

Author SHA1 Message Date
Kerwin
f1b85cf892
Fix the missing project information in the process instance timeout alarm (#7324) 2021-12-10 18:55:10 +08:00
Kerwin
3fadfca085
fix in the process alarm message, projectid is 0 (#7298) 2021-12-10 12:25:35 +08:00
kezhenxu94
67cc260d52
Split the components into individual package, refactor configurations to use Spring properties (#7160) 
* Split the components into individual package

A follow-up PR will be made to build dedicated Docker images for each
component, so that every component Docker image has minimal jars, which
is easy to maintain and good for security fixes.

* Split the components into individual package

A follow-up PR will be made to build dedicated Docker images for each
component, so that every component Docker image has minimal jars, which
is easy to maintain and good for security fixes.

* Split the components into individual package

A follow-up PR will be made to build dedicated Docker images for each
component, so that every component Docker image has minimal jars, which
is easy to maintain and good for security fixes.
 2021-12-10 11:28:53 +08:00
wind
a80a7b3fcf
[Feature-6963][MasterServer] unified cache manager (#7187) 
* unified cache

* reduce db select

* checkstyle

Co-authored-by: caishunfeng <534328519@qq.com>
 2021-12-07 15:55:22 +08:00
JinYong Li
2792c35d68
[Bug] [API] queryProcessDefinitionByCode bug (#7221) 2021-12-07 14:41:26 +08:00
wind
ba2b2a67c2
[Feature-6988][MasterServer] add cache manager for workflow (#7090) 
* add cache manager for workflow

* [DS-6988][MasterServer] add cache manager for workflow

* cache evict code optimization

* test

Co-authored-by: caishunfeng <534328519@qq.com>
 2021-12-03 23:13:11 +08:00
wangxj3
2b519f31e9
[Feature-#6422] task group queue (#6722) 
* add task group

* modify task group

* pull dev

* add license header

* modify code style

* fix code style

* fix sql error

* fix error

* fix test

* fix test

* fix test

* fix test

* fix code style

* fix ut

* code style

* fix unit test

* test ut

* ut

* add unittest

* test ut

* modify back ut

* majorization code

* fix conflict

* fix ut

Co-authored-by: wangxj <wangxj31>
 2021-12-03 14:02:31 +08:00
JinYong Li
b265297fc8
[Improvement][API] fix 6772 new api bug (#7098) 
* add processDefinition releaseWorkflowAndSchedule

* add ProcessTaskRelationServiceImpl.moveTaskProcessRelation

* add dependency and subprocess judgement

* fix api bug

* fix codestyle
 2021-12-02 14:06:39 +08:00
kezhenxu94
2ed3c0c3a7
Recover unit tests that have not been run for some time (#7054) 2021-12-01 13:55:26 +08:00
Sunny Lei
0dcff1425a
[DS-5229][feat] Implement server port custom config (#6947) 
* [DS-5229][fix] server port custom config
This closes #5229

* [DS-5229][feat] Implement server port custom config
This closes #5229

* [Bug] [readme] Error link to Docker and k8s in readme apache#6802 (#6862)

* [Bug] [readme] Error link to Docker and k8s in readme #6802
1、modify the error link

* [Bug] [readme] Error link to Docker and k8s in readme #6802
1、modify the error link in readme_zh_cn.md

* [DS-6829][WorkerServer] skip create log dir and print log in dryRun model (#6852)

Co-authored-by: caishunfeng <534328519@qq.com>

* [DS-5229][feat] Implement server port custom config
Modify review suggestion
This closes #5229

Co-authored-by: GaoTianDuo <gaotianduo_yewu@cmss.chinamobile.com>
Co-authored-by: wind <caishunfeng@users.noreply.github.com>
Co-authored-by: caishunfeng <534328519@qq.com>
 2021-12-01 00:21:03 +08:00
wind
8e93939121
[DS-6963][MasterServer]add cache manager for tenant and user (#6974) 
* cache manager for tenant and user

* add test

* fix checkstyle

* add dependencies

* fix sonar check

* use spring boot cache configuration

* add license

* check style

* check style

* add license

* change proxy name to processor

* add license header

* check style

* add license file

* add license file

* add license

Co-authored-by: caishunfeng <534328519@qq.com>
 2021-11-30 08:29:25 +08:00
wind
28872d8706
[DS-6891][MasterServer] reduce db operation when process instance running (#6904) 
* remove taskInstanceCacheManager

* remove taskInstanceCacheManager

* [DS-6891][MasterServer] reduce db operation when process instance running

* [DS-6891][MasterServer] reduce db operation when process instance running

* fix test

* fix Transactional method call

* checkstyle

Co-authored-by: caishunfeng <534328519@qq.com>
 2021-11-19 18:31:24 +08:00
JinYong Li
4bec792160
[Fix][Common] rewrite code generate,fix bit shift (#6914) 
* rewrite code generate,fix bit shift

* fix ut

* add algorithm from licenses file

* add algorithm from licenses file

* add algorithm from licenses file

* add algorithm from licenses file

* add algorithm from licenses file

* fix ut
 2021-11-19 16:04:05 +08:00
wind
595e4843d0
[DS-6849][MasterServer] fetch more commands and handle in parallel (#6850) 
* [DS-6849][MasterServer] fetch more commands and handle in parallel

* add return

* handle command with check

* add test

* delete master prefix

Co-authored-by: caishunfeng <534328519@qq.com>
 2021-11-19 12:03:49 +08:00
OS
1160b53940
[Feature][Master] change sub process id to code (#6858) 
* feature 6597 change sub process id to code

* feature 6597 change sub process id to code
 2021-11-15 20:27:19 +08:00
OS
880c1d28ff
[Feature][master] change sub process id to code. (#6846) 
* 6597 change sub process id to code.

* 6597 change sub process id to code.

* 6597 change sub process id to code.

* 6597 change sub process id to code.

* 6597 change sub process id to code.

* 6597 change sub process id to code.

* code style

* fix ut process

* fix ut process
 2021-11-15 01:00:44 +08:00
kezhenxu94
4114cb07f6
Add actuator and enable metrics, add Grafana configs (#6840) 2021-11-14 20:22:13 +08:00
mask
a998415afe
[Feature][datasource] Change DataSource Connection Pool from Druid to HiKariCP (#6490) (#6828) 
* [Feature][datasource] Change DataSource Connection Pool from Druid to HiKariCP (#6490)

* fix pom dep scope

* filter alert

* filter alert

* filter alert

* fix checkstyle

* remove hikaricp version

* update pom

* add properties

* add properties

* rename properties

* fix e2e fail

* rename properties
 2021-11-14 13:02:58 +08:00
Jiajie Zhong
6045728428
[ci] Add checkstyle prohibit rule System.out.println (#6778) 
* [ci] Add checkstyle prohibit rule System.out.println

* Correct RegexpSingleline place
 2021-11-13 16:47:29 +08:00
kezhenxu94
11844415b0
Refactor datasource and task plugins and simplify its usage (#6767) 2021-11-10 18:29:17 +08:00
kezhenxu94
63602bf77c
Clean up the utility codes (#6732) 2021-11-08 23:22:07 +08:00
OS
24bd8a16e1
[Bug][Master] The task of dependent should use taskCode (#6730) 
* fix #6682 The task of dependent should use taskCode

* fix #6682 The task of dependent should use taskCode
 2021-11-07 23:20:35 +08:00
Wenjun Ruan
0afb780718
[Improvement-5807][Dao] Use h2 in UT (#6500) 
* Use H2 in UT

* change scope

* Change profile name to h2

* Add datasource-h2.properties, datasource-postgresql.properties

* Fix UT

* Resolve conflict

* Remove startDatabase
 2021-11-07 23:19:19 +08:00
kezhenxu94
5400abc74f
Refactor registry plugin and simplify its usage (#6712) 2021-11-07 15:18:10 +08:00
wangxj3
dc8b87e473
[Feature-#6268][server-master] Serial execte process (#6185) 
* add serial processInstance

* fix bug

* add test

* fix code style

* fix style code

* add sql

* fix sql error

* add api

* add test

* fix code style

* modify api

* delete column , fix mapper

* fix unimport

* fix test

* fix bug of missing param for Python

* fix code style

* fix test

* fix test

Co-authored-by: wangxj <wangxj31>
 2021-11-05 17:25:45 +08:00
wind
1ebab356f3
[DS-6660][Improvement] Change the registry default timeout (#6663) 
* [DS-6660][Improvement] Change the registry default timeout

* [DS-6660][Improvement] Change the registry default timeout

* fix Double to Integer

Co-authored-by: caishunfeng <534328519@qq.com>
 2021-11-04 22:31:13 +08:00
mask
9b1984c244
[fix-6570][task] add sql task udf resource load (#6570) (#6571) 
* [fix-6570][task] add sql task udf resource load (#6570)

-add sql task udf resource load
-Fix UdfFuncRequest serialization

* fix checkstyle

* add defaultFS in sqlTaskContext

* fix checkstyle
 2021-10-21 15:23:49 +08:00
OS
52a550b6ae
[BUG-6543][Master] process instance state is always running when failure task (#6547) 
* fix-6543: process instance state is always running when failure task exists

* code style

* code style
 2021-10-15 18:04:04 +08:00
wind
50f1766903
[DS-6499][WorkerServer] report busy state when worker is overload (#6512) 
* [DS-6499][WorkerServer] report busy state when worker is overload
 2021-10-13 11:45:56 +08:00
OS
cea8ae6991
6471: cache process definition in master (#6511) 2021-10-13 11:36:27 +08:00
wind
fea7874f95
[DS-6502][MasterServer] fix cmdParam NPE when constructProcessInstance on SCHEDULER mode (#6504) 
Co-authored-by: caishunfeng <534328519@qq.com>
 2021-10-12 14:42:49 +08:00
OS
db04a5b04d
[Feature-6471]Cache Process definition in master (#6485) 
* feature-6471 Cache Process definition in master
 2021-10-11 20:22:58 +08:00
Kerwin
88690462b4
[Fix-6478] [Server] Fix the lack of scheduling time in complement parallelism mode (#6491) 
* fix missing history data

* update code style

* update ExecutorServiceTest test class.
 2021-10-11 20:03:25 +08:00
wind
e091801e05
[Improvement-6474] [MasterServer] schedule time for process instance optimization (#6477) 
* [DS-6474][MasterServer] change to handle schedule time for process instance in WorkflowExecuteThread

* delete all the valid tasks when complement data if id is not null

* checkstyle

Co-authored-by: caishunfeng <534328519@qq.com>
 2021-10-11 15:13:38 +08:00
OS
a502e643bc
fix bug: complement data error (#6460) 2021-10-08 16:56:33 +08:00
OS
a8baa9553f
[Bug-6455][Master]fix bug 6455: cannot stop sub-task (#6458) 
* fix bug: cannot stop the task.

* fix bug: cannot stop the task.

* remove the check thread number
 2021-10-08 14:08:12 +08:00
Hua Jiang
fac6b4afd3
[Fix-5875][API] When I saved the task that had the same name task in another flow ,the service would throw DuplicateKeyException (#6430) 
* using the task code in the dag
 2021-10-08 09:55:54 +08:00
Kirs
ea2a8d26a0
[Improvement#6438][Worker]remove meaningless DB query (#6440) 
org.apache.dolphinscheduler.server.worker.runner.TaskExecuteThread
In the process of task kill, it will query the DB to obtain the task instance object, but this judgment does not make any sense. In fact, the relevant context is already clear, so there is no need to query from DB.
 2021-10-02 13:09:00 +08:00
wind
6dd0e050ce
[DS-6433][Improvement] find one command optimize without mybatis-plus (#6434) 
Co-authored-by: caishunfeng <534328519@qq.com>
 2021-09-30 19:35:28 +08:00
Shukun Zhang
6e453de241
[Feature] workflow level task dry run (#6104) 
* feat workflow level task dry run

* feat workflow level task dry run

* feat workflow level task dry run

* feat workflow level task dry run

* feat workflow level task dry run

* feat workflow level task dry run

* feat workflow level task dry run

* feat workflow level task dry run

* feat workflow level task dry run

* feat workflow level task dry run
 2021-09-30 11:18:06 +08:00
caishunfeng
ea6503b997
[DS-6403][fix] Put DB queries ahead before submit to TaskPriorityQueue (#6406) 
* [DS-6403][fix] Put DB queries ahead before submit to TaskPriorityQueue

* [DS-6403][fix] Put DB queries ahead before submit to TaskPriorityQueue

* [DS-6403][fix] Put DB queries ahead before submit to TaskPriorityQueue

* checkstyle

Co-authored-by: caishunfeng <534328519@qq.com>
 2021-09-29 14:04:35 +08:00
OS
c013b49e72
[Fix-6347][Master-API]fix bug #6347 the first schedule_time is error in complement data (#6389) 
* fix bug #6347  complement data errors
 2021-09-28 17:00:46 +08:00
Kirs
67508d8d1b
update project version to 2.0.0-SNAPSHOT (#6383) 2021-09-28 12:00:43 +08:00
ououtt
652d690976
[DS-6362][fix] fix switch process definition version failed (#6363) 2021-09-27 00:30:59 +08:00
OS
20a3741b24
fix bug #6347 the first schedule_time is error in complement data (#6351) 2021-09-26 11:55:57 +08:00
HanayoZz
4625783479
[Fix-6072][common] Remove Generic method JsonUtils.toMap (#6283) 
* [Fix][common] Remove Generic method JsonUtils.toMap, Replace by parseObj(String json, TypeReference type)
 2021-09-23 23:06:53 +08:00
LiuBodong
c4375a54c1
[Fix][Task] Task lost resource information (#6292) (#6295) 
Co-authored-by: liubodong <liubodong>
 2021-09-23 22:23:17 +08:00
JinYong Li
0b53adeb07
[Fix-6302] [API] Add a new task to an existing workflow, and the task is lost (#6307) 
* fix processDefinition save/update/delete/move/import transaction

* fix ut

* fix processDefinnition release/processInstance update/taskDefinition transaction

* fix processDefinnition release/processInstance update/taskDefinition transaction

* fix 6302 update processDefinition,the task lost

* fix ut

Co-authored-by: JinyLeeChina <297062848@qq.com>
 2021-09-23 15:06:49 +08:00
JinYong Li
7a321505cf
[Fix-6248] [API] fix processDefinition save/update/delete/move/import transaction (#6285) 
* fix processDefinition save/update/delete/move/import transaction

* fix ut

Co-authored-by: JinyLeeChina <297062848@qq.com>
 2021-09-21 23:10:26 +08:00
OS
649723132b
fix bug-6277 submit-time error in parallel tasks. (#6278) 2021-09-20 14:57:53 +08:00