test/DolphinScheduler
1
0
Fork 0
mirror of https://gitee.com/dolphinscheduler/DolphinScheduler.git synced 2024-12-03 12:48:46 +08:00
Code Issues Actions 10 Packages Projects Releases Wiki Activity
7,317 Commits 27 Branches 59 Tags 276 MiB
aa18878122
Commit Graph

107 Commits

Author SHA1 Message Date
zhuangchong
a3a9cb9d46 [maven-release-plugin] prepare for next development iteration 2022-12-07 17:23:58 +08:00
zhuangchong
f1aefae5e2 [maven-release-plugin] prepare release 3.1.2 2022-12-07 17:23:57 +08:00
Kerwin
937d636613 Fix execute shell task exception no dolphinscheduler_env.sh file execute permission (#12909) 2022-11-17 21:44:55 +08:00
zhuangchong
48ad9ff412 [maven-release-plugin] prepare for next development iteration 2022-11-08 11:18:29 +08:00
zhuangchong
efc0367999 [maven-release-plugin] prepare release 3.1.1 2022-11-08 11:18:28 +08:00
Wenjun Ruan
fe3e93861f
Add switch for checkOwn (#197) (#12803) 2022-11-08 10:44:17 +08:00
zhuangchong
145c5ac8cb [maven-release-plugin] prepare for next development iteration 2022-11-07 20:43:01 +08:00
zhuangchong
525a6846ac [maven-release-plugin] prepare release 3.1.1 2022-11-07 20:43:00 +08:00
Kerwin
2c93dc6041
cherry-pick (#12608) 
[improvement-#12416] Use CollectionUtils.isEmpty judge the collection isEmpty to avoid NPE
#12449

Co-authored-by: fuchanghai <33984497+fuchanghai@users.noreply.github.com>
 2022-10-29 21:48:37 +08:00
Kerwin
94ef02e9de
cherry-pick [Improvement] Merge spi.utils into common.utils (#12607) 
#12502

Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
 2022-10-29 21:30:28 +08:00
Wenjun Ruan
75f6c416fb Add task executor threads full policy config in worker (#12510) 2022-10-27 14:52:15 +08:00
Wenjun Ruan
a5a849f615 cherry-pick [Bug] Set tenantDir permission 
#12486
 2022-10-27 14:02:57 +08:00
kezhenxu94
eb7b48596c cherry-pick Reorganize some classes in common module, remove duplicate classes 
#12321
 2022-10-27 13:53:41 +08:00
Wenjun Ruan
a17d0cc5d2 cherry-pick [Improvement][WorkerGroup]Remove workerGroup in registry #12217 2022-10-27 13:21:24 +08:00
Kerwin
58ef3cccd7
[3.1.1-prepare]cherry-pick [Improvement] remove log-server and server module #12206 (#12562) 2022-10-27 12:54:38 +08:00
Kerwin
f3277277f0
cherry-pick Refactor LogServiceClient Singleton to avoid repeat creation of NettyClient #11777 (#12542) 
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
 2022-10-26 16:20:13 +08:00
Kerwin
344293102d
[3.1.1-prepare][cherry-pick][Improvement] Add remote task model #11767 (#12541) 2022-10-26 15:44:13 +08:00
Kerwin
2402b8a6ef
Use temurin Java docker images instead of deprecated ones (#12334) (#12421) 
Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>
 2022-10-18 18:51:14 +08:00
Kerwin
d2e56af838
[Bug] [spark-sql] In spark-sql, select both SPARK1 and SPARK2 versions and execute ${SPARK_HOME2}/bin/spark-sql (#11721) (#12420) 
select different versions of spark-sql to execute different versions of spark-sql

Co-authored-by: limaiwang <limaiwang@163.com>
 2022-10-18 18:45:54 +08:00
sgw
703f9991b4 [DS-12154][worker] Optimize the log printing of the worker module (#12183) 
* [DS-12154][worker] Optimize the log printing of the worker module according to the log specification.
 2022-10-18 09:28:07 +08:00
caishunfeng
7e39396a76 [maven-release-plugin] prepare for next development iteration 2022-09-26 21:50:03 +08:00
caishunfeng
ae33ba5947 [maven-release-plugin] prepare release 3.1.0 2022-09-26 21:49:27 +08:00
Daniel Y
2a437607ae [Improvement-11834] Upgrade docker base image to support python3.9 (#11835) 
* use openjdk:8-jre-slim-bullseye instead of buster to support python3.9

* use openjdk:8-jre-slim-bullseye instead of buster across all components
 2022-09-19 15:50:25 +08:00
LiuBodong
0f3b42925f [Fix][Monitor]Monitor UI not show DisakAvailable and MemoryUsage correctly (#11870) 2022-09-19 15:30:49 +08:00
Kengo Seki
52b79b017e [Improvement] Replace commons-lang 2 function invocations with commons-lang3 (#11810) 
* [Improvement] Replace commons-lang 2 function invocations with commons-lang3
 2022-09-19 14:48:17 +08:00
caishunfeng
f034a09d25
[Bug-11650][worker] #11650 fix SQL type task, stop task cause NPE (#11668) (#11958) 
Co-authored-by: 冯剑 <35831367+fengjian1129@users.noreply.github.com>
 2022-09-15 14:24:34 +08:00
kezhenxu94
277f137358
Add Kubernetes configmap reload to all components (#11730) 2022-09-02 12:03:19 +08:00
Wenjun Ruan
67e7f88d8b
Refactor heart beat task, use json to serialize/deserialize (#11702) 
* Refactor heart beat task, use json to serialize/deserialize
 2022-08-31 16:20:23 +08:00
Wenjun Ruan
03e1e6fe45
[Bug] [Worker] Optimize the getAppId method to avoid worker OOM when kill task (#11701) 
* Fix kill job may cause worker oom
 2022-08-31 15:25:01 +08:00
Wenjun Ruan
1b120e3a59
Refactor worker execute task process (#11540) 
* Refactor worker execute task process
 2022-08-26 13:33:51 +08:00
JinYong Li
3f2ca7bca3
[Fix-9980] [Server] fix heartBeatTaskCount bug (#11232) 
* fix heartBeat bug

* modify class name

* fix conflict

Co-authored-by: JinyLeeChina <jiny.li@foxmail.com>
 2022-08-23 11:30:13 +08:00
Wenjun Ruan
3516533017
Remove logger header in task log file (#11555) 2022-08-19 14:01:52 +08:00
Eric Gao
9330d6cfcd
[Doc][Security] Update instructions on worker groups (#11483) 
* Update instructions on worker groups
 2022-08-15 17:44:00 +08:00
Wenjun Ruan
7ff34c3947
[Feature-7024] Add waiting strategy to support master/worker can recover from registry lost (#11368) 
* Add waiting strategy to support master/worker can recover from registry lost

* throw exception when zookeeper registry start failed due to interrupted
 2022-08-13 09:52:03 +08:00
caishunfeng
0464123c2b
[Feature-11223] support stream task (#11350) 
* add task execute type

* update task definition list paging

* update task instance list paging

* stream task start

* [Feature][UI] Some changes to execute task.
    * Set the connection edge to dashed line.
    * Add FLINK_STREAM task.

* add stream task

* flink savepoint and cancel

* fix query bug

* add stream task definition

* add task instance for stream task

* delete stream task definition state

* update api for stream task definition edit

* modify search for stream task instance

* add language

* delete task type search for stream task definition

* change task type search for stream task instance

* add jump button

* add savepoint

* add down log for stream task instance

* ui test

* stream task start

* run DAG

* [Fix][UI] Fix the stream task edgs not to be dashed when filling back.

* [Feature][UI] Remove some fields for FLINK_STREAM.

* add start modal

* add dryRun column for stream task instance

* fix duration

* fix pon

* fix build error

* Add success tip

* add auto sync for stream task instance

* remove forgien key for task instance

* license header

* UT fix

* modify locales

* recover common config

* fix UT

* add doc

Co-authored-by: Amy <amywang0104@163.com>
Co-authored-by: devosend <devosend@gmail.com>
 2022-08-10 21:44:43 +08:00
Wenjun Ruan
8774415197
Split ExecutionStatus to WorkflowExecutionStatus and TaskExecutionStatus (#11340) 2022-08-10 11:00:23 +08:00
Eric Gao
9ca1eb96c4
[Improvement][Metrics] Add metrics for alert server (#11240) 
* [Improvement][Metrics] Add metrics for alert server (#11131)

* Update related docs of metrics

* Add grafana demo dashboards for alert server metrics

* Refactor metric classes with UtilityClass annotation

* Refactor meter names in camelCase for checkstyle
 2022-08-03 15:42:06 +08:00
xuhhui
bfff3a7c5d
fix error (#11206) 2022-07-30 18:20:20 +08:00
zhuxt2015
3701a24d15
[Improvement][Task Log] Task status log print description instead of code (#11009) 
* use execution status instead of status code
 2022-07-22 13:34:31 +08:00
Wenjun Ruan
5e9c7dad23
Add dolphinscheduler-bom to manage the dependency version (#11025) 2022-07-20 10:37:31 +08:00
zhuxt2015
a74d7ef665
[hotfix][Worker] Remove service dependency from worker module (#11008) 
* worker remove service dependency
 2022-07-17 22:16:35 +08:00
Wenjun Ruan
083ab2b5c9
Remove dao in worker (#10994) 2022-07-15 20:07:18 +08:00
Wenjun Ruan
2be1d4bf0a
Fix worker cannot shutdown due to resource close failed or heart beat check failed (#10979) 
* Use try-with-resource to close resource, and add heart error threshold to avoid worker cannot close due to heart beat check failed

* Move heartbeat error threshold to applicaiton.yml
 2022-07-15 20:06:53 +08:00
Wenjun Ruan
cade66a9b6
[Fix-10827] Fix network error cause worker cannot send message to master (#10886) 
* Fix network error cause worker cannot send message to master
 2022-07-12 14:08:42 +08:00
Eric Gao
2f7281c2d2
[Feature][Metrics] Add resource download related metrics for workers (#10749) 
* [Feature][Metrics] Add resource download related metrics for workers (#9324)

* [Feature][Metrics] Fix bugs and add grafana demos for worker resource download metrics (#9324)

* [Feature][Metrics] Add docs to resource related metrics (#9324)

* [Feature][Metrics] Use tags to indicate status in metrics (#9324)

* [Feature][Metrics] Fix demos, docs and remove redundant code (#9324)

* [Feature][Metrics] Remove .pnpm-debug.log (#9324)

* [Feature][Metrics] Fix style check (#9324)

* [Feature][Metrics] Replace KB with bytes for the unit of resource file size in metrics (#9324)

* [Feature][Metrics] Make code neat (#9324)
 2022-07-12 11:44:34 +08:00
Wenjun Ruan
f639a2eed4
[Fix-10854] Fix database restart may lost task instance status (#10866) 
* Fix database update error doesn't rollback the task instance status

* Fix database error may cause workflow dead with running status
 2022-07-11 09:57:00 +08:00
Wenjun Ruan
426567348e
Remove quartz in service (#10748) 
* Remove quartz in service
 2022-07-06 15:43:55 +08:00
Wenjun Ruan
67d14fb7b3
[Fix-10785] Fix state event handle error will not retry (#10786) 
* Fix state event handle error will not retry

* Use state event handler to deal with the event
 2022-07-06 14:53:28 +08:00
WangJPLeo
8f621ff98b
[Optimization] Calculate global parameter and local parameter at master. (#10704) 
* Global parameter and local parameter calculation external expansion.

* k8s task ut fix.

* TimePlaceholderUtils import DateUtils fix

* follow the review comments to fix.

* follow the review comments to fix.

* e2e rerun
 2022-06-30 22:45:25 +08:00
Wenjun Ruan
35b25da863
Validate master/worker config (#10649) 2022-06-28 20:17:43 +08:00