seckiller
89da67ddeb
[fix][worker]:Fix the issue of missing disk usage reporting information in the worker ( #15565 )
...
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-02-06 16:43:29 +08:00
John Huang
69676b445c
[Improvement][UT] Improve Worker registry coverage ( #15380 )
...
Co-authored-by: fuchanghai <changhaifu@apache.org>
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-02-06 10:45:53 +08:00
Wenjun Ruan
4d15932ede
Fix createFile with permission will not work ( #15556 )
2024-02-03 18:00:33 +08:00
Wenjun Ruan
25810a81dc
Exit JVM when OOM ( #15538 )
2024-01-31 13:59:30 +08:00
Wenjun Ruan
64e1e67581
Use DefaultUncaughtExceptionHandler to log the uncached exception ( #15496 )
...
Co-authored-by: fuchanghai <changhaifu@apache.org>
2024-01-18 14:59:17 +08:00
Wenjun Ruan
0b873e6870
[HotFix] Fix createTaskInstanceWorkingDirectory failed if the old path exist ( #15377 )
2024-01-15 11:53:55 +08:00
Wenjun Ruan
7b9c9e0eb2
Enable set ServerLoadProtection fot Master/Worker ( #15439 )
2024-01-14 22:17:58 +08:00
Wenjun Ruan
6c1e001edf
Add config for defaultTenantEnabled ( #15391 )
2024-01-06 18:26:22 +08:00
Gallardot
575b89e2f9
[Bug][Registry] Optimizing waiting strategy ( #15223 )
...
* [Improvement][Registry] Optimizing waiting strategy
Signed-off-by: Gallardot <gallardot@apache.org>
2024-01-02 22:41:50 +09:00
Wenjun Ruan
d6dea4633c
Optimize server startup log ( #15362 )
2023-12-27 20:20:21 +08:00
Wenjun Ruan
5523a62825
Remove taskQueue and looper in worker ( #15292 )
2023-12-27 19:51:45 +08:00
Wenjun Ruan
0e88ea3ac8
Recreate new TaskInstance Working Directory when exist in worker ( #15358 )
2023-12-25 16:00:41 +08:00
Wenjun Ruan
5b6b0ceb31
Fix resource file usage(Delete Resource/ResourceUser which is deprecated) ( #15355 )
2023-12-23 22:47:59 +08:00
Wenjun Ruan
2119e41800
[Improvement] Move delay calculation to Master ( #15278 )
2023-12-06 09:37:14 +08:00
Wenjun Ruan
43f5f24529
Add dolphinscheduler-extract-common module ( #15266 )
2023-12-05 21:06:39 +08:00
Wenjun Ruan
ef2e9c9b67
Set the tenant as the owner in final stage ( #15256 )
2023-12-01 21:49:28 +08:00
Wenjun Ruan
f2d146665c
Use chown to set the file owner ( #15240 )
2023-11-29 16:28:39 +08:00
Wenjun Ruan
d6175f3bbf
Exclude DataSourceAutoConfiguration in worker server ( #15169 )
2023-11-16 19:17:39 +08:00
dbac
d683553324
[Improvement][Master] Calculate the remainTime then we set the delay execution. ( #15012 )
...
* [#15007 ][fix][worker-server] fix bug task delay execution
* [#15007 ][fix][worker-server] fix bug task delay execution
mvn spotless:apply
* [#15007 ][fix][worker-server] fix bug task delay execution
worker fix task delay logic
---------
Co-authored-by: 旺阳 <wang@lqwang.net>
2023-10-11 02:32:07 -05:00
Wenjun Ruan
ab29ae77c1
Exclude DataSourceAutoConfiguration in WorkerServer ( #14957 )
2023-09-23 22:01:54 +08:00
Aaron Wang
73e846d03e
[Improvement-14884][Master] Add overload state in master heartbeat to trigger slot change ( #14887 )
2023-09-14 10:36:42 +08:00
Aaron Wang
b12df72a0c
[Fix][Resource Center] Remove NONE storage type and set LOCAL type as default ( #14661 )
2023-09-07 14:09:15 +08:00
Wenjun Ruan
742f0c3e60
Add workflowInstanceHost and taskInstanceHost in ITaskInstanceExecutionEvent ( #14827 )
2023-08-29 13:32:17 +08:00
Wenjun Ruan
d223d654cc
Add rpc benchmark test ( #14797 )
2023-08-27 20:59:55 +08:00
Wenjun Ruan
55cc77226f
[RPC] Refactor the remote module. ( #14776 )
...
* Add dolphinscheduler-extract-base
* Add dolphinscheduler-extract-master
* Add dolphinscheduler-extract-worker
* Add dolphinscheduler-extract-alert
2023-08-23 09:43:17 +08:00
Wenjun Ruan
93b0283dfe
Set tenant as the the resource file owner ( #13832 )
2023-08-11 00:57:47 +08:00
Wenjun Ruan
3efcf0c5ac
Fix message in MessageRetryRunner might disorder ( #14725 )
2023-08-09 20:19:02 +08:00
Wenjun Ruan
9ce8871158
Support get property from env/jvm/common.properties ( #14645 )
2023-07-28 14:41:58 +08:00
Aaron Wang
ca760aabd4
[Fix-14512][BOM] kubernetes-client version conflict ( #14561 )
2023-07-21 18:17:53 +08:00
Wenjun Ruan
cbd354d134
Add host/port in heartbeat ( #14591 )
2023-07-20 21:26:35 +08:00
Wenjun Ruan
399d0d0ce9
Add log for NetUtils ( #14578 )
2023-07-18 21:17:17 +08:00
Gallardot
9d10de6efa
[Improvement][Task] Mask configYaml in task log ( #14571 )
...
* [Improvement][Task] Mask configYaml in task log
Signed-off-by: Gallardot <gallardot@apache.org>
---------
Signed-off-by: Gallardot <gallardot@apache.org>
Co-authored-by: Aaron Wang <wangweirao16@gmail.com>
2023-07-18 15:02:20 +08:00
Rick Cheng
5a6f1c88a3
[Fix-14503][Worker] Fix the problem of subprocess cannot be killed ( #14504 )
...
Co-authored-by: xiangzihao <460888207@qq.com>
2023-07-17 21:43:34 +08:00
Wenjun Ruan
0246327083
Polish config, expose rpc config in application.yml ( #14501 )
2023-07-11 09:29:31 +08:00
Wenjun Ruan
406aeaf659
Use jvm_args_env.sh to set JVM args ( #14487 )
2023-07-10 17:39:57 +08:00
Eric Gao
e86630bb7b
[Improvement][Metrics] Add more worker related metrics and fix some previous ones ( #14254 )
...
* Add more worker related metrics and fix some previous ones
* update metrics docs
2023-06-01 16:54:48 +08:00
Rick Cheng
428f1559a3
[Improv-14083][Resource Center] Handle deleteSource in OSS / S3 / GCS ( #14084 )
2023-05-22 20:44:43 +08:00
Wenjun Ruan
05df0269f6
Fix task exception might not exist in task instance log ( #14085 )
2023-05-21 09:55:35 +08:00
Aaron Wang
74ed7856bc
[Bug-14105][Task] Replace default tenant when downloading resource ( #14106 )
2023-05-16 18:11:58 +08:00
Aaron Wang
1e3ae7b942
[improv][Resource Center] Implement getResourceFileName in StorageOperator ( #14097 )
2023-05-15 14:02:30 +08:00
Rick Cheng
3eb4c0ed1d
[Improvement-14054][Worker] Kill multiple yarn apps at the same time ( #14055 )
...
* [Improvement-14054][Worker] Kill multiple yarn apps at the same time
* Update dolphinscheduler-task-plugin/dolphinscheduler-task-api/src/main/java/org/apache/dolphinscheduler/plugin/task/api/am/YarnApplicationManager.java
Co-authored-by: Aaron Wang <wangweirao16@gmail.com>
2023-05-06 19:13:12 +08:00
Wenjun Ruan
cf91544b99
Refactor logic task ( #13948 )
2023-04-26 10:01:51 +08:00
klaokai
f6e0f2e471
[Bug] [worker] WorkerConfig output is wrong ( #14000 ) ( #14001 )
2023-04-24 19:04:34 +08:00
Aaron Wang
731123cebc
[Bug][Worker] get worker status bug ( #13992 )
...
* get worker status bug
* modify configuration name
2023-04-24 09:51:47 +08:00
Aaron Wang
45586f71d5
[BUG][Resource Center] Task can not select main JAR ( #13536 )
2023-04-18 18:38:16 +08:00
Wenjun Ruan
61a689aa5a
Use percentage to represent memory/cpu usage ( #13896 )
2023-04-13 15:48:20 +08:00
Rick Cheng
7ee66f2d02
[Feature-13429][Remote Logging] Add support for writing task logs to Google Cloud Storage ( #13777 )
2023-04-06 17:49:33 +08:00
Wenjun Ruan
41a8ba9aab
Support alert server HA ( #13865 )
...
* Support alert server cluster
* Remove WORKER_ALERT_LISTEN_HOST in statefulset-dolphinscheduler-worker.yaml
2023-04-05 17:31:50 +08:00
Aaron Wang
d6fcf60534
[Improvement-13751][Worker] Support real-time pod log collection ( #13752 )
...
* Support real-time pod log collection
* datax task test have to mock ShellCommandExecutor with whole dependencies
* replace \n with EMPTY_STRING
* modify error log msg
* Update dolphinscheduler-task-plugin/dolphinscheduler-task-api/src/main/java/org/apache/dolphinscheduler/plugin/task/api/AbstractCommandExecutor.java
Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
* throw exception in pod log collection thread
---------
Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
2023-04-05 16:13:45 +08:00
Wenjun Ruan
64833952fc
Remove the extra default tenant in workflow save page ( #13867 )
2023-04-04 17:14:40 +08:00
