Commit Graph

8357 Commits

Author SHA1 Message Date
Wenjun Ruan
6862fe7548
Removed unused StateEventHandleFailure (#16052) 2024-05-23 11:35:46 +08:00
Wenjun Ruan
56ecec779c
Fix NettyRemotingClient might throw IllegalMonitorStateException (#16038) 2024-05-22 10:26:04 +08:00
小可耐
0fc2fe971a
[DSIP-39][parameter] Improvement startup parameters/global parameters/project parameters data type (#15967)
* [Improvement][parameter] New data types and type filtering

* [Improvement][parameter] Improvement startup parameters/global parameters data type

* fix api interfaces compatible

* add project parameter data type default value

* [Improvement][parameter] New data types and type filtering

* [Improvement][parameter] Improvement startup parameters/global parameters data type

* fix api interfaces compatible

* add project parameter data type default value

* improvement project code

* remove useless imports

* remove method onClearSearchTaskType

* add parameter doc

* optimisation logic

* code conflict resolution

* code conflict resolution

* [Improvement][Monitor] Show master && worker Busy Or Normal Status and Show Commands table list (#15978)

* update

* test

* add monitor enhance ui

* update

* update

* update doc

* fix spotless

* update

* update

* Update dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/DataAnalysisController.java

Co-authored-by: Wenjun Ruan <wenjun@apache.org>

* Update dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/DataAnalysisController.java

Co-authored-by: Wenjun Ruan <wenjun@apache.org>

* Update dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/ErrorCommandMapper.java

Co-authored-by: Wenjun Ruan <wenjun@apache.org>

* Update dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/ErrorCommandMapper.xml

Co-authored-by: Wenjun Ruan <wenjun@apache.org>

* Update dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/CommandMapper.java

Co-authored-by: Wenjun Ruan <wenjun@apache.org>

* Update dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/ErrorCommandMapper.xml

Co-authored-by: Wenjun Ruan <wenjun@apache.org>

* update

* fix spotless

* update

---------

Co-authored-by: Wenjun Ruan <wenjun@apache.org>

* [Improvement][Monitor] Add UT for montor (#15998)

* formatting Code
2024-05-21 15:39:43 +08:00
TianXinCoord
cf400b86ba
Correction of Typos in the Chinese Document Appendix for Task Parameters (#16033)
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-05-21 14:00:52 +08:00
Rick Cheng
dbd790ddac
[Fix][CI] fix the ci error of Values.datasource.profile (#16031)
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2024-05-21 11:16:30 +08:00
Wenjun Ruan
e4df01a033
Increase block-until-connected in ZookeeperRegistryTestCase (#16041) 2024-05-21 10:32:15 +08:00
Wenjun Ruan
b184364561
Fix WorkerTaskExecutorThreadPool#isOverload is incorrect (#16027) 2024-05-20 19:13:53 +08:00
Wenjun Ruan
5463d0255a
Fix rpc channel leak due to concurrent operation (#16021)
* Fix rpc channel leak due to concurrent operation

* Throw channel create failed exception

---------

Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-05-20 12:10:33 +08:00
jiabei
7b7a33337f
docs: fix type in contribute.md (#16022)
Co-authored-by: jxbwhe <iswanghe@aliyun.com>
2024-05-18 21:34:08 +08:00
Wenjun Ruan
cb0b4fa52f
Deprecated 2.0.9/3.0.6 upgrade test (#16012) 2024-05-17 19:17:54 +08:00
Wenjun Ruan
4e73fc28b0
Use random port in registry IT (#16014) 2024-05-17 17:35:44 +08:00
JohnHuang
79f90a87f8
[Feature][Helm] make api/master/worker/alert application.yaml configurable (#15922) 2024-05-17 15:23:21 +08:00
Wenjun Ruan
61915a2d5d
Fix workflow instance restart failed due to duplicate key in varpool (#16001) 2024-05-16 11:32:27 +08:00
旺阳
68a190674f
[Improvement][Monitor] Add UT for montor (#15998) 2024-05-15 14:44:04 +08:00
旺阳
0e5cb664bb
[Improvement][Monitor] Show master && worker Busy Or Normal Status and Show Commands table list (#15978)
* update

* test

* add monitor enhance ui

* update

* update

* update doc

* fix spotless

* update

* update

* Update dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/DataAnalysisController.java

Co-authored-by: Wenjun Ruan <wenjun@apache.org>

* Update dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/DataAnalysisController.java

Co-authored-by: Wenjun Ruan <wenjun@apache.org>

* Update dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/ErrorCommandMapper.java

Co-authored-by: Wenjun Ruan <wenjun@apache.org>

* Update dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/ErrorCommandMapper.xml

Co-authored-by: Wenjun Ruan <wenjun@apache.org>

* Update dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/CommandMapper.java

Co-authored-by: Wenjun Ruan <wenjun@apache.org>

* Update dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/ErrorCommandMapper.xml

Co-authored-by: Wenjun Ruan <wenjun@apache.org>

* update

* fix spotless

* update

---------

Co-authored-by: Wenjun Ruan <wenjun@apache.org>
2024-05-15 13:58:59 +08:00
Gee
4f4a13da02
[Fix-15992][server] maintain slot index by iterate MasterPriorityQueue orderly (#15993) 2024-05-15 11:44:27 +08:00
Wenjun Ruan
1a4537ecbd
Fix database schema change check failed with pg in ci (#15995)
Co-authored-by: xiangzihao <460888207@qq.com>
2024-05-15 10:36:27 +08:00
John Bampton
8d6e9eecfc
docs: fix spelling (#15996) 2024-05-15 09:41:02 +08:00
Eric Gao
dc306bfa1d
[Improvement] Use safe constructor with snake yaml (#15758) 2024-05-14 18:04:04 +08:00
licc6
680d2e0b05
Repair resource configuration exist typo (#15988)
Co-authored-by: fuchanghai <changhaifu@apache.org>
2024-05-13 19:47:47 +08:00
Wenjun Ruan
7c8fa9b48c
Add IntegretionTest for registry module (#15981) 2024-05-13 16:35:37 +08:00
Wenjun Ruan
3446fd8ab1
EMR task support replace params placeholder (#15975)
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2024-05-10 17:34:26 +08:00
Gallardot
ace20f96c2
[Bug] [Helm] No DB Nodes Exist (#15970) 2024-05-10 15:43:00 +08:00
JohnHuang
ad1a6af4fb
Add link to ETCD/JDBC Registry Guideline (#15597) 2024-05-09 16:52:37 +08:00
cntiger
60b019b729
[Improvement] Fix the git url command injection in pytorch task(#15873) (#15950)
* fix the git url command injection danger(#15873)

* [Improvement] Fix the git url command injection in pytorch,format code style task(#15873)

---------

Co-authored-by: cntigers <Xiaohu4321@>
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-05-09 14:50:27 +08:00
Zzih96
5c569b705c
[fix-15907] Fix get remote shell exit code is incorrect (#15911)
* ./mvnw spotless:apply

* Update dolphinscheduler-task-plugin/dolphinscheduler-task-remoteshell/src/main/java/org/apache/dolphinscheduler/plugin/task/remoteshell/RemoteExecutor.java

Co-authored-by: Wenjun Ruan <wenjun@apache.org>

---------

Co-authored-by: 詹子恒 <ziheng.zhan@longbridge.sg>
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-05-09 14:15:06 +08:00
刘阳阳(liuyangyang)
ba5de75829
Add tenantCode propagation to DynamicCommandUtils.createCommand (#15956) 2024-05-09 13:06:22 +08:00
Wenjun Ruan
8d336def61
[DSIP-35][Alert] Refactor the alert thread model (#15932) 2024-05-09 12:23:01 +08:00
privking
bbca37d03e
[FIX] Completed tasks cannot be re-executed in a workflow instance (#15884)
* fix bug: Failed to resume stopped workflow instance

* Revert "fix bug: Failed to resume stopped workflow instance"

This reverts commit 1546e9d5a51178a94bedd18a718b15431355428b.

* fix bug : Completed tasks cannot be re-executed in a workflow instance

---------

Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-05-09 11:36:56 +08:00
xiangzihao
8426d2346c
[HotFix] [CI] Temporary skipping mergeable check (#15958)
* temporary skipping mergeable check
2024-05-09 11:01:05 +08:00
Wenjun Ruan
99d8276be7
Optimizing the scope of RPC base classes (#15946)
* Optimizing the scope of RPC base classes

* Fix UT
2024-05-08 11:46:09 +08:00
JohnHuang
d218b021e7
[DSIP-25][Remote Logging] Split remote logging configuration (#15826)
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-05-06 11:33:16 +08:00
Evan Sun
fa6ea8ba7b
[TEST] increase coverage of project workergroup relation service (#15944)
Co-authored-by: abzymeinsjtu <abzymeinsjtu@B-54Q8MD6R-0244.local>
2024-05-06 10:02:04 +08:00
Evan Sun
ebcdaeb9ac
[TEST] increase coverage of project preference service test (#15939) 2024-04-29 21:03:01 +08:00
Wenjun Ruan
0a11cd21bd
Fix jar path is not correct in java task (#15906) 2024-04-29 18:07:05 +08:00
Wenjun Ruan
647cbae400
[DSIP-32][Master] Add command fetcher strategy for master fetch command (#15900) 2024-04-29 16:14:23 +08:00
calvin
5a6c6c37f0
[Improvement-15910][UI] Supposed to provide a default value for the custom parallelism when using the mode of parallel execution. (#15912)
* worked out the issue

* imrpove the parallism strategy

* imrpove the parallism strategy

* merge from dev
2024-04-28 18:47:20 +08:00
小可耐
7a55adeae9
[Improvement-15919][datasource] Improvement datasource get name (#15920) 2024-04-28 11:02:44 +08:00
DaqianLiao
b29965bdce
[Fix] In updateWorkerNodes method, the workerNodeInfoWriteLock should be used. #15898 (#15903)
Co-authored-by: answerliao <answerliao@tencent.com>
2024-04-26 11:24:05 +08:00
Wenjun Ruan
d1135eabc7
[DSIP-34] Change required_approving_review_count to 2 (#15914) 2024-04-25 19:39:56 +08:00
Wenjun Ruan
446f6ba72b
Fix auto create tennat concurrently will cause the task failed (#15909) 2024-04-25 18:14:48 +08:00
Wenjun Ruan
b3b8c0784d
Fix kill dynamic task doesn't kill the wait to run workflow instances (#15896) 2024-04-25 11:05:51 +08:00
Evan Sun
1d13ef09c3
[TEST] increase coverage of project parameter service test (#15905)
Co-authored-by: abzymeinsjtu <abzymeinsjtu@B-54Q8MD6R-0244.local>
2024-04-24 23:31:56 +08:00
旺阳
59f060e278
[Improvement] Fix alert code smell 2024-04-23 15:10:08 +08:00
Wenjun Ruan
e9d85914d7
Fix queryByTypeAndJobId might error due to multiple result (#15883) 2024-04-22 17:41:17 +08:00
Wenjun Ruan
e2c8b080f9
[DSIP-31] Reduce the thread pool size of hikari (#15890) 2024-04-22 17:05:24 +08:00
John Huang
6c78c8ec9a
[Improvement][Spark] Support Local Spark Cluster (#15589)
* [Improvement][Spark] Support Local Spark Cluster

* remote default local from deploy mode

---------

Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-04-22 15:05:59 +08:00
privking
e66441a2c9
[FIX] Fix cannot recover a stopped workflow instance (#15880) 2024-04-20 07:32:18 +08:00
Wenjun Ruan
285c5a8eb5
[DSIP-28] Donnot scan whole bean under classpath (#15874) 2024-04-19 18:12:40 +08:00
Gallardot
a9decc911f
[Bug][Helm] fix image.registry (#15860)
Signed-off-by: Gallardot <gallardot@apache.org>
Co-authored-by: fuchanghai <changhaifu@apache.org>
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-04-19 16:52:17 +08:00