Commit Graph

542 Commits

Author SHA1 Message Date
xiangzihao
bec678a8b9
[DSIP-78][Data Quality] Remove data quality module (#16794) 2024-11-20 09:20:41 +08:00
小可耐
5f319e5183
[Fix-16705] [Built-in Parameter] Fix project name and definition name variable parameter not resolved (#16715) 2024-10-25 15:29:57 +08:00
Wenjun Ruan
62d33a799a
[DSIP-75] Support auto add license header by spotless (#16643) 2024-09-22 16:27:29 +08:00
xiangzihao
fee4955724
[DSIP-68] Unify the table structure and table fields in backend and frontend ui sections (#16544) 2024-09-04 21:22:34 +08:00
xiangzihao
4c195063f9
[Improvement-16516] Uniformly name process in backend code as workflow (#16524) 2024-08-29 10:45:22 +08:00
Wenjun Ruan
20a0e50cce
[DSIP-67] Use command to trigger workflow instance rather generate workflow instance (#16523) 2024-08-28 19:27:40 +08:00
Wenjun Ruan
9448806af0
[DSIP-61][Master] Refactor thread pool and state event orchestration in master (#16327) 2024-08-26 15:19:39 +08:00
xiangzihao
d9244ca22f
[DSIP-57][Audit] Revert PR 14981 code (#16370) 2024-07-26 14:54:44 +08:00
xiangzihao
761323f3e1
[DSIP-53] Suggest add process lineage parsing for dependent task (#16288)
* add dsip 53
2024-07-11 21:25:06 +08:00
Wenjun Ruan
8f6df8d840
[DSIP-54][Master] Use ClusterManager to manage the cluster in master (#16263) 2024-07-05 11:55:00 +08:00
xiangzihao
e8566818b0
[DSIP-50][UDF Manage] Remove unused udf manage function (#16212)
* Remove unused udf manage function
2024-06-26 21:08:48 +08:00
Wenjun Ruan
ec6a3e0727
[DSIP-46] Unify the logic task params data structure between front-end and back-end (#16193) 2024-06-25 13:32:13 +08:00
Wenjun Ruan
b34fe46044
Fix task cannot use workflow's environment (#16199) 2024-06-24 20:27:25 +08:00
Wenjun Ruan
5df4b1cbc3
Fix condition task will cause workflow instance failed (#16152) 2024-06-19 11:28:35 +08:00
sleo
26f2aadb06
Rename variable QUEUE_MAX_SIZE to QUEUE_INITIAL_SIZE (#16040) 2024-05-24 20:09:10 +08:00
小可耐
0fc2fe971a
[DSIP-39][parameter] Improvement startup parameters/global parameters/project parameters data type (#15967)
* [Improvement][parameter] New data types and type filtering

* [Improvement][parameter] Improvement startup parameters/global parameters data type

* fix api interfaces compatible

* add project parameter data type default value

* [Improvement][parameter] New data types and type filtering

* [Improvement][parameter] Improvement startup parameters/global parameters data type

* fix api interfaces compatible

* add project parameter data type default value

* improvement project code

* remove useless imports

* remove method onClearSearchTaskType

* add parameter doc

* optimisation logic

* code conflict resolution

* code conflict resolution

* [Improvement][Monitor] Show master && worker Busy Or Normal Status and Show Commands table list (#15978)

* update

* test

* add monitor enhance ui

* update

* update

* update doc

* fix spotless

* update

* update

* Update dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/DataAnalysisController.java

Co-authored-by: Wenjun Ruan <wenjun@apache.org>

* Update dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/DataAnalysisController.java

Co-authored-by: Wenjun Ruan <wenjun@apache.org>

* Update dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/ErrorCommandMapper.java

Co-authored-by: Wenjun Ruan <wenjun@apache.org>

* Update dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/ErrorCommandMapper.xml

Co-authored-by: Wenjun Ruan <wenjun@apache.org>

* Update dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/CommandMapper.java

Co-authored-by: Wenjun Ruan <wenjun@apache.org>

* Update dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/ErrorCommandMapper.xml

Co-authored-by: Wenjun Ruan <wenjun@apache.org>

* update

* fix spotless

* update

---------

Co-authored-by: Wenjun Ruan <wenjun@apache.org>

* [Improvement][Monitor] Add UT for montor (#15998)

* formatting Code
2024-05-21 15:39:43 +08:00
Gee
4f4a13da02
[Fix-15992][server] maintain slot index by iterate MasterPriorityQueue orderly (#15993) 2024-05-15 11:44:27 +08:00
Wenjun Ruan
647cbae400
[DSIP-32][Master] Add command fetcher strategy for master fetch command (#15900) 2024-04-29 16:14:23 +08:00
Wenjun Ruan
e9d85914d7
Fix queryByTypeAndJobId might error due to multiple result (#15883) 2024-04-22 17:41:17 +08:00
Wenjun Ruan
285c5a8eb5
[DSIP-28] Donnot scan whole bean under classpath (#15874) 2024-04-19 18:12:40 +08:00
Wenjun Ruan
76d059810a
Forbidden forcess success a task instance in a running workflow instance (#15855) 2024-04-16 16:19:57 +08:00
Wenjun Ruan
27d0563fe4
Bind processId to constructor CodeGenerator (#15848) 2024-04-16 10:58:43 +08:00
旺阳
226345512e
[DSIP-26][Audit log] Audit log improvement design (#15554) 2024-04-15 13:40:35 +08:00
旺阳
5050a8e1e6
[Improve] Fix typo on ProcessServiceImpl (#15817) 2024-04-09 14:38:47 +08:00
Rick Cheng
b652967c53
[Improvement-15719] Remove the useless methods in ProcessUtils (#15720) 2024-03-18 12:22:59 +08:00
Frank-Liao
e97f6a41ed
Check whether the key exists when "start param to overwrite global param" (#15676) (#15677)
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-03-15 17:33:18 +08:00
Wenjun Ruan
1b32e28cc8
Set the workflow instance ready state to running in failover (#15572) 2024-02-06 15:25:50 +08:00
Wenjun Ruan
43a06525a2
Fix Recover WorkflowInstance will casue workflow Instance state is success but task insatnce is killed/paused (#15574) 2024-02-06 10:09:30 +08:00
Jay Chung
01eb8f834f
fix: start param for wf not work (#15544)
* fix: start param for wf not work

fix: #15280

* fix test
2024-02-05 16:17:30 +08:00
Wenjun Ruan
4d15932ede
Fix createFile with permission will not work (#15556) 2024-02-03 18:00:33 +08:00
Wenjun Ruan
89742332a8
Fix TaskGroupQueue will never be wakeup due to wakeup failed at one time (#15528) 2024-02-01 17:13:58 +08:00
Wenjun Ruan
44c356db4d
TaskGroupPriority only compare When TaskGroup is same (#15486) 2024-01-18 10:59:54 +08:00
Wenjun Ruan
0b873e6870
[HotFix] Fix createTaskInstanceWorkingDirectory failed if the old path exist (#15377) 2024-01-15 11:53:55 +08:00
Gallardot
a405abec9e
[Bug][Master]serial_wait strategy workflow unable to wake up (#15270)
* fix: serial_wait strategy workflow unable to wake up

Signed-off-by: Gallardot <gallardot@apache.org>

* fix: serial_wait strategy workflow unable to wake up

Signed-off-by: Gallardot <gallardot@apache.org>

---------

Signed-off-by: Gallardot <gallardot@apache.org>
Co-authored-by: fuchanghai <changhaifu@apache.org>
2024-01-12 17:53:24 +08:00
Wenjun Ruan
5b6b0ceb31
Fix resource file usage(Delete Resource/ResourceUser which is deprecated) (#15355) 2023-12-23 22:47:59 +08:00
Wenjun Ruan
43f5f24529
Add dolphinscheduler-extract-common module (#15266) 2023-12-05 21:06:39 +08:00
Wenjun Ruan
9c7ad6df0a
[Improvement] Clean up Scheduler logic (#15198)
* Clean up Scheduler logic

* Update dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProcessDefinitionServiceImpl.java

Co-authored-by: 旺阳 <wang@lqwang.net>

---------

Co-authored-by: 旺阳 <wang@lqwang.net>
2023-11-24 10:53:23 +08:00
旺阳
5afd94136c
[Fix][Data Quality] Change t_ds_dq_rule_input_entry field name fix PostgreSQL not support value issue (#14992)
* change data quality field name

* fix-dao

* fix-dao

* fix-dao

* fix-dao

* update

* test

* update

* update
2023-11-13 11:04:30 +08:00
Wei Xiaonan
c0ed68121d
[Feature-14832][Listener]Implementation of Listener Mechanism (#14981)
* first commit

* 1. sql: sync ddl
2. front-end: change to ternary expression
3. back-end: correct license header in ListenerEvent.java

* test case

* frontend: remove unnecessary console

* fix unit test

* remove log depends on user-provided value

* fix dolphinscheduler_postgresql.sql

* sync database schema

* fix unit test

* fix unit test

* fix some NIT.

* extract GLOBAL_ALERT_GROUP_ID into variable

* fix ddl bug

* add column task_type in t_ds_fav_task in upgrade/3.2.0_schema

* add unit test
2023-10-29 20:28:05 +08:00
Chenshuai
4b5eb1a2bb
[Improvement-14894][dolphinscheduler-alert] alert message contains the operator (#14910)
* feat: add operator in alert info

* fix: optimize get user info directly

* fix: also optimize get user info directly

* fix: code style

* fix: code style

* fix: update test case

* fix: code check style

---------

Co-authored-by: Jim Chen <chenshuai19950725@gmail.com>
Co-authored-by: David Zollo <dailidong66@gmail.com>
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
Co-authored-by: xiangzihao <460888207@qq.com>
2023-09-25 10:50:28 +08:00
Chenshuai
6dcc26f52e
feat: add task priority in alert info (#14919)
Co-authored-by: Jim Chen <chenshuai19950725@gmail.com>
Co-authored-by: Aaron Wang <wangweirao16@gmail.com>
2023-09-15 17:30:25 +08:00
Aaron Wang
73e846d03e
[Improvement-14884][Master] Add overload state in master heartbeat to trigger slot change (#14887) 2023-09-14 10:36:42 +08:00
Jay Chung
3fa69d2db8
[Bug][Worker] Fix sql/shell task doesn't replace parameter (#14876)
Fix parameter assign error, fix #14869 which including
* shell task param not work, botch local and global
* others task chain assign not work
2023-09-08 10:09:16 +08:00
Wenjun Ruan
d223d654cc
Add rpc benchmark test (#14797) 2023-08-27 20:59:55 +08:00
Wenjun Ruan
55cc77226f
[RPC] Refactor the remote module. (#14776)
* Add dolphinscheduler-extract-base
* Add dolphinscheduler-extract-master
* Add dolphinscheduler-extract-worker
* Add dolphinscheduler-extract-alert
2023-08-23 09:43:17 +08:00
Aaron Wang
a491e3a849
fix: null app-info-path cause NPE (#14752) 2023-08-19 13:28:45 +08:00
Wenjun Ruan
db62ce0e47
Fix serial mode will cause NPE in Workflow bootstrap (#14703) 2023-08-04 23:43:57 +08:00
fuchanghai
b391b74df4
[fix-#14537] the branch that needs to be executed overlaps with another branch, it may not be able to complete the normal execution (#14563) 2023-08-01 11:04:31 +08:00
KingsleyY
4a80e3a021
[DS-14566][master] fix serial_wait command_param does not replace global_params (#14606)
This closes #14566

Co-authored-by: yangxin3 <yangxin3@newhope.cn>
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
Co-authored-by: xiangzihao <460888207@qq.com>
2023-07-25 13:36:35 +08:00
Wenjun Ruan
e6d94632b9
Add WorkflowExecuteContext (#14544) 2023-07-14 18:40:01 +08:00