Wenjun Ruan
0bfd7d66d9
db: Modify name len to 255 ( #13788 )
2023-03-24 09:40:09 +08:00
Aaron Wang
8895921d87
[Improvement-13651] Remove useless resources_task ( #13721 )
2023-03-23 22:52:23 +08:00
陈家名
c2d5774790
[Improvement] SQL optimization queryTheHighestPriorityTasks ( #13671 )
...
Co-authored-by: chenjiaming <chenjiaming@kezaihui.com>
2023-03-05 22:35:24 +08:00
Wenjun Ruan
4351a25f2a
Add execute function to handle the workflow instance operation ( #13610 )
2023-02-24 13:47:41 +08:00
fuchanghai
c406f81e9c
[improve-#13597] fix the method name ( #13599 )
2023-02-22 10:53:51 +08:00
Wenjun Ruan
133d1733eb
Alter title in t_ds_alert to 512 ( #13535 )
2023-02-09 09:41:07 +08:00
seedscoder
8d12dc0702
[Improvement-13491] Use lombok @Slf4j annotation to generate logger ( #13509 )
2023-02-07 20:32:53 +08:00
Stalary
b9ef4a7177
FIX: qrtz table use uppercase ( #13494 )
...
* FIX: use uppercase
2023-02-02 11:25:27 +08:00
Stalary
5fb6035f0d
fix: 3.1.1_schema mysql ddl sql error ( #13485 )
2023-02-01 21:11:05 +08:00
Wenjun Ruan
af9091a98a
Use convert to set the exist column collate to utf8_bin ( #13479 )
2023-02-01 18:40:18 +08:00
Wenjun Ruan
d10eaa8d81
Change mysql COLLATE to utf8_bin ( #13469 )
2023-02-01 14:08:31 +08:00
Wenjun Ruan
713fdbc444
Split the upgrade code ( #13436 )
...
1. Split the upgrade code into different version
2. Log the dml/ddl sql in origin friendly format
3. Fix ddl of 3.0.0 and 3.1.0
4. Fix search resource regular expression injection
2023-01-20 20:48:07 +08:00
qianli2022
8be32d4145
[Feature][Api] When use api to run a process we want get processInstanceId ( #13184 )
...
* add sql
* add mapper
* add dao
* add excutor
Co-authored-by: qianl4 <qianl4@cicso.com>
2023-01-18 17:58:32 +08:00
jackfanwan
8439b5dc69
[fix-12721] Fix cannot modify the upstream task in task definition page. ( #12722 )
2023-01-15 19:58:47 +08:00
Wenjun Ruan
483330273d
Remove entity in QueryWrapper constructor ( #13396 )
2023-01-14 20:57:03 +08:00
Wenjun Ruan
98dfd64444
Clear the task group data when delete a project/workflowInstance ( #13383 )
2023-01-12 21:51:51 +08:00
calvin
4f73fe1bd8
fix this issue with missing a comma in the TaskInstanceMapper.xml ( #13359 )
2023-01-09 14:43:34 +08:00
Kerwin
fde2c8073e
Fix that when the timing data is not configured with environmental information, the timing management does not display the data. ( #13339 )
2023-01-05 12:15:02 +08:00
Wenjun Ruan
a074f7e2e6
Delete workflow will delete workflow instance ( #13336 )
2023-01-04 22:22:03 +08:00
Aaron Wang
ccad56e88e
[Improvement][Master] Validate same content of input file when using task cache ( #13298 )
...
* support file content checksum
* fix inject null storageOperate bug
2023-01-03 11:38:13 +08:00
Wenjun Ruan
8a479927f3
Add projectCode in t_ds_process_instance and t_ds_task_instance to remove join ( #13284 )
2023-01-03 09:52:28 +08:00
陈家名
5077fa6aae
[Improvement][dao] Optimized countInstanceStateByProjectCodes ( #13273 )
2022-12-28 14:31:21 +08:00
Jay Chung
6627c38be5
[fix][dml] Also query workflow without tenant by given name ( #13075 )
...
before this, once remove user tenant, all users workflow
can not be found anymore, this patch fix this problem, make
this entrypoint can query workflow without tenant to its
users
2022-12-28 14:14:29 +08:00
Wenjun Ruan
fd3afd84ad
Remove alert when delete workflow instance ( #13281 )
2022-12-28 11:40:12 +08:00
XCL
a75a69f3ec
[Improvement][api]Start the downstream workflow when it is online ( #13257 )
...
Start the downstream workflow when it is online
2022-12-26 11:58:37 +08:00
Wenjun Ruan
04785dc799
Add mybatis in mysql registry module ( #13275 )
2022-12-26 11:09:25 +08:00
陈家名
8870464c02
[Improvement][api] Optimize batch query in process-list interface ( #13222 )
...
Co-authored-by: chenjiaming <chenjiaming@kezaihui.com>
2022-12-23 23:33:42 +08:00
calvin
a98cae7065
[Fix-13260][API] Ironed out the issue with lacking the value of user's name when querying the list of workflow definitions. ( #13261 )
...
* fix the issue with lacking user name
2022-12-23 19:22:08 +08:00
calvin
a235d3cd6a
[Improvement][UI] Support to display extra columns such as 'Worker Group' and 'Environment Name' on the page of cron manage. ( #13267 )
2022-12-23 15:24:59 +08:00
calvin
915bdf481d
fix the issue with throwing the sql exception ( #13231 )
2022-12-20 17:54:47 +08:00
陈家名
7643d2840a
[Improvement][api-sql] Optimize the sql for paging query project ( #13172 )
...
* Optimizing sql
* add version join condition
Co-authored-by: chenjiaming <chenjiaming@kezaihui.com>
2022-12-19 08:51:21 +08:00
JieguangZhou
66e20271ad
[Feature][Master] Add task caching mechanism to improve the running speed of repetitive tasks ( #13194 )
...
* Supports task instance cache operation
* add task plugin cache
* use SHA-256 to generate key
* Update dolphinscheduler-dao/src/main/resources/sql/dolphinscheduler_mysql.sql
Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>
* Update dolphinscheduler-dao/src/main/resources/sql/dolphinscheduler_postgresql.sql
Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>
* Optimizing database Scripts
* Optimize clear cache operation
Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>
2022-12-18 18:17:09 +08:00
insist777
631e3a81d1
[Feature][API] New restful API for taskInstance ( #13070 )
2022-12-14 14:47:36 +08:00
JieguangZhou
e4b9b67255
Allow execute task in workflow instance ( #13103 )
2022-12-13 16:43:44 +08:00
Wenjun Ruan
f5801a0e1c
Fix task group might be rob by multiple task ( #13115 )
2022-12-11 22:58:21 +08:00
Yann Ann
6ef74073cc
[Improve-13001]migrate commons-collections -> commons-collections4 ( #13002 )
2022-12-10 23:50:19 +08:00
Wenjun Ruan
66b8a81ad8
Revert "Update dolphinscheduler_ddl.sql ( #13081 )" ( #13117 )
...
This reverts commit fa3e2a5e1a
2022-12-07 10:14:28 +08:00
jackfanwan
cb8d125e0f
[Improvement] optimization task definition & fix in the task definition list, if one task have more pre task, the task list can't show all task ( #13106 )
...
* fix in the task definition list, if one task have more pre task, the task list can't show all task
* modify code style
* in the task definition, delete search workflow name filter, and fix in the task definition list, if one task have more pre task, the task list can't show all task
* modify code style
* modify code style
* delete useless select sql
* add annotation
Co-authored-by: fanwanlong <fanwanlong@kezaihui.com>
2022-12-07 09:52:53 +08:00
Eric Gao
d9f43adafd
Prevent default alert group from binding alert plugin instances with id 1 and 2 automatically ( #13114 )
2022-12-06 23:10:48 +08:00
Wenjun Ruan
8a152aebc7
Fix task log file might not be clear ( #13102 )
2022-12-06 11:23:20 +08:00
Wenjun Ruan
fed1e85fb8
Fix the invalidate task instance never be deleted ( #13091 )
2022-12-04 18:38:54 +08:00
darkz
fa3e2a5e1a
Update dolphinscheduler_ddl.sql ( #13081 )
...
https://github.com/apache/dolphinscheduler/pull/12826
for this pr
2022-12-03 22:11:08 +08:00
Jay Chung
f5a59982b3
[fix] Python gateway can not upload to resource center ( #13042 )
...
in #12076 we refactor our resource center, remove the resource table,
it is a good refactor but it failed python api upload, this patch try
to fix python api upload function
ref: #12076
2022-12-02 10:11:57 +08:00
insist777
1064680ded
[feat][API] New restful API for workflow state ( #13031 )
2022-12-01 14:21:01 +08:00
insist777
a0eb45b9e4
[Feature][API] New restful API for workflowInstance ( #12990 )
2022-11-28 12:58:46 +08:00
Kerwin
f9a0ecbc81
[Bug-12997][API] Fix that the end time is not reset when the workflow instance reruns. ( #12998 )
...
* Fix that the end time is not reset when the workflow instance reruns.
2022-11-27 15:08:30 +08:00
rickchengx
43719f7c15
[Improvement-12763][UT] Improve testListAuthorizedProjects() in ProjectMapperTest ( #12768 )
2022-11-27 13:15:38 +08:00
rickchengx
7336afaa65
[Fix-12916] Add permission check when query or download log ( #12917 )
2022-11-25 17:59:28 +08:00
Kerwin
31021730ec
The task instance list is sorted by submission time ( #12974 )
2022-11-24 18:59:53 +08:00
Wenjun Ruan
d99ba29b66
Fix master cluster may loop command unbalanced ( #12891 )
...
(cherry picked from commit 3b2b86661be76b7c1404a910c865d78b7936313d)
2022-11-16 10:20:22 +08:00
Guanhua Li
83f9588eb0
[Improvement-12856][UT] Refactor duplicated code fragment in UdfFuncMapperTest ( #12857 )
2022-11-15 22:59:11 +08:00
Kerwin
713046b043
Modify the table t_ds_worker_group to add a description field in the postgresql upgrade script ( #12883 )
2022-11-15 17:26:41 +08:00
rickchengx
7c711e622f
[Improvement-12850][UT] Improve the ut of DataSourceMapperTest ( #12851 )
2022-11-12 23:57:40 +08:00
jackfanwan
f249f2b257
[fix-12675]edit workflow related task, workflow's task version change ( #12692 )
...
* edit workflow related task, workflow's task version change
* Update dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/enums/Status.java
Co-authored-by: fanwanlong <fanwanlong@kezaihui.com>
Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
2022-11-06 20:49:07 +08:00
jackfanwan
913525343e
modify sql error ( #12717 )
2022-11-05 21:44:01 +08:00
jackfanwan
7d0e2cbbb9
when delete workflow, delete related task ( #12681 )
...
* when delete workflow, delete related task
Co-authored-by: fanwanlong <fanwanlong@kezaihui.com>
2022-11-04 14:40:15 +08:00
ZhenjiLiu
7cdb926a5f
[Improvement][Batch Query] Batch query ProcessDefinitions belongs to need failover ProcessInstance. ( #12506 )
2022-11-03 09:15:19 +08:00
Yiming Guo
dc8d18cf87
[Feature-12040][api][ui] Add authorization management of read and write permissions for project center ( #12048 )
...
[Feature-12040][api][ui] Add authorization management of read and write permissions for project center
2022-10-27 18:01:51 +08:00
Tq
4b4d0b92a3
[FIX][DAO-12277] change t_ds_fav_task column task_name to task_type and related code ( #12276 )
...
* change t_ds_fav_task column task_name to task_type and related code
* change task_name to task_type
* change FavTask taskName to taskType, taskType to taskCategory
Co-authored-by: labbomb <739955946@qq.com>
2022-10-27 09:36:26 +08:00
HanayoZz
489e7fe4e2
[Feature-10495][Resource Center] Resource Center Refactor ( #12076 )
...
* resource center refactor - S3 services connection
Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
2022-10-26 13:53:44 +08:00
Hengdong Gong
3030419ee1
[fix](dolphinscheduler-dao) fix upgrade to 3.1.0 sql missing field ( #12314 ) ( #12315 )
2022-10-26 12:32:05 +08:00
Yann Ann
04aa125ba2
[Refactor]migrate some task dao functions from ProcessServiceImpl to dao modules. ( #12505 )
...
* migrate some task DAO to dao modules.
* Update dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/repository/impl/ProcessInstanceMapDaoImpl.java
Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
2022-10-26 11:51:58 +08:00
陈家名
97a0771f21
[fix][sql] Add unique key to process_definition_log avoid TooManyResultExpection ( #12503 )
...
* Add unique key to t_ds_process_definition_log
2022-10-25 14:20:37 +08:00
caishunfeng
dedff70f90
[Improvement] Merge spi.utils into common.utils ( #12502 )
...
* improve JSONUtil
* merge spi.utils into common.utils
* split constants && remove some copy utils method
* remove StringUtils
2022-10-23 21:23:54 +08:00
Wenjun Ruan
b5193ee4e5
Remove equals in User to fix UT ( #12487 )
2022-10-22 08:59:10 +08:00
Kerwin
1436ad65fc
[Bug-12410] [API]Fix the worker list result in workflow definition only has default ( #12411 )
...
* Fix the worker list result in workflow definition only has default
* Fix tenant, k8s namespace similar permission issues.
* code clean in the k8s namespace mapper
* delete redundant sql method
2022-10-21 13:54:28 +08:00
Eric Gao
d46e4659d9
[Improvement][Style] Fix remaining style errors and wildcard imports ( #12412 )
...
* Fix remaining style errors and wildcard imports
* Upgrade spotless to the latest stable version, exclude .github folder from spotless check and fix dead link
* Fix spotless matching scope of markdown files
2022-10-19 11:08:04 +08:00
Kerwin
c02e08bb3d
[BUG-12396][Schedule] Fixed that the workflow definition scheduling that has been online after the version upgrade does not execute ( #12397 )
...
* Fixed that the workflow definition scheduling that has been online after the version upgrade does not execute.
2022-10-17 19:45:39 +08:00
fuchanghai
22103f975e
[improvement] Migrate all UT cases from jUnit4 to jUnit5 in dao module ( #12328 )
...
* [improvement] Migrate all UT cases from jUnit4 to jUnit5 in dao module
Co-authored-by: Jackie <‘2875334588@qq.com’>
2022-10-13 15:26:39 +08:00
kezhenxu94
db1d319159
Reorganize some classes in common module, remove duplicate classes ( #12321 )
2022-10-13 10:23:27 +08:00
Yann Ann
17cd644506
[Fix-#11669][Workflow Instance Page] Fix the duration in Workflow Instance page. ( #12264 )
...
* fix process duration issue
* use processInstance.getState().isFinished() replace new isFinish function
* add WorkflowUtils
2022-10-13 09:50:31 +08:00
Jiajie Zhong
0e1c8d8153
[Feature][API] New restful API task and task relation ( #12116 )
2022-10-13 09:41:03 +08:00
jegger
04e1b88c1c
[fix] Fix error problem on h2 startup data quality rule management page ( #12108 )
...
Co-authored-by: jegger <zhangbq3306@163.com>
2022-10-11 16:52:35 +08:00
Ivan
98a8b5383e
[ fix : #11590 ][upgrade] Fix the bugs when I upgrade ds from v1.3.9 to v3.0.0 ,such as file cannot be found or column not found etc ( #11619 )
...
* fix the bugs which upgrade ds metabase from v1.3.9 to v3.0.0 with mysql
* [Fix-11590][upgrade] Fix the bugs when I upgrade ds from v1.3.9 to v3.0.0 ,such as file cannot be found or column not found etc
* fix : #11590 add process_instance_code_state_index with table t_ds_process_instance to improve query efficiency for the method of findLastProcessInterval
* Fix the bug that cannot find the files when upgrade ds version with postgresql
Co-authored-by: fan.wang <fan.wang@jiduauto.com>
2022-10-11 10:20:05 +08:00
Yann Ann
60c43d5d32
just add WorkerGroupMapper UT ( #12242 )
2022-10-06 20:10:48 +08:00
Yann Ann
3c4f9380d1
just correct wrong word ( #12174 )
2022-10-05 17:08:48 +08:00
fuchanghai
e27c79974d
[fix-#11753] send alert error alert data id ( #11774 )
...
* [fix-#11753] send alert error alert data id
Co-authored-by: fuchanghai <changhai.fu@marketingforce.com>
2022-09-29 15:34:40 +08:00
caishunfeng
151c2d2c70
fix upgrade sql ( #12141 )
2022-09-26 15:21:52 +08:00
labbomb
64f4cb4f3e
refactor: dag-menu refactor ( #11965 )
...
* refactor: dag-menu refactor
* feat: add i18n support for the dag-menu
* feat: add e2e class
Co-authored-by: devosend <devosend@gmail.com>
Co-authored-by: Tq <tianqitobethefirst@gmail.com>
2022-09-26 11:06:39 +08:00
Jiajie Zhong
82ddd72e4a
[Feature][API] New restful API for workflow and schedule ( #11912 )
...
* [feat] New restful API for workflow and schedule
CURD for workflow and schedule, different with exists
API, this new restful api only operate single resource
in each request, and return the latest. For example,
previous workflow should also need to post tasks definition
and tasks relation definition, but this patch will allow
you to create workflow without task relate information
* use checkProjectAndAuthThrowException, and fix CI error
* Update dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/ScheduleV2Controller.java
* change method name
from createProcessDefinitionV2 to createSingleProcessDefinition
from updateProcessDefinitionV2 to updateSingleProcessDefinition
Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
2022-09-23 22:25:19 +08:00
rickchengx
08a4c7981f
[improve] Remove the spark version of spark task ( #11860 )
2022-09-21 20:27:28 +08:00
limaiwang
ed1d1e8856
add(dependent node's add unprivileged project list): dependent node's add unprivileged project list ( #12073 )
2022-09-21 09:28:22 +08:00
juzimao
c8884e4f2f
and task instance date range search ( #11963 )
2022-09-19 17:50:24 +08:00
insist777
5b384f3fab
[Feature-11473][Task]Support test task ( #11670 )
...
* [improve] support test tasks
* [improve] support test tasks
* [improve] support test tasks
* [improve] support test tasks
* [improve] support test tasks
* Update TaskExecuteThread.java
* try solve e2e q
* try solve e2e q
* try solve e2e q
* try solve e2e q
* try solve e2e q
* try solve e2e q
* try solve e2e q
* try solve e2e q
* try solve e2e q
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* Update DataSource.java
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* Update messages_zh_CN.properties
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* Update messages.properties
* Update messages_en_US.properties
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
2022-09-17 13:40:56 +08:00
Wenjun Ruan
864a90820d
Fix insertOrUpdate plugin may failed due to concurrent operation ( #11471 )
2022-09-16 15:39:04 +08:00
Wenjun Ruan
d833a28b2e
Fix cannot save processDefinition ( #11931 )
2022-09-16 13:54:18 +08:00
Wenjun Ruan
4ad34483a5
Fix alert_send_status may throw duplicate key exception and add limit for query alert ( #11953 )
2022-09-16 13:26:33 +08:00
caishunfeng
925e2fa551
[Bug][Workflow Definition] fix ordinary users can not create depend task ( #11961 )
2022-09-16 09:12:33 +08:00
wendongdi
e70c501f9a
[Fix][db] fix init&upgrade mysql-meta-schema bugs ( #11887 )
2022-09-14 18:03:07 +08:00
juzimao
b1068e27ba
Optimize date range condition ( #11911 )
2022-09-14 16:55:10 +08:00
Wenjun Ruan
8a47785f42
Fix listing processDefinition the schedulerReleaseState will never be null ( #11888 )
2022-09-13 10:07:12 +08:00
Yann Ann
e20f17a7b7
[Improvement-#11768][Monitor] Support monitor h2 database in monitor page ( #11813 )
2022-09-09 09:43:56 +08:00
Stalary
37325b4c34
[Bug](dependent) Dependent downstream trigger error when schedule cycle not day. ( #11734 )
...
* FIX: dependent
* FIX: version
* MOD: for review
2022-09-08 15:08:10 +08:00
caishunfeng
2e61c76c22
[Improvement] Add remote task model ( #11767 )
2022-09-08 13:41:36 +08:00
kezhenxu94
efedc9a53c
Bump up dependencies to fix cves ( #11765 )
2022-09-08 09:05:05 +08:00
fuchanghai
7c0c1f4b78
Update dolphinscheduler_ddl.sql fix alert t_ds_user add time_zone is not correct ( #11795 )
2022-09-06 23:14:41 +08:00
Tq
4283cfd7a1
[fix][API]fix task types sequence ( #11775 )
...
* fix task types sequence
* fix mapper table name t_ds_fav_task
* fix sequence
2022-09-05 20:29:03 +08:00
caishunfeng
f8d46a26c1
[Feature-11530] add state history for process instance ( #11757 )
...
* [Feature] add state history for process instance (#97 )
* add state history for process instance
* upsertProcessInstance
* remove unuse method
* fix UT
Co-authored-by: caishunfeng <534328519@qq.com>
2022-09-03 10:14:05 +08:00
Tq
b3cc8a55b9
[feature][API]feature: add task type list and enable user to add task type to fav ( #11727 )
...
* feature: add task type list and enable user to add task type to fav
* make DependentExecute.java variable protected
* add standalone server fileSet
2022-09-02 11:40:23 +08:00
juzimao
d0f5e7edf3
optimize the process instance query, change the date time range ( #11719 )
2022-08-31 19:53:41 +08:00
