Commit Graph

5584 Commits

Author SHA1 Message Date
Hua Jiang
a0955a078a
[Fix-7019][MasterServer] Fix the issue with failing to save and run the process that contains the switch task. (#7046) 2021-11-30 09:59:48 +08:00
wind
8e93939121
[DS-6963][MasterServer]add cache manager for tenant and user (#6974)
* cache manager for tenant and user

* add test

* fix checkstyle

* add dependencies

* fix sonar check

* use spring boot cache configuration

* add license

* check style

* check style

* add license

* change proxy name to processor

* add license header

* check style

* add license file

* add license file

* add license

Co-authored-by: caishunfeng <534328519@qq.com>
2021-11-30 08:29:25 +08:00
EdwardYang
88cd37ff0f
[Improvement][API][num-1] save schedule when workflow is offline in interface 'createEmptyProcessDefinition' (#7047)
* [Improvement] save schedule when workflow is offline

* [Improvement] rollback transactional when creating/updating schedule failed.

* [Improvement] resolve merge conflict

* [Improvement] do not set projectName and processDefinitionName when saving schedule

* [Improvement] do not set projectName and processDefinitionName when saving schedule

Co-authored-by: edward-yang <yangjianh210@gmail.com>
2021-11-29 21:32:23 +08:00
zwZjut
b815da6158
to #7001: delete certain edge in processDefinition (#7038)
* to #7001: delete certain edge in processDefinition

* to #7001: delete certain edge in processDefinition

* to #7001: delete certain edge in processDefinition

* to #7001: delete certain edge in processDefinition

* to #7001: fix checkstyle

* to #7001: delete edge by projectCode,processDefinitionCode,preTaskCode and postTaskCode

* to #7001: add UT

Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
2021-11-29 20:51:58 +08:00
JinYong Li
1e9973299f
add processDefinition releaseWorkflowAndSchedule (#7044) 2021-11-29 20:13:49 +08:00
zwZjut
f865c26322
to #7000: implement deleteTaskProcessRelation (#7043)
* to #7000: implement deleteTaskProcessRelation

* to #7000: ! CollectionUtils.isEmpty -> CollectionUtils.isNotEmpty

* to #7000: fix bug

* to #7000: add UT

Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
2021-11-29 20:11:47 +08:00
JinYong Li
c5bde33ac0
[Upgrade][install] upgrade compatibility exception (#7030)
* upgrade compatibility exception

* upgrade compatibility exception
2021-11-29 15:07:47 +08:00
EdwardYang
f30ee5956e
[Improvement][API][num-1 & 2] Implement createEmptyProcessDefinition & updateProcessDefinitionBasicInfo (#6984)
* [feature] finish interface createEmptyProcessDefinition

* [feature] finish interface updateProcessDefinitionBasicInfo

* [feature] use class Schedule replace ScheduleDto and delete it.

* [feature] delete chinese comment.

* [Improvement] change '.*' to single form of import

Co-authored-by: 熠然 <yangjianhua@cai-inc.com>
Co-authored-by: edward-yang <yangjianh210@gmail.com>
2021-11-29 10:30:48 +08:00
Jiajie Zhong
03ee12bccb
[community] Add new release version to issue template (#7021) 2021-11-28 19:04:45 +08:00
Zonglei Dong
4c4ebb8924
Refactor code, add class import. (#7028) 2021-11-28 10:35:13 +08:00
sparklezzz
42e20cbf60
fix(kerboros): set correct classloader for kerboros conf load in plugin tasks (#7027)
Co-authored-by: xudong.zhang <xudong.zhang@nio.com>
2021-11-28 09:25:51 +08:00
zwZjut
a8aed0f5f7
to #7005: transform params to SPRING_DATASOURCE_XXX (#7007)
* to #7005: transform params to SPRING_DATASOURCE_XXX

* to #7005: exclude h2

Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
2021-11-26 16:57:41 +08:00
lvshaokang
1a88b1863b
[Improvement-5785][common] Fix code format (#6864)
* [DS-5785][style] Fix common module code format

* [DS-5785][style] Fix common module code format
2021-11-26 14:43:23 +08:00
xuhhui
1c6ef50036
[improvement] improve install.sh if then statement (#6782)
* [Fix] Executing install.sh failed when password and others contain @

* Update install.sh

Co-authored-by: Jiajie Zhong <zhongjiajie955@hotmail.com>

Co-authored-by: Kirs <acm_master@163.com>
Co-authored-by: Jiajie Zhong <zhongjiajie955@hotmail.com>
2021-11-26 14:42:14 +08:00
kezhenxu94
ef0cb46661
Fix shell tools for database schema cannot run (#7003) 2021-11-26 11:10:38 +08:00
wind
e6fe39ea47
[DS-6961][MasterServer] batch dispatch (#6962)
* [DS-6961][MasterServer] batch dispatch

* fix test

Co-authored-by: caishunfeng <534328519@qq.com>
2021-11-25 12:55:37 +08:00
JinYong Li
5a04b8d49a
update schdule api (#6977) 2021-11-25 10:22:41 +08:00
kezhenxu94
2de4a3cbb5
Add a demo docker-compose for testing the meter module (#6938) 2021-11-24 17:22:30 +08:00
zwZjut
1edd2bd0f6
to #6957 implement interface deleteUpstreamRelation deleteDownstreamRelation (#6966)
* to #6957

* to #6957

* to #6957

* to #6957: group by process definition code

* to #6957: fix sonar

* to #6957: fix sonar

* to #6957: add ut , add preTaskVersion

* to #6957: fix style

* to #6957: use org.apache.dolphinscheduler.spi.utils.StringUtils

Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
2021-11-24 17:12:47 +08:00
JinYong Li
e51a2a1642
[Improvement][API][num-8] add ProcessTaskRelationServiceImpl.createProcessTaskRelation (#6970)
* add ProcessTaskRelationServiceImpl.createProcessTaskRelation

* code style
2021-11-24 15:06:12 +08:00
lyq
c78fd24776
bug about dolphinscheduler-remote SendAsync (#6972)
* bug about SendAsync
2021-11-24 09:53:22 +08:00
时光
360464e279
[Improvement][API][num-6] TaskDefinition releaseTaskDefinition (#6965)
* finished releaseTaskDefinition

* add releaseTaskDefinition unit test

* change import

* add import

* bug fix

* change import

Co-authored-by: 时光 <caizhedong@cai-inc.com>
2021-11-23 18:20:13 +08:00
Jiajie Zhong
d93248acbc
[python] Add custom exception (#6935) 2021-11-23 16:59:38 +08:00
Jiajie Zhong
41e8836c91
[python] Add task sql (#6968)
* [python] Add task sql

* Add java gateway function doc
2021-11-23 16:58:00 +08:00
zhang
abde2d7d28
update task_definition_log field typ (#6967)
* update task_definition_log field typ

update task_definition_log  field task_params type,modify text to longtext to be consistent with the main table

* update t_ds_task_definition_log field task_params

update t_ds_task_definition_log field task_params
2021-11-23 14:26:20 +08:00
songjianet
982030bf38
[Feature][Project Process] Delete vertical scroll bar. (#6958) 2021-11-23 10:45:11 +08:00
Kerwin
38b14410ab
[Fix-6941][API] Fix the failure of querying details of alert group (#6942)
* Fix the failure of querying details of alert group

* code cleanup
2021-11-23 09:28:27 +08:00
LinKai
ad33d43498
[Improvement][Dao] SQL optimization - limit one piece of data #5061 (#6940)
* [Feature][unittest] Recover UT in AlertPluginManagerTest.java #6619

* use throws exception instead of try-catch to catch exceptions in AlertPluginManagerTest.java

* change the assert result from assertNull to assertNotNull

* [Improvement][Dao] SQL optimization - limit one piece of data #5069

* Improve the code to A rate

Co-authored-by: linkai <linkai@bigo.sg>
2021-11-22 20:27:29 +08:00
songjianet
da35c17cc0
[Feature][Workflow relationship] Increase the multilingual judgment of the state. (#6954) 2021-11-22 13:47:14 +08:00
zwZjut
199a84aa05
[fix #6878] implement interface queryUpstreamRelation queryDownstreamRelation and add UT (#6888)
* fix: #6878, implement interface queryUpstreamRelation queryDownstreamRelation and add UT

* fix: #6878, implement interface queryUpstreamRelation queryDownstreamRelation and add UT

* fix: #6878, add license

* fix: #6878, code style

* fix: #6878, code style

* fix: #6878, fix Duplication

* fix: #6878

* fix: #6878 , fix style

* to #6878: fix style

* to #6878: fix style

* to #6878: add buildTaskDefinition to overwrite equals and hashCode

Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
2021-11-22 11:02:15 +08:00
wind
e6239e808b
[DS-6829][WorkerServer] skip create log dir and print log in dryRun model (#6852)
Co-authored-by: caishunfeng <534328519@qq.com>
2021-11-21 17:51:56 +08:00
GaoTianDuo
1b6b5268b6
[Bug] [readme] Error link to Docker and k8s in readme apache#6802 (#6862)
* [Bug] [readme] Error link to Docker and k8s in readme #6802
1、modify the error link

* [Bug] [readme] Error link to Docker and k8s in readme #6802
1、modify the error link in readme_zh_cn.md
2021-11-21 17:50:19 +08:00
songjianet
d2666d6739
[Feature][Workflow relationship] The page displays null as -. (#6934)
* [Feature][Workflow relationship] The page displays `null` as `-`.

* [Feature][Workflow relationship] The page displays `null` as `-`.
2021-11-20 22:59:18 +08:00
songjianet
024aacacb3
[Feature][Workflow relationship] Improve English support. (#6936)
* [Feature][Workflow relationship] Improve English support.

* [Feature][Workflow relationship] Improve English support.
2021-11-20 22:58:30 +08:00
ououtt
1fdbc3270c
[DS-6931][fix] swagger failed to start due to improper use of request param annotation (#6933)
Co-authored-by: eye <eye.gu@aloudata.com>
2021-11-19 19:03:34 +08:00
wind
28872d8706
[DS-6891][MasterServer] reduce db operation when process instance running (#6904)
* remove taskInstanceCacheManager

* remove taskInstanceCacheManager

* [DS-6891][MasterServer] reduce db operation when process instance running

* [DS-6891][MasterServer] reduce db operation when process instance running

* fix test

* fix Transactional method call

* checkstyle

Co-authored-by: caishunfeng <534328519@qq.com>
2021-11-19 18:31:24 +08:00
wangyizhi
f2d242c7e7
[Feature-6918][UI] DAG page interaction optimization (#6919)
* [Feature] Add grid layout and optimize DAG style

* [Improvement] Add process definition status label and code optimization

* [Feature] add DAG node search bar

* [Feature] Add DAG scale bar

* [Improvement] Open pre-tasks settings for all tasks

* [Fix] Fix ut

* [Fix] Replace /deep/ with ::v-deep

* [Fix] Fix code smell
2021-11-19 18:08:51 +08:00
Jiajie Zhong
6e812be14d
[python] Add ut cover process definitions separate mode (#6908)
* [python] Add ut cover process definitions separate mode

* Remove unused print syntax

* Fix test error, cause raw_script not exists anymore
2021-11-19 17:24:19 +08:00
Jiajie Zhong
0dce68edd7
[python] Add task type python http (#6906)
* [python] Add task type python http

* Fix unittest error

* Fix UT error
2021-11-19 16:37:22 +08:00
JinYong Li
4bec792160
[Fix][Common] rewrite code generate,fix bit shift (#6914)
* rewrite code generate,fix bit shift

* fix ut

* add algorithm from licenses file

* add algorithm from licenses file

* add algorithm from licenses file

* add algorithm from licenses file

* add algorithm from licenses file

* fix ut
2021-11-19 16:04:05 +08:00
wind
595e4843d0
[DS-6849][MasterServer] fetch more commands and handle in parallel (#6850)
* [DS-6849][MasterServer] fetch more commands and handle in parallel

* add return

* handle command with check

* add test

* delete master prefix

Co-authored-by: caishunfeng <534328519@qq.com>
2021-11-19 12:03:49 +08:00
songjianet
1be080237b
[Fix-6896] fix the problem that the name search cannot be used. (#6911) 2021-11-19 11:46:45 +08:00
Jiajie Zhong
f9a130d73e
Use docker.scarf.sh to track docker user info (#6913)
* Use docker.scarf.sh to track docker user info

* Revert ci docker image change
2021-11-19 11:45:37 +08:00
songjianet
eb01f2e617
[Feature][Process definition] add search function to select in batch operation dialog. (#6915) 2021-11-19 11:17:03 +08:00
Kerwin
802fc498b5
[Fix-#6605][Common] Fix the situation where the obtained string property is empty. (#6661)
* Fix the situation where the obtained string property is empty.

* recovery test

* update code style.

* megre dev.
2021-11-19 10:58:17 +08:00
songjianet
00813b0a69
[Improvement-6889] split the routing module to increase subsequent maintainability. (#6892) 2021-11-19 09:38:22 +08:00
KyoYang
80dcf9b03e
[Improvement-#5293]upgrade cron-utils version 5.0.5 -> 9.1.3 (#6857)
* fix complement data retry bug; issues: [Bug] [Master] A bug on task retry mechanism #6613

Co-authored-by: yangqiyu <yangqiyu@deepexi.com>
2021-11-18 14:32:32 +08:00
Jeff Zhan
3101218c73
[Improvement][API][num-5] The workflow of the main table version is not allowed to be deleted (#6893)
* feature:deleteProcessDefinitionVersion

* pref:delete duplicate enum definition

Co-authored-by: 北笙 <“zhanqian@cai-inc.com”>
2021-11-18 14:30:58 +08:00
JinYong Li
fc6a5116c4
[Improvement][API][num-7] add task delete check (#6887)
* upgrade add sub_process

* add interface method

* add interface method

* fix ut

* add task delete check

* add task delete check
2021-11-18 10:51:29 +08:00
yihong
861aaaf971
[FIX-#6505][Dao] upgrade the MySQL driver package for building MySQL jdbcUrl (#6708)
* replace com.mysql.jdbc.Driver everywhere with com.mysql.cj.jdbc.Driver
2021-11-18 08:39:11 +08:00