Commit Graph

5521 Commits

Author SHA1 Message Date
ouyangyewei
27ea43d5df
to feature #7110 (#7111)
Co-authored-by: ouyangyewei <yewei.oyyw@alibaba-inc.com>
2021-12-04 21:43:14 +08:00
Zonglei Dong
fa8ccd3855
Refactor code, add class import, keep ‘logger’ variable names uniform. (#7164) 2021-12-04 21:18:43 +08:00
ouyangyewei
db090685a8
to feature #7169 (#7170)
Co-authored-by: ouyangyewei <yewei.oyyw@alibaba-inc.com>
2021-12-04 19:24:09 +08:00
zwZjut
af9ed35295
[Feature][MasterServer] add cache for workGroup and schedule #6987 (#7161)
* to #6987

* to #6987

* to #6987

* to #6987

* to #6987

* to #6987: @Param -> @Name

* to #6987: fix Sonar

* to #6987: fix Sonar

Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
2021-12-04 16:59:05 +08:00
BaoLiang
0e75bd8008
[BUG][SQL] fix the process list page load slowly. (#7166)
* fix #7601 move ddl to 2.0.1

* fix #7601 move ddl to 2.0.1
2021-12-04 15:53:18 +08:00
GaoTianDuo
ed9fca6c97
[Bug] [dolphinscheduler-api] The pages of project management, workflow instance and task instance are accessed slowly #7061 (#7139)
1、I found that the SQL behind this interface is very slow. The main table is large, but there is no index to use. By add the index, the all mode in the query plan can be changed into a more efficient ref mode. If this optimization still fails to meet the requirements, please contact me and I will continue to optimize。
2、modify index start_time_index in init script and upgrade scripts.
2021-12-04 13:13:30 +08:00
wind
ba2b2a67c2
[Feature-6988][MasterServer] add cache manager for workflow (#7090)
* add cache manager for workflow

* [DS-6988][MasterServer] add cache manager for workflow

* cache evict code optimization

* test

Co-authored-by: caishunfeng <534328519@qq.com>
2021-12-03 23:13:11 +08:00
zwZjut
1f184440ca
[UT#7157]Fix Ut Run Error (#7158)
* to #7157

* to #7157

Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
2021-12-03 22:01:26 +08:00
Hua Jiang
226bed1768
fix this issue (#7120) 2021-12-03 16:56:30 +08:00
zwZjut
2bf73603bb
[Feature][dolphinscheduler-api] access control of taskDefinition and taskInstance in project to #7081 (#7082)
* to #7081

* fix #7081

* to #7081

Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
2021-12-03 16:56:08 +08:00
Kerwin
3de182dc08
Optimize the display of the pop-up window for creating worker groups (#7148) 2021-12-03 16:55:58 +08:00
zwZjut
8ebe060658
[Feature][dolphinscheduler-api] access control of processDefinition and processInstance in project #7073 (#7080)
* to #7073

* to #7073

* to #7073

Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
2021-12-03 16:55:20 +08:00
zwZjut
d09d68e109
to #7065: fix ExecutorService and schedulerService (#7072)
Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
2021-12-03 16:54:39 +08:00
songjianet
99f6ab3847
[Feature][Workflow relationship] Format name. (#7143) 2021-12-03 14:43:45 +08:00
wangxj3
2b519f31e9
[Feature-#6422] task group queue (#6722)
* add task group

* modify task group

* pull dev

* add license header

* modify code style

* fix code style

* fix sql error

* fix error

* fix test

* fix test

* fix test

* fix test

* fix code style

* fix ut

* code style

* fix unit test

* test ut

* ut

* add unittest

* test ut

* modify back ut

* majorization code

* fix conflict

* fix ut

Co-authored-by: wangxj <wangxj31>
2021-12-03 14:02:31 +08:00
Devosend
f480b8adf8
[Feature-6930] [Python]Add workflow as code task type sub_process (#7022)
* [python] add subProcess task

* refactor python gateway server and task for subprocess

* add function comment for getProcessDefinition

* change process_definition usage of Subprocess task
2021-12-03 10:26:45 +08:00
songjianet
3304edee47
[Feature][Workflow relationship] Format time. (#7123) 2021-12-03 10:02:33 +08:00
songjianet
791ed6de77
[Feature][Workflow relationship] Improve English support. (#7129) 2021-12-03 01:05:45 +08:00
Hua Jiang
6beae191e2
[Fix-7103] Fix the issue with the sqoop task missing the verification of props. (#7119)
* add the verification of parameters for the sqoop task
2021-12-02 19:33:51 +08:00
Kerwin
9fffbdb316
add TaskProcessorFactoryTest Ignore. (#7130) 2021-12-02 19:15:57 +08:00
zwZjut
c42531e39c
to #7108 (#7109) 2021-12-02 16:34:38 +08:00
wangyizhi
e0623973e0
[Fix-7083] [UI] The switch task is missing the else branch (#7084) 2021-12-02 14:35:36 +08:00
JinYong Li
b265297fc8
[Improvement][API] fix 6772 new api bug (#7098)
* add processDefinition releaseWorkflowAndSchedule

* add ProcessTaskRelationServiceImpl.moveTaskProcessRelation

* add dependency and subprocess judgement

* fix api bug

* fix codestyle
2021-12-02 14:06:39 +08:00
Hua Jiang
d7eb7453e5
[Fix-7070][UI] Remedy the issue with no saving the timeout's strategy into the database. (#7107) 2021-12-01 21:03:49 +08:00
wangyizhi
173a385618
[Improvement-7092][UI] Optimize DAG to adapt to situations without locations (#7102) 2021-12-01 16:40:11 +08:00
songjianet
b3efcdfeba
[Fix][Create Warning Instance Edit] Fix the problem of incomplete display of the edit form. (#7096) 2021-12-01 14:41:14 +08:00
张俊杰
463fc76bb7
[Fix-6809][dolphinscheduler-task-sql] Fix sql task can't send alert (#7089) 2021-12-01 14:08:40 +08:00
kezhenxu94
2ed3c0c3a7
Recover unit tests that have not been run for some time (#7054) 2021-12-01 13:55:26 +08:00
Sunny Lei
0dcff1425a
[DS-5229][feat] Implement server port custom config (#6947)
* [DS-5229][fix] server port custom config
This closes #5229

* [DS-5229][feat] Implement server port custom config
This closes #5229

* [Bug] [readme] Error link to Docker and k8s in readme apache#6802 (#6862)

* [Bug] [readme] Error link to Docker and k8s in readme #6802
1、modify the error link

* [Bug] [readme] Error link to Docker and k8s in readme #6802
1、modify the error link in readme_zh_cn.md

* [DS-6829][WorkerServer] skip create log dir and print log in dryRun model (#6852)

Co-authored-by: caishunfeng <534328519@qq.com>

* [DS-5229][feat] Implement server port custom config
Modify review suggestion
This closes #5229

Co-authored-by: GaoTianDuo <gaotianduo_yewu@cmss.chinamobile.com>
Co-authored-by: wind <caishunfeng@users.noreply.github.com>
Co-authored-by: caishunfeng <534328519@qq.com>
2021-12-01 00:21:03 +08:00
zwZjut
100a9ba3fc
to #7055: tenant-auto-create: true (#7056)
Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
2021-11-30 16:44:13 +08:00
kezhenxu94
37ba1eb5ad
Add E2E tests for some core features (#7025) 2021-11-30 13:07:55 +08:00
ouyangyewei
7b20f4c239
return domain after creating success (#7050)
Co-authored-by: ouyangyewei <yewei.oyyw@alibaba-inc.com>
2021-11-30 11:50:32 +08:00
zhang
c2585a1170
[BUG] master task log can't output repair (#7029)
* update task_definition_log field typ

update task_definition_log  field task_params type,modify text to longtext to be consistent with the main table

* update t_ds_task_definition_log field task_params

update t_ds_task_definition_log field task_params

* [FIX:211128] master task log Can't output repair

Co-authored-by: crane <zhangbq3306@163.com>
2021-11-30 11:04:32 +08:00
wind
be3bd4c83d
[DS-7004][MasterServer]fix master still work when it lose zk connection (#7045)
Co-authored-by: caishunfeng <534328519@qq.com>
2021-11-30 11:00:17 +08:00
JinYong Li
6e55394783
[Improvement][API][num-9] add ProcessTaskRelationServiceImpl.moveTaskProcessRelation (#7053)
* add processDefinition releaseWorkflowAndSchedule

* add ProcessTaskRelationServiceImpl.moveTaskProcessRelation

* add dependency and subprocess judgement
2021-11-30 10:24:28 +08:00
Kerwin
f0630bc9a0
Fix view task status module on project homepage cannot read property label of undefined (#6956) 2021-11-30 10:00:54 +08:00
Kerwin
631a31b6ac
fix token delete request 404 (#6979) 2021-11-30 10:00:22 +08:00
Hua Jiang
a0955a078a
[Fix-7019][MasterServer] Fix the issue with failing to save and run the process that contains the switch task. (#7046) 2021-11-30 09:59:48 +08:00
wind
8e93939121
[DS-6963][MasterServer]add cache manager for tenant and user (#6974)
* cache manager for tenant and user

* add test

* fix checkstyle

* add dependencies

* fix sonar check

* use spring boot cache configuration

* add license

* check style

* check style

* add license

* change proxy name to processor

* add license header

* check style

* add license file

* add license file

* add license

Co-authored-by: caishunfeng <534328519@qq.com>
2021-11-30 08:29:25 +08:00
EdwardYang
88cd37ff0f
[Improvement][API][num-1] save schedule when workflow is offline in interface 'createEmptyProcessDefinition' (#7047)
* [Improvement] save schedule when workflow is offline

* [Improvement] rollback transactional when creating/updating schedule failed.

* [Improvement] resolve merge conflict

* [Improvement] do not set projectName and processDefinitionName when saving schedule

* [Improvement] do not set projectName and processDefinitionName when saving schedule

Co-authored-by: edward-yang <yangjianh210@gmail.com>
2021-11-29 21:32:23 +08:00
zwZjut
b815da6158
to #7001: delete certain edge in processDefinition (#7038)
* to #7001: delete certain edge in processDefinition

* to #7001: delete certain edge in processDefinition

* to #7001: delete certain edge in processDefinition

* to #7001: delete certain edge in processDefinition

* to #7001: fix checkstyle

* to #7001: delete edge by projectCode,processDefinitionCode,preTaskCode and postTaskCode

* to #7001: add UT

Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
2021-11-29 20:51:58 +08:00
JinYong Li
1e9973299f
add processDefinition releaseWorkflowAndSchedule (#7044) 2021-11-29 20:13:49 +08:00
zwZjut
f865c26322
to #7000: implement deleteTaskProcessRelation (#7043)
* to #7000: implement deleteTaskProcessRelation

* to #7000: ! CollectionUtils.isEmpty -> CollectionUtils.isNotEmpty

* to #7000: fix bug

* to #7000: add UT

Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
2021-11-29 20:11:47 +08:00
JinYong Li
c5bde33ac0
[Upgrade][install] upgrade compatibility exception (#7030)
* upgrade compatibility exception

* upgrade compatibility exception
2021-11-29 15:07:47 +08:00
EdwardYang
f30ee5956e
[Improvement][API][num-1 & 2] Implement createEmptyProcessDefinition & updateProcessDefinitionBasicInfo (#6984)
* [feature] finish interface createEmptyProcessDefinition

* [feature] finish interface updateProcessDefinitionBasicInfo

* [feature] use class Schedule replace ScheduleDto and delete it.

* [feature] delete chinese comment.

* [Improvement] change '.*' to single form of import

Co-authored-by: 熠然 <yangjianhua@cai-inc.com>
Co-authored-by: edward-yang <yangjianh210@gmail.com>
2021-11-29 10:30:48 +08:00
Jiajie Zhong
03ee12bccb
[community] Add new release version to issue template (#7021) 2021-11-28 19:04:45 +08:00
Zonglei Dong
4c4ebb8924
Refactor code, add class import. (#7028) 2021-11-28 10:35:13 +08:00
sparklezzz
42e20cbf60
fix(kerboros): set correct classloader for kerboros conf load in plugin tasks (#7027)
Co-authored-by: xudong.zhang <xudong.zhang@nio.com>
2021-11-28 09:25:51 +08:00
zwZjut
a8aed0f5f7
to #7005: transform params to SPRING_DATASOURCE_XXX (#7007)
* to #7005: transform params to SPRING_DATASOURCE_XXX

* to #7005: exclude h2

Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
2021-11-26 16:57:41 +08:00
lvshaokang
1a88b1863b
[Improvement-5785][common] Fix code format (#6864)
* [DS-5785][style] Fix common module code format

* [DS-5785][style] Fix common module code format
2021-11-26 14:43:23 +08:00