Commit Graph

5534 Commits

Author SHA1 Message Date
wind
ff9bc806ac
[Bug-7206] [MasterServer] fix process isntance always running when task timeout (#7207)
* fix timeout

* add task timeout map to avoid repeated timeout event

* split task check list for retry and timeout

Co-authored-by: caishunfeng <534328519@qq.com>
2021-12-06 18:20:56 +08:00
Hua Jiang
705ba74f52
[Fix-7195] The file path of 2.0.1-schema used a wrong separator. (#7200) 2021-12-06 17:29:11 +08:00
JinYong Li
c7b06b0079
fix worker group display (#7208) 2021-12-06 15:44:28 +08:00
Kerwin
36ebfe0984
[Implement][server]TaskProcessor code optimization (#7146)
* TaskProcessor code optimization

* fix code style

* update Collectors.toMap
2021-12-06 14:15:09 +08:00
Kerwin
8f77fbbeb8
[Fix-7154][API] Fix MultipartFile resource [file] cannot be resolved to absolute file… (#7155)
* fix MultipartFile resource [file] cannot be resolved to absolute file path

* update testCopyFile.
2021-12-06 14:13:53 +08:00
ouyangyewei
3c8d1c6da9
to feature #7191 (#7194)
Co-authored-by: ouyangyewei <yewei.oyyw@alibaba-inc.com>
2021-12-06 07:17:33 +08:00
zwZjut
05aef279c6
[Feature][dolphinscheduler-api] add queryLog and downloadTaskLog with projectCode #7153 (#7192)
* [Feature][dolphinscheduler-api] add queryLog and downloadTaskLog with projectCode #7153

* [Feature][dolphinscheduler-api] add queryLog and downloadTaskLog with projectCode #7153

* [Feature][dolphinscheduler-api] add queryLog and downloadTaskLog with projectCode #7153

Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
2021-12-05 23:57:46 +08:00
SbloodyS
d83735ab51
[Fix-7037][dolphincheduler-server] process instance can not be killed when complement data (#7140)
* fix bug 7037

* update processComplementData

Co-authored-by: SbloodyS <sbloodys@qq.com>
2021-12-05 15:03:52 +08:00
ouyangyewei
3361d763c4
to feature #7180 (#7185)
Co-authored-by: ouyangyewei <yewei.oyyw@alibaba-inc.com>
2021-12-05 14:13:23 +08:00
Manhua
2dc09c627f
[Fix-5551][common] Fix ExecutionStatus for hive-on-Tez application (#6289)
* add ENDED state

* Update HadoopUtils.java
2021-12-05 14:03:29 +08:00
SbloodyS
8d39bf14b1
[Fix-7093][dolphincheduler-server] fix dependent node stuck in running (#7116)
* fix bug 7093

* remove redundant else

Co-authored-by: SbloodyS <sbloodys@qq.com>
2021-12-05 13:52:48 +08:00
songjianet
eb07a2b1c9
[Fix][Warning instance manage] Expand the support of i18n. (#7184) 2021-12-05 11:23:50 +08:00
songjianet
2fa3157e6e
[Feature][Workflow] Main package. (#7177) 2021-12-04 21:54:59 +08:00
ouyangyewei
27ea43d5df
to feature #7110 (#7111)
Co-authored-by: ouyangyewei <yewei.oyyw@alibaba-inc.com>
2021-12-04 21:43:14 +08:00
Zonglei Dong
fa8ccd3855
Refactor code, add class import, keep ‘logger’ variable names uniform. (#7164) 2021-12-04 21:18:43 +08:00
ouyangyewei
db090685a8
to feature #7169 (#7170)
Co-authored-by: ouyangyewei <yewei.oyyw@alibaba-inc.com>
2021-12-04 19:24:09 +08:00
zwZjut
af9ed35295
[Feature][MasterServer] add cache for workGroup and schedule #6987 (#7161)
* to #6987

* to #6987

* to #6987

* to #6987

* to #6987

* to #6987: @Param -> @Name

* to #6987: fix Sonar

* to #6987: fix Sonar

Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
2021-12-04 16:59:05 +08:00
BaoLiang
0e75bd8008
[BUG][SQL] fix the process list page load slowly. (#7166)
* fix #7601 move ddl to 2.0.1

* fix #7601 move ddl to 2.0.1
2021-12-04 15:53:18 +08:00
GaoTianDuo
ed9fca6c97
[Bug] [dolphinscheduler-api] The pages of project management, workflow instance and task instance are accessed slowly #7061 (#7139)
1、I found that the SQL behind this interface is very slow. The main table is large, but there is no index to use. By add the index, the all mode in the query plan can be changed into a more efficient ref mode. If this optimization still fails to meet the requirements, please contact me and I will continue to optimize。
2、modify index start_time_index in init script and upgrade scripts.
2021-12-04 13:13:30 +08:00
wind
ba2b2a67c2
[Feature-6988][MasterServer] add cache manager for workflow (#7090)
* add cache manager for workflow

* [DS-6988][MasterServer] add cache manager for workflow

* cache evict code optimization

* test

Co-authored-by: caishunfeng <534328519@qq.com>
2021-12-03 23:13:11 +08:00
zwZjut
1f184440ca
[UT#7157]Fix Ut Run Error (#7158)
* to #7157

* to #7157

Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
2021-12-03 22:01:26 +08:00
Hua Jiang
226bed1768
fix this issue (#7120) 2021-12-03 16:56:30 +08:00
zwZjut
2bf73603bb
[Feature][dolphinscheduler-api] access control of taskDefinition and taskInstance in project to #7081 (#7082)
* to #7081

* fix #7081

* to #7081

Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
2021-12-03 16:56:08 +08:00
Kerwin
3de182dc08
Optimize the display of the pop-up window for creating worker groups (#7148) 2021-12-03 16:55:58 +08:00
zwZjut
8ebe060658
[Feature][dolphinscheduler-api] access control of processDefinition and processInstance in project #7073 (#7080)
* to #7073

* to #7073

* to #7073

Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
2021-12-03 16:55:20 +08:00
zwZjut
d09d68e109
to #7065: fix ExecutorService and schedulerService (#7072)
Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
2021-12-03 16:54:39 +08:00
songjianet
99f6ab3847
[Feature][Workflow relationship] Format name. (#7143) 2021-12-03 14:43:45 +08:00
wangxj3
2b519f31e9
[Feature-#6422] task group queue (#6722)
* add task group

* modify task group

* pull dev

* add license header

* modify code style

* fix code style

* fix sql error

* fix error

* fix test

* fix test

* fix test

* fix test

* fix code style

* fix ut

* code style

* fix unit test

* test ut

* ut

* add unittest

* test ut

* modify back ut

* majorization code

* fix conflict

* fix ut

Co-authored-by: wangxj <wangxj31>
2021-12-03 14:02:31 +08:00
Devosend
f480b8adf8
[Feature-6930] [Python]Add workflow as code task type sub_process (#7022)
* [python] add subProcess task

* refactor python gateway server and task for subprocess

* add function comment for getProcessDefinition

* change process_definition usage of Subprocess task
2021-12-03 10:26:45 +08:00
songjianet
3304edee47
[Feature][Workflow relationship] Format time. (#7123) 2021-12-03 10:02:33 +08:00
songjianet
791ed6de77
[Feature][Workflow relationship] Improve English support. (#7129) 2021-12-03 01:05:45 +08:00
Hua Jiang
6beae191e2
[Fix-7103] Fix the issue with the sqoop task missing the verification of props. (#7119)
* add the verification of parameters for the sqoop task
2021-12-02 19:33:51 +08:00
Kerwin
9fffbdb316
add TaskProcessorFactoryTest Ignore. (#7130) 2021-12-02 19:15:57 +08:00
zwZjut
c42531e39c
to #7108 (#7109) 2021-12-02 16:34:38 +08:00
wangyizhi
e0623973e0
[Fix-7083] [UI] The switch task is missing the else branch (#7084) 2021-12-02 14:35:36 +08:00
JinYong Li
b265297fc8
[Improvement][API] fix 6772 new api bug (#7098)
* add processDefinition releaseWorkflowAndSchedule

* add ProcessTaskRelationServiceImpl.moveTaskProcessRelation

* add dependency and subprocess judgement

* fix api bug

* fix codestyle
2021-12-02 14:06:39 +08:00
Hua Jiang
d7eb7453e5
[Fix-7070][UI] Remedy the issue with no saving the timeout's strategy into the database. (#7107) 2021-12-01 21:03:49 +08:00
wangyizhi
173a385618
[Improvement-7092][UI] Optimize DAG to adapt to situations without locations (#7102) 2021-12-01 16:40:11 +08:00
songjianet
b3efcdfeba
[Fix][Create Warning Instance Edit] Fix the problem of incomplete display of the edit form. (#7096) 2021-12-01 14:41:14 +08:00
张俊杰
463fc76bb7
[Fix-6809][dolphinscheduler-task-sql] Fix sql task can't send alert (#7089) 2021-12-01 14:08:40 +08:00
kezhenxu94
2ed3c0c3a7
Recover unit tests that have not been run for some time (#7054) 2021-12-01 13:55:26 +08:00
Sunny Lei
0dcff1425a
[DS-5229][feat] Implement server port custom config (#6947)
* [DS-5229][fix] server port custom config
This closes #5229

* [DS-5229][feat] Implement server port custom config
This closes #5229

* [Bug] [readme] Error link to Docker and k8s in readme apache#6802 (#6862)

* [Bug] [readme] Error link to Docker and k8s in readme #6802
1、modify the error link

* [Bug] [readme] Error link to Docker and k8s in readme #6802
1、modify the error link in readme_zh_cn.md

* [DS-6829][WorkerServer] skip create log dir and print log in dryRun model (#6852)

Co-authored-by: caishunfeng <534328519@qq.com>

* [DS-5229][feat] Implement server port custom config
Modify review suggestion
This closes #5229

Co-authored-by: GaoTianDuo <gaotianduo_yewu@cmss.chinamobile.com>
Co-authored-by: wind <caishunfeng@users.noreply.github.com>
Co-authored-by: caishunfeng <534328519@qq.com>
2021-12-01 00:21:03 +08:00
zwZjut
100a9ba3fc
to #7055: tenant-auto-create: true (#7056)
Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
2021-11-30 16:44:13 +08:00
kezhenxu94
37ba1eb5ad
Add E2E tests for some core features (#7025) 2021-11-30 13:07:55 +08:00
ouyangyewei
7b20f4c239
return domain after creating success (#7050)
Co-authored-by: ouyangyewei <yewei.oyyw@alibaba-inc.com>
2021-11-30 11:50:32 +08:00
zhang
c2585a1170
[BUG] master task log can't output repair (#7029)
* update task_definition_log field typ

update task_definition_log  field task_params type,modify text to longtext to be consistent with the main table

* update t_ds_task_definition_log field task_params

update t_ds_task_definition_log field task_params

* [FIX:211128] master task log Can't output repair

Co-authored-by: crane <zhangbq3306@163.com>
2021-11-30 11:04:32 +08:00
wind
be3bd4c83d
[DS-7004][MasterServer]fix master still work when it lose zk connection (#7045)
Co-authored-by: caishunfeng <534328519@qq.com>
2021-11-30 11:00:17 +08:00
JinYong Li
6e55394783
[Improvement][API][num-9] add ProcessTaskRelationServiceImpl.moveTaskProcessRelation (#7053)
* add processDefinition releaseWorkflowAndSchedule

* add ProcessTaskRelationServiceImpl.moveTaskProcessRelation

* add dependency and subprocess judgement
2021-11-30 10:24:28 +08:00
Kerwin
f0630bc9a0
Fix view task status module on project homepage cannot read property label of undefined (#6956) 2021-11-30 10:00:54 +08:00
Kerwin
631a31b6ac
fix token delete request 404 (#6979) 2021-11-30 10:00:22 +08:00