Commit Graph

701 Commits

Author SHA1 Message Date
Wenjun Ruan
acd3d3fab1
Fix TaskGroup cannot work and will cause master dead loop (#11254)
* Fix TaskGroup cannot work and will cause master dead loop

* Remove acquireTaskGroupAgain in ProcessServiceImpl
2022-08-04 14:43:45 +08:00
Wenjun Ruan
8e21c38c00
Write alert result into db (#11221) 2022-08-01 13:04:27 +08:00
zhuxt2015
4d9db3451e
[Bug] [API] If the single log length is long, it will causes the View Log page to hang (#10612) 2022-07-26 10:14:45 +08:00
jackfanwan
ccb7641468
修改数据质量查询的权限判断 (#10977)
Co-authored-by: fanwanlong <fanwanlong@kezaihui.com>
2022-07-25 09:35:53 +08:00
WangJPLeo
fd59f0bb32
[Fix-11031] AccessToken can only be used by the creator. (#11032)
* AccessToken can only be used by the creator.

* supplement ut

* checkstyle fix
2022-07-20 16:32:03 +08:00
Wenjun Ruan
5e9c7dad23
Add dolphinscheduler-bom to manage the dependency version (#11025) 2022-07-20 10:37:31 +08:00
Jiajie Zhong
81930e5420
[python] Fix tasks with multiple upstream and workflow query error (#10941)
* when task with more than one upstreams, mapper
   TaskDefinitionMapper method queryByName will return
   more than one record, and failed the mybatis result
   type, so we have to add `limit 1` to it to
* add multiple runs of example in integrate test
* Change from subprocess.Popen to subprocess.call_check
  in integrating test which will raise an error when failed
2022-07-14 22:06:07 +08:00
Assert
4ff8eaa996
[Fix-10595][api] Fix Dependent Node Only View Projects Created By Self (#10834)
* [Fix-10595][api] Fix Dependent Node Only View Projects Created By Self
2022-07-14 19:03:34 +08:00
SongTao Zhuang
99a134febb
fix: typo task definition. (#10958) 2022-07-14 11:40:46 +08:00
WangJPLeo
9f34a837b8
[Fix-10909] Returns executionStatus for each process instance status and access token module support. (#10922)
* Returns executionStatus for each process instance status and access token module support
2022-07-13 20:45:52 +08:00
Jiajie Zhong
0db9bbd538
[fix] delete workflow or task used by other tasks (#10873)
currently, users can delete process definitions used
in other sub-process tasks or in other dependent tasks.
but this change will break the dependence of those task
and failed DAG, this patch add validation of those
delete behavior to avoid this error
2022-07-12 21:22:06 +08:00
Jiajie Zhong
a6154220dc
[fix] update queue and tenant with same code error (#10877)
in ae6aa53f I use the same logic test to create or update
tenant and queue which failed create with the same value,
this patch will fix it. and also I use the constructor to reduce
the new object logic here
2022-07-11 11:06:30 +08:00
Wenjun Ruan
f639a2eed4
[Fix-10854] Fix database restart may lost task instance status (#10866)
* Fix database update error doesn't rollback the task instance status

* Fix database error may cause workflow dead with running status
2022-07-11 09:57:00 +08:00
WangJPLeo
3756a233d8
[Fix-10863] Command execution result data statistics remove userId condition. (#10864)
* Command execution result data statistics remove userId condition.

* e2e rerun
2022-07-10 17:17:35 +08:00
Jiajie Zhong
ae6aa53f96
[python] Fix submit and run error (#10792)
because we add permission, so it failed out
createQueue method. this patch fix it and
do some refactor of our tenant and queue
validator code
2022-07-08 10:54:41 +08:00
qianli2022
1e48bbc5c6
[Feature-10629][Improvement] Support multi cluster environments - namespace use cluster (#10650)
* server code

* namespace ui

* fix dao test

* fix test

* fix

* docs and image update

Co-authored-by: qianl4 <qianl4@cicso.com>
2022-07-07 16:05:52 +08:00
WangJPLeo
b8a90d801e
[Improvement] Resource authority module database table unique index increased. (#10807)
* Resource authority module database table unique index increased.

* add unique in sql file

* rerun

* ut fix

* e2e rerun

* dao un fix
2022-07-07 14:35:50 +08:00
WangJPLeo
d2fe16d252
[Optimization] Config module resource relation fix. (#10718)
* Config module resource relation fix.

* Boolean judge fix

* e2e rerun

* e2e rerun

* e2e rerun

* ArrayList fix

* change variable name.

* Add transaction processing on create data resource method
2022-07-05 15:51:23 +08:00
Wenjun Ruan
1b86394d3c
Optimize sql script use insert ignore (#10700) 2022-06-30 19:04:57 +08:00
WangJPLeo
bc4ceda4cf
TaskGroup remove query func permission check. (#10557) 2022-06-23 10:00:31 +08:00
WangJPLeo
20348578a2
[Feature] [Permission]Data permission module added and Query class api remove verification. (#10504)
* Data permission module added

* ut fix.

* queueController ut fix.

* queue service ut fix.

* e2e rerun
2022-06-22 15:53:36 +08:00
chuxing
0dd6f4008e
[fix][master]: fix condition npe (#10478)
* [fix][master]: fix condition npe

* [fix][master]: fix condition npe

Co-authored-by: huaangcg <huangcg@getui.com>
2022-06-17 13:40:11 +08:00
Wenjun Ruan
ad2646ff1f
Fix TaskProcessorFactory#getTaskProcessor get common processor is not thread safe (#10479)
* Fix TaskProcessorFactory#getTaskProcessor get common processor is not thread safe
2022-06-16 21:46:18 +08:00
SmallProgrammer
7aa9fa4965
[imporve] Add index to table t_ds_relation_process_instance (#10461) 2022-06-16 15:24:05 +08:00
qianli2022
ff065d8e5b
[Feature][Improvement] Support multi cluster environments - k8s type (#10096)
* service code

* [Feature][UI] Add front-end for cluster manage

* fix e2e

* remove comment on cluster controller

* doc

* img

* setting e2e.yaml

* test

* rerun e2e

* fix bug from comment

* Update index.tsx

use Nspace instead of css.

* Update index.tsx

Remove the style.

* Delete index.module.scss

Remove the useless file.

Co-authored-by: qianl4 <qianl4@cicso.com>
Co-authored-by: William Tong <weitong@cisco.com>
Co-authored-by: Amy0104 <97265214+Amy0104@users.noreply.github.com>
2022-06-15 13:39:20 +08:00
陈家名
b86dc53ad1
[fix][Python] Support same task name in project defferent process definition (#10428)
close: #10431
2022-06-14 13:53:18 +08:00
WangJPLeo
9782fe4e7c
[Fix-10367] Permission check related key adjustment. (#10368)
* Permission check related key adjustment.

* resource create or upload fix.

* Analysis Permission fix.

* Ut coverage add.

* rerun

* source permission fix.

* ut coverage is achieved

* Reference permission package path modification

* k8s path bug fix.

* k8s path bug fix.

* Corrected controller method name and rerun.
2022-06-09 19:10:34 +08:00
xiangzihao
b1fb17119b
[Feature-6758][Task] Add limit resource usage for tasks base on cgroup (#10373) 2022-06-08 19:42:05 +08:00
WangJPLeo
e3e39cbdea
[Fix] Privilege Reconstruction Data Query Fix & Resource Creation Post Oper… (#10313)
* Privilege Reconstruction Data Query Fix & Resource Creation Post Operations.

* Resource Retry

* e2e retry

* create project log add

* project list query log add

* clean project log

* delete delay

* delete delay

* remove post handle

* project e2e

* browser refresh

* browser refresh

* e2e fix

* e2e browser refresh

* rowk flow e2e fix

* mapper deduplication

* udf e2e

* e2e
2022-06-01 20:29:10 +08:00
kezhenxu94
d80cf21456
Clean up unused dependencies and packaging issues (#9944) 2022-05-31 15:22:41 +08:00
Wenjun Ruan
3a9e67325e
Fix database initialize late in standalone. (#10296) 2022-05-31 11:52:00 +08:00
WangJPLeo
a4948f58e6
[Feature][Permission] Reconstruction of permissions of resource center and monitoring center. (#10307)
* Reconstruction of permissions of resource center and monitoring center.

* clear local logs.

* resource query fix
2022-05-31 08:14:45 +08:00
hstdream
53ab6f7b92
[Feature][API] Modify the permissions of project management, security center, data source center and data quality module. (#10289)
* Resource userName is null

* [Fix-10082]Resource userName is null and udf function manager add a username

* [Fix-10082]Resource userName is null and udf function manager add a username

* [Fix-10082]Resource userName is null and udf function manager add a username

* [Feature][API] Modify the permissions of project management, security center, data source center and data quality module.

* [Feature][API] Modify the permissions of project management, security center, data source center and data quality module.

* [Feature][API] Modify the permissions of project management, security center, data source center and data quality module.

Co-authored-by: houshitao <shitaohou@163.com>
2022-05-30 23:32:25 +08:00
WangJPLeo
7ac8ea493c
[Feature][Permission] Project management module authority control refactoring, authority abstract interface setting. (#10261)
* Create process definition with task group and task group priority cannot be save into db

* [Feature][API] Permission control of project

* [Feature][API] Permission control of project

* [Feature][API] Permission control of project

* [Feature][API] Permission control of project

* [Feature][API] Permission control of project

* [Feature][API] Permission control of project

* [Feature][API] Permission control of project

* [Feature][API] Permission control of project

* [Feature][API] Permission control of project

* [Feature][API] Permission control of project

* [Feature][API] Permission control of project

* [Feature][API] Permission control of project

* [Feature][API] Permission control of project

* [Feature][API] Permission control of project

* [Feature][API] Permission control of project

* [Feature][API] Permission control of project

* The result of the assertion cannot be changed. The only difference is the judgment condition.

* test recovery.

* controller test fix.

* ProjectE2ETest fix.

* WebElement import.

* Prioritize problem solving.

* Judging the length of the set fix

Co-authored-by: houshitao <shitaohou@163.com>
Co-authored-by: hstdream <33045461+hstdream@users.noreply.github.com>
2022-05-29 16:14:16 +08:00
Jiajie Zhong
d8db2b57d2
[fix] Task group queue query all list error (#10237)
Add support query all task queue by not given specific groupId

close: #10102
2022-05-25 16:33:23 +08:00
longtb
bd34e66b8a
[Fix-10076] queryUserByToken use server timezone (#10177)
* [Fix-10076] queryUserByToken use server timezone

* [Fix-10076] queryUserByToken use server timezone
2022-05-23 09:54:11 +08:00
BaoLiang
ee2b855ced
[BUG][TaskGroup] Task group does not take effect (#10093)
* fix 10092: Task group does not take effect

* fix 10092: Task group does not take effect

* fix 10092: Task group does not take effect
2022-05-18 18:40:36 +08:00
hstdream
701b535ae6
[Fix-10082]Resource userName is null and udf function manager add a username (#10064)
* Resource userName is null

* [Fix-10082]Resource userName is null and udf function manager add a username

* [Fix-10082]Resource userName is null and udf function manager add a username

* [Fix-10082]Resource userName is null and udf function manager add a username

Co-authored-by: houshitao <shitaohou@163.com>
2022-05-18 11:33:08 +08:00
hstdream
cd70ad584a
Create process definition with task group and task group priority cannot be save into db (#10078)
Co-authored-by: houshitao <shitaohou@163.com>
2022-05-18 09:41:03 +08:00
qianli2022
bc1c15b142
[Feature-8252][doc] K8s and namespace manager docs and web page update (#9881)
* test

* test

* doc

* fix image error

* fix

* Update docs/docs/en/guide/security.md

* Update docs/docs/en/guide/security.md

* Update docs/docs/en/guide/security.md

* Update docs/docs/zh/guide/security.md

* fix doc

Co-authored-by: qianl4 <qianl4@cicso.com>
Co-authored-by: William Tong <weitong@cisco.com>
Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com>
2022-05-11 16:40:30 +08:00
Jiajie Zhong
316b611826
[chore] Combine version 206 and 210 sql upgrade to 300 (#9959) 2022-05-10 11:28:56 +08:00
WangJPLeo
fb0f96ed94
[Fix-9868] A task flow definition isolates the runs of different execution strategies by version numbers. (#9869)
* The thread cache task flow definition should get the latest version.

* Coverage on New Code

* Coverage on New Code

* Coverage on New Code

* use an existing method.

* Increase unit test coverage.

* Task flow definitions enforce policy isolation.
2022-05-03 22:52:42 +08:00
Town
fcb32e4c3c
[fix] fix query access token for specified user failed (#9855) 2022-04-30 11:07:00 +08:00
WangJPLeo
ab2d535e54
The node ID under serial is greater than the current ID. (#9820)
Co-authored-by: WangJPLeo <wangjipeng@whaleops.com>
2022-04-28 11:00:33 +08:00
caishunfeng
70dae6945f
revert delete dumplicate task_definition_log ddl (#8567)
Co-authored-by: caishunfeng <534328519@qq.com>
2022-04-27 15:19:40 +08:00
leiwingqueen
c837580c68
[bug][db] Table relation_project_user have duplicate record (#9536)
* update sql ddl
* fix syntax err
2022-04-27 12:04:08 +08:00
WangJPLeo
f839b527bf
Task group queue supports fuzzy search. (#9784) 2022-04-26 18:03:55 +08:00
WangJPLeo
5c0be8a3d7
A task instance that normally queries the serial wait state. (#9777)
Co-authored-by: WangJPLeo <wangjipeng@whaleops.com>
2022-04-26 15:47:01 +08:00
WangJPLeo
7bcec7115a
[Fix-9717] The failure policy of the task flow takes effect (#9718)
* Failure policy takes effect.

* Coverage on New Code

* correct description logic

* Compatible with all scenarios

* clearer logic

Co-authored-by: WangJPLeo <wangjipeng@whaleops.com>
2022-04-25 15:29:18 +08:00
yimaixinchen
fb11525e49
[chore] Correct the java doc in funtion DagHelperTest generateDag2 (#9602)
Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com>
2022-04-23 10:31:48 +08:00