Commit Graph

963 Commits

Author SHA1 Message Date
WangJPLeo
996790ce9e
[Improvement-9609][Worker]The resource download method is selected according to the configurati… (#9636)
* The resource download method is selected according to the configuration and the service startup verification is added.

* common check CI fix

* Startup check changed to running check

* code smell

* Coordinate resources to increase test coverage.

* Split resource download method.

* Unit Test Coverage

Co-authored-by: WangJPLeo <wangjipeng@whaleops.com>
2022-04-22 11:45:49 +08:00
WangJPLeo
ae84900329
[Fix-9617]New task group project name drop-down data is displayed according to user type. (#9625)
* New task group project name drop-down data by user type

* remove unused import

* avoid import *

Co-authored-by: WangJPLeo <wangjipeng@whaleops.com>
2022-04-20 18:24:45 +08:00
qianli2022
165d7aa51f
[Feature][Task] K8s namespace auth manager (#9303)
* k8s auth

* remove log

* fix test

* use constants

* use constants K8S_LOCAL_TEST_CLUSTER

* simple auth get

* change test

* add namespace authorize in user page

* prettier code

* change test data

Co-authored-by: qianl4 <qianl4@cicso.com>
Co-authored-by: William Tong <weitong@cisco.com>
2022-04-20 18:23:23 +08:00
JinYong Li
2aa191014d
fix 9584 (#9585) 2022-04-20 16:10:04 +08:00
WangJPLeo
e2ec489042
[Fix-9610] Sub-workflow status check is limited to SUB_PROCESS components (#9611)
* Sub-workflow status check is limited to SUB_PROCESS components

* Sub-workflow status check is limited to SUB_PROCESS components

Co-authored-by: WangJPLeo <wangjipeng@whaleops.com>
2022-04-20 15:04:27 +08:00
xiangzihao
a3bf10c88d
[Feature][API] Refactor get dependent node information api (#9591)
* feature_8922

* fix comment error

* remove unused import

* fix unused import
2022-04-20 14:53:40 +08:00
Tq
c5b7e5adff
[Bug] [API-9558]fix homepage task instance count method to use submit time to recount (#9559)
* fix homepage task instance count method to use submit time to recount

* fix homepage task instance count method to use submit time to recount

* fix homepage task instance count method to use submit time to recount

* fix homepage task instance count method JUNIT

* fix homepage task instance count method JUNIT

* fix homepage task instance count method JUNIT
2022-04-19 15:23:57 +08:00
WangJPLeo
b1d57dbce4
Check the status of the child process when the parent process is running (#9567)
Co-authored-by: WangJPLeo <wangjipeng@whaleops.com>
2022-04-18 20:27:11 +08:00
calvin
adcc43fd7e
[Improve][API] Allowed the non-root user to create the task group. (#9523)
* create a new branch from dev

* fix this issue

* merge from dev
2022-04-16 21:25:56 +08:00
WangJPLeo
95ccc0c855
[Fix-9516] [Statistics] Statistics Manager shows data should belongs to current user (#9515) 2022-04-15 18:25:25 +08:00
BaoLiang
fea9ce391b
Update inappropriate characters (#9413) 2022-04-10 21:09:48 +08:00
Tq
e2759a8f42
[Feature] [ALERT-9406]add new properties to alert class (#9408)
* add new properties to Alert.java and do minor changes to comments

* fix Integer to int

* fix Integer to int

* fix sql files

* fix not null properties to default null
2022-04-08 23:54:34 +08:00
caishunfeng
ab0357027d
[Improvement] change method access (#9390)
* change method to protected

* change method access
2022-04-07 21:51:21 +08:00
Tq
f186b0d391
[Bug][API-9364]fix ProcessInstance wrong alert group id (#9383)
* fix ProcessInstance wrong alert group id

* change  createComplementCommandList method to protected
2022-04-07 19:19:38 +08:00
Eric Gao
fd6b43bc81
[Dev] Switch version in pom.xml to dev-SNAPSHOT (#9223) (#9299) 2022-04-07 18:15:19 +08:00
Jiajie Zhong
3457cee960
[python] Migrate pythonGatewayServer into api server (#9372)
Currently the size of our distribute package is up to
800MB, this patch is migrate python gateway server into
api server

The distribute package size before and after this patch is:

```sh
# before
796M   apache-dolphinscheduler-2.0.4-SNAPSHOT-bin.tar.gz

# after
647M   apache-dolphinscheduler-2.0.4-SNAPSHOT-bin.tar.gz
```
2022-04-07 14:41:15 +08:00
xiangzihao
1679f15a50
[FIX-9355] Fix scheduleTime of start-process-instance api in api-doc (#9359)
* fix #9355

* fix #9355

* fix ut error

* fix ut error
2022-04-06 21:34:22 +08:00
Tq
4873ec8a45
fix task definition page info total count (#9354) 2022-04-06 09:57:20 +08:00
zchong
4a29c6a6c8
[Improvement-9338][API] show more create datasource exception message (#9336)
* Update DataSourceServiceImpl.java

fix  error message miss

* Update DataSourceServiceImpl.java

import optional jar
2022-04-06 09:17:29 +08:00
mans2singh
4b3401a418
[dolphinscheduler-api][task-group-controller] Corrected notes for create, update and queryTaskGroupByStatus (#9176) 2022-04-01 10:31:41 +08:00
worry
b4b5241797
[DS-9266][feat][server]Implement show dolphinscheduler banner at launch (#9273)
- add banner.txt
- modify application.yaml

This closes #9266
2022-03-30 16:45:24 +08:00
caishunfeng
994de4c902
[Bug-9045][Server] improve log detail error if task instance host is null (#9205)
* improve log detail error if task instance host is null

* test

Co-authored-by: caishunfeng <534328519@qq.com>
2022-03-28 11:36:56 +08:00
caishunfeng
263791a63e
[Bug-9137][Server] fix timezone init (#9219)
Co-authored-by: caishunfeng <534328519@qq.com>
2022-03-28 09:38:51 +08:00
nobolity
0e3cafec1d
[Feature-8612][RESOURCE] extend s3 to the storage of ds (#8637)
* feat(resource  manager): extend s3 to the storage of ds

1.fix some spell question
2.extend the type of storage
3.add the s3utils
to manager resource
4.automatic inject the storage in addition to your
config

* fix(resource  manager): update the dependency

* fix(resource  manager): extend s3 to the storage of ds

fix the constant of hadooputils

* fix(resource  manager): extend s3 to the storage of ds

1.fix some spell question
2.delete the import *

* fix(resource  manager):

merge  the unitTest:
1.TenantServiceImpl
2.ResourceServiceImpl
3.UserServiceImpl

* fix(resource  manager): extend s3 to the storage of ds

merge the resourceServiceTest

* fix(resource  manager): test  cancel the test method

createTenant verifyTenant

* fix(resource  manager): merge the code  follow the check-result of sonar

* fix(resource  manager): extend s3 to the storage of ds

fit the spell question

* fix(resource  manager): extend s3 to the storage of ds

revert the common.properties

* fix(resource  manager): extend s3 to the storage of ds

update the storageConfig with None

* fix(resource  manager): extend s3 to the storage of ds

fix the judge of resourceType

* fix(resource  manager): extend s3 to the storage of ds

undo the compile-mysql

* fix(resource  manager): extend s3 to the storage of ds

delete hadoop aws

* fix(resource  manager): extend s3 to the storage of ds

update the know-dependencies to delete aws 1.7.4
update the e2e
file-manager common.properties

* fix(resource  manager): extend s3 to the storage of ds

update the aws-region

* fix(resource  manager): extend s3 to the storage of ds

fix the storageconfig init

* fix(resource  manager): update e2e docker-compose

update e2e docker-compose

* fix(resource  manager): extend s3 to the storage of ds

revent the e2e common.proprites

print the resource type in propertyUtil

* fix(resource  manager): extend s3 to the storage of ds
1.println the properties

* fix(resource  manager): println the s3 info

* fix(resource  manager): extend s3 to the storage of ds

delete the info  and upgrade the s3 info to e2e

* fix(resource  manager): extend s3 to the storage of ds

add the bucket init

* fix(resource  manager): extend s3 to the storage of ds

1.fix some spell question
2.delete the import *

* fix(resource  manager): extend s3 to the storage of ds

upgrade the s3 endpoint

* fix(resource  manager): withPathStyleAccessEnabled(true)

* fix(resource  manager): extend s3 to the storage of ds

1.fix some spell question
2.delete the import *

* fix(resource  manager): upgrade the  s3client builder

* fix(resource  manager): correct  the s3 point to s3client

* fix(resource  manager): update the constant BUCKET_NAME

* fix(resource  manager): e2e  s3 endpoint -> s3:9000

* fix(resource  manager): extend s3 to the storage of ds

1.fix some spell question
2.delete the import *

* style(resource  manager): add info to createBucket

* style(resource  manager): debug the log

* ci(resource  manager): test

test s3

* ci(ci): add INSERT INTO dolphinscheduler.t_ds_tenant (id, tenant_code, description, queue_id, create_time, update_time) VALUES(1, 'root', NULL, 1, NULL, NULL); to h2.sql

* fix(resource  manager): update the h2 sql

* fix(resource  manager): solve to delete the tenant

* style(resource  manager): merge the style end delete the unuse s3 config

* fix(resource  manager): extend s3 to the storage of ds

UPDATE the rename resources when s3

* fix(resource  manager): extend s3 to the storage of ds

1.fix the code style of QuartzImpl

* fix(resource  manager): extend s3 to the storage of ds

1.impoort restore_type to CommonUtils

* fix(resource  manager): update the work thread

* fix(resource  manager): update  the baseTaskProcessor

* fix(resource  manager): upgrade dolphinscheduler-standalone-server.xml

* fix(resource  manager): add  user Info to dolphinscheduler_h2.sql

* fix(resource  manager): merge  the resourceType to NONE

* style(upgrade the log level to info):

* fix(resource  manager): sysnc the h2.sql

* fix(resource  manager): update the merge the user tenant

* fix(resource  manager): merge the resourcesServiceImpl

* fix(resource  manager):

when the storage is s3 ,that the directory can't be renamed

* fix(resource  manager): in s3 ,the directory cannot be renamed

* fix(resource  manager): delete the deleteRenameDirectory in E2E

* fix(resource  manager): check the style and  recoverd the test

* fix(resource  manager): delete the log.print(LoginUser)
2022-03-25 13:28:13 +08:00
calvin
7c5bebea98
[Fix-8544][API] The folder's size can't be updated when creating, updating or deleting a resource file int the folder. (#9107) 2022-03-23 18:58:41 +08:00
Tq
3c377ff54a
[Bug-9094][API] fix default time-zone conf (#9095)
* fix the API server default time-zone conf

* fix the Python server default time-zone conf
2022-03-22 19:38:46 +08:00
calvin
9b163249d6
[Fix-9025] [UI Next] Rectify this issue about failing to save a process when the user is ordinary user. (#9063)
* modify the back-edn api

* fix this issue
2022-03-21 21:52:07 +08:00
yangyunxi
b7cf8b49c5
AlertPluginInstanceController verifyGroupName method code optimizatio… (#8920)
* AlertPluginInstanceController verifyGroupName method code optimization #8863
Co-authored-by: Kerwin <37063904+zhuangchong@users.noreply.github.com>
2022-03-16 10:42:03 +08:00
caishunfeng
60fbbcb790
[Feature-8530][API] add system default timezone if no user timezone when get userInfo (#8899)
Co-authored-by: caishunfeng <534328519@qq.com>
2022-03-15 17:46:41 +08:00
Kerwin
a1bef43981
[Feature-8579][Task] Task back-end plugin optimization, new plugins only need to modify the plugin's own module (#8584)
* Task plug-in optimization, task-related classes are migrated to the task plugin module

* remove tasktype enum class.

* fix code style.

* fix code style

* fix code style

* fix ProcessServiceTest.testUpdateResourceInfo test class

* fix code style

* fix test task plugin manager check parameters null

* fix test ut

* update aws license

* change TaskTimeoutStrategy class
2022-03-15 15:16:36 +08:00
calvin
c7e80e42d7
[Fix-8836][UI Next][V1.0.0-Alpha] Rectify this issue about creating a file or directory with a wrong path. (#8892)
* fix this issue

* fix this issue

* fix code style

* fix code style
2022-03-15 14:19:19 +08:00
songjianet
db64dcade8
Revert "[Fix-8836][UI Next][V1.0.0-Alpha] Rectify this issue about creating a file or directory with a wrong path. (#8880)" (#8888)
This reverts commit e4dcf3d2e0.
2022-03-15 10:13:37 +08:00
calvin
e4dcf3d2e0
[Fix-8836][UI Next][V1.0.0-Alpha] Rectify this issue about creating a file or directory with a wrong path. (#8880)
* fix this issue

* fix this issue
2022-03-15 09:35:14 +08:00
Tq
1f83140558
fix create&update wrokflow cron APIs, verify end time must bigger than start time (#8871) 2022-03-14 16:40:07 +08:00
kezhenxu94
d3cbf8eeaf
[UI-NEXT] Fix wrong packaging locations (#8851) 2022-03-14 15:26:15 +08:00
ouyangyewei
5fa1c4e7e3
[Feature][dolphinscheduler-api] Return domain object after creating success (#8825)
Co-authored-by: ouyangyewei <yewei.oyyw@alibaba-inc.com>
2022-03-13 21:20:36 +08:00
zhanqian
e466f54c5d
issue 8645 (#8811)
Co-authored-by: 北笙 <“zhanqian@cai-inc.com”>
Co-authored-by: zhanqian <zhanqian@cai-inc.com>
2022-03-10 20:01:44 +08:00
Sunny Lei
2aed250ed4
[DS-7896][fix] Fixed the problem of duplicating project names when authorizing projects (#8453)
This closes #7896
2022-03-10 16:58:02 +08:00
caishunfeng
ddd4474189
change quartz executor to be managed by spring (#8774)
Co-authored-by: caishunfeng <534328519@qq.com>
2022-03-09 21:36:38 +08:00
xiangzihao
fc334b7b13
[DS-8580] Fix ui-next serve router for / path (#8748) 2022-03-08 11:09:08 +08:00
kezhenxu94
d2a360480a
[CI][UI NEXT] Package new UI into release tar and remove old UI from it (#8670)
* [CI][UI NEXT] Package new UI into release tar and remove old UI from it

* Update e2e.yml
2022-03-07 11:20:24 +08:00
xiangzihao
2ab8c1ca7d
[Fix-8616][WorkFlowLineage] work flow lineage search result missing data (#8684)
* fix bug_8616

* remove meaningless query column

* fix code smell

* fix remaining problems
2022-03-04 14:05:59 +08:00
kezhenxu94
e04bdba22f
Adapt web configurations for UI next (#8639) 2022-03-02 12:10:38 +08:00
xiangzihao
3cfb3270fe
[Fix-8581][API-Doc] Add some missing chinese translation (#8588) 2022-03-01 15:14:52 +08:00
yimaixinchen
a72742b9de
remove unuse import jsonFormat (#8548) 2022-02-25 22:05:05 +08:00
xiangzihao
891a002bea
[Feature-#8373][MasterServer] Dependent tasks can re-run automatically in the case of complement (#8496)
* first add feature_8373

* fix code smell

* add blank line

* fix some problems

* fix unit test error
2022-02-25 10:34:02 +08:00
caishunfeng
84f2cfec39
[Feature][Server] Custom timezone - Add timezone handler unified (#8477)
* date convert of timezone

* remove @JsonFormat

* add unit test

* fix time preview in scheduler

* optimization & add env config

Co-authored-by: caishunfeng <534328519@qq.com>
2022-02-24 18:20:34 +08:00
Sunny Lei
1ffb5d6e8d
[DS-7896][fix] Fixed when a non-admin user creates a udf function, resources other than .jar will be displayed in the UDF resource (#8458)
This closes #7892
2022-02-24 18:17:25 +08:00
kezhenxu94
fa19006a18
Add a property to enable/disable metrics module (#8484) 2022-02-24 10:21:47 +08:00
calvin
c590ad43d7
fix this issue (#8509) 2022-02-24 08:50:03 +08:00