Commit Graph

980 Commits

Author SHA1 Message Date
Paul Zhang
8562f6a878
[Feature][Log]Add timezone information in log output (#9913) 2022-05-06 17:31:44 +08:00
songjianet
7a7807e520
[Feature] Remove old ui. (#9909)
* [Feature] Remove old ui.

* [Docs] Add license file for screenfull.
2022-05-06 14:04:35 +08:00
litiliu
e41de35e17
[Fix-9856]Failed to upload the file but get misleading error message (#9861)
* [fix-9856]Failed to upload the file because the full name was too long, but get misleading message

* Update Status.java

Co-authored-by: litiliu <litiliu@cisco.com>
Co-authored-by: lidongdai <dailidong66@gmail.com>
2022-05-04 21:20:14 +08:00
caishunfeng
5d5125d507
[Bug-9829][Api] Fix schedule timezone (#9830) 2022-04-29 10:17:34 +08:00
陈家名
828034f8b9
[improve][python] Specify warning group when deploy process-definition (#9773) 2022-04-27 22:02:14 +08:00
WangJPLeo
d2fe85d7da
[Fix-9796] The task queue can be executed normally. (#9797)
* Task group queue run manually

* Test repetition.

* Reference package is not available*.

Co-authored-by: WangJPLeo <wangjipeng@whaleops.com>
2022-04-27 17:23:48 +08:00
leiwingqueen
c837580c68
[bug][db] Table relation_project_user have duplicate record (#9536)
* update sql ddl
* fix syntax err
2022-04-27 12:04:08 +08:00
hstdream
341e220bf0
[Bug][Api] Failed to rerun the workflow instance (#9795) 2022-04-26 23:04:38 +08:00
Tq
8b3984b20e
fix query issue when update environment (#9752) 2022-04-25 17:46:36 +08:00
WangJPLeo
ea002452b0
Subtask component treemap problem fix. (#9755)
Co-authored-by: WangJPLeo <wangjipeng@whaleops.com>
2022-04-25 17:07:15 +08:00
Jiajie Zhong
de50f43de6
[common] Make dolphinscheduler_env.sh work when start server (#9726)
* [common] Make dolphinscheduler_env.sh work

* Change dist tarball `dolphinscheduler_env.sh` location
  from `bin/` to `conf/`, which users could finish their
  change configuration operation in one single directory.
  and we only need to add `$DOLPHINSCHEDULER_HOME/conf`
  when we start our sever instead of adding both
  `$DOLPHINSCHEDULER_HOME/conf` and `$DOLPHINSCHEDULER_HOME/bin`
* Change the `start.sh`'s path of `dolphinscheduler_env.sh`
* Change the setting order of `dolphinscheduler_env.sh`
* `bin/env/dolphinscheduler_env.sh` will overwrite the `<server>/conf/dolphinscheduler_env.sh`
when start the server using `bin/dolphinsceduler_daemon.sh` or `bin/install.sh`
* Change the related docs
2022-04-25 15:35:43 +08:00
caishunfeng
0176f4bf61
[Bug-9737][Api] fix task plugin load in api (#9744)
* fix task plugin load in api

* task plugin loading by event
2022-04-25 13:26:34 +08:00
gaojun2048
ebc4253d50
[fix][Service] BusinessTime should format with schedule timezone (#9714)
* BusinessTime should format with schedule timezone

* fix test error

* fix test error

* fix test error
2022-04-24 19:21:21 +08:00
Tq
a51b710b1c
fix alert msg and change primitive to String to avoid wrong format (#9689) 2022-04-24 13:29:27 +08:00
Mr.An
29a0ea32c6
[Fix] Support more generic tenant code when create tenant (#9634) 2022-04-23 18:41:03 +08:00
WangJPLeo
387ebe5bb0
Project management batch deletion should give a specific description if it fails. (#9669)
Co-authored-by: WangJPLeo <wangjipeng@whaleops.com>
2022-04-22 18:17:22 +08:00
exmy
267b307632
[improve][api] Support to upload file without file type suffix (#9553) 2022-04-22 14:42:41 +08:00
WangJPLeo
996790ce9e
[Improvement-9609][Worker]The resource download method is selected according to the configurati… (#9636)
* The resource download method is selected according to the configuration and the service startup verification is added.

* common check CI fix

* Startup check changed to running check

* code smell

* Coordinate resources to increase test coverage.

* Split resource download method.

* Unit Test Coverage

Co-authored-by: WangJPLeo <wangjipeng@whaleops.com>
2022-04-22 11:45:49 +08:00
WangJPLeo
ae84900329
[Fix-9617]New task group project name drop-down data is displayed according to user type. (#9625)
* New task group project name drop-down data by user type

* remove unused import

* avoid import *

Co-authored-by: WangJPLeo <wangjipeng@whaleops.com>
2022-04-20 18:24:45 +08:00
qianli2022
165d7aa51f
[Feature][Task] K8s namespace auth manager (#9303)
* k8s auth

* remove log

* fix test

* use constants

* use constants K8S_LOCAL_TEST_CLUSTER

* simple auth get

* change test

* add namespace authorize in user page

* prettier code

* change test data

Co-authored-by: qianl4 <qianl4@cicso.com>
Co-authored-by: William Tong <weitong@cisco.com>
2022-04-20 18:23:23 +08:00
JinYong Li
2aa191014d
fix 9584 (#9585) 2022-04-20 16:10:04 +08:00
WangJPLeo
e2ec489042
[Fix-9610] Sub-workflow status check is limited to SUB_PROCESS components (#9611)
* Sub-workflow status check is limited to SUB_PROCESS components

* Sub-workflow status check is limited to SUB_PROCESS components

Co-authored-by: WangJPLeo <wangjipeng@whaleops.com>
2022-04-20 15:04:27 +08:00
xiangzihao
a3bf10c88d
[Feature][API] Refactor get dependent node information api (#9591)
* feature_8922

* fix comment error

* remove unused import

* fix unused import
2022-04-20 14:53:40 +08:00
Tq
c5b7e5adff
[Bug] [API-9558]fix homepage task instance count method to use submit time to recount (#9559)
* fix homepage task instance count method to use submit time to recount

* fix homepage task instance count method to use submit time to recount

* fix homepage task instance count method to use submit time to recount

* fix homepage task instance count method JUNIT

* fix homepage task instance count method JUNIT

* fix homepage task instance count method JUNIT
2022-04-19 15:23:57 +08:00
WangJPLeo
b1d57dbce4
Check the status of the child process when the parent process is running (#9567)
Co-authored-by: WangJPLeo <wangjipeng@whaleops.com>
2022-04-18 20:27:11 +08:00
calvin
adcc43fd7e
[Improve][API] Allowed the non-root user to create the task group. (#9523)
* create a new branch from dev

* fix this issue

* merge from dev
2022-04-16 21:25:56 +08:00
WangJPLeo
95ccc0c855
[Fix-9516] [Statistics] Statistics Manager shows data should belongs to current user (#9515) 2022-04-15 18:25:25 +08:00
BaoLiang
fea9ce391b
Update inappropriate characters (#9413) 2022-04-10 21:09:48 +08:00
Tq
e2759a8f42
[Feature] [ALERT-9406]add new properties to alert class (#9408)
* add new properties to Alert.java and do minor changes to comments

* fix Integer to int

* fix Integer to int

* fix sql files

* fix not null properties to default null
2022-04-08 23:54:34 +08:00
caishunfeng
ab0357027d
[Improvement] change method access (#9390)
* change method to protected

* change method access
2022-04-07 21:51:21 +08:00
Tq
f186b0d391
[Bug][API-9364]fix ProcessInstance wrong alert group id (#9383)
* fix ProcessInstance wrong alert group id

* change  createComplementCommandList method to protected
2022-04-07 19:19:38 +08:00
Eric Gao
fd6b43bc81
[Dev] Switch version in pom.xml to dev-SNAPSHOT (#9223) (#9299) 2022-04-07 18:15:19 +08:00
Jiajie Zhong
3457cee960
[python] Migrate pythonGatewayServer into api server (#9372)
Currently the size of our distribute package is up to
800MB, this patch is migrate python gateway server into
api server

The distribute package size before and after this patch is:

```sh
# before
796M   apache-dolphinscheduler-2.0.4-SNAPSHOT-bin.tar.gz

# after
647M   apache-dolphinscheduler-2.0.4-SNAPSHOT-bin.tar.gz
```
2022-04-07 14:41:15 +08:00
xiangzihao
1679f15a50
[FIX-9355] Fix scheduleTime of start-process-instance api in api-doc (#9359)
* fix #9355

* fix #9355

* fix ut error

* fix ut error
2022-04-06 21:34:22 +08:00
Tq
4873ec8a45
fix task definition page info total count (#9354) 2022-04-06 09:57:20 +08:00
zchong
4a29c6a6c8
[Improvement-9338][API] show more create datasource exception message (#9336)
* Update DataSourceServiceImpl.java

fix  error message miss

* Update DataSourceServiceImpl.java

import optional jar
2022-04-06 09:17:29 +08:00
mans2singh
4b3401a418
[dolphinscheduler-api][task-group-controller] Corrected notes for create, update and queryTaskGroupByStatus (#9176) 2022-04-01 10:31:41 +08:00
worry
b4b5241797
[DS-9266][feat][server]Implement show dolphinscheduler banner at launch (#9273)
- add banner.txt
- modify application.yaml

This closes #9266
2022-03-30 16:45:24 +08:00
caishunfeng
994de4c902
[Bug-9045][Server] improve log detail error if task instance host is null (#9205)
* improve log detail error if task instance host is null

* test

Co-authored-by: caishunfeng <534328519@qq.com>
2022-03-28 11:36:56 +08:00
caishunfeng
263791a63e
[Bug-9137][Server] fix timezone init (#9219)
Co-authored-by: caishunfeng <534328519@qq.com>
2022-03-28 09:38:51 +08:00
nobolity
0e3cafec1d
[Feature-8612][RESOURCE] extend s3 to the storage of ds (#8637)
* feat(resource  manager): extend s3 to the storage of ds

1.fix some spell question
2.extend the type of storage
3.add the s3utils
to manager resource
4.automatic inject the storage in addition to your
config

* fix(resource  manager): update the dependency

* fix(resource  manager): extend s3 to the storage of ds

fix the constant of hadooputils

* fix(resource  manager): extend s3 to the storage of ds

1.fix some spell question
2.delete the import *

* fix(resource  manager):

merge  the unitTest:
1.TenantServiceImpl
2.ResourceServiceImpl
3.UserServiceImpl

* fix(resource  manager): extend s3 to the storage of ds

merge the resourceServiceTest

* fix(resource  manager): test  cancel the test method

createTenant verifyTenant

* fix(resource  manager): merge the code  follow the check-result of sonar

* fix(resource  manager): extend s3 to the storage of ds

fit the spell question

* fix(resource  manager): extend s3 to the storage of ds

revert the common.properties

* fix(resource  manager): extend s3 to the storage of ds

update the storageConfig with None

* fix(resource  manager): extend s3 to the storage of ds

fix the judge of resourceType

* fix(resource  manager): extend s3 to the storage of ds

undo the compile-mysql

* fix(resource  manager): extend s3 to the storage of ds

delete hadoop aws

* fix(resource  manager): extend s3 to the storage of ds

update the know-dependencies to delete aws 1.7.4
update the e2e
file-manager common.properties

* fix(resource  manager): extend s3 to the storage of ds

update the aws-region

* fix(resource  manager): extend s3 to the storage of ds

fix the storageconfig init

* fix(resource  manager): update e2e docker-compose

update e2e docker-compose

* fix(resource  manager): extend s3 to the storage of ds

revent the e2e common.proprites

print the resource type in propertyUtil

* fix(resource  manager): extend s3 to the storage of ds
1.println the properties

* fix(resource  manager): println the s3 info

* fix(resource  manager): extend s3 to the storage of ds

delete the info  and upgrade the s3 info to e2e

* fix(resource  manager): extend s3 to the storage of ds

add the bucket init

* fix(resource  manager): extend s3 to the storage of ds

1.fix some spell question
2.delete the import *

* fix(resource  manager): extend s3 to the storage of ds

upgrade the s3 endpoint

* fix(resource  manager): withPathStyleAccessEnabled(true)

* fix(resource  manager): extend s3 to the storage of ds

1.fix some spell question
2.delete the import *

* fix(resource  manager): upgrade the  s3client builder

* fix(resource  manager): correct  the s3 point to s3client

* fix(resource  manager): update the constant BUCKET_NAME

* fix(resource  manager): e2e  s3 endpoint -> s3:9000

* fix(resource  manager): extend s3 to the storage of ds

1.fix some spell question
2.delete the import *

* style(resource  manager): add info to createBucket

* style(resource  manager): debug the log

* ci(resource  manager): test

test s3

* ci(ci): add INSERT INTO dolphinscheduler.t_ds_tenant (id, tenant_code, description, queue_id, create_time, update_time) VALUES(1, 'root', NULL, 1, NULL, NULL); to h2.sql

* fix(resource  manager): update the h2 sql

* fix(resource  manager): solve to delete the tenant

* style(resource  manager): merge the style end delete the unuse s3 config

* fix(resource  manager): extend s3 to the storage of ds

UPDATE the rename resources when s3

* fix(resource  manager): extend s3 to the storage of ds

1.fix the code style of QuartzImpl

* fix(resource  manager): extend s3 to the storage of ds

1.impoort restore_type to CommonUtils

* fix(resource  manager): update the work thread

* fix(resource  manager): update  the baseTaskProcessor

* fix(resource  manager): upgrade dolphinscheduler-standalone-server.xml

* fix(resource  manager): add  user Info to dolphinscheduler_h2.sql

* fix(resource  manager): merge  the resourceType to NONE

* style(upgrade the log level to info):

* fix(resource  manager): sysnc the h2.sql

* fix(resource  manager): update the merge the user tenant

* fix(resource  manager): merge the resourcesServiceImpl

* fix(resource  manager):

when the storage is s3 ,that the directory can't be renamed

* fix(resource  manager): in s3 ,the directory cannot be renamed

* fix(resource  manager): delete the deleteRenameDirectory in E2E

* fix(resource  manager): check the style and  recoverd the test

* fix(resource  manager): delete the log.print(LoginUser)
2022-03-25 13:28:13 +08:00
calvin
7c5bebea98
[Fix-8544][API] The folder's size can't be updated when creating, updating or deleting a resource file int the folder. (#9107) 2022-03-23 18:58:41 +08:00
Tq
3c377ff54a
[Bug-9094][API] fix default time-zone conf (#9095)
* fix the API server default time-zone conf

* fix the Python server default time-zone conf
2022-03-22 19:38:46 +08:00
calvin
9b163249d6
[Fix-9025] [UI Next] Rectify this issue about failing to save a process when the user is ordinary user. (#9063)
* modify the back-edn api

* fix this issue
2022-03-21 21:52:07 +08:00
yangyunxi
b7cf8b49c5
AlertPluginInstanceController verifyGroupName method code optimizatio… (#8920)
* AlertPluginInstanceController verifyGroupName method code optimization #8863
Co-authored-by: Kerwin <37063904+zhuangchong@users.noreply.github.com>
2022-03-16 10:42:03 +08:00
caishunfeng
60fbbcb790
[Feature-8530][API] add system default timezone if no user timezone when get userInfo (#8899)
Co-authored-by: caishunfeng <534328519@qq.com>
2022-03-15 17:46:41 +08:00
Kerwin
a1bef43981
[Feature-8579][Task] Task back-end plugin optimization, new plugins only need to modify the plugin's own module (#8584)
* Task plug-in optimization, task-related classes are migrated to the task plugin module

* remove tasktype enum class.

* fix code style.

* fix code style

* fix code style

* fix ProcessServiceTest.testUpdateResourceInfo test class

* fix code style

* fix test task plugin manager check parameters null

* fix test ut

* update aws license

* change TaskTimeoutStrategy class
2022-03-15 15:16:36 +08:00
calvin
c7e80e42d7
[Fix-8836][UI Next][V1.0.0-Alpha] Rectify this issue about creating a file or directory with a wrong path. (#8892)
* fix this issue

* fix this issue

* fix code style

* fix code style
2022-03-15 14:19:19 +08:00
songjianet
db64dcade8
Revert "[Fix-8836][UI Next][V1.0.0-Alpha] Rectify this issue about creating a file or directory with a wrong path. (#8880)" (#8888)
This reverts commit e4dcf3d2e0.
2022-03-15 10:13:37 +08:00
calvin
e4dcf3d2e0
[Fix-8836][UI Next][V1.0.0-Alpha] Rectify this issue about creating a file or directory with a wrong path. (#8880)
* fix this issue

* fix this issue
2022-03-15 09:35:14 +08:00