Commit Graph

6343 Commits

Author SHA1 Message Date
nobolity
0e3cafec1d
[Feature-8612][RESOURCE] extend s3 to the storage of ds (#8637)
* feat(resource  manager): extend s3 to the storage of ds

1.fix some spell question
2.extend the type of storage
3.add the s3utils
to manager resource
4.automatic inject the storage in addition to your
config

* fix(resource  manager): update the dependency

* fix(resource  manager): extend s3 to the storage of ds

fix the constant of hadooputils

* fix(resource  manager): extend s3 to the storage of ds

1.fix some spell question
2.delete the import *

* fix(resource  manager):

merge  the unitTest:
1.TenantServiceImpl
2.ResourceServiceImpl
3.UserServiceImpl

* fix(resource  manager): extend s3 to the storage of ds

merge the resourceServiceTest

* fix(resource  manager): test  cancel the test method

createTenant verifyTenant

* fix(resource  manager): merge the code  follow the check-result of sonar

* fix(resource  manager): extend s3 to the storage of ds

fit the spell question

* fix(resource  manager): extend s3 to the storage of ds

revert the common.properties

* fix(resource  manager): extend s3 to the storage of ds

update the storageConfig with None

* fix(resource  manager): extend s3 to the storage of ds

fix the judge of resourceType

* fix(resource  manager): extend s3 to the storage of ds

undo the compile-mysql

* fix(resource  manager): extend s3 to the storage of ds

delete hadoop aws

* fix(resource  manager): extend s3 to the storage of ds

update the know-dependencies to delete aws 1.7.4
update the e2e
file-manager common.properties

* fix(resource  manager): extend s3 to the storage of ds

update the aws-region

* fix(resource  manager): extend s3 to the storage of ds

fix the storageconfig init

* fix(resource  manager): update e2e docker-compose

update e2e docker-compose

* fix(resource  manager): extend s3 to the storage of ds

revent the e2e common.proprites

print the resource type in propertyUtil

* fix(resource  manager): extend s3 to the storage of ds
1.println the properties

* fix(resource  manager): println the s3 info

* fix(resource  manager): extend s3 to the storage of ds

delete the info  and upgrade the s3 info to e2e

* fix(resource  manager): extend s3 to the storage of ds

add the bucket init

* fix(resource  manager): extend s3 to the storage of ds

1.fix some spell question
2.delete the import *

* fix(resource  manager): extend s3 to the storage of ds

upgrade the s3 endpoint

* fix(resource  manager): withPathStyleAccessEnabled(true)

* fix(resource  manager): extend s3 to the storage of ds

1.fix some spell question
2.delete the import *

* fix(resource  manager): upgrade the  s3client builder

* fix(resource  manager): correct  the s3 point to s3client

* fix(resource  manager): update the constant BUCKET_NAME

* fix(resource  manager): e2e  s3 endpoint -> s3:9000

* fix(resource  manager): extend s3 to the storage of ds

1.fix some spell question
2.delete the import *

* style(resource  manager): add info to createBucket

* style(resource  manager): debug the log

* ci(resource  manager): test

test s3

* ci(ci): add INSERT INTO dolphinscheduler.t_ds_tenant (id, tenant_code, description, queue_id, create_time, update_time) VALUES(1, 'root', NULL, 1, NULL, NULL); to h2.sql

* fix(resource  manager): update the h2 sql

* fix(resource  manager): solve to delete the tenant

* style(resource  manager): merge the style end delete the unuse s3 config

* fix(resource  manager): extend s3 to the storage of ds

UPDATE the rename resources when s3

* fix(resource  manager): extend s3 to the storage of ds

1.fix the code style of QuartzImpl

* fix(resource  manager): extend s3 to the storage of ds

1.impoort restore_type to CommonUtils

* fix(resource  manager): update the work thread

* fix(resource  manager): update  the baseTaskProcessor

* fix(resource  manager): upgrade dolphinscheduler-standalone-server.xml

* fix(resource  manager): add  user Info to dolphinscheduler_h2.sql

* fix(resource  manager): merge  the resourceType to NONE

* style(upgrade the log level to info):

* fix(resource  manager): sysnc the h2.sql

* fix(resource  manager): update the merge the user tenant

* fix(resource  manager): merge the resourcesServiceImpl

* fix(resource  manager):

when the storage is s3 ,that the directory can't be renamed

* fix(resource  manager): in s3 ,the directory cannot be renamed

* fix(resource  manager): delete the deleteRenameDirectory in E2E

* fix(resource  manager): check the style and  recoverd the test

* fix(resource  manager): delete the log.print(LoginUser)
2022-03-25 13:28:13 +08:00
Tq
97807390a2
fix alert instance proxy configs when no username and password (#9171) 2022-03-25 09:43:59 +08:00
sparklezzz
934d0ad063
[Feat]Restore presto and spark sql datasource in datasource plugin module (#7619)
* feat(datasource-plugin): restore presto and spark sql datasource

* feat(datasource-plugin): restore presto and spark sql datasource

* feat(datasource for spark/presto): fix pom and simplify code

Co-authored-by: xudong.zhang <xudong.zhang@nio.com>
2022-03-24 22:48:55 +08:00
gaojun2048
9996f7a4fa
update some method to protected (#9169) 2022-03-24 21:43:13 +08:00
Amy0104
64d449952d
[Fix][UI Next][V1.0.0-Alpha] Fix the unlimited label in the language pack configuration in DATAX. (#9167) 2022-03-24 18:43:57 +08:00
Devosend
6e6407cb18
[Fix][UI Next][V1.0.0-Alpha] Home white screen bug (#9161)
* fix timezone label bug

* modify default timezone

* support filter for timimg modal
2022-03-24 18:14:03 +08:00
Amy0104
327a60efd3
[Fix][UI Next][V1.0.0-Alpha]Fix the default value of custom template incorrect in DATAX. (#9163) 2022-03-24 18:12:43 +08:00
Amy0104
8ebd24a420
[Fix][UI Next][V1.0.0-Alpha] Fix the title and alarm group not hidden when the sql type is non query in SQL. (#9157) 2022-03-24 17:31:06 +08:00
Amy0104
2d444e56b3
[Fix][UI Next][V1.0.0-Alpha] Remove the resources in SQL. (#9153) 2022-03-24 16:57:50 +08:00
songjianet
647b0990aa
[Fix][UI Next][V1.0.0-Alpha] Fix the upstream Tasks field UI display extra long. (#9148)
* [Fix][UI Next][V1.0.0-Alpha] Fix the upstream Tasks field UI display extra long.
2022-03-24 16:35:29 +08:00
Amy0104
0868a81061
[Fix][UI Next][V1.0.0-Alpha] Remove the timeout settings switch in HTTP. (#9149) 2022-03-24 16:02:26 +08:00
Jiajie Zhong
5289b09817
[python] Fix change exists pd attribute user error (#9140)
* [python] Fix change exists pd attribute user error

* Remove attribute from class ProcessDefinition, cause it should to
  object User instead of object ProcessDefinition.
* Grant project to user if attribute user change for exists ProcessDefinition

close: #8751

* Add py.test conftest.py for package integration
2022-03-24 15:24:02 +08:00
labbomb
da2e85e9ec
[BUG][UI Next]fix the problem about Batch delete not working (#9080)
* fix the problem about Batch delete not working while click the border of the button

* try to fix e2e

Co-authored-by: Jiajie Zhong <zhongjiajie955@hotmail.com>
2022-03-24 14:43:02 +08:00
Amy0104
926fad861e
[Fix][UI Next][V1.0.0-Alpha]Fix the resources options incorrect in FLINK. (#9146) 2022-03-24 13:57:56 +08:00
Amy0104
73ca831cb4
[Fix][UI Next][V1.0.0-Alpha] Add direct and type into the custom par… (#9081)
* [Fix][UI Next][V1.0.0-Alpha] Add direct and type  into the custom parameters  in shell.

* [Fix][UI Next][V1.0.0-Alpha] Add the classes for e2e.
2022-03-24 12:09:25 +08:00
Tq
b9448c3f6f
fix email alert session configs (#9142) 2022-03-24 09:50:50 +08:00
calvin
49de0f092c
[Improve-9109][UI Next][V1.0.0-Alpha] When there's not any relations of workflows this page will display prompts. (#9133) 2022-03-23 22:08:34 +08:00
Amy0104
9c33c3e692
[Fix][UI Next][V1.0.0-Alpha] Fix the Flink Version and App Name not hidden when the deploy mode is local in FLINK. (#9135) 2022-03-23 22:07:34 +08:00
Devosend
d3cbb011c6
[Fix][UI Next][V1.0.0-Alpha] Alarm Instance console error (#9141) 2022-03-23 22:05:47 +08:00
songjianet
ddfd4a5762
[Fix][UI Next][V1.0.0-Alpha] Fix the user token edit bug. (#9123)
* [Fix][UI Next][V1.0.0-Alpha] Fix the user token edit bug.
2022-03-23 19:00:53 +08:00
Kerwin
25fc1dcb5f
Fix the exception of task getting external resource data during execution (#9134) 2022-03-23 19:00:09 +08:00
calvin
7c5bebea98
[Fix-8544][API] The folder's size can't be updated when creating, updating or deleting a resource file int the folder. (#9107) 2022-03-23 18:58:41 +08:00
Amy0104
08ea1aa701
[Fix][UI Next][V1.0.0-Alpha] Fix the deploy mode with an extra client option in FLINK. (#9131) 2022-03-23 18:24:28 +08:00
Amy0104
ecd3ce4196
[Fix][UI Next][V1.0.0-Alpha] Fix the resources options incorrect in SPARK. (#9128) 2022-03-23 17:49:30 +08:00
calvin
8db65fcf1e
[Improve-9118][UI Next][V1.0.0-Alpha] Extend the column of the resource name on the page of the UDF resource management. (#9119) 2022-03-23 16:58:02 +08:00
Amy0104
d2138c7dfa
[Fix][UI Next][V1.0.0-Alpha]Fix the SQL Parameter and UDF function not shown when the datasource type is HIVE. (#9120) 2022-03-23 16:34:09 +08:00
songjianet
c69d9f8a7b
[Fix][UI Next][V1.0.0-Alpha] Fix there are multiple hover prompts. (#9116)
* [Fix][UI Next][V1.0.0-Alpha] Fix there are multiple hover prompts.
2022-03-23 16:00:27 +08:00
calvin
aae328e3a1
[Improve-9108][UI Next][V1.0.0-Alpha] Rectify the issue about having a few misspelled words in the resource management. (#9117) 2022-03-23 14:58:54 +08:00
Amy0104
7022b181f4
[Fix][UI Next][V1.0.0-Alpha] Fix the send email and the log not shown (#9112) 2022-03-23 12:40:42 +08:00
wangyizhi
8d1974b9bc
[Fix][UI Next][V1.0.0-Alpha] Fix the task icons not shown in dag page (#9104) 2022-03-23 12:36:15 +08:00
Devosend
184691ab17
[Fix][UI Next][V1.0.0-Alpha] Fix udf function auth bug (#9111) 2022-03-23 12:34:58 +08:00
Devosend
0c5c9cabc3
[Fix][UI Next][V1.0.0-Alpha]Fix udf function upload internationalization bug (#9110) 2022-03-23 12:27:00 +08:00
Tq
3c377ff54a
[Bug-9094][API] fix default time-zone conf (#9095)
* fix the API server default time-zone conf

* fix the Python server default time-zone conf
2022-03-22 19:38:46 +08:00
Devosend
0b2923d577
[Fix][UI Next][V1.0.0-Alpha] Resource authorization failed (#9092)
* fix resources auth bug

* support batch select for resource auth
2022-03-22 18:56:41 +08:00
songjianet
d641c8d4ea
[Fix][UI Next][V1.0.0-Alpha] Fix the problem that the fields in the workflow relationship are disordered and the time is empty. (#9091) 2022-03-22 18:49:15 +08:00
worry
23b24508ab
[Fix-9078][UI Next][V1.0.0-Alpha] The interface for adding DingTalk alert instance is not fully internationalized (#9079) 2022-03-22 16:07:42 +08:00
songjianet
117963f7c8
[Fix][UI Next][V1.0.0-Alpha] Fix the worker page name. (#9086) 2022-03-22 16:05:42 +08:00
Devosend
5762a5c86f
[Fix][UI Next][V1.0.0-Alpha] Workflow upload button internationalization (#9083) 2022-03-22 15:13:04 +08:00
Amy0104
936ea99303
[Fix][UI Next][V1.0.0-Alpha]Fix default display of node name in sub process node. (#9075) 2022-03-22 15:06:55 +08:00
xiangzihao
197accc601
[Feature][e2e] Suggest e2e test adapt M1 chip (#9077) 2022-03-22 14:11:50 +08:00
caishunfeng
852597c9bf
[Bug-9070][Server]fix taskExecutionContext json convert error (#9072)
* [Bug-9070][Server]fix taskExecutionContext json convert error

* test

* add license header

Co-authored-by: caishunfeng <534328519@qq.com>
2022-03-22 14:02:07 +08:00
Devosend
f5ef410d06
[Fix][UI Next][V1.0.0-Alpha] fix queue display bug in tenant modal (#9064)
* fix queue always display first queue bug in tenant modal

* modify queueId init
2022-03-22 11:27:22 +08:00
Amy0104
bf1f959f58
[Feature][UI Next][V1.0.0-Alpha] Adjust all table column widths in th… (#9062)
* [Feature][UI Next][V1.0.0-Alpha] Adjust all table column widths in this project.

* [Fix][UI Next][V1.0.0-Alpha] Fix workflow delete handle (#9057)

* [Fix][UI Next][V1.0.0-Alpha] User Manage Queue display error (#9060)

* fix package bug (#9051)

* [Fix][UI Next][V1.0.0-Alpha] Fix the monitoring center does not support multiple worker. (#9061)

* [Fix][UI Next][V1.0.0-Alpha] Remove the unused Ref.

* [Fix-9025] [UI Next] Rectify this issue about failing to save a process when the user is ordinary user. (#9063)

* modify the back-edn api

* fix this issue

* [Fix][UI Next][V1.0.0-Alpha]Change the column-config file name to column-width-config and COLUMN_CONFIG to COLUMN_WIDTH_CONFIG.

* [Bug][UI Next] Fix style errors (#9067)

* [Feature][UI Next][V1.0.0-Alpha] Adjust all table column widths in this project.

* [Fix][UI Next][V1.0.0-Alpha] Remove the unused Ref.

* [Fix][UI Next][V1.0.0-Alpha]Change the column-config file name to column-width-config and COLUMN_CONFIG to COLUMN_WIDTH_CONFIG.

Co-authored-by: Devosend <devosend@gmail.com>
Co-authored-by: zixi0825 <sunzhaohe0825@gmail.com>
Co-authored-by: songjianet <1778651752@qq.com>
Co-authored-by: calvin <jianghuachinacom@163.com>
Co-authored-by: labbomb <739955946@qq.com>
2022-03-22 10:36:40 +08:00
labbomb
dac39921d8
[Bug][UI Next] Fix style errors (#9067) 2022-03-22 10:01:07 +08:00
calvin
9b163249d6
[Fix-9025] [UI Next] Rectify this issue about failing to save a process when the user is ordinary user. (#9063)
* modify the back-edn api

* fix this issue
2022-03-21 21:52:07 +08:00
songjianet
659c610457
[Fix][UI Next][V1.0.0-Alpha] Fix the monitoring center does not support multiple worker. (#9061) 2022-03-21 20:00:25 +08:00
zixi0825
63a10037a6
fix package bug (#9051) 2022-03-21 19:57:03 +08:00
Devosend
bae9f16b6c
[Fix][UI Next][V1.0.0-Alpha] User Manage Queue display error (#9060) 2022-03-21 18:56:49 +08:00
Devosend
cfe2473e57
[Fix][UI Next][V1.0.0-Alpha] Fix workflow delete handle (#9057) 2022-03-21 18:50:26 +08:00
xiangzihao
e0e7701341
[E2E] [CI] Fix flaky e2e test cause by #9036 (#9053) 2022-03-21 17:30:33 +08:00