Commit Graph

175 Commits

Author SHA1 Message Date
Tq
c7435a51ac
[Feature][task-plugins] aws datasync task plugins (#11865)
* submit datasync task

* add aws software package license
2022-10-09 10:47:01 +08:00
caishunfeng
40090d459e
[Improvement] remove log-server and server module (#12206)
* remove log-server module

* remove dolphinscheduler-server module
2022-09-30 16:41:41 +08:00
JieguangZhou
308f632fcc
[Feature][TASK-PLUGIN] AWS Database Migration Service (DMS) (#11868)
* add aws dms
2022-09-30 10:16:16 +08:00
Tq
cf5a8894e2
[BUG][BE-12113]fix htrace dependency conflicts (#12126)
* fix htrace dependency conflicts

* replace htrace-core with hbase-noop-htrace
2022-09-26 13:49:08 +08:00
Eric Gao
66958b91d7
[Feature][Resource Center] Add support for Alibaba Cloud OSS as storage of resource center (#11708)
* Add support for Alibaba Cloud OSS as storage of resource center

* Add dependency licenses

* Refactor code, add unit tests and related docs
2022-09-20 22:44:06 +08:00
songjianet
8cddb10773
[Feat][UI] Update package. (#12046)
* [Feat][UI] Update package.

* [Feat][UI] Update package.
2022-09-19 20:58:41 +08:00
kezhenxu94
147e0ae8d7
Remove duplicate Spring version management (#11899) 2022-09-11 20:50:07 +08:00
kezhenxu94
105773f54d
Remove htrace dependencies (#11881)
* Remove htrace dependencies

* Update pom.xml
2022-09-10 22:54:38 +08:00
Kengo Seki
3664d85143
[Improve] Upgrade Hadoop to 3.2.4 (#11832) 2022-09-09 10:47:03 +08:00
kezhenxu94
efedc9a53c
Bump up dependencies to fix cves (#11765) 2022-09-08 09:05:05 +08:00
wjf
2eb8b9f7f0
[Feature][Registry] Support etcd as registry (#10981) 2022-09-06 22:25:19 +08:00
fuchanghai
554562bfa9
[fix-#11691] Add lz4-java in clickhouse datasource. (#11697) 2022-09-05 09:15:58 +08:00
kezhenxu94
277f137358
Add Kubernetes configmap reload to all components (#11730) 2022-09-02 12:03:19 +08:00
kezhenxu94
13e4720aa2
Upgrade spring boot and swagger (#11574) 2022-08-22 10:27:40 +08:00
kezhenxu94
f6ef628030
Support reading application properties from kubernetes config map (#11402) 2022-08-16 20:25:17 +08:00
Devosend
91a944f967
fix nodeSize is not a function bug (#11200) 2022-07-29 17:39:45 +08:00
Wenjun Ruan
c7789bf049
Add base loop task execotor and http template parser (#11137)
* Add base loop task execotor and http template parser

* Add JsonPathUtils
2022-07-26 14:51:50 +08:00
JieguangZhou
af7b359333
[Feature][Task Plugin] Add SageMaker Pipeline task plugin for MLOps scenario (#10826) (#10935)
* add sagemaker pipeline task plugin

[DOC] add Sagemaker task plugin document

change license and fix dependencies

* [fix] Optimize the code

* Update dolphinscheduler-task-plugin/dolphinscheduler-task-sagemaker/src/main/java/org/apache/dolphinscheduler/plugin/task/sagemaker/SagemakerParameters.java

Optimization parameter judgment

Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>

* [fix] nips

Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
2022-07-21 18:16:58 +08:00
Chris Ho
0cf31232de
[Feature][dependencies] Bump spring-core from 5.3.12 to 5.3.19 (#10857) 2022-07-09 15:19:46 +08:00
songjianet
41698011b3
[Docs] Updated license. (#10838) 2022-07-07 21:00:32 +08:00
songjianet
36bba3fcdf
[Docs] Upgrade license. (#10365) 2022-06-06 11:16:06 +08:00
PJ Fanning
d044e0479d
issue-10356: upgrade logback to fix cve (#10357) 2022-06-03 19:21:40 +08:00
PJ Fanning
b6350280e6
issues-10354: upgrade commons-io to fix CVE (#10355) 2022-06-03 17:17:52 +08:00
kezhenxu94
d80cf21456
Clean up unused dependencies and packaging issues (#9944) 2022-05-31 15:22:41 +08:00
songjianet
cbf0010b83
[Docs] Upgrade license. (#9967) 2022-05-10 15:29:19 +08:00
songjianet
7a7807e520
[Feature] Remove old ui. (#9909)
* [Feature] Remove old ui.

* [Docs] Add license file for screenfull.
2022-05-06 14:04:35 +08:00
songjianet
083050a107
[Docs][UI Next][V1.0.0-Beta] Add @types-lodash license. (#9883) 2022-05-05 11:49:49 +08:00
Kerwin
4f2b9ac886
[Improve][common] code clean in the common module (#9435) 2022-04-29 10:34:03 +08:00
Tq
14144e8d2e
[Bug][dep] Bump httpclient version to 4.5.13 (#9631) 2022-04-27 12:11:11 +08:00
PJ Fanning
f45fe85703
upgrade commons-compress to 1.21 (#9540) 2022-04-20 17:42:14 +08:00
PJ Fanning
7fecb92fc2
use secure version of postgresql (#9573)
* use secure version of postgresql

* Update known-dependencies.txt
2022-04-20 17:14:47 +08:00
songjianet
3ea942dbf8
[Docs] Add license file for screenfull. (#9581) 2022-04-19 16:10:34 +08:00
Eric Gao
a5bbf7852d
[Feature][Task-Plugin]Add zeppelin task-plugin to support Apache Zeppelin (#9327) 2022-04-17 22:10:10 +08:00
Kerwin
4221ee2433
Add python module dependency in the dist module (#9506) 2022-04-15 13:53:37 +08:00
Jiajie Zhong
3d78859fe5
[python] Recover python release properties (#9444)
This patch recovers the properties `python.sign.skip=false`
when the combined profile `release,python` is used.

also close: #9433
2022-04-12 09:42:59 +08:00
Eric Gao
fd6b43bc81
[Dev] Switch version in pom.xml to dev-SNAPSHOT (#9223) (#9299) 2022-04-07 18:15:19 +08:00
Jiajie Zhong
3457cee960
[python] Migrate pythonGatewayServer into api server (#9372)
Currently the size of our distribute package is up to
800MB, this patch is migrate python gateway server into
api server

The distribute package size before and after this patch is:

```sh
# before
796M   apache-dolphinscheduler-2.0.4-SNAPSHOT-bin.tar.gz

# after
647M   apache-dolphinscheduler-2.0.4-SNAPSHOT-bin.tar.gz
```
2022-04-07 14:41:15 +08:00
songjianet
81f9b876f9
[Build] Update NaiveUI. (#9306) 2022-03-31 20:41:36 +08:00
nobolity
0e3cafec1d
[Feature-8612][RESOURCE] extend s3 to the storage of ds (#8637)
* feat(resource  manager): extend s3 to the storage of ds

1.fix some spell question
2.extend the type of storage
3.add the s3utils
to manager resource
4.automatic inject the storage in addition to your
config

* fix(resource  manager): update the dependency

* fix(resource  manager): extend s3 to the storage of ds

fix the constant of hadooputils

* fix(resource  manager): extend s3 to the storage of ds

1.fix some spell question
2.delete the import *

* fix(resource  manager):

merge  the unitTest:
1.TenantServiceImpl
2.ResourceServiceImpl
3.UserServiceImpl

* fix(resource  manager): extend s3 to the storage of ds

merge the resourceServiceTest

* fix(resource  manager): test  cancel the test method

createTenant verifyTenant

* fix(resource  manager): merge the code  follow the check-result of sonar

* fix(resource  manager): extend s3 to the storage of ds

fit the spell question

* fix(resource  manager): extend s3 to the storage of ds

revert the common.properties

* fix(resource  manager): extend s3 to the storage of ds

update the storageConfig with None

* fix(resource  manager): extend s3 to the storage of ds

fix the judge of resourceType

* fix(resource  manager): extend s3 to the storage of ds

undo the compile-mysql

* fix(resource  manager): extend s3 to the storage of ds

delete hadoop aws

* fix(resource  manager): extend s3 to the storage of ds

update the know-dependencies to delete aws 1.7.4
update the e2e
file-manager common.properties

* fix(resource  manager): extend s3 to the storage of ds

update the aws-region

* fix(resource  manager): extend s3 to the storage of ds

fix the storageconfig init

* fix(resource  manager): update e2e docker-compose

update e2e docker-compose

* fix(resource  manager): extend s3 to the storage of ds

revent the e2e common.proprites

print the resource type in propertyUtil

* fix(resource  manager): extend s3 to the storage of ds
1.println the properties

* fix(resource  manager): println the s3 info

* fix(resource  manager): extend s3 to the storage of ds

delete the info  and upgrade the s3 info to e2e

* fix(resource  manager): extend s3 to the storage of ds

add the bucket init

* fix(resource  manager): extend s3 to the storage of ds

1.fix some spell question
2.delete the import *

* fix(resource  manager): extend s3 to the storage of ds

upgrade the s3 endpoint

* fix(resource  manager): withPathStyleAccessEnabled(true)

* fix(resource  manager): extend s3 to the storage of ds

1.fix some spell question
2.delete the import *

* fix(resource  manager): upgrade the  s3client builder

* fix(resource  manager): correct  the s3 point to s3client

* fix(resource  manager): update the constant BUCKET_NAME

* fix(resource  manager): e2e  s3 endpoint -> s3:9000

* fix(resource  manager): extend s3 to the storage of ds

1.fix some spell question
2.delete the import *

* style(resource  manager): add info to createBucket

* style(resource  manager): debug the log

* ci(resource  manager): test

test s3

* ci(ci): add INSERT INTO dolphinscheduler.t_ds_tenant (id, tenant_code, description, queue_id, create_time, update_time) VALUES(1, 'root', NULL, 1, NULL, NULL); to h2.sql

* fix(resource  manager): update the h2 sql

* fix(resource  manager): solve to delete the tenant

* style(resource  manager): merge the style end delete the unuse s3 config

* fix(resource  manager): extend s3 to the storage of ds

UPDATE the rename resources when s3

* fix(resource  manager): extend s3 to the storage of ds

1.fix the code style of QuartzImpl

* fix(resource  manager): extend s3 to the storage of ds

1.impoort restore_type to CommonUtils

* fix(resource  manager): update the work thread

* fix(resource  manager): update  the baseTaskProcessor

* fix(resource  manager): upgrade dolphinscheduler-standalone-server.xml

* fix(resource  manager): add  user Info to dolphinscheduler_h2.sql

* fix(resource  manager): merge  the resourceType to NONE

* style(upgrade the log level to info):

* fix(resource  manager): sysnc the h2.sql

* fix(resource  manager): update the merge the user tenant

* fix(resource  manager): merge the resourcesServiceImpl

* fix(resource  manager):

when the storage is s3 ,that the directory can't be renamed

* fix(resource  manager): in s3 ,the directory cannot be renamed

* fix(resource  manager): delete the deleteRenameDirectory in E2E

* fix(resource  manager): check the style and  recoverd the test

* fix(resource  manager): delete the log.print(LoginUser)
2022-03-25 13:28:13 +08:00
kezhenxu94
7cf79da5d9
Reduce the size of distribution tar (#9019) 2022-03-20 16:42:06 +08:00
Jiajie Zhong
fd5e79bd80
[python] Separate py from main dist and auto add gpg sign (#8999)
* Add new profile `python` to separate build python
  package from `release`. Now someone who does not interested
  in python api could build package without python environment
* Add auto gpg asc sign when run `mvn instal -Ppython`
* Add property to skip gpg sign `python.sign.skip`
* Set `python.sign.skip=false` sign python dist by default

Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>
2022-03-19 22:07:24 +08:00
songjianet
f726730d93
[Docs][UI-Next] Add date-fns-tz license. (#8907) 2022-03-15 18:26:49 +08:00
kezhenxu94
d3cbf8eeaf
[UI-NEXT] Fix wrong packaging locations (#8851) 2022-03-14 15:26:15 +08:00
Jiajie Zhong
62e12c73dc
[python] Add LICENSE and NOTICE to python dist package (#8768)
* Add LICENSE and NOTICE to Python API
* Add new license dir `python-api-license` to keep all
  Python API's LICENSE files
* Change LICENSE in distribute package
* Add content about should add LICENSE when new dependence
  adding
2022-03-09 19:50:43 +08:00
songjianet
c661827752
[Docs][UI-Next] Add js-cookie license. (#8762) 2022-03-09 12:15:16 +08:00
songjianet
d3a8fa1d98
[Docs][UI Next][V1.0.0-Alpha] Add license file. (#8703)
* [Docs][UI-Next] Add license file.

* [Docs][UI-Next] Add license file.

* [Docs][UI-Next] Add license file.

* [Docs][UI-Next] Add license file.
2022-03-08 14:21:51 +08:00
kezhenxu94
d2a360480a
[CI][UI NEXT] Package new UI into release tar and remove old UI from it (#8670)
* [CI][UI NEXT] Package new UI into release tar and remove old UI from it

* Update e2e.yml
2022-03-07 11:20:24 +08:00
BaoLiang
44bb654c5e
Revert "upgrade mybatis-plus to 3.4.3 (#8515)" (#8659)
This reverts commit ca6d148a1b.
2022-03-02 16:10:39 +08:00
ronyang1985
20ee9a37fe
[Feature-8485][task] Add EMR task plugin to support submit task to AWS EMR (#8503)
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
2022-03-01 11:43:37 +08:00
songjianet
8ed2eeb86c
[Docs][UI Next] Add package license. (#8545)
* [Docs][UI-Next] Add package license.

* [Docs][UI Next] Add package license.
2022-02-26 17:10:21 +08:00