Commit Graph

7213 Commits

Author SHA1 Message Date
Eric Gao
66958b91d7
[Feature][Resource Center] Add support for Alibaba Cloud OSS as storage of resource center (#11708)
* Add support for Alibaba Cloud OSS as storage of resource center

* Add dependency licenses

* Refactor code, add unit tests and related docs
2022-09-20 22:44:06 +08:00
Wenjun Ruan
26dad5e2a8
Fix update TaskDefinition error (#12060) 2022-09-20 22:23:16 +08:00
jackfanwan
0bf639b851
[improvement]Add Set cluster name (#12058)
* Add Set cluster name

* add unit test

Co-authored-by: fanwanlong <fanwanlong@kezaihui.com>
2022-09-20 21:26:05 +08:00
insist777
955e8ebe0f
[Improve] Fix support test tasks UT (#12062) 2022-09-20 15:46:29 +08:00
金鑫
649015bfff
[Bug][ApiServer] When modifying the task definition, inserting the task definition log table will report the bug of primary key conflict #12037
Co-authored-by: 金鑫 <jinyanhui@huansi.net>
2022-09-20 14:42:21 +08:00
Eric Gao
e1b55db72a
[Improvement][Test] Give an example on replacing Powermock with Mockito (#11588)
* Give an example on replacing powermock with mockito

* Remove redundant comments

* Refactoring related UTs for better readability.
2022-09-20 09:48:02 +08:00
rickchengx
b3bbf69b2e
[Improvement-11291][sql] Improvement the log output while executing multiple sql statements (#11292) 2022-09-20 09:30:27 +08:00
songjianet
8cddb10773
[Feat][UI] Update package. (#12046)
* [Feat][UI] Update package.

* [Feat][UI] Update package.
2022-09-19 20:58:41 +08:00
Kerwin
89d55bc9b7
Fix link errors in release documentation. (#12049) 2022-09-19 20:01:23 +08:00
juzimao
c8884e4f2f
and task instance date range search (#11963) 2022-09-19 17:50:24 +08:00
JieguangZhou
ad683c3c42
[feat][python] Support MLflow task in python api (#11962) 2022-09-18 16:28:18 +08:00
limfff
c24ad9adb6
[Fix] Fixed an error importing project Maven (#12026) 2022-09-18 10:11:20 +08:00
JieguangZhou
bbc228e93f
[Feature][PyDolphinScheduler] Support DVC task in pyds #11922 (#11941)
* add dvc task in pyds

* add BaseDVC class
2022-09-17 16:55:51 +08:00
JieguangZhou
e50a32d9cc
[feat][python] Support OpenMLDB task in python api (#11944)
Using the whole word zookeeper instead of short cut zk
Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com>
2022-09-17 15:24:17 +08:00
insist777
5b384f3fab
[Feature-11473][Task]Support test task (#11670)
* [improve] support test tasks

* [improve] support test tasks

* [improve] support test tasks

* [improve] support test tasks

* [improve] support test tasks

* Update TaskExecuteThread.java

* try solve e2e q

* try solve e2e q

* try solve e2e q

* try solve e2e q

* try solve e2e q

* try solve e2e q

* try solve e2e q

* try solve e2e q

* try solve e2e q

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* Update DataSource.java

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* Update messages_zh_CN.properties

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* Update messages.properties

* Update messages_en_US.properties

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks

* [Feature] support test tasks
2022-09-17 13:40:56 +08:00
Eric Gao
a44817fc46
[Doc][Style] Fix doc format once for all (#12006) 2022-09-17 11:33:32 +08:00
sgw
1a7c6eabf3
[Improvement-11773][api] Optimize the log printing of the api module according… (#11782)
* [DS-11773][api] Optimize the log printing of the api module according to the log specification doc.
- Resolve conflict.

* [DS-11773][api] Optimize the log printing of the api module according to the log specification doc.
- fix ProcessDefinitionServiceImpl.java dependency

* [DS-11773][api] Optimize the log printing of the api module according to the log specification doc.
- Accept proposed change in ResourcePermissionCheckServiceImpl.java

* [DS-11773][api] Optimize the log printing of the api module according to the log specification doc.
- Update the UT case in TenantServiceTest.java, UsersServiceTest.java, TaskGroupServiceTest.java.
- Update log printing in TaskGroupServiceImpl.java.
- Fix NPE of log parameter in ExecutorServiceImpl.java when call python api.

* [DS-11773][api] Optimize the log printing of the api module according to the log specification doc.
- Fix NPE of log in WorkerGroupServiceImpl.java.
- Fix log parameter missing bug in ResourcesServiceImpl.java.

* [DS-11773][api] Optimize the log printing of the api module according to the log specification doc.
- Fix bugs and handle some vulnerability codes.

* [DS-11773][api] Optimize the log printing of the api module according to the log specification doc.
- Fix bugs and handle some vulnerability codes.

* [DS-11773][api] Optimize the log printing of the api module according to the log specification doc.
- Fix e2e User test.

* [DS-11773][api] Optimize the log printing of the api module according to the log specification doc.
- Fix codeQL check & SonarCloud Code Analysis.

* [DS-11773][api] Optimize the log printing of the api module according to the log specification doc.
- Resolve import conflict.

* [DS-11773][api] Optimize the log printing of the api module according to the log specification doc.
- Fix codeQL check & SonarCloud Code Analysis.
2022-09-17 11:28:04 +08:00
JieguangZhou
e05f14ba23
[feat][python] Add sagemaker yaml example (#11925) 2022-09-17 10:34:06 +08:00
JieguangZhou
5202e5cfc6
[feat][python] Support Pytorch task in python api (#11975)
Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com>
2022-09-16 20:24:38 +08:00
Wenjun Ruan
864a90820d
Fix insertOrUpdate plugin may failed due to concurrent operation (#11471) 2022-09-16 15:39:04 +08:00
Wenjun Ruan
d833a28b2e
Fix cannot save processDefinition (#11931) 2022-09-16 13:54:18 +08:00
jackfanwan
71b4087421
Example Change the regular expression of the data source password (#11459) 2022-09-16 13:34:22 +08:00
kezhenxu94
5811b84fcc
Add validations of possible malicious keys (#11966) 2022-09-16 13:32:59 +08:00
Wenjun Ruan
4ad34483a5
Fix alert_send_status may throw duplicate key exception and add limit for query alert (#11953) 2022-09-16 13:26:33 +08:00
rickchengx
5b6bebf7d5
[fix][doc] Update the registry related configuration in values.yaml (#11444) 2022-09-16 10:38:27 +08:00
ZhaoGuodong
86291bb5e6
[Feature-10273][Datasource] Add dataSource for AWS Athena (#11157) 2022-09-16 09:59:20 +08:00
caishunfeng
925e2fa551
[Bug][Workflow Definition] fix ordinary users can not create depend task (#11961) 2022-09-16 09:12:33 +08:00
kezhenxu94
b73a9511a7
Remove sensitive configs from dolphinscheduler_env.sh (#11917) 2022-09-15 19:03:47 +08:00
itzhang89
511149f644
[fix-10938]: use dot to replace source and make the default env work in shell (#11625)
* [fix-10938]: use the dot to replace source and make the default env work in the sh
Co-authored-by: Kerwin <37063904+zhuangchong@users.noreply.github.com>
2022-09-15 15:05:09 +08:00
Wenjun Ruan
9813ded981
Add debug logs for properties read from common.properties (#11910) 2022-09-15 10:53:19 +08:00
Eric Gao
a88cb02ee8
[improve][Doc] Put default task parameters in a new file (#11776)
* Put default task parameters in a new file
* Add links to task plugin default parameters appendix docs
* Update Chinese docs
2022-09-15 10:32:10 +08:00
Stalary
bd0fa79e37
[Bug][Dependent]: Id also clone due to duplicate when use dependent mode. (#11929) 2022-09-15 10:00:38 +08:00
Yann Ann
e938fdbe96
[Bug] [Master] WorkflowExecuteRunnable will face a infinite loop #11838 (#11864)
* [Bug] [Master] WorkflowExecuteRunnable will face a infinite loop #11838
2022-09-15 09:46:30 +08:00
xuhhui
dec6197a67
[Feature][Api] Refactor&Add api.controller.QueueV2Controller (#11389)
* [Feature][Api] Refactor&Add api.controller.QueueV2Controller

* update review
2022-09-15 09:37:12 +08:00
caishunfeng
6868876a29
fix workflow keep running when task fail (#11930) 2022-09-15 09:06:10 +08:00
wendongdi
e70c501f9a
[Fix][db] fix init&upgrade mysql-meta-schema bugs (#11887) 2022-09-14 18:03:07 +08:00
juzimao
b1068e27ba
Optimize date range condition (#11911) 2022-09-14 16:55:10 +08:00
Wenjun Ruan
d3a77c68e6
Make LogServiceClient Singleton (#11777) 2022-09-14 16:18:08 +08:00
caishunfeng
0deafba12c
fix procedure task param pass (#11919) 2022-09-14 13:44:01 +08:00
Daniel Y
009cb68011
[Improvement-11834] Upgrade docker base image to support python3.9 (#11835)
* use openjdk:8-jre-slim-bullseye instead of buster to support python3.9

* use openjdk:8-jre-slim-bullseye instead of buster across all components
2022-09-14 09:29:45 +08:00
JieguangZhou
892d867270
[Doc][Task Plugin] Fix MLflow task plugin doc (#11905)
* update mlflow doc

* fix punctuations
2022-09-13 16:20:03 +08:00
He Zhao
ca22c083b3
[Fix] [Quartz] cron did not work as expected (#11851)
* [Fix] [Quartz] cron did not work as expected

* [Fix] [Quartz] make the value of batchTriggerAcquisitionMaxCount to 1(default value)

Co-authored-by: hezhao2 <hezhao2@cisco.com>
2022-09-13 16:05:40 +08:00
rickchengx
120d8f696d
[Fix-11877][UI] Fix the problem that the environment cannot be deleted on the UI of some tasks (#11878) 2022-09-13 13:54:55 +08:00
Wenjun Ruan
8a47785f42
Fix listing processDefinition the schedulerReleaseState will never be null (#11888) 2022-09-13 10:07:12 +08:00
fuchanghai
070b8c2546
[fix-#11830]fix doc about sub-process's child node describe (#11837)
* [fix-#11830]fix doc

* Update docs/docs/en/guide/task/sub-process.md
2022-09-13 10:05:59 +08:00
limaiwang
25b78a8003
[Bug] [spark-sql] In spark-sql, select both SPARK1 and SPARK2 versions and execute ${SPARK_HOME2}/bin/spark-sql (#11721)
select different versions of spark-sql to execute different versions of spark-sql
2022-09-12 10:35:27 +08:00
kezhenxu94
147e0ae8d7
Remove duplicate Spring version management (#11899) 2022-09-11 20:50:07 +08:00
shiyu li
cc5f710147
[helm] fix worker template persistentVolumeClaim link (#11248) 2022-09-11 20:46:55 +08:00
W.XING
1164cb776d
Add chunjun in dolphinscheduler-task-all/pom.xml (#11375) 2022-09-11 19:14:05 +08:00
ouyangyewei
db46e4786a
Add default constructor (#11858) 2022-09-11 19:12:03 +08:00