test/DolphinScheduler
1
0
Fork 0
mirror of https://gitee.com/dolphinscheduler/DolphinScheduler.git synced 2024-12-12 12:25:16 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity
7,468 Commits 27 Branches 59 Tags 275 MiB
d803658abd
Commit Graph

1198 Commits

Author SHA1 Message Date
insist777
528f45acc5
[improve]Source skip check (#12900) 
* Source skip check
 2022-11-17 14:23:12 +08:00
Assert
2dbc79693e
[feature][task plugin] Add new task plugin for apache linkis (#12693) 2022-11-17 10:14:36 +08:00
Jay Chung
f20c9b3102
[chore][python] Change name from process definition to workflow (#12918) 
only change its name in python gateway server code, incluing

* Function name: all related to process definition
* Parameter name and comment related

ref: apache/dolphinscheduler-sdk-python#22
 2022-11-16 21:23:39 +08:00
Zzih
0186413a9f
[Feature][Api] Refactor org.apache.dolphinscheduler.api.controller.TaskInstanceController (#11747) 
* Feature ：Refactor org.apache.dolphinscheduler.api.controller.TaskInstanceController
 2022-11-16 18:32:42 +08:00
Jay Chung
6d8befa075
[fix] Add token as authentication for python gateway (#12893) 
separate from #6407. Authentication, add secret to ensure only trusted people could connect to gateway.

fix: #8255
 2022-11-14 18:43:08 +08:00
Tq
d324cce6f1
[Feature-12526][Api] add dynamic task function: shell task (#12840) 
* submit dynamic task code

* submit shell.json

* add dynamic-task-type-config.yaml to standalone server
 2022-11-14 13:52:44 +08:00
rickchengx
813c44b22b
[Improvement-12838][UT] Improve the ut of datasource (#12839) 2022-11-14 11:30:10 +08:00
Kerwin
e3cf72cbcd
[Fix-12832][API] Fix update worker group exception group name already exists. (#12874) 2022-11-14 11:20:51 +08:00
Jay Chung
87a88e3662
[feat] Support set execute type to pydolphinscheduler (#12871) 
Up to now, we can only submit workflow with parallel mode. this patch give users ability specific execute type to workflow
 2022-11-12 10:31:08 +08:00
JieguangZhou
229c554912
[feature][task] Add Kubeflow task plugin for MLOps scenario (#12843) 2022-11-11 16:08:38 +08:00
Jay Chung
1347a8f94c
[chore] Separate Python API into another repository (#12779) 
Currently, our Python API code is a module in apache/dolphinscheduler codebase,
each time users change Python API code, they need to run all requests CI check
for dolphinscheduler and Python API, But if the user does only change Python
code, it could be merged if Python API CI pass and do not dependent on others CI.

Besides, we release Python API as the same version of dolphinscheduler. It is
easy for user to match Python API version. But when Python API does not change
any code, but dolphinscheduler release a bugfix version, Python API has to
release the new version to match dolphinscheduler. This happened when we
released Python API 2.0.6 and 2.0.7. 2.0.6 and 2.0.7 is bugfix version, and
Python API does not change any code, so the PyPI package is the same.

Separate Python API also makes our code more sense, we will have more
distinguished code in dolphinscheduler and Python API new repository.
Have separate issue tracker and changelog for information to users.

ref PR in other repository: apache/dolphinscheduler-sdk-python#1

see more detail in mail thread: https://lists.apache.org/thread/4z7l5l54c4d81smjlk1n8nq380p9f0oo
 2022-11-09 21:20:15 +08:00
jackfanwan
641a2559ea
[fix-12687]when timeout_notify_strategy is null, modify timeout_notify_strategy (#12689) 
* According to timeoutFlag, assign timeoutNotifyStrategy value

Co-authored-by: fanwanlong <fanwanlong@kezaihui.com>
 2022-11-09 11:48:12 +08:00
houshitao
04c6b5e413
[Bug][Api] Added param description. (#12813) 
Co-authored-by: houshitao <shitaohou@163.com>
 2022-11-08 21:17:40 +08:00
jackfanwan
23ead4f0e8
[Bug] [dolphinscheduler-api] in the task define list, when edit task, should show pre node list (#12695) 
* backend : add pre node list

* resolve workflow error

Co-authored-by: fanwanlong <fanwanlong@kezaihui.com>
 2022-11-08 13:31:20 +08:00
Eric Gao
6d79414491
Add instructions on how to configure logging level for debugging purpose (#12699) 2022-11-07 10:30:56 +08:00
JiPeng Wang
606a5aeee5
[Bug] Resource default auth function disabled false. (#12467) 
* resource function disable check fix

Co-authored-by: Kerwin <37063904+zhuangchong@users.noreply.github.com>
 2022-11-06 21:15:27 +08:00
jackfanwan
f249f2b257
[fix-12675]edit workflow related task, workflow's task version change (#12692) 
* edit workflow related task, workflow's task version change

* Update dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/enums/Status.java

Co-authored-by: fanwanlong <fanwanlong@kezaihui.com>
Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
 2022-11-06 20:49:07 +08:00
Aaron Wang
04e23ed88f
Update Status.java (#12724) 2022-11-05 21:09:17 +08:00
jackfanwan
7d0e2cbbb9
when delete workflow, delete related task (#12681) 
* when delete workflow, delete related task

Co-authored-by: fanwanlong <fanwanlong@kezaihui.com>
 2022-11-04 14:40:15 +08:00
JieguangZhou
a47b4d0672
[Feature][Task] Transfer files between tasks (#12552) 
* add data transfer between tasks

* add delete DATA_TRANSFER API

* convert Result to DeleteDataTransferResponse

* add api UT

* fix final

* fix doc
 2022-11-03 16:33:39 +08:00
rickchengx
c69376e74c
[Improvement-12658][Permission] Improve the readability of setPerm() in grantProjectbyCode() (#12660) 2022-11-03 09:29:04 +08:00
rickchengx
44e0935f56
[Improvement-12650][Permission] Improve the check of resourcePermissionCheck() (#12652) 2022-11-03 09:12:56 +08:00
rickchengx
3ff328c961
[Improvement-12623][Permission]Remove the unused projectIds in operationPermissionCheck() (#12624) 2022-11-02 11:20:00 +08:00
calvin
d84f1ef269
[Improvement][UI] Support to view the process variables on the page of DAG. (#12609) 2022-10-30 20:31:40 +08:00
旺阳
fb9761f27a
Fix alert status spelling error (#12592) 
* fix alert status spelling error

* fix-ut
 2022-10-29 21:33:03 +08:00
kezhenxu94
065d5caccc
Only expose necessary actuator endpoints (#12571) 2022-10-28 07:40:32 +08:00
Yiming Guo
dc8d18cf87
[Feature-12040][api][ui] Add authorization management of read and write permissions for project center (#12048) 
[Feature-12040][api][ui] Add authorization management of read and write permissions for project center
 2022-10-27 18:01:51 +08:00
houshitao
4a13148bfc
[Bug] [API] Before deleting a worker group, check whether there is environment that reference the worker group. (#12534) 
* [Bug] [API] Before deleting a worker group, check whether there is environment that reference the worker group.

Co-authored-by: houshitao <shitaohou@163.com>
 2022-10-27 16:34:08 +08:00
Tq
4b4d0b92a3
[FIX][DAO-12277] change t_ds_fav_task column task_name to task_type and related code (#12276) 
* change t_ds_fav_task column task_name to task_type and related code

* change task_name to task_type

* change FavTask taskName to taskType, taskType to taskCategory

Co-authored-by: labbomb <739955946@qq.com>
 2022-10-27 09:36:26 +08:00
Yann Ann
ed209bdf82
[Refactor] Migrate all workergroup-related interface functions from ProcessServiceImpl (#12493) 
* Migrate all workergroup-related interface functions from ProcessServiceImpl
 2022-10-26 19:18:32 +08:00
houshitao
547aa437ab
[Bug] [API] The workflow definition and the tenant in the workflow instance are inconsistent. (#12533) 
* [Bug] [API] The workflow definition and the tenant in the workflow instance are inconsistent

* fix  NPE.

Co-authored-by: houshitao <shitaohou@163.com>
 2022-10-26 16:11:24 +08:00
HanayoZz
489e7fe4e2
[Feature-10495][Resource Center] Resource Center Refactor (#12076) 
* resource center refactor - S3 services connection

Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
 2022-10-26 13:53:44 +08:00
Yann Ann
04aa125ba2
[Refactor]migrate some task dao functions from ProcessServiceImpl to dao modules. (#12505) 
* migrate some task DAO to dao modules.

* Update dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/repository/impl/ProcessInstanceMapDaoImpl.java

Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
 2022-10-26 11:51:58 +08:00
Yann Ann
af9374c2c2
fill up missing i18 properties (#12532) 2022-10-26 11:39:26 +08:00
rickchengx
233bbdb09e
[Fix-12451][k8s] Read the kubeconfig from cluster conf (#12452) 2022-10-25 11:25:30 +08:00
fuchanghai
77e29b59b2
[Improvement] add springdoc-openapi-ui (#12379) 
Co-authored-by: fuchanghai <‘2875334588@qq.com’>
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
 2022-10-23 23:17:53 +08:00
caishunfeng
dedff70f90
[Improvement] Merge spi.utils into common.utils (#12502) 
* improve JSONUtil

* merge spi.utils into common.utils

* split constants && remove some copy utils method

* remove StringUtils
 2022-10-23 21:23:54 +08:00
Yann Ann
b936b882bb
[Refactor] Migrate all command-related interface functions from ProcessServiceImpl (#12474) 
* migrate all command-related interface functions to CommonService
 2022-10-22 12:39:25 +08:00
Yann Ann
1384d8f4fa
set @Transactional rollbackFor (#12445) 2022-10-22 11:33:17 +08:00
Wenjun Ruan
b5193ee4e5
Remove equals in User to fix UT (#12487) 2022-10-22 08:59:10 +08:00
Kerwin
1436ad65fc
[Bug-12410] [API]Fix the worker list result in workflow definition only has default (#12411) 
* Fix the worker list result in workflow definition only has default

* Fix tenant, k8s namespace similar permission issues.

* code clean in the k8s namespace mapper

* delete redundant sql method
 2022-10-21 13:54:28 +08:00
fuchanghai
f0a34cddec
[improvement-#12416] Use CollectionUtils.isEmpty judge the collection isEmpty to avoid NPE (#12449) 2022-10-20 14:26:56 +08:00
Eric Gao
d46e4659d9
[Improvement][Style] Fix remaining style errors and wildcard imports (#12412) 
* Fix remaining style errors and wildcard imports

* Upgrade spotless to the latest stable version, exclude .github folder from spotless check and fix dead link

* Fix spotless matching scope of markdown files
 2022-10-19 11:08:04 +08:00
Kerwin
fccbe5593a
When the workflow definition is copied, the operation user of the timed instance is changed to the current user. (#12400) 2022-10-18 08:56:19 +08:00
kezhenxu94
4ea7eec750
Use temurin Java docker images instead of deprecated ones (#12334) 2022-10-17 15:46:21 +08:00
Kerwin
55004bebe0
[Improvement-12391][API] Workflow definitions that contain logical task nodes support the copy function (#12392) 
* Workflow definitions that contain logical task nodes support the copy function.
 2022-10-17 10:28:58 +08:00
Eric Gao
2f37da0dbc
[Improvement][Test] Block the usage of powermock and move mockito dependencies from sub-modules to root pom (#12311) 
* move mockito dependencies from sub-modules to root pom

* Add check in CI to block the usage of powermock
 2022-10-14 12:06:51 +08:00
Eric Gao
224508850e
[Migrate][Test] Migrate all UT cases from jUnit 4 to jUnit 5 in alert and api module (#12337) 
* Migrate all UT cases from jUnit 4 to jUnit 5 in alert and api module

* Fix potential problems
 2022-10-13 14:40:57 +08:00
kezhenxu94
db1d319159
Reorganize some classes in common module, remove duplicate classes (#12321) 2022-10-13 10:23:27 +08:00
Yann Ann
17cd644506
[Fix-#11669][Workflow Instance Page] Fix the duration in Workflow Instance page. (#12264) 
* fix process duration issue

* use processInstance.getState().isFinished() replace new isFinish function

* add WorkflowUtils
 2022-10-13 09:50:31 +08:00