Commit Graph

933 Commits

Author SHA1 Message Date
Wenjun Ruan
673687d540
[DSIP-73] Add dolphinscheduler-task-executor module to unify the task execution logic (#16790)
Some checks are pending
E2E-K8S / E2E-K8S-Execute (push) Blocked by required conditions
E2E-K8S / E2E-K8S-Result (push) Blocked by required conditions
E2E / E2E-Path-Filter (push) Waiting to run
E2E / E2E-Build (push) Blocked by required conditions
E2E / ${{ matrix.case.name }} (map[class:org.apache.dolphinscheduler.e2e.cases.ClickhouseDataSourceE2ETest name:ClickhouseDataSourceE2ETest]) (push) Blocked by required conditions
E2E / ${{ matrix.case.name }} (map[class:org.apache.dolphinscheduler.e2e.cases.ClusterE2ETest name:ClusterE2ETest]) (push) Blocked by required conditions
E2E / ${{ matrix.case.name }} (map[class:org.apache.dolphinscheduler.e2e.cases.EnvironmentE2ETest name:EnvironmentE2ETest]) (push) Blocked by required conditions
E2E / ${{ matrix.case.name }} (map[class:org.apache.dolphinscheduler.e2e.cases.FileManageE2ETest name:FileManageE2ETest]) (push) Blocked by required conditions
E2E / ${{ matrix.case.name }} (map[class:org.apache.dolphinscheduler.e2e.cases.HiveDataSourceE2ETest name:HiveDataSourceE2ETest]) (push) Blocked by required conditions
E2E / ${{ matrix.case.name }} (map[class:org.apache.dolphinscheduler.e2e.cases.MysqlDataSourceE2ETest name:MysqlDataSourceE2ETest]) (push) Blocked by required conditions
E2E / ${{ matrix.case.name }} (map[class:org.apache.dolphinscheduler.e2e.cases.PostgresDataSourceE2ETest name:PostgresDataSourceE2ETest]) (push) Blocked by required conditions
E2E / ${{ matrix.case.name }} (map[class:org.apache.dolphinscheduler.e2e.cases.ProjectE2ETest name:ProjectE2ETest]) (push) Blocked by required conditions
E2E / ${{ matrix.case.name }} (map[class:org.apache.dolphinscheduler.e2e.cases.QueueE2ETest name:QueueE2ETest]) (push) Blocked by required conditions
E2E / ${{ matrix.case.name }} (map[class:org.apache.dolphinscheduler.e2e.cases.SqlServerDataSourceE2ETest name:SqlServerDataSourceE2ETest]) (push) Blocked by required conditions
E2E / ${{ matrix.case.name }} (map[class:org.apache.dolphinscheduler.e2e.cases.TenantE2ETest name:TenantE2ETest]) (push) Blocked by required conditions
E2E / ${{ matrix.case.name }} (map[class:org.apache.dolphinscheduler.e2e.cases.TokenE2ETest name:TokenE2ETest]) (push) Blocked by required conditions
E2E / ${{ matrix.case.name }} (map[class:org.apache.dolphinscheduler.e2e.cases.UserE2ETest name:UserE2ETest]) (push) Blocked by required conditions
E2E / ${{ matrix.case.name }} (map[class:org.apache.dolphinscheduler.e2e.cases.WorkerGroupE2ETest name:WorkerGroupE2ETest]) (push) Blocked by required conditions
E2E / ${{ matrix.case.name }} (map[class:org.apache.dolphinscheduler.e2e.cases.WorkflowE2ETest name:WorkflowE2ETest]) (push) Blocked by required conditions
E2E / ${{ matrix.case.name }} (map[class:org.apache.dolphinscheduler.e2e.cases.WorkflowHttpTaskE2ETest name:WorkflowHttpTaskE2ETest]) (push) Blocked by required conditions
E2E / ${{ matrix.case.name }} (map[class:org.apache.dolphinscheduler.e2e.cases.WorkflowJavaTaskE2ETest name:WorkflowJavaTaskE2ETest]) (push) Blocked by required conditions
E2E / ${{ matrix.case.name }} (map[class:org.apache.dolphinscheduler.e2e.cases.tasks.PythonTaskE2ETest name:PythonTaskE2ETest]) (push) Blocked by required conditions
E2E / ${{ matrix.case.name }} (map[class:org.apache.dolphinscheduler.e2e.cases.tasks.ShellTaskE2ETest name:ShellTaskE2ETest]) (push) Blocked by required conditions
E2E / E2E (push) Blocked by required conditions
publish-docker / build (push) Waiting to run
publish-helm-chart / build (push) Waiting to run
Test / Unit-Test-Path-Filter (push) Waiting to run
Test / Unit-Test (11) (push) Blocked by required conditions
Test / Unit-Test (8) (push) Blocked by required conditions
Test / Unit Test (push) Blocked by required conditions
2024-11-29 15:25:54 +08:00
Wenjun Ruan
20e9ffac66
[Improvement-16835][Worker] Clear empty directory under task working path after task instance finish (#16836) 2024-11-26 10:19:21 +08:00
xiangzihao
bec678a8b9
[DSIP-78][Data Quality] Remove data quality module (#16794) 2024-11-20 09:20:41 +08:00
xiangzihao
f3b5d1f57d
[Feature-16801][UI] Add copy path of resource file in task definition (#16803) 2024-11-18 17:12:18 +08:00
Mighten Dai
9024dcad3e
[Feature-16396][storage-plugin] Add Tencent Cloud COS Storage Plugin (#16565) 2024-09-27 17:03:20 +08:00
xiangzihao
d67ea805f9
[DSIP-38] Add plugin directory in binary package (#16613) 2024-09-23 09:59:34 +08:00
Wenjun Ruan
62d33a799a
[DSIP-75] Support auto add license header by spotless (#16643) 2024-09-22 16:27:29 +08:00
Wenjun Ruan
3c944bccee
[Fix-16338][Test] Fix Master integration test case has been skipped in ci (#16637) 2024-09-21 15:31:14 +08:00
xiangzihao
fee4955724
[DSIP-68] Unify the table structure and table fields in backend and frontend ui sections (#16544) 2024-09-04 21:22:34 +08:00
xiangzihao
4c195063f9
[Improvement-16516] Uniformly name process in backend code as workflow (#16524) 2024-08-29 10:45:22 +08:00
Wenjun Ruan
20a0e50cce
[DSIP-67] Use command to trigger workflow instance rather generate workflow instance (#16523) 2024-08-28 19:27:40 +08:00
Wenjun Ruan
9448806af0
[DSIP-61][Master] Refactor thread pool and state event orchestration in master (#16327) 2024-08-26 15:19:39 +08:00
xiangzihao
a6504412cf
[DSIP-60][Http task plugin] Refactor http task plugin (#16413) 2024-08-06 23:39:30 +08:00
xiangzihao
d9244ca22f
[DSIP-57][Audit] Revert PR 14981 code (#16370) 2024-07-26 14:54:44 +08:00
zuo
8d1680d2f0
[Fix-16279] Support Java 8 date/time type java.time.OffsetDateTime (#16304)
* [Fix-16279] Support Java 8 date/time type `java.time.OffsetDateTime`
2024-07-15 21:14:51 +08:00
Wenjun Ruan
2db80981ca
[DSIP-56] Refactor JDBC registry support session timeout and data change event (#16287)
* Support sessionTimeout in jdbc registry
* Add registry table to initialize sql
2024-07-12 14:12:28 +08:00
xiangzihao
761323f3e1
[DSIP-53] Suggest add process lineage parsing for dependent task (#16288)
* add dsip 53
2024-07-11 21:25:06 +08:00
Wenjun Ruan
8f6df8d840
[DSIP-54][Master] Use ClusterManager to manage the cluster in master (#16263) 2024-07-05 11:55:00 +08:00
xiangzihao
225e969d2d
[DSIP-51][Task] Remove unused pigeon task plugin (#16219) 2024-06-28 09:51:02 +08:00
xiangzihao
e8566818b0
[DSIP-50][UDF Manage] Remove unused udf manage function (#16212)
* Remove unused udf manage function
2024-06-26 21:08:48 +08:00
Wenjun Ruan
ec6a3e0727
[DSIP-46] Unify the logic task params data structure between front-end and back-end (#16193) 2024-06-25 13:32:13 +08:00
Wenjun Ruan
b34fe46044
Fix task cannot use workflow's environment (#16199) 2024-06-24 20:27:25 +08:00
Wenjun Ruan
edeb1b2458
[DSIP-45] Polish the Storage SPI (#16141) 2024-06-17 15:24:56 +08:00
Wenjun Ruan
7ae9ca5a33
Revert "[Improvement] Abnormal characters check (#15824)" (#16102)
This reverts commit e5e7749251.
2024-05-31 19:45:12 +08:00
Wenjun Ruan
af55f94cc0
Fix sudo enable is false will still check tenant exist (#16071)
(cherry picked from commit c5baa75ef93243c6bf88194eb3ad94a48173a003)
2024-05-27 18:34:14 +08:00
Wenjun Ruan
f754611509
[DSIP-42] Add dolphinscheduler-aws-authentication module (#16043) 2024-05-24 14:15:03 +08:00
Eric Gao
dc306bfa1d
[Improvement] Use safe constructor with snake yaml (#15758) 2024-05-14 18:04:04 +08:00
Wenjun Ruan
7c8fa9b48c
Add IntegretionTest for registry module (#15981) 2024-05-13 16:35:37 +08:00
Wenjun Ruan
8d336def61
[DSIP-35][Alert] Refactor the alert thread model (#15932) 2024-05-09 12:23:01 +08:00
JohnHuang
d218b021e7
[DSIP-25][Remote Logging] Split remote logging configuration (#15826)
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-05-06 11:33:16 +08:00
Wenjun Ruan
446f6ba72b
Fix auto create tennat concurrently will cause the task failed (#15909) 2024-04-25 18:14:48 +08:00
旺阳
59f060e278
[Improvement] Fix alert code smell 2024-04-23 15:10:08 +08:00
Wenjun Ruan
285c5a8eb5
[DSIP-28] Donnot scan whole bean under classpath (#15874) 2024-04-19 18:12:40 +08:00
旺阳
d306f1d04b
Refactor record audit log logic (#15881) 2024-04-19 15:20:31 +08:00
XinXing
3fe9fd45b1
[Fix-15706] Seatunnel improvement (#15852)
* fix_seatunnel_15706

* CodeFormat

* Change to use JSONUtils

* Constants moved to constant code

* Delete empty lines

* Delete empty lines
2024-04-18 10:36:18 +08:00
Wenjun Ruan
27d0563fe4
Bind processId to constructor CodeGenerator (#15848) 2024-04-16 10:58:43 +08:00
旺阳
226345512e
[DSIP-26][Audit log] Audit log improvement design (#15554) 2024-04-15 13:40:35 +08:00
BaiJv
e5e7749251
[Improvement] Abnormal characters check (#15824)
* abnormal characters check

* add test case

* remove error log

* fix code style

* fix import
2024-04-12 10:06:32 +08:00
Wenjun Ruan
66df5d4b90
Split cpuUsage to systemCpuUsage and jvmCpuUsage (#15803) 2024-04-09 14:04:24 +08:00
John Huang
ac0189a636
[DSIP-24][RemoteLogging]Support AbsRemoteLogHandler (#15769) 2024-04-01 09:50:49 +08:00
John Huang
d39bdcb165
[RemoteLogging] Move init into loghandler (#15780)
Co-authored-by: 旺阳 <wang@lqwang.net>
2024-03-31 21:47:26 +08:00
caishunfeng
99a2461785
[Improvement] change js engine (#15747)
* change js engine

* add dependent jar
2024-03-22 11:31:47 +08:00
Wenjun Ruan
56a834b48b
[Bug] Disable delete a specific workflow version when exist workflow instance under this version which is not finish (#15730) 2024-03-21 17:07:03 +08:00
zer0e
82ffdbfb30
fix: create linux user with home dir (#15670) 2024-03-05 11:30:23 +08:00
Wenjun Ruan
4541e74e53
Fix create parent directory will cause FileAlreadyExistsException (#15576)
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-02-06 11:59:07 +08:00
Jay Chung
91d56f4860
fix: data quality may fail in docker mode (#15563) 2024-02-05 17:14:59 +08:00
Jay Chung
8efaa9fa1f
fix: data quality can not use (#15551)
* fix: data quality can not use

fix: #15468, #15249, #14858

---------

Co-authored-by: Rick Cheng <rickchengx@gmail.com>
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2024-02-05 09:54:06 +08:00
Wenjun Ruan
4d15932ede
Fix createFile with permission will not work (#15556) 2024-02-03 18:00:33 +08:00
Wenjun Ruan
64e1e67581
Use DefaultUncaughtExceptionHandler to log the uncached exception (#15496)
Co-authored-by: fuchanghai <changhaifu@apache.org>
2024-01-18 14:59:17 +08:00
Wenjun Ruan
0b873e6870
[HotFix] Fix createTaskInstanceWorkingDirectory failed if the old path exist (#15377) 2024-01-15 11:53:55 +08:00