Commit Graph

881 Commits

Author SHA1 Message Date
xiangzihao
d6690fa89c
[E2E] add udf manage test (#8203) 2022-01-27 16:17:32 +08:00
zixi0825
4ba6d667fc
[Feature][DataQuality] Add Data quality Module #4283 (#6718)
* add data quality module

* add license

* add package configuration in dist pom

* fix license and jar import bug

* replace apache/skywalking-eyes@9bd5feb SHA

* refacotr jbdc-connector and writer

* modify parameter name in HiveConnector

* fix checkstyle error

* fix checkstyle error in dolphinschesuler-dist

* fix checkstyle error in dolphinschesuler-dist

* fix checkstyle error in dolphinschesuler-dist

* fix duplicate code bug

* fix code style bug

* fix code smells

* add dq relevant enums and parameter

* replace apache/skywalking-eyes@9bd5feb SHA

* fix Constants bug

* remove the unused class

* add unit test

* fix code style error

* add unit test

* refactor data quality common entity

* fix code style error

* add unit test

* close e2e test

* fix code smell bug

* modify dataquality enum value to 14 in TaskType

* add data qualtiy task

* update

* add getDatasourceOptions interface

* fix checkstyle

* close e2e test

* add data quality task ui

* update skywalking-eyes SHA

* fix style

* fix eslint error

* fix eslint error

* test e2e

* add unit test and alter dataquality task result

* fix checkstyle

* fix process service test error

* add unit test and fix code smells

* fix checkstyle error

* fix unit test error

* fix checkstyle error

* change execute sql type name

* revert ui pom.xml

* fix data quality task error

* fix checkstyle error

* fix dq task src_connector_type ui select bug

* fix spark rw postgresql bug

* change mysql driver scope

* fix form-create json bug

* fix code smell

* fix DolphinException Bug

* fix ui validate rule and Alert title

* fix target connection param bug

* fix threshold validate change

* add rule input entry index

* change statistic_comparison_check logic

* remove check type change

* add DateExpressionReplaceUtil

* fix null point expetion

* fix null point expetion

* fix test error

* add more sql driver

* fix test error and remove DateExprReplaceUtil

* add get datasource tables and columns

* add get datasource tables and columns

* remove hive-jdbc in pom.xml

* fix code smells

* update sql

* change the pom.xml

* optimize multi_table_accuracy ui

* fix v-show error

* fix code smells

* update sql

* [Feature][DataQuality] Add data quality task ui (#5054)

* add data quality task ui

* update skywalking-eyes SHA

* fix style

* fix eslint error

* fix eslint error

* test e2e

* fix dq task src_connector_type ui select bug

* fix threshold validate change

* remove check type change

* add get datasource tables and columns

* optimize multi_table_accuracy ui

* fix v-show error

* fix code smells

Co-authored-by: sunchaohe <sunzhaohe@linklogis.com>

* [Feature][DataQuality] Add data quality module  (#4830)

* add data quality module

* add license

* add package configuration in dist pom

* fix license and jar import bug

* replace apache/skywalking-eyes@9bd5feb SHA

* refacotr jbdc-connector and writer

* modify parameter name in HiveConnector

* fix checkstyle error

* fix checkstyle error in dolphinschesuler-dist

* fix checkstyle error in dolphinschesuler-dist

* fix checkstyle error in dolphinschesuler-dist

* fix duplicate code bug

* fix code style bug

* fix code smells

* update

* close e2e test

* fix spark rw postgresql bug

* change mysql driver scope

* add more sql driver

* remove hive-jdbc in pom.xml

* change the pom.xml

Co-authored-by: sunchaohe <sunzhaohe@linklogis.com>

* [Feature][DataQuality] Add data quality task backend (#4883)

* add dq relevant enums and parameter

* replace apache/skywalking-eyes@9bd5feb SHA


Co-authored-by: sunchaohe <sunzhaohe@linklogis.com>

* refactor data_quality_module

* add header license

* data quality module refactor

* fix unit test error

* fix checkstyle error

* fix unit test error

* fix checkstyle error

* fix unit test error

* fix code smell

* fix check style

* fix unit test error

* task statistics value add unique code

* fix unit test error

* fix checkstyle error

* fix checkstyle

* fix security hotspot

* fix unit test error

* fix security hotspot

* fix check

* add data quality task error handling

* fix unit test error

* add unit test

* add unit test

* optimize data quality result alert

* fix unit test

* fix sql script error

* fix bug

* update sql script

* fix checkstyle

* add license

* fix checkstyle

* fix checkstyle

* fix unit test

* add jacoco dependencies

* fix unit test

* fix unit test

* add jacoco dependencies

* add unit test

* add unit test

* add license

* fix checkstyle

* fix pom

* fix checkstyle

* fix checkstyle

* merge dev

* fix ui error

* fix pom error

* fix pom error

* fix test error

* fix test error

* mssql-jdbc exclude azure-keyvault

* fix test error

* merge dev and add unit test

* add notes

* rollback the CollectionUtils

* fix

* update sql

* fix

* fix

* fix query rule page error

* change dq.jar path

* fix sql error

* fix ui error

* fix(dq): jar path&task enum description

* add notes on DataQualityApplication

* fix dq result jump error

* fix(ui): page condition

* feat(ui): add show error output path

* change version

* remove all chinese word in sql

* merge

Co-authored-by: sunchaohe <sunzhaohe@linklogis.com>
2022-01-27 15:55:45 +08:00
xiangzihao
7b22c16a85
[E2E]Fix file manage e2e error (#8214) 2022-01-27 14:59:22 +08:00
yc322
fb9f4a1339
[Feature-5822][api] Add audit log (#6322)
* rebase

* update audit log type name

* fix the quoted property error

* fix bug

* update audit global switch default value

* fix wrong ddl comment

* fix wrong code style

* update audit log search sql

* add licenses

* add unit test

* fix wrong code style

* fix bugs

* fix wrong code style

* add test case

* add test case

* add license

* Add unit test in pom.xml

* resolve code smell problem

* mysql and postgresql ddl update

* update audit log path and audit page query

* update audit configuration

* update audit log schema, add resource type and resource id

* update audit query and list page

* resolve audit log front page bugs and log query error

* update audit configuration

* resolve wrong configuration

* update AuditLogControllerTest

* update application.yaml

* resolve code smell

* update standalone configuration

* resolve bugs

Co-authored-by: yc322 <1393242780@qq.com>
2022-01-26 09:53:18 +08:00
xiangzihao
30b8b4a93e
[E2E][CI] Add renaming file management E2E tests (#8160) 2022-01-26 09:45:21 +08:00
Kerwin
103ccb01da
change version to 2.0.4-SNAPSHOT (#8188) 2022-01-25 15:17:17 +08:00
Kerwin
1fd748f376
[Feature-8138][Task] add at@ msg in the dingtalk task plugin (#8139) 2022-01-21 17:09:39 +08:00
Devosend
a77b0ef93b
[Fix-8098] [UI] Fix resource manage bug (#8107) 2022-01-20 10:19:02 +08:00
xiangzihao
e7f480bdca
[E2E][CI] Add DataSource E2E test (#8085) 2022-01-19 17:29:02 +08:00
songjianet
c201c156f3
[Feature][Dag] Number of failed retries and failed retry interval use input component. (#8104) 2022-01-18 19:09:56 +08:00
songjianet
9faeef5877
[Feature][Workflow Instance] Add Workflow instalce batch delete reload table list. (#8082) 2022-01-17 15:10:00 +08:00
Jiajie Zhong
a061cf136c
[e2e] Add new test of security environment (#8048)
* [e2e] Add new test of security environment

ref: #7281

* Fix frontend bug

* Fix order of UT

* Fix delete and edit bottom class

* Try to fix

* Try to fix
2022-01-15 09:13:04 +08:00
songjianet
0c353d69e2
[Fix][Dag] Timeout alert. (#8049) 2022-01-14 18:34:21 +08:00
xiangzihao
e45e78d803
[Fix-7513] Fix identical numbers and add positive integer judgment (#8032)
* fix bug_7513

* fix bug_7513

* fix bug_7513

* fix bug_7513

* fix bug_7513

* fix bug_7513

* fix bug_7513

* fix bug_7513

* fix bug_7513

* fix bug_7513

* fix bug_7513

* fix bug_7513

* fix bug_7513

* fix bug_7513

* fix bug_7513

* test

* test

* test

* test

* test

* chinese change
2022-01-14 15:45:53 +08:00
calvin
677482795c
fix a little issue (#8039) 2022-01-14 14:33:57 +08:00
Kerwin
b9b0722b4b
Fix query workflow history version information error (#7988) 2022-01-13 14:04:36 +08:00
Kerwin
cb8c875fa8
[Feature-7804][Task] DingTalk alert plugin adds signature. (#7805)
* DingTalk alarm component adds signature.

* fix code style.

* fix code style.

* add dingtalk document link

* fix DingTalkAlertChannelFactoryTest

* update NAME_DING_TALK_SECRET
2022-01-12 15:24:25 +08:00
Kerwin
64efc65edf
Create or update users to add tenants cannot be empty verification. (#7594) 2022-01-12 14:38:41 +08:00
xiangzihao
5be8e0531c
[E2E] [Resource] add file manage case (#7906) 2022-01-12 14:29:03 +08:00
calvin
8cac0d8f94
[Fix-7288] Not allow to delete the default alarm group. (#7932) 2022-01-12 14:12:30 +08:00
wind
3edc58cbdd
[E2E] Add queue e2e test (#7941) 2022-01-12 10:31:17 +08:00
calvin
24eb40c2b2
[Fix-7857] The user won't be allowed to disable their own account. (#7891) 2022-01-08 18:01:22 +08:00
janeHe13
0b38d0e826
[E2E]add worker group manage e2e case (#7879)
* add worker group manage e2e case

* add worker group manage e2e case

* Update dolphinscheduler-e2e/dolphinscheduler-e2e-case/src/test/java/org/apache/dolphinscheduler/e2e/cases/WorkerGroupE2ETest.java

Co-authored-by: janeHe13 <438044805@qq.com>
Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com>
2022-01-07 21:24:03 +08:00
Jiajie Zhong
9b6b381b93
[deploy] Split Ambari plugin to independent repo (#7880)
fix: #7036
2022-01-07 21:10:48 +08:00
Kerwin
5172862a0a
Add seatunnel task plugin (#7131) 2022-01-07 20:51:11 +08:00
wangyizhi
984665a5ef
[Feature-7018][UI] Add list edit view from task definition (#7852)
* Performance optimization of DEPENDENT task

* fix eslint

* [Feature] Improve task definition list

* feat: Task definition list

* change update interface
2022-01-07 10:38:56 +08:00
xiangzihao
70ab2b886f
fix bug_7499 (#7739)
Co-authored-by: SbloodyS <sbloodys@qq.com>
2022-01-04 10:18:57 +08:00
xiangzihao
d9496efc38
[Feature-7777][UI] process definition page add create username (#7778)
* feature_7777
2022-01-04 10:16:24 +08:00
xiangzihao
0dfbbc8983
[Feature-5951][Improvement][UI] When creating an alarm instance, the alarm instance name input box prompts optimization (#7770) 2022-01-04 09:13:20 +08:00
calvin
4203304b08
[Fix-7493] Support searching for task node in the DAG. (#7766)
* add a select to support fuzzy search
2022-01-01 12:46:40 +08:00
xiangzihao
c58dbefaa5
[Fix-7459][Sqoop] running error in parallel (#7752)
* fix bug_7459

* fix bug_7459

* fix bug_7459

* fix bug_7459

* fix bug_7459

Co-authored-by: SbloodyS <sbloodys@qq.com>
2021-12-31 16:26:07 +08:00
calvin
1f97a62411
[Fix-7510] Remedy the issue with clicking the breadcrumb on the page of file resources. (#7747) 2021-12-31 13:14:07 +08:00
wangyizhi
52a74396dd
[Improvement-6151] Performance optimization of DEPENDENT task (#7725)
* Performance optimization of DEPENDENT task

* fix eslint
2021-12-30 18:20:20 +08:00
xiangzihao
1cd5162e4e
fix bug_7512 (#7716) 2021-12-30 10:34:10 +08:00
xiangzihao
a86c08e32c
feature_7513 (#7707)
Co-authored-by: SbloodyS <sbloodys@qq.com>
2021-12-29 18:22:47 +08:00
Jiajie Zhong
abf3d001fc
[UI] Fix wrong dialogs for process definition's version info (#7662)
We should use process_definition.update_time instead of create time
to make more sence.

close: #7660
2021-12-29 17:25:00 +08:00
calvin
5ab0135d18
[Fix-7345] Support the whole operations for the task group and the task group queue. (#7678)
* add task group

* modify task group

* add task group queue

* add task group queue

* add task group queue

* add task group queue

* add task group priority

* fix a few issue with api

* add task group queue

* modify task group queue

* modify task group

* fix a little bit issue

* fix a little bug in the page of managing task group
2021-12-29 11:40:44 +08:00
kezhenxu94
269a62418f
Add E2E case for user manage (#7652) 2021-12-28 14:02:06 +08:00
songjianet
86137e05ea
[Feature][Workflow Move] Display none workflow move btn. (#7666) 2021-12-28 13:27:52 +08:00
Assert
e172015cd3
fix conflict pr 6997 (#7571)
Co-authored-by: shangeyao <sgy960921>
2021-12-27 12:03:27 +08:00
xiangzihao
82db009781
fix bug_7564 (#7625) 2021-12-26 19:45:15 +08:00
kezhenxu94
03a71297dd
Unify the element selector rules for E2E (#7409)
* Unify the element selector rules for E2E

* Fix typo in docker swarm
2021-12-22 14:53:39 +08:00
calvin
fb31dcd59c
fix this bug (#7461) 2021-12-17 14:40:40 +08:00
songjianet
1f1edb2f23
[Fix][Project List UI] Fix the calling interface that keeps decreasing by one after manually setting the pageNo parameter. (#7399) 2021-12-14 16:31:08 +08:00
kezhenxu94
67cc260d52
Split the components into individual package, refactor configurations to use Spring properties (#7160)
* Split the components into individual package

A follow-up PR will be made to build dedicated Docker images for each
component, so that every component Docker image has minimal jars, which
is easy to maintain and good for security fixes.

* Split the components into individual package

A follow-up PR will be made to build dedicated Docker images for each
component, so that every component Docker image has minimal jars, which
is easy to maintain and good for security fixes.

* Split the components into individual package

A follow-up PR will be made to build dedicated Docker images for each
component, so that every component Docker image has minimal jars, which
is easy to maintain and good for security fixes.
2021-12-10 11:28:53 +08:00
百岁
f21eb8f9fe
[ISSUE 6444]Optimize task node helper document in workflow toolbar (#7226)
* [ISSUE 6444]Optimize task node helper document in workflow toolbar

* fix some review issue than submit by DS

* refact by add guard judgement when type is undefined

* move method helpUrlEnable and helpUrl nest in 'methods' body
2021-12-08 15:09:01 +08:00
songjianet
3ca2ee9ac8
[Bug][dolphinscheduler ui] When the edit time is repaired, the alarm group display. (#7262) 2021-12-08 14:24:53 +08:00
songjianet
eb07a2b1c9
[Fix][Warning instance manage] Expand the support of i18n. (#7184) 2021-12-05 11:23:50 +08:00
songjianet
2fa3157e6e
[Feature][Workflow] Main package. (#7177) 2021-12-04 21:54:59 +08:00
Hua Jiang
226bed1768
fix this issue (#7120) 2021-12-03 16:56:30 +08:00