Commit Graph

4996 Commits

Author SHA1 Message Date
Chouc
3193837f5a [DS-5559][fix][Master Server] Master Server was shutdown but the process still in system (#5588)
* Close spring context to destory beans that has running thread
2021-06-15 10:30:26 +08:00
Shiwen Cheng
309deaefe8 [Fix-5581][SQL] Specific key was too long, max key length is 767 bytes for varchar(256) in some mysql with innodb_large_prefix=OFF (#5582) 2021-06-15 10:30:26 +08:00
JinyLeeChina
a561a618af
[Feature][JsonSplit-api] api of processDefinition create/update (#5602)
* processDefinition create/update

* fix codeStyle

* fix codeStyle

* fix ut

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-06-11 09:43:05 +08:00
Squidyu
cc9e5d5d34
Add:Name verification remove the first and last spaces. (#5574)
Co-authored-by: Squid <2824638304@qq.com>
2021-06-02 12:01:01 +08:00
Shiwen Cheng
a5a0c7c5f8
[Fix-5570][Worker] Fix worker.groups in worker.properties is still commented and improve install scripts and configurations (#5571) 2021-06-02 11:21:01 +08:00
ruanwenjun
f8ecb536b7
[Improvement][Alert] server down will send repetitive message #5525 (#5529)
* [Improvement][Alert] server down will send repetitive message #5525

* add ut
2021-06-01 10:21:46 +08:00
geosmart
60af52fb2b
[Improvement][ApiDoc] fix-queue controller verifyQueue param (#5558)
Co-authored-by: wanggang <wanggy01@servyou.com.cn>
2021-05-31 21:07:21 +08:00
geosmart
56273319a2
[Bug][Master] remove check with executePath when kill yarn job (#5552)
* [Bug][Master] remove check with executePath when kill yarn job

* [Bug][Master] init execute path if not exist

* [Bug][Master] move createWorkDirIfAbsent out side of executePath calculate  part

Co-authored-by: wanggang <wanggy01@servyou.com.cn>
2021-05-29 12:14:46 +08:00
ruanwenjun
842c5400e6
[Improvement][Server] Must restart master if Zk reconnect (#5210) (#5211) 2021-05-25 05:03:29 +08:00
ruanwenjun
9d0c816cee
[Bug][worker] task throw ConcurrentModifiedException #5528 (#5530)
* [Bug][worker] task throw ConcurrentModifiedException #5528

* fix code smell
2021-05-25 04:53:56 +08:00
Shiwen Cheng
b537f4eb26
[Improvement][Docker/K8s] Support more configurations and service accesses, reduce the duplication configurations, improve image for python and the compatibility of python home (#5393)
* [Improvement][Docker] Support more configurations and improve image for python

* [Improvement][K8s] Support more configurations, more service accesses, and reduce the duplication configurations

* [Improvement][Python] Improve the compatibility of python home

* [Improvement][Install] Fix install config and install.sh

* [Improvement][Install] Fix workflow ut

* [Improvement][Docker] Optimize docker-swarm/check script

* [Improvement][DB] Update default username and password of database

* Update comments in master/worker.properties

* Specify version for all images

* [Improvement][Docker] Optimize PS1 and WORKDIR

* [Improvement][SQL] Reduce heap size to 64m in create-dolphinscheduler.sh and upgrade-dolphinscheduler.sh

* [Fix-5431][K8s] Fix master and worker cannot get the right address with custom DNS
2021-05-24 00:03:35 +08:00
zhuangchong
9ba4ffbe48
[Improvement-5527][api-server] failed find any kerberos (#5533)
* fix failed find any kerberos.

* increase code coverage.

* update common utils test.

* update common utils test.

* update common utils test.

* fix exception type.
2021-05-23 23:43:59 +08:00
QuakeWang
46660b58ed
[Test] Add SparkParameters test (#5478)
* Create SparkparametersTest.java

Add a test case

* Update and rename SparkparametersTest.java to SparkParametersTest.java

* Update pom.xml

add SparkParametersTest.java in pom file
2021-05-22 23:56:39 +08:00
zhuangchong
802f4af159
fix dist packaging lacks alert jar. (#5536) 2021-05-22 23:02:55 +08:00
Kirs
8a1d849701
[ut]ignore plugin ut (#5532) 2021-05-21 10:36:08 +08:00
Shuyang Wu
d9f7d1f63e
feat: add contributor over time graph to README (#5531) 2021-05-21 10:24:55 +08:00
zhuangchong
1b7dcea610
[Improvement-5440][Maven] Project module maven dependency optimization (#5441)
* Optimize dependencies between modules.

* update module-alert dependency.

* add repository in the pom xml.

* update Central Repository in the pom.xml.
2021-05-19 10:05:00 +08:00
Yao WANG
bc22ae7c91
[Improvement][Task] Remove deprecated TaskRecordDao and simply the after() method in the AbstractTask class (#5492)
* remove the depreated data quality module

* fix lint

Co-authored-by: Yao <ywang46@paypal.com>
2021-05-18 17:00:03 +08:00
ruanwenjun
018f5c89f6
fix sql symbol error (#5462) 2021-05-18 10:20:18 +08:00
ruanwenjun
a925f64571
[Feature][api] schedule add timezone support #5259 (#5465)
* [Feature][api] schedule add timezone support #5259

* import moment-timezone

* fix code smell
2021-05-17 21:08:20 +08:00
ruanwenjun
68301db6b9
[Improvement][Api] Upload resource to remote failed, the local tmp file need to be cleared #5475 (#5476) 2021-05-17 11:13:14 +08:00
lcdrs
d04f4b6053
[Implement][Common] Modify the plug-in loading log to print messages
Co-authored-by: caoping <caoping@staff.weibo.com>
2021-05-13 16:26:21 +08:00
Kirs
a3e5dfc146
Merge pull request #5390 from apache/json_split
[Feature-4417] Process definition json split
2021-05-12 18:44:30 +08:00
baoliang
dd80f98196 update sleep exception 2021-05-12 16:50:39 +08:00
baoliang
1c72a4b9d8 Merge remote-tracking branch 'upstream/dev' into json_split 2021-05-12 10:32:54 +08:00
wen-hemin
5079600c98
fix: field length (#5446)
Co-authored-by: wen-hemin <wenhemin@apache.com>
2021-05-12 10:15:01 +08:00
ruanwenjun
5556e47a85
[Feature][Plugin-Email] Numeric data type support in email attachment #5424 (#5426) 2021-05-11 20:53:27 +08:00
baoliang
7aa8b518ac Merge remote-tracking branch 'upstream/dev' into json_split
# Conflicts:
#	dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/master/runner/MasterExecThread.java
#	dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/utils/DependentExecute.java
2021-05-11 13:55:27 +08:00
baoliang
2dd0d936fb Merge remote-tracking branch 'upstream/dev' into json_split
# Conflicts:
#	dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/master/runner/MasterExecThread.java
#	dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/utils/DependentExecute.java
2021-05-11 11:45:40 +08:00
baoliang
156dadac69 Merge remote-tracking branch 'upstream/dev' into json_split
# Conflicts:
#	dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/master/runner/MasterExecThread.java
#	dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/utils/DependentExecute.java
2021-05-11 10:08:28 +08:00
ruanwenjun
fbef290cc5
[Bug][Service] All tasks that need to send attachment will fail due to chennel idle (#5442) 2021-05-11 03:03:52 +08:00
JinyLeeChina
515fa4912e
[Feature][JsonSplit] Fix ProcessService ut (#5435)
* Fix ProcessService ut

* codeStyle

* Fix AlertPlugin ut

* Fix AlertPlugin ut

* Fix server module ut

* Fix server module ut

* Fix api ut

* codeStyle

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-05-10 23:17:56 +08:00
ruanwenjun
25727d2d95
[Improvement][Alert] Replace HSSFWorkbook (#5400)
* [Improvement][Alert] Replace HSSFWorkbook

* add license
2021-05-10 09:57:49 +08:00
zhuangchong
9c77faa8ac
[Improvement-5058][Master] The master-server creates the DB data source twice (#5139)
* reduce the number of data sources created twice by master server.

* update ProcessAlertManagerTest test class to increase coverage.

* add DolphinSchedulerPluginLoaderTest test ignore.

Co-authored-by: zhuang chong
2021-05-10 09:56:17 +08:00
zhuangchong
0d8011f9c1
Cleanup of useless files in the binary package lib directory. (#5430) 2021-05-10 09:44:17 +08:00
JinyLeeChina
847865fba6
[Feature][JsonSplit] Fix dao ut (#5433)
* update taskParams/add task delayTime/fix conditionType bug

* update codeStyle for merge to dev

* fix process lineage bug

* fix codeStyle for json_split

* Fix dao ut

* Fix dao ut

* codeStyle

* modify postgreSQL

* fix TaskInstanceMapper ut

* fix ProjectMapper ut

* fix ProcessDefinitionDao ut

* datasource

* fix postgreSQL

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-05-09 13:53:14 +08:00
JinyLeeChina
f7b71fb9ca
[Feature][JsonSplit] Fix codeStyle for json_split (#5432)
* update taskParams/add task delayTime/fix conditionType bug

* update codeStyle for merge to dev

* fix process lineage bug

* fix codeStyle for json_split

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-05-09 00:01:09 +08:00
QuakeWang
f88bbbd637
[Improvement] Create RollViewLogRequestCommandTest.java (#5417)
* Create RollViewLogRequestCommandTest.java

Add a test case about RollViewLogRequestCommand

* Update pom.xml

add the test files in the parent pom
2021-05-07 20:58:23 +08:00
JinyLeeChina
363426de71
[Feature][JsonSplit] fix process lineage bug (#5418)
* update taskParams/add task delayTime/fix conditionType bug

* update codeStyle for merge to dev

* fix process lineage bug

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-05-07 16:37:49 +08:00
baoliang
c850b85484 Merge remote-tracking branch 'upstream/dev' into json_split
# Conflicts:
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/ProcessDefinitionController.java
#	dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/task/procedure/ProcedureTask.java
2021-05-07 01:38:53 +08:00
baoliang
b594afc94c Merge remote-tracking branch 'upstream/dev' into json_split
# Conflicts:
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/ProcessDefinitionController.java
#	dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/task/procedure/ProcedureTask.java
2021-05-07 01:38:31 +08:00
ruanwenjun
71f39fa875
[Improvement][Dao] Database add connection check (#5408) (#5409) 2021-05-06 09:41:44 +08:00
ruanwenjun
0bdf15efb9
[Improvement][DAO] Add Unique key in t_ds_alertgroup and t_ds_datasource (#5373)
* [Improvement][DAO] Add Unique key to t_ds_worker_group t_ds_datasource

* Add exception catch

* add ddl

* fix ut
2021-05-06 09:32:28 +08:00
ruanwenjun
97f08429f4
[Improvement] Change the status box text (#5421) 2021-05-06 09:22:51 +08:00
Hayden Zhou
2114b63216
replace access log by AccessLogAnnotation (#5315)
* replace access log by AccessLogAnnotaion

* fix codesytle error

* fix RequestContextHolder.getRequestAttributes may be null

* fix code style

* accessLogAnnotation addd ignore params

* fix style

* reduce code smells

* fix code style

* fix logtext if request is ignore

* add trance id

* fix code style

* add comment

* optimize

* fix code style

* fix code

Co-authored-by: hayden <hayden@haydendeMacBook-Pro.local>
2021-05-04 22:40:46 +08:00
QuakeWang
22c62372a6
Create GetLogBytesResponseCommandTest.java (#5413)
Create a test case
2021-05-01 23:47:43 +08:00
ruanwenjun
ee95d07a90
[Feature][Alert plugin] Support slack alert (#5382) (#5384)
* [Feature][Alert plugin] Support slack alert (#5382)

* fix code smell

* add markdown table
2021-04-30 10:11:32 +08:00
zhuangchong
88a07f7b55
[Feature-4093][server] Support for stored procedures and stored function calls and data source supports DB2 (#4094)
* Support for stored procedures and stored function calls and data source supports DB2.


Co-authored-by: zhuangchong <zhuangchong8@163.com>
2021-04-29 18:29:44 +08:00
BoYiZhang
2a59ed092c
[Improvement-#3476][UI]Resource files support copy path operations (#5348)
* Improvement-5346 :  Resource files support copy path operations

* Improvement-5346 :  Resource files support copy path operations

* this close #5346

The current resource file does not support copying paths

When using the path is very easy to output errors. Add copy button. Support copy operation

* fix code smell
2021-04-29 16:29:29 +08:00
Shiwen Cheng
0f3eed9d00
[Fix-5386][Alert] Fix ERROR: relation t_ds_plugin_define does not exist (#5387) 2021-04-29 16:17:01 +08:00