Commit Graph

6994 Commits

Author SHA1 Message Date
Devosend
7821d3ce7e fix nodeSize is not a function bug (#11200)
(cherry picked from commit 91a944f967)
2022-08-02 18:52:52 +08:00
Jiajie Zhong
9281f2b650 [ci] Make cluster test work on release branch (#11199)
When I try to release the 3.0.0 on branch `3.0.0-prepare`,
I find out cluster-test error due to version change of
tarball, I think we should make cluster test work on both
dev and prepare release branch, so I add this patch

(cherry picked from commit 93a73b1292)
2022-08-02 18:52:29 +08:00
Jiajie Zhong
ab73121401 [python] Add directly install without clone source code (#11179)
Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>

(cherry picked from commit 9e692f919a)
2022-08-02 18:51:46 +08:00
Tq
59f3bce232 add Feishu alert doc and configure email alert doc to the docsdev.js (#11173)
add Feishu alert doc and configure email alert doc to the docsdev.js

(cherry picked from commit 9859b5da1f)
2022-08-02 18:50:29 +08:00
caishunfeng
a8b1b61bd2 [Bug-11071] fix local params (#11184)
* fix local params

* add CI case

(cherry picked from commit 818648df7d)
2022-08-02 18:49:53 +08:00
pinkhello
f7ed6ef850 [Fix] Parameter values disappear when editing Zeppelin task (#11115)
(cherry picked from commit 2e81f30298)
2022-08-02 18:48:15 +08:00
Jiajie Zhong
dd34a1da67 [fix] data quality dml error 2022-08-02 18:07:39 +08:00
Jiajie Zhong
3915c4f40a [fix] Make DDL work in version 3.0
* Add missing database ddl
* Because postgres supported `if not exist` and `if exists` syntax
  so we can use bare SQL directly instead of store process
2022-08-02 16:26:34 +08:00
Sheldon
91da5be260 [fix] 300 error cherry-pick and other bug 2022-07-29 11:22:33 +08:00
Sheldon
1f55cc89c1 [Fix-10665] [S3] Fix s3 download method (#10675)
when overwrite param is true and dest file is exist, throw exception
make dirs for ancestor directories of the source file path if it is not exists
To describe the error more clearly, change "the file isn`t exists" to specific error message

Co-authored-by: sheldonliu <sheldonliu>
(cherry picked from commit efd68f4805)
2022-07-28 11:22:31 +08:00
caishunfeng
034475bfbe [Bug-11101] fix task failover NPE (#11168)
(cherry picked from commit 6c7e00c4aa)
2022-07-28 11:14:33 +08:00
Jiajie Zhong
88712b423a [fix] Some cherry pick error 2022-07-28 11:13:48 +08:00
陈家名
4da1240b0e [Python] Supports creating or editing resources. (#10823)
(cherry picked from commit 59cd86157f)
2022-07-28 11:13:48 +08:00
Jiajie Zhong
e4ab488426 [python] Fix tasks with multiple upstream and workflow query error (#10941)
* when task with more than one upstreams, mapper
   TaskDefinitionMapper method queryByName will return
   more than one record, and failed the mybatis result
   type, so we have to add `limit 1` to it to
* add multiple runs of example in integrate test
* Change from subprocess.Popen to subprocess.call_check
  in integrating test which will raise an error when failed

(cherry picked from commit 81930e5420)
2022-07-28 11:13:48 +08:00
Jiajie Zhong
c67f9ee85d [fix] update queue and tenant with same code error (#10877)
in ae6aa53f I use the same logic test to create or update
tenant and queue which failed create with the same value,
this patch will fix it. and also I use the constructor to reduce
the new object logic here

(cherry picked from commit a6154220dc)
2022-07-28 11:13:48 +08:00
Jiajie Zhong
b2ad6f3f8e [python] Integrate test run strategy change (#10825)
this patch change python integrate test strategy,
before that, only content change in dir
dolphinscheduler-python will run the test. And it
will fail our python api startter when some code
change in java gateway. So we should start the
integrate test except docs change.

and in the future, we may add python ci to
.asf.yaml to make sure it pass

close: #10488
(cherry picked from commit 598290487c)
2022-07-28 11:13:48 +08:00
Jiajie Zhong
d3f28c8411 [python] Fix submit and run error (#10792)
because we add permission, so it failed out
createQueue method. this patch fix it and
do some refactor of our tenant and queue
validator code

(cherry picked from commit ae6aa53f96)
2022-07-28 11:13:48 +08:00
caishunfeng
cec5e88319
[Bug-11071] Fix task local params NPE (#11166)
* fix local param NPE

* delete empty files
2022-07-28 09:43:58 +08:00
xuhhui
acc2fba6cd [doc] Fix bugs in doc installation, design and glossary (#11084)
(cherry picked from commit 8b50578bdb)
2022-07-27 22:43:17 +08:00
Jiajie Zhong
6bab63cc7b [python] fix doc build error (#10810)
(cherry picked from commit 113a67122f)
2022-07-27 22:36:34 +08:00
Jiajie Zhong
6f5ea6e411 [python] Correct some docs for python api (#10772)
* Remove add leading `$` from command
* Add copy button from code block
* Using `python -m pip` to install package avoid error

(cherry picked from commit b4eee7edb6)
2022-07-27 22:34:40 +08:00
sneh-wha
c4c4771435 [doc] Change subprocess dependent (#10637)
(cherry picked from commit 1f46078ab8)
2022-07-27 22:29:03 +08:00
Eric Gao
4324ddd28e [docs] Add links to metrics docs (#10583)
(cherry picked from commit 16144a8858)
2022-07-27 22:24:30 +08:00
Stalary
fe8525744d [doc] Add run the task alone doc. (#10581)
(cherry picked from commit b2df357814)
2022-07-27 22:23:23 +08:00
xiangzihao
b88b522827 [doc] Add doc about how to use healthcheck endpoints (#10592)
(cherry picked from commit 33cc68ed4a)
2022-07-27 22:20:13 +08:00
xiangzihao
b943c003a6 [doc] Add docs about complement dependent mode (#10587)
(cherry picked from commit b21e211d2c)
2022-07-27 22:19:27 +08:00
陈家名
dd4b99298a [python] Support using full-name definitions for resources (#10551)
(cherry picked from commit 7970573365)
2022-07-27 22:13:04 +08:00
Devosend
4f37b7c377
fix space validate for create folder bug (#11106) (#11172)
(cherry picked from commit 2f2de7f90e)
2022-07-27 17:16:58 +08:00
caishunfeng
9742502ea3 fix k8s master and worker start error (#11156) 2022-07-27 09:39:15 +08:00
Devosend
ebdc34b25f [Fix][UI] Add duplicate of file suffix (#11136) 2022-07-27 09:38:34 +08:00
zhuxt2015
1b0f8c04d3 [Bug] [API] If the single log length is long, it will causes the View Log page to hang (#10612) 2022-07-27 09:38:25 +08:00
Devosend
4622a8036a [Fix][UI] fix bug where The Gantt chart does not show the execution status (#11135)
* fix bug where The Gantt chart does not show the execution status of all components

* modify round method
2022-07-27 09:38:02 +08:00
Wenjun Ruan
b791e9158e [Fix-10918] Close datasource when expire from guava cache (#11120)
* Close datasource when expire from guava cache
* Remove duplicate datasource in HiveDataSourceClient
2022-07-27 09:37:18 +08:00
songjianet
8581a464db [Fix][UI] Fix the problem of too large spacing in operation. (#11090)
* [Fix][UI] Fix the problem of too large spacing in operation.

* [Fix][UI] Fix the problem of too large spacing in operation.
2022-07-27 09:35:04 +08:00
Devosend
9164ec8bf1 fix view variable display bug (#11100) 2022-07-27 09:34:43 +08:00
Amy0104
34d2c513c5 [Fix][UI] Widen the segment separator input box. (#11096) 2022-07-27 09:34:30 +08:00
xiangzihao
dba1d076c1 add oracle dependency (#11095) 2022-07-27 09:34:05 +08:00
Assert
7e1d8c2d86 [Feature-10594][ui] Data Quality Result Display Optimize (#10835) 2022-07-20 14:56:36 +08:00
pinkhello
59d8f222c8 [Fix-10539] [Bug] [UI] Copy task a to B, change the user-defined para… (#10611)
* fixed 10539: [Bug] [ui]Copy task a to B, change the user-defined parameters of Task B, and change the parameters of task a

* fixed 10539: [Bug] [ui]Copy task a to B, change the user-defined parameters of Task B, and change the parameters of task a

* feat: reactor code

* feat: remove console.info

* fixed: reactor deepclone

* feat: code style
2022-07-20 11:40:43 +08:00
caishunfeng
b9e3187ad5 [Feature-10871] add workflow executing data query (#10875)
* add workflow executing data query
* fix sonar check for interrupted
2022-07-19 16:10:42 +08:00
Amy0104
bb6aaf755e [Fix][UI] Set the next node to not required. (#10932)
* [Fix][UI] Set the next node to not required.

* [Fix][UI] Remove the unuse i
2022-07-19 15:37:14 +08:00
Amy0104
47782ba555 [Fix][UI] Add validator to the next node field of the switch task. (#10898) 2022-07-19 15:37:02 +08:00
Wenjun Ruan
5a2ea0b76b Fix compile error 2022-07-19 12:30:26 +08:00
Wenjun Ruan
648d469b62 Return status 2022-07-19 12:27:49 +08:00
Wenjun Ruan
f3250bf5fa Fix worker cannot shutdown due to resource close failed or heart beat check failed (#10979)
* Use try-with-resource to close resource, and add heart error threshold to avoid worker cannot close due to heart beat check failed

* Move heartbeat error threshold to applicaiton.yml

(cherry picked from commit 2be1d4bf0a)
2022-07-19 12:23:20 +08:00
Wenjun Ruan
17f0be5bfb Kill task when we do master failover to avoid task lost (#10997)
(cherry picked from commit 998e4d74dd)
2022-07-19 12:22:26 +08:00
Wenjun Ruan
44ddb6908e Fix kill yarn job error when failover caused by doesn't set ProcessDefinition (#10948)
(cherry picked from commit b245e7c973)
2022-07-19 12:22:13 +08:00
Wenjun Ruan
812d7a8f26 Fix taskInstance's host is not worker nettyServer address (#10926)
* Fix taskInstance's host is not worker nettyServer address

* Remove unnecessary mock

(cherry picked from commit df0416c193)
2022-07-19 12:21:59 +08:00
Wenjun Ruan
527ee472fb Catch exception when check state in StateWheelExecuteThread (#10908)
* Catch exception when check state

(cherry picked from commit 2a67866718)
2022-07-19 12:21:20 +08:00
Wenjun Ruan
9ee20cffef [Fix-10827] Fix network error cause worker cannot send message to master (#10886)
* Fix network error cause worker cannot send message to master

(cherry picked from commit cade66a9b6)
2022-07-19 12:20:18 +08:00