test/DolphinScheduler
1
0
Fork 0
mirror of https://gitee.com/dolphinscheduler/DolphinScheduler.git synced 2024-12-04 05:09:48 +08:00
Code Issues Actions 10 Packages Projects Releases Wiki Activity
7,364 Commits 27 Branches 59 Tags 276 MiB
63244c8065
Commit Graph

420 Commits

Author SHA1 Message Date
kezhenxu94
31ba49ac02
Update doc according to recent update of Spring / Swagger (#11755) 2022-09-02 19:06:15 +08:00
kezhenxu94
277f137358
Add Kubernetes configmap reload to all components (#11730) 2022-09-02 12:03:19 +08:00
Evans Sun
5fabce783a
[DOC] improve zk digest doc (#11703) 2022-08-30 21:07:04 +08:00
Eric Gao
4a37345436
[Feature][Task Plugin] Support hive cli task plugin (#11651) 
* Add hive cli task plugin

* Add docs for hive-cli task plugin

* Fix hive-cli task related front-end nits and checkstyle

* Fix exception handling logic
 2022-08-30 10:45:51 +08:00
旺阳
6e4d974b91
[doc] Correct E2E Doc, fix WorkerGroupPage typo (#11629) 2022-08-28 17:44:06 +08:00
Yann Ann
eda9f3d50c
[DOC] should notice that need setting the full path when call the resource file #11620 (#11621) 2022-08-26 10:58:56 +08:00
fuchanghai
ce8db1c5fb
[improvement-#11630]fix document about common.properties (#11653) 2022-08-26 10:50:31 +08:00
Wenjun Ruan
b96d69701a
[Improvement-#11613] Add spi priority factory (#11614) 
* Add spi priority factory

* Add doc

* Add override log

* Use lombok

* Add comment
 2022-08-24 19:51:47 +08:00
Eric Gao
3e73829924
Fix homepage email subscription link (#11622) 
* Fix homepage email link bug

* Replace dev with latest
 2022-08-24 09:25:23 +08:00
Eric Gao
ff4d2c9460
[Doc][Bug] Fix resource center docs for standalone mode (#11600) 
* Fix resource center docs for standalone mode

* Remove useless images
 2022-08-23 09:46:01 +08:00
labbomb
5813a61f09
feat: update slack (#11578) 2022-08-22 11:03:50 +08:00
monster
3e360a8ecb
[Improvement-11550] [Doc]Document content update (#11577) 
* [Improvement-11550] [Doc]Document content update

* Update pictures

* fix

* Update Process word to Workflow

Co-authored-by: David <davidzollo365@gmail.com>
 2022-08-22 00:00:42 +08:00
Eric Gao
277c78010f
Enable spotless to fix markdown checkstyle and refactor faq.md as an example (#11384) 2022-08-18 16:48:22 +08:00
JieguangZhou
a974ba74ab
[Feature] New task plugin Pytorch (#11498) 
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
 2022-08-18 10:20:37 +08:00
sgw
8ec2f79248
[DS-11481][docs] Add log specification document to contribution guidelines. (#11484) 2022-08-17 21:57:02 +08:00
lugela
e5d06c40a5
[Feature-10182][task-datax] (#11387) 
In the process of realizing datax synchronization, use custom json for better connection to enable kerberos-authenticated clusters, such as using plug-ins such as hdfs and hbase
 2022-08-17 10:01:20 +08:00
Eric Gao
9330d6cfcd
[Doc][Security] Update instructions on worker groups (#11483) 
* Update instructions on worker groups
 2022-08-15 17:44:00 +08:00
Qi Zhang
a4bdadbb84
[doc] Use Rainbond deployment DolphinScheduler Cluster (#10951) 
* [doc] Use Rainbond deployment DolphinScheduler Cluster

Signed-off-by: Qi Zhang <smallqi1@163.com>

* Update docs/docs/en/guide/installation/rainbond.md

* Update docs/docs/en/guide/installation/rainbond.md

* Update docs/docs/zh/guide/installation/rainbond.md

* Update docs/docs/zh/guide/installation/rainbond.md

* [doc] modify rainbond doc and add image

Signed-off-by: Qi Zhang <smallqi1@163.com>

* fix: img link

Signed-off-by: Qi Zhang <smallqi1@163.com>

* fix: modify proposal

Signed-off-by: Qi Zhang <smallqi1@163.com>

* docs: move rainbond docs to integration

Signed-off-by: Qi Zhang <smallqi1@163.com>

Signed-off-by: Qi Zhang <smallqi1@163.com>
Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com>
 2022-08-15 14:33:37 +08:00
Wenjun Ruan
7ff34c3947
[Feature-7024] Add waiting strategy to support master/worker can recover from registry lost (#11368) 
* Add waiting strategy to support master/worker can recover from registry lost

* throw exception when zookeeper registry start failed due to interrupted
 2022-08-13 09:52:03 +08:00
Eric Gao
e3c6ace81d
[Doc][DSIP] Move DSIP docs to the right place (#11442) (#11443) 2022-08-12 11:20:12 +08:00
Eric Gao
9d6fc92af9
[Doc][Resources] Instruct users to use local storage if they have remote storage mounted to local (#11435) 
* [Doc][Resources] Instruct users to use local storage if they have remote storage mounted to local (#11427)

* Remove dead link in pyds README

* Add hyperlinks for docs

Co-authored-by: Jiajie Zhong <zhongjiajie955@hotmail.com>
 2022-08-12 00:18:43 +08:00
labbomb
de4e4af844
feat: Modifying Slack Links (#11416) 
* feat: Modifying Slack Links

* append

Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com>
 2022-08-11 13:11:23 +08:00
Jiajie Zhong
26afd655cb
[python] Add multiple version of document (#11391) 
* Using sphinx-multiversion to support multiple version
of python api document
* change python api sidebar link
* fix whitelist_externals deprecated and add git

close: #8227
 2022-08-11 12:02:29 +08:00
Shuo Zhang
50380e008f
[doc] Add how obtain IP address from network card in FAQ (#11311) 
This pull request update the FAQ doc (faq.md) by adding how to obtain the IP address
from the specified network card, which is part of relevant information created when 
the master service and worker service are registered with zookeeper.
 2022-08-11 09:49:47 +08:00
Jiajie Zhong
278371bcb9
[doc] Recover traffic configuration (#11288) 
some config about traffic is deleted by
accident in #11113 this patch recover it
 2022-08-11 09:46:43 +08:00
caishunfeng
0464123c2b
[Feature-11223] support stream task (#11350) 
* add task execute type

* update task definition list paging

* update task instance list paging

* stream task start

* [Feature][UI] Some changes to execute task.
    * Set the connection edge to dashed line.
    * Add FLINK_STREAM task.

* add stream task

* flink savepoint and cancel

* fix query bug

* add stream task definition

* add task instance for stream task

* delete stream task definition state

* update api for stream task definition edit

* modify search for stream task instance

* add language

* delete task type search for stream task definition

* change task type search for stream task instance

* add jump button

* add savepoint

* add down log for stream task instance

* ui test

* stream task start

* run DAG

* [Fix][UI] Fix the stream task edgs not to be dashed when filling back.

* [Feature][UI] Remove some fields for FLINK_STREAM.

* add start modal

* add dryRun column for stream task instance

* fix duration

* fix pon

* fix build error

* Add success tip

* add auto sync for stream task instance

* remove forgien key for task instance

* license header

* UT fix

* modify locales

* recover common config

* fix UT

* add doc

Co-authored-by: Amy <amywang0104@163.com>
Co-authored-by: devosend <devosend@gmail.com>
 2022-08-10 21:44:43 +08:00
insist777
e9ccaffbe0
[doc] Refine the deployment documentation (#11361) 2022-08-09 19:35:49 +08:00
insist777
8ad6e10da5
[doc] Improve the document introduction of complement (#11275) 2022-08-09 19:35:25 +08:00
Jiajie Zhong
c4fdfc7a55
[doc] Fixed hadoop link to avoid dead link (#11363) 
https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/CommandsManual.html#jar
point to the latest docs and will be changed more frequent than fixed link
https://hadoop.apache.org/docs/r3.2.4/hadoop-project-dist/hadoop-common/CommandsManual.html#jar.
So I change to the fixed link
 2022-08-09 13:34:49 +08:00
Jiajie Zhong
2865b17ed4
[release] New release version 3.0.0 (#11356) 2022-08-09 09:47:15 +08:00
xiangzihao
983f9b41bf
[Doc] Add tools/libs to the init database steps (#11345) 2022-08-08 21:53:23 +08:00
Jiajie Zhong
d16c63528f
[community] Remove stackoverflow ref link in website (#11342) 
* Our community do not have official support for stack overflow any more
* Change the order side for Slack and mailing list, to avoid some users
  just want to ask question but join mailing list, and then unsubscribe
  found there are too many emails in mailing list
 2022-08-08 11:24:09 +08:00
Eric Gao
3aa9f2ea25
[Bug][Doc] Fix typos in docs which causes links to upgrade section to fail (#11333) 
* [Bug][Doc] Fix typo in docs

* Remove dead links and update related docs
 2022-08-06 20:50:48 +08:00
Eric Gao
6a02870926
[Feature][style] Add spotless maven plugin for automatic style fix. (#11272) 
* [Feature][style] Add spotless maven plugin for automatic style fix (#10963)

* Fix spotless ratchet configuration

* Remove license-check and decrease line length threshold value

* Update related docs

* Remove checkstyle and add pre-commit hook

* Test updated pre-commit hook

* Replace checkstyle with spotless in CI

* Remove reviewdog
 2022-08-06 19:29:23 +08:00
Jiajie Zhong
bf5f7a88c5
[doc] migrate community and development docs into main repo (#11313) 2022-08-05 17:58:39 +08:00
rickchengx
39186b1a6d
[Doc] update the configuration doc (#11113) 2022-08-04 11:55:05 +08:00
Jiajie Zhong
dc3ab6af11
[doc] Add notice when environment can not select (#11276) 
ref: #11249

Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
 2022-08-04 11:08:11 +08:00
Eric Gao
9ca1eb96c4
[Improvement][Metrics] Add metrics for alert server (#11240) 
* [Improvement][Metrics] Add metrics for alert server (#11131)

* Update related docs of metrics

* Add grafana demo dashboards for alert server metrics

* Refactor metric classes with UtilityClass annotation

* Refactor meter names in camelCase for checkstyle
 2022-08-03 15:42:06 +08:00
阿福Chris
01fc6c8a3a
[doc] Fix installation typo (#11220) 
Co-authored-by: xiangzihao <460888207@qq.com>
 2022-08-01 10:03:53 +08:00
fuchanghai
d73ad2e7d5
[BUG-11181] [Document] text check (#11183) 2022-07-29 23:49:46 +08:00
xuhhui
513f336015
[Feature] Add new task type chunjun (#10937) 2022-07-28 19:25:29 +08:00
Tq
9859b5da1f
add Feishu alert doc and configure email alert doc to the docsdev.js (#11173) 
add Feishu alert doc and configure email alert doc to the docsdev.js
 2022-07-28 17:21:32 +08:00
Jiajie Zhong
de5c22b276
[doc] Add new page incompatible change (#11153) 
Add incompatibles and give some hint to user when then
want to upgrade to specific version.

ref: #10607
 2022-07-27 11:15:09 +08:00
xuhhui
8b50578bdb
[doc] Fix bugs in doc installation, design and glossary (#11084) 2022-07-26 18:08:19 +08:00
qinchaofeng
c28774e5dd
[Doc]Chinese translation updated (#11146) 
* Update python.md Chinese translations

* Update zeppelin.md Chinese translations
 2022-07-26 14:33:38 +08:00
Eric Gao
0e15ce3389
[Improvement][Metrics] Switch to use tags to indicate task / workflow execution status for metrics (#11128) 
* [Improvement][Metrics] Switch to use tags to indicate task / workflow execution status for metrics (#10867)

* Update docs and grafana demo dashboards
 2022-07-26 13:44:02 +08:00
JieguangZhou
952aee80e7
[fix] py sagemaker doc (#11109) 2022-07-22 22:57:50 +08:00
xiangzihao
5b429d29b8
add oracle dependency (#11095) 2022-07-22 09:49:20 +08:00
JieguangZhou
af7b359333
[Feature][Task Plugin] Add SageMaker Pipeline task plugin for MLOps scenario (#10826) (#10935) 
* add sagemaker pipeline task plugin

[DOC] add Sagemaker task plugin document

change license and fix dependencies

* [fix] Optimize the code

* Update dolphinscheduler-task-plugin/dolphinscheduler-task-sagemaker/src/main/java/org/apache/dolphinscheduler/plugin/task/sagemaker/SagemakerParameters.java

Optimization parameter judgment

Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>

* [fix] nips

Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
 2022-07-21 18:16:58 +08:00
Eric Gao
f689220290
[Feature][Task Plugin] Increase zeppelin task stability in production (#11010) 
* [Feature][Task Plugin] Increase zeppelin task stability in production (#10584)

* Add front-end and update docs for the production mode of zeppelin task

* Fix minor front-end bug of zeppelin task plugin

* Refactor ZeppelinParameters with lombok

* Fix formatting

* Replace @Data with @Getter, @Setter, @ToString to avoid decrease in test coverage
 2022-07-20 14:16:47 +08:00
Eric Gao
a38fa34d43
[Feature][Task Plugin] Enable users to switch endpoints in zeppelin tasks (#10925) 
* [Feature][Task Plugin] Enable users to switch endpoints in zeppelin tasks (#9814)
 2022-07-15 09:26:28 +08:00
Jiajie Zhong
08cfbc73e0
[doc] Add missing doc change by snehal (#10892) 2022-07-12 14:16:00 +08:00
Eric Gao
2f7281c2d2
[Feature][Metrics] Add resource download related metrics for workers (#10749) 
* [Feature][Metrics] Add resource download related metrics for workers (#9324)

* [Feature][Metrics] Fix bugs and add grafana demos for worker resource download metrics (#9324)

* [Feature][Metrics] Add docs to resource related metrics (#9324)

* [Feature][Metrics] Use tags to indicate status in metrics (#9324)

* [Feature][Metrics] Fix demos, docs and remove redundant code (#9324)

* [Feature][Metrics] Remove .pnpm-debug.log (#9324)

* [Feature][Metrics] Fix style check (#9324)

* [Feature][Metrics] Replace KB with bytes for the unit of resource file size in metrics (#9324)

* [Feature][Metrics] Make code neat (#9324)
 2022-07-12 11:44:34 +08:00
ZhaoGuodong
8eaf5a2309
[Feature-10219][EMR] EMR supports use <add-Steps> to add steps to an existing cluster (#10657) 
* Add the ProgramType parameter to distinguish task types
* EmrAddStepsTask supports Add-Steps
* UI supports Add-Steps
* EmrTask modify the name of the class to EmrJobFlowTask
* add ERM Task abstract base class AbstractEmrTask
* add testcase for EmrAddStepsTask
* Modifying help Documents
 2022-07-10 10:15:25 +08:00
Luke Yan
057a9f6b24
[Fix-10830][Doc] Change the start or install script command sh in the document to bash (#10831) 
* Change sh to bash

Change the script execution command sh to bash

* Change sh to bash

Change the script execution command sh to bash

* Change sh to bash

Change the script execution command sh to bash

* Change sh to bash

Change the script execution command sh to bash

* Change sh to bash

Co-authored-by: luke <luke.yan@china-entercom.net>
 2022-07-09 15:41:56 +08:00
Eric Gao
1387fb3ccf
[Doc][Upgrade] Fix tiny error in upgrade docs (#10847) 
* [Doc][Upgrade] Fix tiny error in upgrade docs (#10846)

* [Doc][Upgrade] Add useSSL=false and fix typo in upgrade docs (#10846)
 2022-07-09 14:53:05 +08:00
Jiajie Zhong
503cf179eb
[doc] 206 drop down doc not work (#10855) 2022-07-08 23:24:34 +08:00
Jiajie Zhong
d29592da1d
New version release 2.0.6 (#10851) 
* New version release 2.0.6

* remove verison 1.3.8 and 1.3.9
 2022-07-08 20:34:14 +08:00
qianli2022
1e48bbc5c6
[Feature-10629][Improvement] Support multi cluster environments - namespace use cluster (#10650) 
* server code

* namespace ui

* fix dao test

* fix test

* fix

* docs and image update

Co-authored-by: qianl4 <qianl4@cicso.com>
 2022-07-07 16:05:52 +08:00
Eric Gao
71f0168510
[Feature] Enable users to create python env from requirements.txt (#10658) 2022-07-06 15:56:39 +08:00
旺阳
2d8b4932b3
[doc] Correct custom jdbc driver for K8s Doc (#10768) 2022-07-06 10:57:14 +08:00
sneh-wha
fa1458b3e0
[doc] Modify features, hw, intro and homepage docs. (#10713) 2022-07-05 21:58:05 +08:00
sneh-wha
91b59dee54
[doc] Modified dq, monitor, security, resources (#10715) 2022-07-05 20:50:37 +08:00
Kerwin
bd1be6cf36
[Task] Added Apache SeaTunnel 2.x task node (#10480) 2022-07-05 19:21:30 +08:00
aiwenmo
5b0347e8da
[Task] Add Dinky task to better support the development and execution of FlinkSQL (#10640) 2022-07-05 10:07:24 +08:00
xiangzihao
7aad6a6515
[doc] Unify data source switching of the docs during installation (#10688) 2022-07-04 16:28:14 +08:00
sneh-wha
6cf9d088ff
[doc] Change tasks docs (#10751) 2022-07-04 11:29:59 +08:00
sneh-wha
423cc5b9be
[doc] Modified project doc. (#10711) 2022-07-04 10:23:25 +08:00
Devosend
4f0a02eb43
[chore] update by release version (#10742) 
* update 3.0.0-beta-2 doc

* update history-versions doc
 2022-07-04 10:05:29 +08:00
sneh-wha
fb8d0c858b
Merging general settings to faq (#10716) 
* Merging general settings to faq

* also change the zh position

Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com>
 2022-07-01 17:18:02 +08:00
sneh-wha
fb90a0e558
[doc] Modify quick start. (#10712) 2022-07-01 16:51:07 +08:00
sneh-wha
0cdae777ab
[doc] Modified alert doc files (#10714) 2022-07-01 16:29:00 +08:00
Paul Zhang
4a3c3e7797
[feature][task-flink] Support Flink application mode (#9577) 2022-07-01 12:20:38 +08:00
sneh-wha
65ebdbef98
[doc] Change tasks doc (#10639) 2022-07-01 01:46:04 +08:00
sneh-wha
7898f41dc0
[doc] Change datasource doc (#10660) 2022-06-30 23:33:59 +08:00
Stalary
a45fc5b032
[doc] Add doc for pre-sql and post-sql (#10685) 2022-06-30 20:01:23 +08:00
caishunfeng
c2f1886306
[doc] Update metadata and design doc (#10600) 2022-06-30 17:48:17 +08:00
sneh-wha
1cdbbc1ba5
[doc] Modified parameter doc files. (#10659) 2022-06-29 14:42:37 +08:00
sneh-wha
1f46078ab8
[doc] Change subprocess dependent (#10637) 2022-06-27 19:30:35 +08:00
sneh-wha
a14b96b091
[doc] Modify Shell task file (#10634) 2022-06-27 17:22:09 +08:00
Jiajie Zhong
cd51a22940
[doc] Correct installPath when using related path (#10597) 
correct the doc and add hint to setting

ref: #10511
 2022-06-27 11:02:19 +08:00
Eric Gao
16144a8858
[docs] Add links to metrics docs (#10583) 2022-06-24 17:27:40 +08:00
Stalary
b2df357814
[doc] Add run the task alone doc. (#10581) 2022-06-24 16:59:22 +08:00
xiangzihao
33cc68ed4a
[doc] Add doc about how to use healthcheck endpoints (#10592) 2022-06-24 16:56:26 +08:00
xiangzihao
b21e211d2c
[doc] Add docs about complement dependent mode (#10587) 2022-06-24 16:33:10 +08:00
xiangzihao
773ebc9e52
fix 10513 (#10564) 2022-06-23 13:44:22 +08:00
houshitao
1b1a567fec
[doc] Correct the complement add docs (#10521) 2022-06-23 12:02:28 +08:00
JieguangZhou
3918140e7e
[fix][doc] Add DVC task plugin doc link (#10529) 2022-06-23 10:01:49 +08:00
Eric Gao
0aba68f335
[Improvement][Metrics] Update some metrics names in grafana-demo dashboards (#10432) (#10552) 2022-06-22 23:05:54 +08:00
Eric Gao
cc06eaaf54
[Improvement][Metrics] Apply micrometer naming convention to metrics (#10477) 
* Apply micrometer naming convention to worker metrics
* Apply micrometer naming convention all current metrics
* Fix remaining metrics names, update English docs and add Chinese docs
* Fix metrics names in grafana-demo dashboards
 2022-06-21 14:27:06 +08:00
JieguangZhou
36e20cdfc8
[Feature][Task Plugin] Add DVC task plugin for MLops scenario (#10372) (#10407) 2022-06-21 13:13:46 +08:00
旺阳
6e4b2e6992
[Fix]Recovery Traffic Control Config (#10505) 
* recovery traffic config

* run ci

* update
 2022-06-20 18:43:48 +08:00
Eric Gao
64cee03fed
[Feature] Enable users to switch and install conda env in jupyter task (#10337) 2022-06-20 14:47:44 +08:00
xiangzihao
704e633659
[Doc] Add api test doc (#10499) 2022-06-19 13:55:23 +08:00
Eric Gao
4be3b877c3
[feature][task] Enable zeppelin schedule a whole zeppelin note (#10434) 2022-06-18 13:34:24 +08:00
guodong
2c227ab384
[Feature-10363][server] S3 Resource center supports bucket customization (#10433) 
* [Feature-10363][server] S3 Resource center supports bucket customization

* Modify the configuration. Sort out HDFS and S3 configurations to distinguish them
* Modify the configuration comment to clarify the configuration meaning
* Modify official documents to add missing configuration descriptions and clarify configuration meanings and usage
* Modify the configuration file in e2E to the correct definition, dolphinscheduler-e2e/dolphinscheduler-e2e-case/src/test/resources/docker/file-manage/common.properties
* Modified code to support bucket customization
* Check that the bucket on personal permission, Otherwise, the startup fails
* S3 increase support resource. Storage. Upload. Base. The base path path
* Change S3Utils implementation style to be consistent with HadoopUtils

* [Feature-10363][server] S3 Resource center supports bucket customization

* Modify code review comments
 2022-06-18 10:20:01 +08:00
liubo1990
b8d5e78994
[Doc] Add how to set the alarm mailbox in faq (#10401) 2022-06-17 21:16:38 +08:00
旺阳
6515c66208
[Improve] Add LDAP user not exsitst action config (#10451) 2022-06-17 14:18:05 +08:00
qianli2022
ff065d8e5b
[Feature][Improvement] Support multi cluster environments - k8s type (#10096) 
* service code

* [Feature][UI] Add front-end for cluster manage

* fix e2e

* remove comment on cluster controller

* doc

* img

* setting e2e.yaml

* test

* rerun e2e

* fix bug from comment

* Update index.tsx

use Nspace instead of css.

* Update index.tsx

Remove the style.

* Delete index.module.scss

Remove the useless file.

Co-authored-by: qianl4 <qianl4@cicso.com>
Co-authored-by: William Tong <weitong@cisco.com>
Co-authored-by: Amy0104 <97265214+Amy0104@users.noreply.github.com>
 2022-06-15 13:39:20 +08:00
Jiajie Zhong
b655aa9b1f
[doc] Correct resource in HDFS HA cluster (#10445) 2022-06-15 11:08:18 +08:00