mirror of
https://gitee.com/dolphinscheduler/DolphinScheduler.git
synced 2024-11-30 19:27:38 +08:00
2e1d6975fd
* [Build] Optimize long release name and remove unused nginx release * [Build] Upgrade spring boot version * [Improvement] Improve code style
103 lines
7.3 KiB
Markdown
103 lines
7.3 KiB
Markdown
Dolphin Scheduler Official Website
|
||
[dolphinscheduler.apache.org](https://dolphinscheduler.apache.org)
|
||
============
|
||
|
||
[](https://www.apache.org/licenses/LICENSE-2.0.html)
|
||
[](https://github.com/apache/dolphinscheduler)
|
||
[](https://codecov.io/gh/apache/dolphinscheduler/branch/dev)
|
||
[](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler)
|
||
[](https://twitter.com/dolphinschedule)
|
||
[](https://join.slack.com/t/asf-dolphinscheduler/shared_invite/zt-omtdhuio-_JISsxYhiVsltmC5h38yfw)
|
||
|
||
|
||
|
||
[](https://starchart.cc/apache/dolphinscheduler)
|
||
|
||
[](README.md)
|
||
[](README_zh_CN.md)
|
||
|
||
## Design Features
|
||
|
||
DolphinScheduler is a distributed and extensible workflow scheduler platform with powerful DAG visual interfaces, dedicated to solving complex job dependencies in the data pipeline and providing various types of jobs available `out of the box`.
|
||
|
||
Its main objectives are as follows:
|
||
|
||
- Associate the tasks according to the dependencies of the tasks in a DAG graph, which can visualize the running state of the task in real-time.
|
||
- Support various task types: Shell, MR, Spark, SQL (MySQL, PostgreSQL, hive, spark SQL), Python, Sub_Process, Procedure, etc.
|
||
- Support scheduling of workflows and dependencies, manual scheduling to pause/stop/recover task, support failure task retry/alarm, recover specified nodes from failure, kill task, etc.
|
||
- Support the priority of workflows & tasks, task failover, and task timeout alarm or failure.
|
||
- Support workflow global parameters and node customized parameter settings.
|
||
- Support online upload/download/management of resource files, etc. Support online file creation and editing.
|
||
- Support task log online viewing and scrolling and downloading, etc.
|
||
- Have implemented cluster HA, decentralize Master cluster and Worker cluster through Zookeeper.
|
||
- Support the viewing of Master/Worker CPU load, memory, and CPU usage metrics.
|
||
- Support displaying workflow history in tree/Gantt chart, as well as statistical analysis on the task status & process status in each workflow.
|
||
- Support back-filling data.
|
||
- Support multi-tenant.
|
||
- Support internationalization.
|
||
- More features waiting for partners to explore...
|
||
|
||
## What's in DolphinScheduler
|
||
|
||
Stability | Accessibility | Features | Scalability |
|
||
-- | -- | -- | --
|
||
Decentralized multi-master and multi-worker | Visualization of workflow key information, such as task status, task type, retry times, task operation machine information, visual variables, and so on at a glance. | Support pause, recover operation | Support customized task types
|
||
support HA | Visualization of all workflow operations, dragging tasks to draw DAGs, configuring data sources and resources. At the same time, for third-party systems, provide API mode operations. | Users on DolphinScheduler can achieve many-to-one or one-to-one mapping relationship through tenants and Hadoop users, which is very important for scheduling large data jobs. | The scheduler supports distributed scheduling, and the overall scheduling capability will increase linearly with the scale of the cluster. Master and Worker support dynamic adjustment.
|
||
Overload processing: By using the task queue mechanism, the number of schedulable tasks on a single machine can be flexibly configured. Machine jam can be avoided with high tolerance to numbers of tasks cached in task queue. | One-click deployment | Support traditional shell tasks, and big data platform task scheduling: MR, Spark, SQL (MySQL, PostgreSQL, hive, spark SQL), Python, Procedure, Sub_Process | |
|
||
|
||
## User Interface Screenshots
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
## QuickStart in Docker
|
||
|
||
Please referer the official website document: [QuickStart in Docker](https://dolphinscheduler.apache.org/en-us/docs/latest/user_doc/docker-deployment.html)
|
||
|
||
## QuickStart in Kubernetes
|
||
|
||
Please referer the official website document: [QuickStart in Kubernetes](https://dolphinscheduler.apache.org/en-us/docs/latest/user_doc/kubernetes-deployment.html)
|
||
|
||
## How to Build
|
||
|
||
```bash
|
||
./mvnw clean install -Prelease
|
||
```
|
||
|
||
Artifact:
|
||
|
||
```
|
||
dolphinscheduler-dist/target/apache-dolphinscheduler-${latest.release.version}-bin.tar.gz: Binary package of DolphinScheduler
|
||
dolphinscheduler-dist/target/apache-dolphinscheduler-${latest.release.version}-src.tar.gz: Source code package of DolphinScheduler
|
||
```
|
||
|
||
## Thanks
|
||
|
||
DolphinScheduler is based on a lot of excellent open-source projects, such as Google guava, guice, grpc, netty, ali bonecp, quartz, and many open-source projects of Apache and so on.
|
||
We would like to express our deep gratitude to all the open-source projects used in Dolphin Scheduler. We hope that we are not only the beneficiaries of open-source, but also give back to the community. Besides, we hope everyone who have the same enthusiasm and passion for open source could join in and contribute to the open-source community!
|
||
|
||
## Get Help
|
||
|
||
1. Submit an [issue](https://github.com/apache/dolphinscheduler/issues/new/choose)
|
||
1. Subscribe to this mailing list: https://dolphinscheduler.apache.org/en-us/community/development/subscribe.html, then email dev@dolphinscheduler.apache.org
|
||
|
||
## Community
|
||
|
||
You are very welcome to communicate with the developers and users of Dolphin Scheduler. There are two ways to find them:
|
||
1. Join the Slack channel by [this invitation link](https://join.slack.com/t/asf-dolphinscheduler/shared_invite/zt-omtdhuio-_JISsxYhiVsltmC5h38yfw).
|
||
2. Follow the [Twitter account of Dolphin Scheduler](https://twitter.com/dolphinschedule) and get the latest news on time.
|
||
|
||
## How to Contribute
|
||
|
||
The community welcomes everyone to contribute, please refer to this page to find out more: [How to contribute](https://dolphinscheduler.apache.org/en-us/community/development/contribute.html).
|
||
|
||
## License
|
||
|
||
Please refer to the [LICENSE](https://github.com/apache/dolphinscheduler/blob/dev/LICENSE) file.
|