Migrated repository
Go to file
xujiaqiang 159179ac95
[Feature-15146][dolphinscheduler-task-sqoop] add sqoop source/target type (#15146)
* task list: sgoop node params  optimize

* security.ts  add  alarm_instance params

* 1 add SqoopTask params
2 add alert plugin aliyun-voice

* add license header

* commit sqhoop optimize

* pnpm-locl.yaml supplement annotation

* remove irrelevent commit.

* Code specification optimization

* optimize sqoop task ui

* Merge Code

* add the license header to  pnpm-locl.yaml

* format the code

* format the code

* Fix sqoop task echo error

---------

Co-authored-by: xujiaqiang <xujiaqiang@aimatech.com>
Co-authored-by: xujiaqiang <“xujiaqiangwz@163.com”>
Co-authored-by: David Zollo <dailidong66@gmail.com>
2023-12-13 17:12:05 +08:00
.github Fix docs style is incorrect by CI pass (#15167) 2023-11-25 22:32:38 +08:00
.idea Add icon in idea (#12829) 2022-11-10 18:56:28 +08:00
.mvn/wrapper [Improve][CI]Upgrade maven-wrapper to 3.1.0 (#8557) 2022-02-27 19:54:23 +08:00
deploy Add deploy on Terraform on README (#15189) 2023-11-20 16:19:07 +08:00
docs [Feature-15248][dolphinscheduler-alert-plugins] add alert plugin aliyun-voice (#15248) 2023-12-13 15:56:37 +08:00
dolphinscheduler-alert [Feature-15248][dolphinscheduler-alert-plugins] add alert plugin aliyun-voice (#15248) 2023-12-13 15:56:37 +08:00
dolphinscheduler-aop fix aop typo error (#13047) 2022-11-30 14:00:06 +08:00
dolphinscheduler-api Add dolphinscheduler-extract-common module (#15266) 2023-12-05 21:06:39 +08:00
dolphinscheduler-api-test [Improvement][API Test] Add api tests for process instance (#14514) 2023-07-14 11:07:41 +08:00
dolphinscheduler-bom [Feature-15248][dolphinscheduler-alert-plugins] add alert plugin aliyun-voice (#15248) 2023-12-13 15:56:37 +08:00
dolphinscheduler-common [Improvement-15260][dolphinscheduler-datasource-hana] add hana related dependencies (#15260) 2023-12-12 17:11:27 +08:00
dolphinscheduler-dao [Fix #15129] [Dependent] The date rules of the dependent node are ambiguous. (#15289) 2023-12-08 16:18:33 +08:00
dolphinscheduler-dao-plugin Remove mapper usage in tools (#15073) 2023-10-25 22:58:47 +08:00
dolphinscheduler-data-quality [Feature][Data Quality] Data Quality Support Choose Database (#14406) 2023-07-12 09:00:28 +08:00
dolphinscheduler-datasource-plugin [Improvement-15260][dolphinscheduler-datasource-hana] add hana related dependencies (#15260) 2023-12-12 17:11:27 +08:00
dolphinscheduler-dist [Feature-15248][dolphinscheduler-alert-plugins] add alert plugin aliyun-voice (#15248) 2023-12-13 15:56:37 +08:00
dolphinscheduler-e2e [Improvement] Refactoring K8S task plugin with connections managed in connection center (#14977) 2023-10-31 17:07:24 +08:00
dolphinscheduler-extract Add dolphinscheduler-extract-common module (#15266) 2023-12-05 21:06:39 +08:00
dolphinscheduler-master [Fix #15129] [Dependent] The date rules of the dependent node are ambiguous. (#15289) 2023-12-08 16:18:33 +08:00
dolphinscheduler-meter [Improvement][Metrics] Add more worker related metrics and fix some previous ones (#14254) 2023-06-01 16:54:48 +08:00
dolphinscheduler-microbench Add rpc benchmark test (#14797) 2023-08-27 20:59:55 +08:00
dolphinscheduler-registry [Improvement][Alert] Add a test send feature when creating an alert instance (#15163) 2023-11-20 10:03:37 +08:00
dolphinscheduler-scheduler-plugin [Improvement-13932][API] Tenant parameters and Worker group parameters are put together in the task startup parameters (#13935) 2023-04-18 14:56:38 +08:00
dolphinscheduler-service Add dolphinscheduler-extract-common module (#15266) 2023-12-05 21:06:39 +08:00
dolphinscheduler-spi [Improvement] Refactoring K8S task plugin with connections managed in connection center (#14977) 2023-10-31 17:07:24 +08:00
dolphinscheduler-standalone-server Fix home page workflow instance miss status (#15193) 2023-11-22 10:33:24 +08:00
dolphinscheduler-storage-plugin [Fix][Resource Center] Remove NONE storage type and set LOCAL type as default (#14661) 2023-09-07 14:09:15 +08:00
dolphinscheduler-task-plugin [Feature-15146][dolphinscheduler-task-sqoop] add sqoop source/target type (#15146) 2023-12-13 17:12:05 +08:00
dolphinscheduler-tools Remove API Result in Service (#15181) 2023-11-20 17:04:03 +08:00
dolphinscheduler-ui [Feature-15146][dolphinscheduler-task-sqoop] add sqoop source/target type (#15146) 2023-12-13 17:12:05 +08:00
dolphinscheduler-worker [Improvement] Move delay calculation to Master (#15278) 2023-12-06 09:37:14 +08:00
images doc: Change readme and standalone docker quick start (#14002) 2023-04-25 09:35:49 +08:00
licenses [Fix][Common] rewrite code generate,fix bit shift (#6914) 2021-11-19 16:04:05 +08:00
script [improvement][install.sh] fix the install Error "Command not found rmr" & su… (#13163) 2023-01-12 18:07:58 +08:00
style [feat][ci] Add pre-commit for easy spotless config (#12192) 2022-09-29 10:10:53 +08:00
tools [Feature-15248][dolphinscheduler-alert-plugins] add alert plugin aliyun-voice (#15248) 2023-12-13 15:56:37 +08:00
.asf.yaml Disable dependabot pull requests (#14596) 2023-07-20 14:00:09 +08:00
.dlc.json [Feature][DataSource] add oceanbase datasource (#13675) 2023-03-07 10:12:58 +08:00
.flake8 [release] version 3.0.3 (#13131) 2022-12-08 16:57:17 +08:00
.gitattributes Create .gitattributes 2019-03-29 23:51:32 +08:00
.gitignore Remove dao module in worker (#13242) 2022-12-22 12:25:29 +08:00
.gitmodules [Feature][style] Add spotless maven plugin for automatic style fix. (#11272) 2022-08-06 19:29:23 +08:00
.licenserc.yaml Add Terraform deploy manifests for quick setup in AWS (#13252) 2022-12-22 16:55:32 +08:00
.pre-commit-config.yaml [chore] Separate Python API into another repository (#12779) 2022-11-09 21:20:15 +08:00
CONTRIBUTING.md doc: Change readme and standalone docker quick start (#14002) 2023-04-25 09:35:49 +08:00
doap.rdf Add doap.rdf (#14883) 2023-09-09 22:40:36 +08:00
LICENSE [Improve][CI]Upgrade maven-wrapper to 3.1.0 (#8557) 2022-02-27 19:54:23 +08:00
lombok.config add lombok config (#11212) 2022-08-02 11:46:58 +08:00
mvnw [Improve][CI]Upgrade maven-wrapper to 3.1.0 (#8557) 2022-02-27 19:54:23 +08:00
mvnw.cmd [Improve][CI]Upgrade maven-wrapper to 3.1.0 (#8557) 2022-02-27 19:54:23 +08:00
NOTICE [NOTICE]Update the NOTICE year (#14272) 2023-06-06 10:25:33 +08:00
pom.xml Add IT for dolphinscheduler-tools module (#15043) 2023-10-20 15:39:42 +08:00
README_zh_CN.md [document]Modify the inconsistencies of grammar and wording in README_zh_CN.md (#14471) 2023-07-07 18:48:54 +08:00
README.md Add deploy on Terraform on README (#15189) 2023-11-20 16:19:07 +08:00

Apache Dolphinscheduler

License codecov Quality Gate Status Twitter Follow Slack Status CN doc

About

Apache DolphinScheduler is the modern data orchestration platform. Agile to create high performance workflow with low-code. It is also provided powerful user interface, dedicated to solving complex task dependencies in the data pipeline and providing various types of jobs available out of the box

The key features for DolphinScheduler are as follows:

  • Easy to deploy, provide four ways to deploy which including Standalone, Cluster, Docker and Kubernetes.
  • Easy to use, workflow can be created and managed by four ways, which including Web UI, Python SDK, Yaml file and Open API
  • Highly reliable and high availability, decentralized architecture with multi-master and multi-worker, native supports horizontal scaling.
  • High performance, its performance is N times faster than other orchestration platform and it can support tens of millions of tasks per day
  • Cloud Native, DolphinScheduler supports orchestrating multi-cloud/data center workflow, and supports custom task type
  • Versioning both workflow and workflow instance(including tasks)
  • Various state control of workflow and task, support pause/stop/recover them in any time
  • Multi-tenancy support
  • Others like backfill support(Web UI native), permission control including project, resource and data source

QuickStart

User Interface Screenshots

  • Homepage: Project and workflow overview, including the latest workflow instance and task instance status statistics. home

  • Workflow Definition: Create and manage workflow by drag and drop, easy to build and maintain complex workflow, support bulk of tasks out of box. workflow-definition

  • Workflow Tree View: Abstract tree structure could clearer understanding of the relationship between tasks workflow-tree

  • Data source: Manage support multiple external data sources, provide unified data access capabilities for such as MySQL, PostgreSQL, Hive, Trino, etc. data-source

  • Monitor: View the status of the master, worker and database in real time, including server resource usage and load, do quick health check without logging in to the server. monitor

Suggestions & Bug Reports

Follow this guide to report your suggestions or bugs.

Contributing

The community welcomes everyone to contribute, please refer to this page to find out more: How to contribute, find the good first issue in here if you are new to DolphinScheduler.

Community

Welcome to join the Apache DolphinScheduler community by:

Landscapes



  

DolphinScheduler enriches the CNCF CLOUD NATIVE Landscape.