mirror of
https://gitee.com/dolphinscheduler/DolphinScheduler.git
synced 2024-12-02 04:08:31 +08:00
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
This commit is contained in:
parent
5fe9b5c316
commit
0a88528deb
@ -63,8 +63,12 @@ Please refer to the commit message section.
|
||||
|
||||
### Pull Request Code Style
|
||||
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (DolphinScheduler uses `Spotless` to automatically fix code style and formatting errors,)
|
||||
[//]: # (see [Development Environment Setup](../development-environment-setup.md#code-style) `Code Style` section for details.)
|
||||
|
||||
DolphinScheduler uses `Spotless` to automatically fix code style and formatting errors,
|
||||
see [Code Style](../development-environment-setup.md#code-style) for details.
|
||||
see [Development Environment Setup](../development-environment-setup.md) `Code Style` section for details.
|
||||
|
||||
### Question
|
||||
|
||||
|
@ -11,7 +11,10 @@ ChunJun task type for executing ChunJun programs. For ChunJun nodes, the worker
|
||||
|
||||
## Task Parameters
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) `Default Task Parameters` section for default parameters.)
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters
|
||||
|
||||
| **Parameter** | **Description** |
|
||||
|-------------------|---------------------------------------------------------------------------------------------------------------------------|
|
||||
|
@ -9,7 +9,10 @@ Condition is a conditional node, that determines which downstream task should ru
|
||||
|
||||
## Task Parameters
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) `Default Task Parameters` section for default parameters.)
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
|
||||
|
||||
| **Parameter** | **Description** |
|
||||
|------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
|
@ -11,7 +11,10 @@ DataX task type for executing DataX programs. For DataX nodes, the worker will e
|
||||
|
||||
## Task Parameters
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) `Default Task Parameters` section for default parameters.)
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
|
||||
|
||||
| **Parameter** | **Description** |
|
||||
|--------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
|
@ -11,7 +11,10 @@ Dependent nodes are **dependency check nodes**. For example, process A depends o
|
||||
|
||||
## Task Parameter
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) `Default Task Parameters` section for default parameters.)
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
|
||||
|
||||
| **Parameter** | **Description** |
|
||||
|------------------|---------------------------------------------|
|
||||
|
@ -12,7 +12,10 @@ it will call `Dinky API` to trigger dinky task. Click [here](http://www.dlink.to
|
||||
|
||||
## Task Parameter
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) `Default Task Parameters` section for default parameters.)
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
|
||||
|
||||
| **Parameter** | **Description** |
|
||||
|---------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
|
@ -20,7 +20,10 @@ The plugin provides the following three functions:
|
||||
|
||||
## Task Parameters
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) `Default Task Parameters` section for default parameters.)
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
|
||||
|
||||
| **Parameter** | **Description** |
|
||||
|-----------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
|
@ -15,7 +15,10 @@ Using [aws-java-sdk](https://aws.amazon.com/cn/sdk-for-java/) in the background
|
||||
|
||||
## Task Parameters
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) `Default Task Parameters` section for default parameters.)
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
|
||||
|
||||
| **Parameter** | **Description** |
|
||||
|-------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
|
@ -15,7 +15,10 @@ Flink task type, used to execute Flink programs. For Flink nodes:
|
||||
|
||||
## Task Parameters
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) `Default Task Parameters` section for default parameters.)
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
|
||||
|
||||
| **Parameter** | **Description** |
|
||||
|-------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---|
|
||||
|
@ -24,7 +24,10 @@ You could choose between these two based on your needs.
|
||||
|
||||
## Task Parameters
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) `Default Task Parameters` section for default parameters.)
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
|
||||
|
||||
| **Parameter** | **Description** |
|
||||
|------------------------------|------------------------------------------------------------------------------------------------------|
|
||||
|
@ -11,7 +11,10 @@ This node is used to perform http type tasks such as the common POST and GET req
|
||||
|
||||
## Task Parameters
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) `Default Task Parameters` section for default parameters.)
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
|
||||
|
||||
| **Parameter** | **Description** |
|
||||
|----------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------|---|--------------------|---|---|-------------------------|-----------------------------------------------------------------------------------------------|
|
||||
|
@ -10,25 +10,18 @@ This node is for executing java-type tasks and supports using files and jar pack
|
||||
|
||||
# Task Parameters
|
||||
|
||||
| **Parameter** | **Description** |
|
||||
|--------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
| Node Name | The name of the set task. The node name in a workflow definition is unique. |
|
||||
| Run Flag | Indicates whether the node is scheduled properly and turns on the kill switch, if not needed. |
|
||||
| Description | Describes the functionality of the node. |
|
||||
| Task Priority | When the number of worker threads is insufficient, the worker executes tasks according to the priority. When the priority is the same, the worker executes tasks by order. |
|
||||
| Worker Group | The group of machines who execute the tasks. If selecting `Default`, DolphinScheduler will randomly choose a worker machine to execute the task. |
|
||||
| Environment Name | Configure the environment in which the task runs. |
|
||||
| Number Of Failed Retries | Number of resubmitted tasks that failed. You can choose the number in the drop-down menu or fill it manually. |
|
||||
| Failed Retry Interval | the interval between the failure and resubmission of a task. You can choose the number in the drop-down menu or fill it manually. |
|
||||
| Delayed Execution Time | the amount of time a task is delayed, in units. |
|
||||
| Timeout Alarm | Check timeout warning, timeout failure, when the task exceeds the“Timeout length”, send a warning message and the task execution fails. |
|
||||
| Module Path | pick Java 9 + 's modularity feature, put all resources into-module-path, and require that the JDK version in your worker supports modularity. |
|
||||
| Main Parameter | Java program main method entry parameter. |
|
||||
| Java VM Parameters | JVM startup parameters. |
|
||||
| Script | You need to write Java code if you use the Java run type. The public class must exist in the code without writing a package statement. |
|
||||
| Resources | External JAR packages or other resource files that are added to the classpath or module path and can be easily retrieved in your JAVA script. |
|
||||
| Custom parameter | A user-defined parameter that is part of HTTP and replaces `${ variable }` in the script . |
|
||||
| Pre Tasks | Selects a pre-task for the current task and sets the pre-task as the upstream of the current task. |
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) `Default Task Parameters` section for default parameters.)
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
|
||||
|
||||
| **Parameter** | **Description** |
|
||||
|--------------------|-----------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
| Module Path | pick Java 9 + 's modularity feature, put all resources into-module-path, and require that the JDK version in your worker supports modularity. |
|
||||
| Main Parameter | Java program main method entry parameter. |
|
||||
| Java VM Parameters | JVM startup parameters. |
|
||||
| Script | You need to write Java code if you use the Java run type. The public class must exist in the code without writing a package statement. |
|
||||
| Resources | External JAR packages or other resource files that are added to the classpath or module path and can be easily retrieved in your JAVA script. |
|
||||
|
||||
## Example
|
||||
|
||||
|
@ -102,7 +102,10 @@ papermill==2.3.4
|
||||
|
||||
## Task Parameters
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) `Default Task Parameters` section for default parameters.)
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
|
||||
|
||||
| **Parameter** | **Description** |
|
||||
|---------------------------|-------------------------------------------------------------------|
|
||||
|
@ -11,7 +11,10 @@ K8S task type used to execute a batch task. In this task, the worker submits the
|
||||
|
||||
## Task Parameters
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) `Default Task Parameters` section for default parameters.)
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
|
||||
|
||||
| **Parameter** | **Description** |
|
||||
|------------------|------------------------------------------------------------------------------------------------------------------|
|
||||
|
@ -11,7 +11,10 @@ MapReduce(MR) task type used for executing MapReduce programs. For MapReduce nod
|
||||
|
||||
## Task Parameters
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) `Default Task Parameters` section for default parameters.)
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
|
||||
|
||||
### General
|
||||
|
||||
|
@ -28,7 +28,10 @@ The MLflow plugin currently supports and will support the following:
|
||||
|
||||
## Task Parameters and Example
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) `Default Task Parameters` section for default parameters.)
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
|
||||
|
||||
| **Parameter** | **Description** |
|
||||
|----------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
@ -100,6 +103,7 @@ You can now use this feature to run all MLFlow projects on Github (For example [
|
||||
## Environment to Prepare
|
||||
|
||||
### Conda Environment
|
||||
|
||||
Please install [anaconda](https://docs.continuum.io/anaconda/install/) or [miniconda](https://docs.conda.io/en/latest/miniconda.html#installing) in advance.
|
||||
|
||||
**Method A:**
|
||||
@ -113,7 +117,6 @@ Add the following content to the file:
|
||||
export PATH=/opt/anaconda3/bin:$PATH
|
||||
```
|
||||
|
||||
|
||||
**Method B:**
|
||||
|
||||
You need to enter the admin account to configure a conda environment variable.
|
||||
@ -153,3 +156,4 @@ ml.mlflow.preset_repository=https://github.com/apache/dolphinscheduler-mlflow
|
||||
# mlflow task plugin preset repository version
|
||||
ml.mlflow.preset_repository_version="main"
|
||||
```
|
||||
|
||||
|
@ -14,7 +14,10 @@ OpenMLDB task plugin used to execute tasks on OpenMLDB cluster.
|
||||
|
||||
## Task Parameters
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) `Default Task Parameters` section for default parameters.)
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
|
||||
|
||||
| **Parameter** | **Description** |
|
||||
|-------------------|--------------------------------------------------------------------------------------------------------------|
|
||||
|
@ -11,7 +11,10 @@ Pigeon is a task used to trigger remote tasks, acquire logs or status by calling
|
||||
|
||||
## Task Parameters
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) `Default Task Parameters` section for default parameters.)
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
|
||||
|
||||
| **Parameter** | **Description** |
|
||||
|------------------|---------------------------------------|
|
||||
|
@ -12,7 +12,10 @@ it will generate a temporary python script, and executes the script by the Linux
|
||||
|
||||
## Task Parameter
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) `Default Task Parameters` section for default parameters.)
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
|
||||
|
||||
| **Parameter** | **Description** |
|
||||
|-------------------|--------------------------------------------------------------------------------------------------------------|
|
||||
|
@ -19,7 +19,10 @@ The task plugin picture is as follows
|
||||
|
||||
![pytorch](../../../../img/tasks/demo/pytorch_en.png)
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) `Default Task Parameters` section for default parameters.)
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
|
||||
|
||||
#### Run time parameters
|
||||
|
||||
|
@ -20,7 +20,10 @@ DolphinScheduler SageMaker task plugin features are as follows:
|
||||
|
||||
## Task Example
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) `Default Task Parameters` section for default parameters.)
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
|
||||
|
||||
Here are some specific parameters for the SagaMaker plugin:
|
||||
|
||||
|
@ -12,16 +12,19 @@ Click [here](https://seatunnel.apache.org/) for more information about `Apache S
|
||||
|
||||
## Task Parameter
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
|
||||
- Engine: Supports FLINK and SPARK
|
||||
- FLINK
|
||||
- Run model: supports `run` and `run-application` modes
|
||||
- Option parameters: used to add the parameters of the Flink engine, such as `-m yarn-cluster -ynm seatunnel`
|
||||
- SPARK
|
||||
- Deployment mode: specify the deployment mode, `cluster` `client` `local`
|
||||
- Master: Specify the `Master` model, `yarn` `local` `spark` `mesos`, where `spark` and `mesos` need to specify the `Master` service address, for example: 127.0.0.1:7077
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) `Default Task Parameters` section for default parameters.)
|
||||
|
||||
> Click [here](https://seatunnel.apache.org/docs/2.1.2/command/usage) for more information on the usage of `Apache SeaTunnel command`
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
|
||||
- Engine: Supports FLINK and SPARK
|
||||
- FLINK
|
||||
- Run model: supports `run` and `run-application` modes
|
||||
- Option parameters: used to add the parameters of the Flink engine, such as `-m yarn-cluster -ynm seatunnel`
|
||||
- SPARK
|
||||
- Deployment mode: specify the deployment mode, `cluster` `client` `local`
|
||||
- Master: Specify the `Master` model, `yarn` `local` `spark` `mesos`, where `spark` and `mesos` need to specify the `Master` service address, for example: 127.0.0.1:7077
|
||||
|
||||
> Click [here](https://seatunnel.apache.org/docs/2.1.2/command/usage) for more information on the usage of `Apache SeaTunnel command`
|
||||
|
||||
- Custom Configuration: Supports custom configuration or select configuration file from Resource Center
|
||||
|
||||
|
@ -11,7 +11,10 @@ Shell task type, used to create a shell type task and execute a series of shell
|
||||
|
||||
## Task Parameters
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) `Default Task Parameters` section for default parameters.)
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
|
||||
|
||||
| **Parameter** | **Description** |
|
||||
|------------------------|-----------------------------------------------------------------------------------------------------------|
|
||||
|
@ -15,7 +15,10 @@ Spark task type for executing Spark application. When executing the Spark task,
|
||||
|
||||
## Task Parameters
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) `Default Task Parameters` section for default parameters.)
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
|
||||
|
||||
| **Parameter** | **Description** |
|
||||
|----------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
|
@ -15,7 +15,10 @@ Refer to [datasource-setting](../howto/datasource-setting.md) `DataSource Center
|
||||
|
||||
## Task Parameter
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) `Default Task Parameters` section for default parameters.)
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
|
||||
|
||||
| **Parameter** | **Description** |
|
||||
|-------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
|
@ -10,7 +10,10 @@
|
||||
|
||||
## Task Parameters
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) `Default Task Parameters` section for default parameters.)
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
|
||||
|
||||
| **Parameter** | **Description** |
|
||||
|-------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
|
@ -11,7 +11,10 @@ The sub-process node is to execute an external workflow definition as a task nod
|
||||
|
||||
## Task Parameter
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) `Default Task Parameters` section for default parameters.)
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
|
||||
|
||||
| **Parameter** | **Description** |
|
||||
|---------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
|
@ -15,7 +15,10 @@ The switch is a conditional judgment node, decide the branch executes according
|
||||
|
||||
## Task Parameters
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) `Default Task Parameters` section for default parameters.)
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
|
||||
|
||||
| **Parameter** | **Description** |
|
||||
|---------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
|
@ -12,7 +12,10 @@ it will call `Zeppelin Client API` to trigger zeppelin notebook paragraph. Click
|
||||
|
||||
## Task Parameters
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) `Default Task Parameters` section for default parameters.)
|
||||
|
||||
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
|
||||
|
||||
| **Parameter** | **Description** |
|
||||
|------------------------------------|------------------------------------------------------------------------------------------------------------------------------------|
|
||||
|
@ -65,8 +65,12 @@ waste time。
|
||||
|
||||
### Pull Request Code Style
|
||||
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (DolphinScheduler使用`Spotless`为您自动修复代码风格和格式问题,)
|
||||
[//]: # (详情见[开发手册](../development-environment-setup.md#代码风格)`代码风格`一栏。)
|
||||
|
||||
DolphinScheduler使用`Spotless`为您自动修复代码风格和格式问题,
|
||||
详情见[代码风格](../development-environment-setup.md#代码风格)。
|
||||
详情见[开发手册](../development-environment-setup.md)`代码风格`一栏。
|
||||
|
||||
### 相关问题
|
||||
|
||||
|
@ -11,7 +11,10 @@ ChunJun 任务类型,用于执行 ChunJun 程序。对于 ChunJun 节点,wor
|
||||
|
||||
## 任务参数
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)`默认任务参数`一栏。)
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
|
||||
|
||||
| **任务参数** | **描述** |
|
||||
|----------|------------------------------------------------------------|
|
||||
|
@ -9,7 +9,10 @@ Conditions 是一个条件节点,根据上游任务运行状态,判断应该
|
||||
|
||||
## 任务参数
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)`默认任务参数`一栏。)
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
|
||||
|
||||
| **任务参数** | **描述** |
|
||||
|----------|---------------------------------------------------------------------------------------------------------------------------------------|
|
||||
|
@ -11,7 +11,10 @@ DataX 任务类型,用于执行 DataX 程序。对于 DataX 节点,worker
|
||||
|
||||
## 任务参数
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)`默认任务参数`一栏。)
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
|
||||
|
||||
| **任务参数** | **描述** |
|
||||
|----------|-------------------------------------------------------------------------------------------------------|
|
||||
|
@ -11,7 +11,10 @@ Dependent 节点,就是**依赖检查节点**。比如 A 流程依赖昨天的
|
||||
|
||||
## 任务参数
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)`默认任务参数`一栏。)
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
|
||||
- 此任务除上述链接中的默认参数外无其他参数。
|
||||
|
||||
## 任务样例
|
||||
|
@ -12,7 +12,10 @@
|
||||
|
||||
## Task Parameter
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)`默认任务参数`一栏。)
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
|
||||
|
||||
| **任务参数** | **描述** |
|
||||
|-------------|---------------------------------------------------------------------|
|
||||
|
@ -17,7 +17,10 @@ DVC 组件用于在DS上使用DVC的数据版本管理功能,帮助用户简
|
||||
|
||||
## 任务样例
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)`默认任务参数`一栏。)
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
|
||||
|
||||
| **任务参数** | **描述** |
|
||||
|----------|-------------------------------|
|
||||
|
@ -10,7 +10,10 @@ Amazon EMR 任务类型,用于在AWS上操作EMR集群并执行计算任务。
|
||||
|
||||
## 任务参数
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)`默认任务参数`一栏。)
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
|
||||
|
||||
| **任务参数** | **描述** |
|
||||
|-------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
|
@ -15,7 +15,10 @@ Flink 任务类型,用于执行 Flink 程序。对于 Flink 节点:
|
||||
|
||||
## 任务参数
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)`默认任务参数`一栏。)
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
|
||||
|
||||
| **任务参数** | **描述** |
|
||||
|-----------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
|
@ -22,7 +22,10 @@
|
||||
|
||||
## 任务参数
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)`默认任务参数`一栏。)
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
|
||||
|
||||
| **任务参数** | **描述** |
|
||||
|---------------|-----------------------------------------------------|
|
||||
|
@ -12,7 +12,10 @@
|
||||
|
||||
## 任务参数
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)`默认任务参数`一栏。)
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
|
||||
|
||||
| **任务参数** | **描述** |
|
||||
|----------|-------------------------------------|
|
||||
|
@ -12,7 +12,10 @@
|
||||
|
||||
## 任务参数
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)`默认任务参数`一栏。)
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
|
||||
|
||||
| **任务参数** | **描述** |
|
||||
|----------|---------------------------------------------------------------|
|
||||
|
@ -101,7 +101,10 @@ papermill==2.3.4
|
||||
|
||||
## 任务参数
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)`默认任务参数`一栏。)
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
|
||||
|
||||
| **任务参数** | **描述** |
|
||||
|---------------------------|------------------------------------|
|
||||
|
@ -11,7 +11,10 @@ kubernetes任务类型,用于在kubernetes上执行一个短时和批处理的
|
||||
|
||||
## 任务参数
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)`默认任务参数`一栏。)
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
|
||||
|
||||
| **任务参数** | **描述** |
|
||||
|----------|-----------------------------------------------------------------|
|
||||
|
@ -11,7 +11,10 @@ MapReduce(MR) 任务类型,用于执行 MapReduce 程序。对于 MapReduce
|
||||
|
||||
## 任务参数
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)`默认任务参数`一栏。)
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
|
||||
|
||||
### JAVA/SCALA 程序
|
||||
|
||||
|
@ -27,7 +27,10 @@ MLflow 组件用于执行 MLflow 任务,目前包含Mlflow Projects,和MLflo
|
||||
|
||||
## 任务样例
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)`默认任务参数`一栏。)
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
|
||||
|
||||
以下是一些MLflow 组件的常用参数
|
||||
|
||||
@ -100,6 +103,7 @@ MLflow 组件用于执行 MLflow 任务,目前包含Mlflow Projects,和MLflo
|
||||
配置文件:/dolphinscheduler/conf/env/dolphinscheduler_env.sh。
|
||||
|
||||
在文件最后添加内容
|
||||
|
||||
```
|
||||
# 配置你的conda环境路径
|
||||
export PATH=/opt/anaconda3/bin:$PATH
|
||||
@ -115,7 +119,6 @@ export PATH=/opt/anaconda3/bin:$PATH
|
||||
|
||||
![mlflow-set-conda-env](../../../../img/tasks/demo/mlflow-set-conda-env.png)
|
||||
|
||||
|
||||
### MLflow service 启动
|
||||
|
||||
确保你已经安装MLflow,可以使用`pip install mlflow`进行安装。
|
||||
@ -134,7 +137,6 @@ mlflow server -h 0.0.0.0 -p 5000 --serve-artifacts --backend-store-uri sqlite://
|
||||
|
||||
![mlflow-server](../../../../img/tasks/demo/mlflow-server.png)
|
||||
|
||||
|
||||
### 内置算法仓库配置
|
||||
|
||||
如果遇到github无法访问的情况,可以修改`commom.properties`配置文件的以下字段,将github地址替换能访问的地址。
|
||||
|
@ -13,7 +13,10 @@ OpenMLDB任务组件可以连接OpenMLDB集群执行任务。
|
||||
|
||||
## 任务样例
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)`默认任务参数`一栏。)
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
|
||||
|
||||
| **任务参数** | **描述** |
|
||||
|-------------|--------------------------------------------------|
|
||||
|
@ -8,7 +8,10 @@ Pigeon任务类型是通过调用远程websocket服务,实现远程任务的
|
||||
|
||||
## 任务参数
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)`默认任务参数`一栏。)
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
|
||||
|
||||
| **任务参数** | **描述** |
|
||||
|----------|-------------------|
|
||||
|
@ -12,7 +12,10 @@ Python 任务类型,用于创建 Python 类型的任务并执行一系列的 P
|
||||
|
||||
## 任务参数
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)`默认任务参数`一栏。)
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
|
||||
|
||||
| **任务参数** | **描述** |
|
||||
|----------|-----------------------------------|
|
||||
|
@ -21,7 +21,10 @@
|
||||
|
||||
### Pytorch参数
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)`默认任务参数`一栏。)
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
|
||||
|
||||
#### 运行参数
|
||||
|
||||
|
@ -18,7 +18,10 @@ DolphinScheduler SageMaker 组件的功能:
|
||||
|
||||
## 任务样例
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)`默认任务参数`一栏。)
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
|
||||
|
||||
| **任务参数** | **描述** |
|
||||
|----------------------|-------------------------------------------------------------------------------------------------------------------------------------|
|
||||
|
@ -12,16 +12,19 @@
|
||||
|
||||
## 任务参数
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
|
||||
- 引擎:支持 FLINK 和 SPARK
|
||||
- FLINK
|
||||
- 运行模型:支持 `run` 和 `run-application` 两种模式
|
||||
- 选项参数:用于添加 Flink 引擎本身参数,例如 `-m yarn-cluster -ynm seatunnel`
|
||||
- SPARK
|
||||
- 部署方式:指定部署模式,`cluster` `client` `local`
|
||||
- Master:指定 `Master` 模型,`yarn` `local` `spark` `mesos`,其中 `spark` 和 `mesos` 需要指定 `Master` 服务地址,例如:127.0.0.1:7077
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)`默认任务参数`一栏。)
|
||||
|
||||
> 点击 [这里](https://seatunnel.apache.org/docs/2.1.2/command/usage) 获取更多关于`Apache SeaTunnel command` 使用的信息
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
|
||||
- 引擎:支持 FLINK 和 SPARK
|
||||
- FLINK
|
||||
- 运行模型:支持 `run` 和 `run-application` 两种模式
|
||||
- 选项参数:用于添加 Flink 引擎本身参数,例如 `-m yarn-cluster -ynm seatunnel`
|
||||
- SPARK
|
||||
- 部署方式:指定部署模式,`cluster` `client` `local`
|
||||
- Master:指定 `Master` 模型,`yarn` `local` `spark` `mesos`,其中 `spark` 和 `mesos` 需要指定 `Master` 服务地址,例如:127.0.0.1:7077
|
||||
|
||||
> 点击 [这里](https://seatunnel.apache.org/docs/2.1.2/command/usage) 获取更多关于`Apache SeaTunnel command` 使用的信息
|
||||
|
||||
- 自定义配置:支持自定义配置或从资源中心选择配置文件
|
||||
|
||||
|
@ -11,7 +11,10 @@ Shell 任务类型,用于创建 Shell 类型的任务并执行一系列的 She
|
||||
|
||||
## 任务参数
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)`默认任务参数`一栏。)
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
|
||||
- 除上述默认参数,此任务没有其他参数
|
||||
|
||||
## 任务样例
|
||||
|
@ -16,7 +16,10 @@ Spark 任务类型用于执行 Spark 应用。对于 Spark 节点,worker 支
|
||||
|
||||
## 任务参数
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)`默认任务参数`一栏。)
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
|
||||
- 程序类型:支持 Java、Scala、Python 和 SQL 四种语言。
|
||||
- Spark 版本:支持 Spark1 和 Spark2。
|
||||
- 主函数的 Class:Spark 程序的入口 Main class 的全路径。
|
||||
|
@ -15,14 +15,15 @@ SQL任务类型,用于连接数据库并执行相应SQL。
|
||||
|
||||
## 任务参数
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)`默认任务参数`一栏。)
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
|
||||
- 数据源:选择对应的数据源
|
||||
- sql类型:支持查询和非查询两种。
|
||||
- 查询:支持 `DML select` 类型的命令,是有结果集返回的,可以指定邮件通知为表格、附件或表格附件三种模板;
|
||||
- 非查询:支持 `DDL`全部命令 和 `DML update、delete、insert` 三种类型的命令;
|
||||
- 分段执行符号:提供在数据源不支持一次执行多段SQL语句时,拆分SQL语句的符号来进行多次调用数据源执行方法。
|
||||
例子:1.当数据源选择Hive数据源时,不需要填写此参数。因为Hive数据源本身支持一次执行多段SQL语句;
|
||||
2.当数据源选择MySQL数据源时,并且要执行多段SQL语句时,需要填写此参数为分号 `;`。因为MySQL数据源不支持一次执行多段SQL语句;
|
||||
- 查询:支持 `DML select` 类型的命令,是有结果集返回的,可以指定邮件通知为表格、附件或表格附件三种模板;
|
||||
- 非查询:支持 `DDL`全部命令 和 `DML update、delete、insert` 三种类型的命令;
|
||||
- 默认采用`;\n`作为SQL分隔符,拆分成多段SQL语句执行。Hive支持一次执行多段SQL语句,故不会拆分。
|
||||
- sql参数:输入参数格式为key1=value1;key2=value2…
|
||||
- sql语句:SQL语句
|
||||
- UDF函数:对于HIVE类型的数据源,可以引用资源中心中创建的UDF函数,其他类型的数据源暂不支持UDF函数。
|
||||
|
@ -19,7 +19,10 @@ END
|
||||
|
||||
## 任务参数
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)`默认任务参数`一栏。)
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
|
||||
|
||||
| **任务参数** | **描述** |
|
||||
|---------------|--------------------------------------------------------------------------------------------------|
|
||||
|
@ -12,7 +12,10 @@
|
||||
|
||||
## 任务参数
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)`默认任务参数`一栏。)
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
|
||||
|
||||
| **任务参数** | **描述** |
|
||||
|----------|----------------------------------------|
|
||||
|
@ -11,7 +11,10 @@ Switch 是一个条件判断节点,依据[全局变量](../parameter/global.md
|
||||
|
||||
## 任务参数
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)`默认任务参数`一栏。)
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
|
||||
|
||||
| **任务参数** | **描述** |
|
||||
|----------|-------------------------------------------------------------------------|
|
||||
|
@ -12,7 +12,10 @@
|
||||
|
||||
## 任务参数
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
|
||||
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
|
||||
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)`默认任务参数`一栏。)
|
||||
|
||||
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
|
||||
|
||||
| **任务参数** | **描述** |
|
||||
|------------------------------------|---------------------------------------------------|
|
||||
|
Loading…
Reference in New Issue
Block a user