mirror of
https://gitee.com/dolphinscheduler/DolphinScheduler.git
synced 2024-11-30 11:17:54 +08:00
[python] Add missing doc about config and connect remote server (#9443)
which includes `configuration`, `run example` `how to connect remote server` close: #9286, #9284, #8917
This commit is contained in:
parent
62284ae689
commit
aaf2042ec4
@ -0,0 +1,96 @@
|
||||
.. Licensed to the Apache Software Foundation (ASF) under one
|
||||
or more contributor license agreements. See the NOTICE file
|
||||
distributed with this work for additional information
|
||||
regarding copyright ownership. The ASF licenses this file
|
||||
to you under the Apache License, Version 2.0 (the
|
||||
"License"); you may not use this file except in compliance
|
||||
with the License. You may obtain a copy of the License at
|
||||
|
||||
.. http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
.. Unless required by applicable law or agreed to in writing,
|
||||
software distributed under the License is distributed on an
|
||||
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
KIND, either express or implied. See the License for the
|
||||
specific language governing permissions and limitations
|
||||
under the License.
|
||||
|
||||
Configuration
|
||||
=============
|
||||
|
||||
Export Configuration File
|
||||
-------------------------
|
||||
|
||||
pydolphinscheduler allows you to change the built-in configurations via CLI or editor you like. pydolphinscheduler
|
||||
integrated built-in configurations in its package, but you could also export it locally by CLI
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ pydolphinscheduler config --init
|
||||
|
||||
And it will create a new YAML file in the path `~/pydolphinscheduler/config.yaml` by default. If you want to export
|
||||
it to another path, you should set `PYDOLPHINSCHEDULER_HOME` before you run command :code:`pydolphinscheduler config --init`.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ export PYDOLPHINSCHEDULER_HOME=<CUSTOM_PATH>
|
||||
$ pydolphinscheduler config --init
|
||||
|
||||
After that, your configuration file will export into `<CUSTOM_PATH>/config.yaml` instead of the default path.
|
||||
|
||||
Change Configuration
|
||||
--------------------
|
||||
|
||||
In section `export configuration file`_ you export the configuration file locally, and as a local file, you could
|
||||
edit it with any editor you like. After you save your change in your editor, the latest configuration will work
|
||||
when you run your workflow code.
|
||||
|
||||
You could also query or change the configuration via CLI :code:`config --get <config>` or :code:`config --get <config> <val>`.
|
||||
Both `--get` and `--set` could be call one or more times in single command, and you could only set the leaf
|
||||
node of the configuration but could get the parent configuration, there are simple examples below:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
# Get single configuration in the leaf node
|
||||
$ pydolphinscheduler config --get java_gateway.address
|
||||
The configuration query as below:
|
||||
|
||||
java_gateway.address = 127.0.0.1
|
||||
|
||||
# Get multiple configuration in the leaf node
|
||||
$ pydolphinscheduler config --get java_gateway.address --get java_gateway.port
|
||||
The configuration query as below:
|
||||
|
||||
java_gateway.address = 127.0.0.1
|
||||
java_gateway.port = 25333
|
||||
|
||||
# Get parent configuration which contain multiple leaf nodes
|
||||
$ pydolphinscheduler config --get java_gateway
|
||||
The configuration query as below:
|
||||
|
||||
java_gateway = ordereddict([('address', '127.0.0.1'), ('port', 25333), ('auto_convert', True)])
|
||||
|
||||
# Set single configuration
|
||||
$ pydolphinscheduler config --set java_gateway.address 192.168.1.1
|
||||
Set configuration done.
|
||||
|
||||
# Set multiple configuration
|
||||
$ pydolphinscheduler config --set java_gateway.address 192.168.1.1 --set java_gateway.port 25334
|
||||
Set configuration done.
|
||||
|
||||
# Set configuration not in leaf node will fail
|
||||
$ pydolphinscheduler config --set java_gateway 192.168.1.1,25334,True
|
||||
Raise error.
|
||||
|
||||
For more information about our CLI, you could see document :doc:`cli`.
|
||||
|
||||
All Configurations
|
||||
------------------
|
||||
|
||||
Here are all our configurations for pydolphinscheduler.
|
||||
|
||||
.. literalinclude:: ../../src/pydolphinscheduler/core/default_config.yaml
|
||||
:language: yaml
|
||||
:lines: 18-
|
||||
|
||||
|
@ -0,0 +1,26 @@
|
||||
.. Licensed to the Apache Software Foundation (ASF) under one
|
||||
or more contributor license agreements. See the NOTICE file
|
||||
distributed with this work for additional information
|
||||
regarding copyright ownership. The ASF licenses this file
|
||||
to you under the Apache License, Version 2.0 (the
|
||||
"License"); you may not use this file except in compliance
|
||||
with the License. You may obtain a copy of the License at
|
||||
|
||||
.. http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
.. Unless required by applicable law or agreed to in writing,
|
||||
software distributed under the License is distributed on an
|
||||
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
KIND, either express or implied. See the License for the
|
||||
specific language governing permissions and limitations
|
||||
under the License.
|
||||
|
||||
How To
|
||||
======
|
||||
|
||||
In this section
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
remote-submit
|
@ -0,0 +1,51 @@
|
||||
.. Licensed to the Apache Software Foundation (ASF) under one
|
||||
or more contributor license agreements. See the NOTICE file
|
||||
distributed with this work for additional information
|
||||
regarding copyright ownership. The ASF licenses this file
|
||||
to you under the Apache License, Version 2.0 (the
|
||||
"License"); you may not use this file except in compliance
|
||||
with the License. You may obtain a copy of the License at
|
||||
|
||||
.. http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
.. Unless required by applicable law or agreed to in writing,
|
||||
software distributed under the License is distributed on an
|
||||
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
KIND, either express or implied. See the License for the
|
||||
specific language governing permissions and limitations
|
||||
under the License.
|
||||
|
||||
Submit Your Code from Different machine
|
||||
=======================================
|
||||
|
||||
Generally, we use pydolphinscheduler as a client to DolphinScheduler, and consider we may change our workflow
|
||||
code frequently, the best practice is running :ref:`python gateway service <start:start python gateway service>`
|
||||
in your server machine and submit the workflow code from your development machine, like a laptop or PC. This behavior
|
||||
is supported by pydolphinscheduler out of box with one or two single command lines.
|
||||
|
||||
Export Configuration File
|
||||
-------------------------
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ pydolphinscheduler config --init
|
||||
|
||||
your could find more detail in :ref:`configuration exporting <config:export configuration file>`
|
||||
|
||||
Run API Server in Other Host
|
||||
----------------------------
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ pydolphinscheduler config --set java_gateway.address <your-api-server-ip-or-hostname>
|
||||
|
||||
your could find more detail in :ref:`configuration setting <config:change configuration>`
|
||||
|
||||
Run API Server in Other Port
|
||||
----------------------------
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ pydolphinscheduler config --set java_gateway.port <your-python-gateway-service-port>
|
||||
|
||||
your could find more detail in :ref:`configuration setting <config:change configuration>`
|
@ -32,7 +32,9 @@ then go and see :doc:`tutorial` for more detail.
|
||||
tutorial
|
||||
concept
|
||||
tasks/index
|
||||
howto/index
|
||||
cli
|
||||
config
|
||||
api
|
||||
|
||||
Indices and tables
|
||||
|
@ -107,6 +107,37 @@ the server is health if keyword `ApiApplicationServer` in the console.
|
||||
yaml config path `python-gateway.enabled : true` in api-server's configuration path in `api-server/conf/application.yaml`.
|
||||
The default value is true and Python gateway service start when api server is been started.
|
||||
|
||||
Run an Example
|
||||
--------------
|
||||
|
||||
Before run an example for pydolphinscheduler, you should get the example code from it source code. You could run
|
||||
single bash command to get it
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ wget https://raw.githubusercontent.com/apache/dolphinscheduler/dev/dolphinscheduler-python/pydolphinscheduler/src/pydolphinscheduler/examples/tutorial.py
|
||||
|
||||
or you could copy-paste the content from `tutorial source code`_. And then you could run the example in your
|
||||
terminal
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ python tutorial.py
|
||||
|
||||
If you want to submit your workflow to a remote API server, which means that your workflow script is different
|
||||
from the API server, you should first change pydolphinscheduler configuration and then submit the workflow script
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ pydolphinscheduler config --init
|
||||
$ pydolphinscheduler config --set java_gateway.address <your-api-server-ip-or-hostname>
|
||||
$ python tutorial.py
|
||||
|
||||
.. note::
|
||||
|
||||
You could see more information in :doc:`config` about all the configurations pydolphinscheduler supported.
|
||||
|
||||
|
||||
What's More
|
||||
-----------
|
||||
|
||||
@ -117,3 +148,4 @@ maybe you could go and play with all :doc:`tasks/index` *PyDolphinScheduler* sup
|
||||
.. _`instructions for all platforms here`: https://wiki.python.org/moin/BeginnersGuide/Download
|
||||
.. _`Apache DolphinScheduler`: https://dolphinscheduler.apache.org
|
||||
.. _`install Apache DolphinScheduler`: https://dolphinscheduler.apache.org/en-us/docs/latest/user_doc/guide/installation/standalone.html
|
||||
.. _`tutorial source code`: https://raw.githubusercontent.com/apache/dolphinscheduler/dev/dolphinscheduler-python/pydolphinscheduler/src/pydolphinscheduler/examples/tutorial.py
|
||||
|
Loading…
Reference in New Issue
Block a user