[fix] Python task can not pass the parameters to downstream task. (#12961)

This commit is contained in:
JieguangZhou 2022-11-22 16:11:45 +08:00 committed by zhuangchong
parent a8f86de597
commit 3802599279
4 changed files with 18 additions and 0 deletions

View File

@ -13,6 +13,7 @@ DolphinScheduler allows parameter transfer between tasks. Currently, transfer di
* [Shell](../task/shell.md)
* [SQL](../task/sql.md)
* [Procedure](../task/stored-procedure.md)
* [Python](../task/python.md)
When defining an upstream node, if there is a need to transmit the result of that node to a dependency related downstream node. You need to set an `OUT` direction parameter to [Custom Parameters] of the [Current Node Settings]. At present, we mainly focus on the SQL and shell nodes to pass parameters downstream.
@ -76,3 +77,11 @@ The result of Node_mysql is as follows:
Even though output is assigned a value of 1 in Node_A's script, the log still shows a value of 100. But according to the principle from [parameter priority](priority.md): `Local Parameter > Parameter Context > Global Parameter`, the output value in Node_B is 1. It proves that the output parameter is passed in the workflow with reference to the expected value, and the query operation is completed using this value in Node_mysql.
But the output value 66 only shows in the Node_A, the reason is that the direction of value is selected as IN, and only when the direction is OUT will it be defined as a variable output.
#### Pass parameter from Python task to downstream
Use `print('${setValue(key=%s)}' % value)`, DolphinScheduler will capture the `${setValue(key=value}` in the output to capture the parameters and pass them downstream.
For example
![img.png](../../../../img/new_ui/dev/parameter/python_context_param.png)

View File

@ -13,6 +13,7 @@ DolphinScheduler 允许在任务间进行参数传递,目前传递方向仅支
* [Shell](../task/shell.md)
* [SQL](../task/sql.md)
* [Procedure](../task/stored-procedure.md)
* [Python](../task/python.md)
当定义上游节点时,如果有需要将该节点的结果传递给有依赖关系的下游节点,需要在【当前节点设置】的【自定义参数】设置一个方向是 OUT 的变量。目前我们主要针对 SQL 和 SHELL 节点做了可以向下传递参数的功能。
@ -76,3 +77,10 @@ Node_mysql 运行结果如下:
虽然在 Node_A 的脚本中为 output 赋值为 1但日志中显示的值仍然为 100。但根据[参数优先级](priority.md)的原则:`本地参数 > 上游任务传递的参数 > 全局参数`,在 Node_B 中输出的值为 1。则证明 output 参数参照预期的值在该工作流中传递,并在 Node_mysql 中使用该值完成查询操作。
但是 value 的值却只有在 Node_A 中输出为 66其原因为 value 的方向选择为 IN只有当方向为 OUT 时才会被定义为变量输出。
#### Python 任务传递参数
使用 `print('${setValue(key=%s)}' % value)`DolphinScheduler会捕捉输出中的 `${setValue(key=value}`来进行参数捕捉,从而传递到下游
![img.png](../../../../img/new_ui/dev/parameter/python_context_param.png)

Binary file not shown.

After

Width:  |  Height:  |  Size: 38 KiB

View File

@ -114,6 +114,7 @@ public class PythonTask extends AbstractTask {
setExitStatusCode(taskResponse.getExitStatusCode());
setProcessId(taskResponse.getProcessId());
setVarPool(shellCommandExecutor.getVarPool());
pythonParameters.dealOutParam(shellCommandExecutor.getVarPool());
} catch (Exception e) {
logger.error("python task failure", e);
setExitStatusCode(TaskConstants.EXIT_CODE_FAILURE);