Commit Graph

116 Commits

Author SHA1 Message Date
baoliang
3434661452 merge from 1.3.3-release 2020-11-11 10:29:51 +08:00
bao liang
7239979167
Merge branch 'dev' into 133-merge-dev 2020-11-09 10:08:19 +08:00
baoliang
381d23ea4f merge from 1.3.3-release 2020-11-06 00:03:20 +08:00
JinyLeeChina
2830ab1d4d
fix #3688, Optimization of table building statements (#3689) 2020-11-03 21:10:50 +08:00
IamMujuziMoses
c7b5956c5a
[Feature-3941][ui] Remove tenant name (#3949)
* FeatureUI-3941: Removed Tenant

* FeatureUI-3941: Removed tenantName in the suggested pages

* FeatureUI-3941: Updated the suggested files

* FeatureUI-3941: Updated the TenantController

* FeatureUI-3941: Updated the TenantController and the pom.xml

* FeatureUI-3941: Updated the TenantControllerTest
2020-10-30 12:18:00 +08:00
wulingqi
cd3dff5f90 fix import dolphinscheduler_mysql.sql insert admin user data 2020-10-26 22:13:49 +08:00
lgcareer
dbc9f149b5 [Fix-#3487][sql] update uc_dolphin_T_t_ds_resources_un 2020-10-15 17:52:45 +08:00
lgcareer
301d728c7b
[Fix-#3487][sql] add dolphinscheduler_dml.sql under 1.3.3_schema (#3907) 2020-10-14 16:28:53 +08:00
yangruochen
7c1ff53a74
[Bug-3845]postgresql need to add IF NOT EXISTS for table t_ds_process… (#3852)
* [Bug-3845]postgresql need to add IF NOT EXISTS for table t_ds_process_definition_version too.

* restart unit test

* rerun unit test
2020-10-08 17:23:40 +08:00
yangruochen
42c88a7c68
[FIX-Bug #3845][Ambari Plugin] Start Ambari report an error: Table 't_ds_process_definition_version' already exists (#3846)
* [Bug][Ambari Plugin] Start Ambari report an error: Table 't_ds_process_definition_version' already exists #3845

* Revert "[Bug][Ambari Plugin] Start Ambari report an error: Table 't_ds_process_definition_version' already exists #3845"

This reverts commit 56e030e6

* [Bug][Ambari Plugin] Start Ambari report an error: Table 't_ds_process_definition_version' already exists #3845
2020-09-28 18:18:52 +08:00
guyinyou
3b581455fc
[Feature]Add Python task "task variable / result transfer" implementation (#3659)
* 增加Python Task的“任务变量/结果传递”实现
Signed-off-by: 古崟佑

* add two files license
Signed-off-by: 古崟佑

* fix 'server/worker/task/AbstractCommandExecutor.java' code style
Signed-off-by: 1941815847Cy4 <1941815847cy4@kuaishou.com>

* update DB
Signed-off-by: 古崟佑

* update DB -- 2
Signed-off-by: 古崟佑

* fix codeStyle
Signed-off-by: 古崟佑

* fix codestyle
Signed-off-by: 古崟佑

* fix codeStyle
Signed-off-by: 古崟佑

* fix codeStyle
Signed-off-by: 古崟佑

* fix codeStyle
Signed-off-by: 古崟佑

* add VarPoolUtils Test
Signed-off-by: 古崟佑

* fix VarPoolUtilsTest codeStyle
Signed-off-by: 古崟佑

* fix VarPoolUtilsTest codeStyle
Signed-off-by: 古崟佑

* fix VarPoolUtilsTest codeStyle
Signed-off-by: 古崟佑

* fix VarPoolUtilsTest codeStyle
Signed-off-by: 古崟佑

* fix VarPoolUtilsTest codeStyle
Signed-off-by: 古崟佑

* add test config for VarPoolUtilsTest
Signed-off-by: 古崟佑

* fix unit test
Signed-off-by: 古崟佑

* fix codeStyle
Signed-off-by: 古崟佑

* fix VarPoolUtilsTest.java
Signed-off-by: 古崟佑

* fix
Signed-off-by: 古崟佑

* change the test class path
Signed-off-by: 古崟佑

* fix
Signed-off-by: 古崟佑

* fix "print the error message"
Signed-off-by: 古崟佑

* fix bug
Signed-off-by: 古崟佑

* fix
Signed-off-by: 古崟佑

Co-authored-by: 1941815847Cy4 <1941815847cy4@kuaishou.com>
2020-09-25 13:20:42 +08:00
muzhongjiang
d6f62f1de4 [Fix-#3487][api、dao] cherry pick from dev to fix that create folder duplicate name under multithreading 2020-09-24 16:16:49 +08:00
lgcareer
1c07892fea
[Fix-3238][docker]Fix that can not create folder in docker with standalone mode (#3741) 2020-09-17 11:36:25 +08:00
muzhongjiang
c42e769ced
[Fix] fix bug #3487 Create folder duplicate name under multithreading (#3697)
* add state

* fixed bug "jackson enum conversion  : InvalidFormatException"

* Word spelling modification
Comment modification
Word spelling modification,Comment modification,Log level modification

* Update EmailManager.java

* Update FlinkParameters.java

* Update SqlTask.java

* fixed  "getNotifyGroupList cache"  bug

* fix bug "Creating folders with multiple threads will result in multiple identical folders #3487"

* fix "Creating folders with multiple threads will result in multiple identical folders" #3487

* fix "Creating folders with multiple threads will result in multiple identical folders" #3487

* fix bug  #3487   Create folder duplicate name under multithreading

* fix bug  #3487   Create folder duplicate name under multithreading

* fix bug  #3487   Create folder duplicate name under multithreading

* fix bug  #3487   Create folder duplicate name under multithreading

Co-authored-by: mzjnumber1@163.com <mzjnumber1@163.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
2020-09-09 14:13:29 +08:00
Yichao Yang
93660f4d61
[Feature-2154][api] Workflow version control (#3053)
* [Feature][api] Workflow version control

* Update messages.properties

* Update messages_en_US.properties

* Update messages_zh_CN.properties
2020-08-25 10:46:58 +08:00
vanilla111
a4ee351a3a
delay execution of tasks and improve some designs (#3427) 2020-08-19 11:36:30 +08:00
lgcareer
efef631b68
[Fix-3423][dao][sql]Fixed that the resource file of the task node can't be found when upgrade from 1.2.0 to 1.3.x (#3454) 2020-08-10 19:05:05 +08:00
lgcareer
0ef75493c4
[Fix-3256][ui] Fix admin user info update error (#3425) (#3428)
* [PROPOSAL-3139] Datasource selection changes from radio to select

* [PROPOSAL-3139] Datasource selection changes from radio to select

* [BUG FIX] issues #3256

* [BUG FIX] issues #3256

* [BUG FIX] issues #3256

* [Fix-3256][ui] Fix admin user info update error

* [Fix-3256][ui] Fix admin user info update error

* [Fix-3256][ui] Fix admin user info update error

* [Fix-3256][ui] Fix admin user info update error

* reset createUser.vue

* [Fix-3256][ui] Fix admin user info update error

Co-authored-by: dailidong <dailidong66@gmail.com>

Co-authored-by: wuchunfu <319355703@qq.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
2020-08-07 14:54:07 +08:00
wuchunfu
f784ce504f
[Fix-3256][ui] Fix admin user info update error (#3425)
* [PROPOSAL-3139] Datasource selection changes from radio to select

* [PROPOSAL-3139] Datasource selection changes from radio to select

* [BUG FIX] issues #3256

* [BUG FIX] issues #3256

* [BUG FIX] issues #3256

* [Fix-3256][ui] Fix admin user info update error

* [Fix-3256][ui] Fix admin user info update error

* [Fix-3256][ui] Fix admin user info update error

* [Fix-3256][ui] Fix admin user info update error

* reset createUser.vue

* [Fix-3256][ui] Fix admin user info update error

Co-authored-by: dailidong <dailidong66@gmail.com>
2020-08-07 10:20:56 +08:00
lgcareer
a3793be7da
[Fix-3390][server]Running hive sql task need find the hdfs path correctly (#3396)
* [Fix-3390][api]Running hive sql task need find the hdfs path correctly

* [Fix-3390][api]Running hive sql task need find the hdfs path correctly

* update soft version
2020-08-04 15:04:56 +08:00
break60
811fd48f0a
Before creating a workflow, clear the canvas (#3388)
* Before creating a workflow, clear the canvas

* [Fix-3256][ui] herry pick commit from dev for Fix admin user info update error (#3306)

Co-authored-by: wuchunfu <319355703@qq.com>
2020-08-04 02:22:00 +00:00
wuchunfu
27b296c45b
[Fix-3256][ui] Fix admin user info update error (#3306)
* [PROPOSAL-3139] Datasource selection changes from radio to select

* [PROPOSAL-3139] Datasource selection changes from radio to select

* [BUG FIX] issues #3256

* [BUG FIX] issues #3256

* [BUG FIX] issues #3256

* [Fix-3256][ui] Fix admin user info update error

* [Fix-3256][ui] Fix admin user info update error

* [Fix-3256][ui] Fix admin user info update error

* [Fix-3256][ui] Fix admin user info update error

* reset createUser.vue

Co-authored-by: dailidong <dailidong66@gmail.com>
2020-08-03 13:50:30 +08:00
Yichao Yang
de819926e9 [Feature-2127][sql-schema] Fix the mysql version limited error (#2127) (#2915) 2020-07-23 15:26:25 +08:00
Rubik-W
a7aa58e6ca
Feature add user state (#2710)
* feature: #1923

* fix:docker sql script add column

* fix: update sql script

* fix: UT bug

* fix: UT bug

* fix: UT bug

* fix: e2e UT bug

* fix: e2e button location

* fix:sql init script to add t_ds_user.state column

Co-authored-by: Rubik-W <whm_777@163.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>
2020-07-15 11:21:38 +08:00
bao liang
657ea4752f
merge from 1.3.1-release (#3161)
* merge from 1.3.1-release

* [bugfix] remove Conflicting configuration sudo and user (#3038)

Co-authored-by: zhangchunyang1024 <zhangchunyang0123@gamil.com>

* remove some standard notices in order to cut down the NOTICE file. (#3101)

* #3084 remove some standard notices in order to cut down the NOTICE file.

* update exclude element sql/soft_version of rat plugin

* update soft_version

* merge from 1.3.1-release

* fix 3112:download udf resource need find the tenant code of resource owner (#3114)

* fix 3112:download udf resource need find the tenant code of resource owner

* fix 2975:download resource need find the tenant code of resource owner

* remove useless code

* merge from 1.3.1-release

* remove some notes (#3127)

Co-authored-by: lenboo <baoliang@analysys.com.cn>

* fix Permission denied if not found dolphinscheder-env.sh on 1.3.1 (#3128)

* fix Permission denied if not found dolphinscheder-env.sh

* update some comments on method

* add PropertyPlaceholderHelper on license (#3129)

* update license header location

* add PropertyPlaceholderHelper on license

* merge from 1.3.1-release

* change from 1.3.0  to 1.3.2

* add Set import

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: zhangchunyang <zhangchunyang0123@gmail.com>
Co-authored-by: zhangchunyang1024 <zhangchunyang0123@gamil.com>
Co-authored-by: lgcareer <18610854716@163.com>
Co-authored-by: lenboo <baoliang@analysys.com.cn>
2020-07-13 16:35:42 +08:00
lgcareer
8e0dbd9848
remove some standard notices in order to cut down the NOTICE file. (#3101)
* #3084 remove some standard notices in order to cut down the NOTICE file.

* update exclude element sql/soft_version of rat plugin

* update soft_version
2020-07-01 10:44:43 +08:00
lenboo
fd76f172c1 Merge remote-tracking branch 'upstream/dev' into dev-merge
# Conflicts:
#	dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/runner/TaskExecuteThread.java
2020-06-30 11:17:14 +08:00
gaojun2048
3cf9afc195 merge 1.3.0-release branch code 2020-06-29 18:42:18 +08:00
dailidong
6d1da61390 fix create table sql exception (#2912)
fix create table sql exception
2020-06-29 17:39:50 +08:00
Yelli
6e20d3d34a modify t_ds_task_instance app_link change sql (#2861) 2020-06-29 17:39:41 +08:00
张世鸣
7cd1874dfb update filename 2020-06-29 17:39:14 +08:00
xingchun-chen
bd766c1633 modify soft_version 1.2.2 to 1.3.0 (#2642)
Co-authored-by: chenxingchun <438044805@qq.com>
2020-06-29 17:39:13 +08:00
lgcareer
54f04a378e init full_name in dml of version 1.2.2 (#2341)
Co-authored-by: dailidong <dailidong66@gmail.com>
2020-06-29 17:36:41 +08:00
lgcareer
82d6dc9e52 new feature for #404 add resource tree function (#2323)
* add create resource directory

* add create resource directory

* update the resource test

* add upgrade sql in version 1.2.2

* Adding request parameter id to update queryResourceListPaging

* set isDirectory value is false default

* add full name to update updateResource

* remove request parameter isDirectory to update createResource method

* update queryResourceListPaging with change get to post

* update updateResource method with remove fullName

* File management list modification (#1976)

* add resource component

* add resource tree visitor

* return json string

* update queryResourceList

* upload file need fullName

* add method rootNode

* Shell task resources and authorization resources (#1989)

* File management list modification

* Shell task resources and authorization resources

* download resource when execute task

* download resource when execute task

* update authorization type

* download resource when execute task

* Spark task resource changes (#1990)

* File management list modification

* Shell task resources and authorization resources

* Spark task resource changes

* download resource when execute task

* update udf function service

* add resource type in ResourceComponent

* UDF resource tree and change DAG style (#2019)

* File management list modification

* Shell task resources and authorization resources

* Spark task resource changes

* UDF resource tree and change DAG style

* add deleteIds method in ResourceMapper and ResourceMapperTest

* Add comments on class and method

* add queryResourceByName method in controller

* update verify-name with change name to full name

* update queryResource with add parameter pid

* update queryResource with add parameter pid

* add resource ids in process definition and delete resource need judge whether it is used by any process definition

* Breadcrumb development (#2033)

* File management list modification

* Shell task resources and authorization resources

* Spark task resource changes

* UDF resource tree and change DAG style

* Breadcrumb development

* Breadcrumb development

* Resource tree bug fix (#2040)

* File management list modification

* Shell task resources and authorization resources

* Spark task resource changes

* UDF resource tree and change DAG style

* Breadcrumb development

* Breadcrumb development

* Resource tree bug fix

* update resource service test

* Fix github action rerun failed

* add status of PARENT_RESOURCE_NOT_EXIST

* Fix github action rerun failed (#2067)

* update resource service test

* Fix github action rerun failed

* add status of PARENT_RESOURCE_NOT_EXIST

* Change crumb position

* Change crumb position (#2068)

* build resource process definition map

* UDF changed to multiple choice

* UDF changed to multiple choice (#2077)

* Change crumb position

* UDF changed to multiple choice

* build resource process definition map (#2076)

* update resource service test

* Fix github action rerun failed

* add status of PARENT_RESOURCE_NOT_EXIST

* build resource process definition map

* update resource name also need update all the children full name

* need add queryResource

* update resource name also need update all the children full name (#2096)

* update resource service test

* Fix github action rerun failed

* add status of PARENT_RESOURCE_NOT_EXIST

* build resource process definition map

* update resource name also need update all the children full name

* need add queryResource

* Limit customization file content to no more than 3000 lines

* Limit customization file content to no more than 3000 lines(#2128)

* Limit customization file content to no more than 3000 lines(#2128) (#2140)

* Change crumb position

* UDF changed to multiple choice

* Limit customization file content to no more than 3000 lines

* Limit customization file content to no more than 3000 lines(#2128)

* add queryResourceJarList

* add queryResourceJarList

* add queryResourceJarList

* add queryResourceJarList (#2192)

* update resource service test

* Fix github action rerun failed

* add status of PARENT_RESOURCE_NOT_EXIST

* build resource process definition map

* update resource name also need update all the children full name

* need add queryResource

* add queryResourceJarList

* add queryResourceJarList

* add queryResourceJarList

* Modify the main jar package

* Modify the main jar package (#2200)

* Change crumb position

* UDF changed to multiple choice

* Limit customization file content to no more than 3000 lines

* Limit customization file content to no more than 3000 lines(#2128)

* Modify the main jar package

* add resource filter in order to get filtered resource

* add comments of resource filter

* update list children by resource

* choose main jar with resource tree (#2220)

* update resource service test

* Fix github action rerun failed

* add status of PARENT_RESOURCE_NOT_EXIST

* build resource process definition map

* update resource name also need update all the children full name

* need add queryResource

* add queryResourceJarList

* add queryResourceJarList

* add queryResourceJarList

* add resource filter in order to get filtered resource

* add comments of resource filter

* update list children by resource

* Return null if query resource list is empty

* update queryResource method change parameter pid to id

* getResouDelete checksum and modify parameter namerceId

* revert .env

* remove parameter pid

* Delete request interface

* go back to the last page

* jar interface call

* Fix issue #2234 and #2228

* change resource name with full name

* Fix issue #2234 and #2228 (#2246)

* update resource service test

* Fix github action rerun failed

* add status of PARENT_RESOURCE_NOT_EXIST

* build resource process definition map

* update resource name also need update all the children full name

* need add queryResource

* add queryResourceJarList

* add queryResourceJarList

* add queryResourceJarList

* add resource filter in order to get filtered resource

* add comments of resource filter

* update list children by resource

* Return null if query resource list is empty

* update queryResource method change parameter pid to id

* revert .env

* remove parameter pid

* Fix issue #2234 and #2228

* change resource name with full name

* Fix list query value error

* remove unauth-file with authorize-resource-tree

* Repair data cannot be echoed

* Repair data cannot be echoed

* execute mr and spark task need query resource name before

* Authorized resource interface replacement

* Authorized resource interface replacement

* Filter UDF resources

* Change parameters

* need query all authorized directory children when create task

* Change normalize.scss import method and animation.scss license modification

* Delete file list update processing

* It's fixed that resource not deleted in hdfs when delete it.

* add tooltips

* add tooltips (#2310)

* Echo workflow name and modify udf management name

* [new feature]add resource tree function

* revert front code in order to be same as dev branch

* revert front code in order to be same as dev branch

* revert common.properties and application.properties

* add super method

* update flink parameter test

* update flink parameter and unit test

* update resource service test

* If resource list is empty,need init it

* update flink parameter test

Co-authored-by: break60 <790061044@qq.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>
2020-06-29 17:36:36 +08:00
Yelli
d7614c059b fix bug: data too long for column 'app_link' (#2146)
* fix bug: data to long for app_link field

* change app_link field length in 1.2.2 ddl

* change app_link field type to text
2020-06-29 17:36:27 +08:00
muzhongjiang
40bbef5e3d
add state (#3040)
Co-authored-by: mzjnumber1@163.com <mzjnumber1@163.com>
2020-06-23 18:47:04 +08:00
lenboo
714ae33e40 merge from dev 2020-06-19 14:20:24 +08:00
Yichao Yang
f7ea4f1978
[Feature-2127][sql-schema] Fix the mysql version limited error for 5.5x - 5.64 (#2911)
Co-authored-by: dailidong <dailidong66@gmail.com>
2020-06-15 23:21:33 +08:00
lenboo
1904a7022a change 1.3.0 to 1.3.0-snapshot 2020-06-10 11:42:15 +08:00
dailidong
0de8e58965
fix create table sql exception (#2912)
fix create table sql exception
2020-06-06 16:38:40 +08:00
Yelli
3cbe865776
modify t_ds_task_instance app_link change sql (#2861) 2020-06-01 17:21:23 +08:00
dailidong
6886ed51c5
fix table T_DS_ALERT (#2854)
* fix worker group config no effect

* remove codehaus janino jar
the license about janino maybe not compatiable with Apache v2

* Merge remote-tracking branch 'upstream/dev-1.3.0' into dev-1.3.0

# Conflicts:
#	dolphinscheduler-server/src/main/resources/config/install_config.conf

* datasource config

* Update datasource.properties

* fix RunConfig bug

* remove param monitor server state

* fix table T_DS_ALERT
2020-06-01 11:11:34 +08:00
lgcareer
470bd59db7
rename worker_group_id to worker_group and set the value of worker_group_name (#2801) 2020-05-25 10:18:01 +08:00
Simon
83006e971e
Merge pull request #17 from apache/dev-1.3.0
Dev 1.3.0
2020-05-14 20:00:52 +08:00
xiaochun.liu
4024465465 fix , 2020-05-14 16:23:07 +08:00
xiaochun.liu
91c544f24f sql 2020-05-14 00:04:52 +08:00
liwenhe1993
29a353e689
Using docker/docker-swarm/docker-compose.yml to run pg and zk in ci_ut.yml (#2692)
* Using docker/docker-swarm/docker-compose.yml to run pg and zk in ci_ut.yml

1. add initdb volume for pg service in docker-compose.yml
2. modify ci_ut.yml

* add sudo in ci_ut.yml

* Query the metadata of indexes is not correct in postgresql/dolphinscheduler_ddl.sql

1. replace information_schema.STATISTICS to pg_stat_all_indexes
2. modify ProcessDefinitionMapperTest.java

* Remove inner zk and pg

* add inner pg
2020-05-12 20:24:17 +08:00
Simon
da4863cbee
Merge pull request #16 from apache/dev-1.3.0
Dev 1.3.0
2020-05-12 18:55:39 +08:00
Simon
01045178b7
Update dolphinscheduler_ddl.sql 2020-05-12 18:35:14 +08:00
bao liang
d99471806e
[bug fix] some bugs from 1.2.0 to 1.3.0 (#2676)
* fix bug: some bugs about worker group from 1.2.0 to 1.3.0

* fix bug: scheduler from 1.2.0 to 1.3.0 do not work

* fix bug: scheduler from 1.2.0 to 1.3.0 do not work

* fix ut bug

Co-authored-by: baoliang <baoliang@analysys.com.cn>
2020-05-12 16:46:16 +08:00