wenjun
4a1303b625
[Improvement][Worker] Kill task error( #5199 ) ( #5212 )
2021-04-06 10:47:53 +08:00
lenboo
027781c55a
Merge remote-tracking branch 'upstream/json_split' into spilit
2021-04-04 11:44:23 +08:00
JinyLeeChina
a8f2894b79
[Feature][JsonSplit] Fix master/processInstance bug ( #5206 )
...
* transform taskCode from long to string
* fix process bug
* code review
* code review
* code review
* Fix master/processInstance bug
Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-04-04 11:26:42 +08:00
bao liang
86ce8f133a
[JsonSplit-4417][master/worker/api]json split- remove json in task instance. ( #5160 )
...
* [json split] refactor json in task instance
* add task_pramas in task_instance
* feature #4417 refactor json split in task instance
* code style
* code style
* code style
* code style
* code style
* update
* update
* update unit test
* update
2021-04-03 17:58:04 +08:00
wenjun
97fb08f0ce
[Improvement][LogServer] Thread pool parameter adjustment #5161 ( #5163 )
...
* [Improvement][LogServer] Thread pool parameter adjustment #5161
* change log request processor thread num 2 * cpuCores + 1
2021-04-02 14:24:37 +08:00
JuFeng Li
e8c9c33d7e
[Fix-5055][UI] Fix can not upload file to correct dir ( #5056 )
...
* fix udf resource can not upload to the correct dir,and when delete udf resource can not refresh page
Co-authored-by: 李巨丰 <lijf@2345.com>
2021-04-01 20:57:46 +08:00
Hayden Zhou
5d7dc1cb2a
[Improvement][API]optimize tenant code validation ( #5093 )
...
* optimize tenant code validation
* fix code style and test err
* fix test error, and remove duplicate tenant code validate
2021-04-01 20:50:14 +08:00
guohaozhang
6b565f0aed
[Fix-#5124] fix change fileName when reupload file ( #5125 )
2021-04-01 20:44:03 +08:00
wenjun
f20bb54896
[Improvement][API] Improvement query all project list ( #5169 ) ( #5192 )
2021-04-01 20:34:49 +08:00
JinyLeeChina
dd36da8d1f
[Feature][JsonSplit] fix process bug ( #5193 )
...
* transform taskCode from long to string
* fix process bug
* code review
* code review
* code review
Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-04-01 17:45:44 +08:00
wanghong1314
817328ee8f
[Improvement-5172][UI]process list add Timing state add online/offline marking color ( #5190 )
...
Co-authored-by: wanghong <wanghong1@xunlei.com>
2021-03-31 20:50:12 +08:00
zhuangchong
e53369318b
[Fix-4721][worker]The shell background starts the YARN task scenario, and the kill function is abnormal ( #4722 )
...
* fix the shell starts the yarn task in the background
* update StringUtils code style.
* solve code smell.
* add method comment in StringUtils class.
* update AlertGroupMapper code style.
* update AlertGroupMapperTest
* update sql script test.
2021-03-30 22:33:49 +08:00
wen-hemin
15b2b1c0ec
[FixBug][Jsonsplit] When to save process definition, upload task code field #4417 ( #5178 )
...
* fix: task page, rename "description" to "desc"
* fix: The default value of field dependence is null.
* fix: frontend compile check
* fix: When create process, the version button is set to unavailable
* fix: sql condition bug
* fix: When to save process definition, upload task code field
Co-authored-by: wen-hemin <wenhemin@apache.com>
2021-03-30 14:28:08 +08:00
lenboo
c8ac6f99fe
Merge remote-tracking branch 'upstream/dev' into spilit
...
# Conflicts:
# dolphinscheduler-common/src/main/resources/common.properties
2021-03-30 11:31:42 +08:00
JinyLeeChina
689106d114
transform taskCode from long to string ( #5174 )
...
Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-03-30 11:07:28 +08:00
KonsonChow
9c3cec5bbd
[DS-4891][docs] update DolphinScheduler slack invitation url ( #5167 )
...
update DolphinScheduler slack invitation url
Co-authored-by: KonsonChow <KonsonChow@gmail.com>
2021-03-29 23:47:52 +08:00
wen-hemin
f960eacdcd
[FixBug][Jsonsplit] sql condition bug #4417 ( #5171 )
...
* fix: task page, rename "description" to "desc"
* fix: The default value of field dependence is null.
* fix: frontend compile check
* fix: When create process, the version button is set to unavailable
* fix: sql condition bug
Co-authored-by: wen-hemin <wenhemin@apache.com>
2021-03-29 22:17:28 +08:00
wen-hemin
0edd783373
[FIxBug][JsonSplit] When create process, the version button is set to unavailable #5137 ( #5168 )
...
* fix: task page, rename "description" to "desc"
* fix: The default value of field dependence is null.
* fix: frontend compile check
* fix: When create process, the version button is set to unavailable
Co-authored-by: wen-hemin <wenhemin@apache.com>
2021-03-29 17:24:01 +08:00
JinyLeeChina
a467fffd7b
[Feature][JsonSplit] fix taskId in json ( #5165 )
...
* modify checkDAGRing and ProcessService method
* merge
* modify dagRing
* modify process instance for project home page
* fix save process bug
* codeStyle
* Fix logical bug in saving process definition
* codeSytle
* Fix bug in interface of queryProcessDefinitionList
* codeSytle
* Fix api bug"
* fix taskId in processDefinitionJson
* fix json taskId
* codeStyle
Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-03-29 14:22:40 +08:00
guohaozhang
a6a1b94df4
[Feature-#5128]Support turning off sudo permissions ( #5129 )
...
* [Feature-#5128]Support turning off sudo permissions
* [Feature-#5128] without sudo permission do not create tenant user
2021-03-29 11:07:39 +08:00
xiaojingXU
9c04396218
[Bug-5132][server] stop the take immediately if an exception occurs in the taskExecuteThread ( #5133 )
...
* [DS-5132][fix] stop the take immediately if an exception occurs in the taskExecuteThread
- let softkill() method return true when the process is stopped,else return false;
2021-03-29 10:41:51 +08:00
wenjun
573252f3e1
[Bug][Server] DB transaction failure #5150 ( #5152 )
2021-03-29 10:30:16 +08:00
wenjun
87ed189ef5
[Improvement] print exception when task execute ( #5159 )
2021-03-29 10:15:18 +08:00
zhuangchong
f6d62a4924
[Improvement-5144][UI] error v-show directives cannot be put on template tags ( #5145 )
...
* fix error v-show directives cannot be put on template tags.
* rerun e2e.
2021-03-26 16:42:13 +08:00
Shiwen Cheng
492b318bd3
[Fix][Server] Fix clear task execute path is related to master ( #5123 )
2021-03-26 10:11:56 +08:00
wenjun
a8b47e1d4f
[Bug][Master] Typo in MasterConfig #5141 ( #5142 )
2021-03-26 09:47:21 +08:00
lenboo
0914699fc5
Merge branch 'json_split' of https://github.com/apache/incubator-dolphinscheduler into spilit
2021-03-26 00:06:26 +08:00
lenboo
f05ba84968
Merge remote-tracking branch 'upstream/dev' into spilit
...
# Conflicts:
# dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/ProcessInstanceMapper.java
# dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/TaskInstanceMapperTest.java
2021-03-25 23:54:01 +08:00
Simon
cd19507271
[feature][jsonsplit]fix copy/export/import processDefinition bugs ( #5146 )
...
* Modify Project and ProjectUser Mapper
* Modify Project and ProjectUser Mapper
* project_code is bigint(20)
* modify ERROR name
* modify saveProcessDefine, remove the duplicate code with createTaskAndRelation
* modify import/export processdefinition, add genProcessData
* fix ut and bug
* code style
* repalce project_id with code
* conflicts solve
* conflicts solve
* conflicts solve
* bugfix
* modify listResources mothod and remove getResourceIds mothod
* 1
* conflicts solve
* modify listResources mothod and remove getResourceIds mothod
* modify listResources mothod and remove getResourceIds mothod
* replace processDefinitionVersion with processDefinitionLog
* codestyle
* codestyle
* add mapper module ut
* codestyle
* fix ProcessInstanceMapperTest
* codestyle
* conflicts solve
* conflicts solve
* conflicts solve
* conflicts solve
* conflicts solve
* fix ProcessInstanceMapperTest
* fix ProjectMapperTest/ProjectUserMapperTest/ScheduleMapperTest
* fix ProjectMapperTest/ProjectUserMapperTest/ScheduleMapperTest
* fix TaskInstanceMapperTest
* add TaskDefinitionLogMapperTest/TaskDefinitionMapperTest and bugfix
* codestyle
* codestyle
* Refactor process lineage
* Refactor process lineage
* codestyle
* codestyle
* Refactor process lineage
* Refactor process lineage
* add WorkFlowLineageMapperTest/WorkFlowLineageServiceTest
* add WorkFlowLineageMapperTest/WorkFlowLineageServiceTest
* add ProcessServiceTest
* 1
* 1
* fix copy/export/import processDefinition bugs
* 1
* codestyle
2021-03-25 18:27:47 +08:00
Simon
3c9b214da7
[Feature][jsonsplit]add ProcessServiceTest ( #5097 )
...
* Modify Project and ProjectUser Mapper
* Modify Project and ProjectUser Mapper
* project_code is bigint(20)
* modify ERROR name
* modify saveProcessDefine, remove the duplicate code with createTaskAndRelation
* modify import/export processdefinition, add genProcessData
* fix ut and bug
* code style
* repalce project_id with code
* conflicts solve
* conflicts solve
* conflicts solve
* bugfix
* modify listResources mothod and remove getResourceIds mothod
* 1
* conflicts solve
* modify listResources mothod and remove getResourceIds mothod
* modify listResources mothod and remove getResourceIds mothod
* replace processDefinitionVersion with processDefinitionLog
* codestyle
* codestyle
* add mapper module ut
* codestyle
* fix ProcessInstanceMapperTest
* codestyle
* conflicts solve
* conflicts solve
* conflicts solve
* conflicts solve
* conflicts solve
* fix ProcessInstanceMapperTest
* fix ProjectMapperTest/ProjectUserMapperTest/ScheduleMapperTest
* fix ProjectMapperTest/ProjectUserMapperTest/ScheduleMapperTest
* fix TaskInstanceMapperTest
* add TaskDefinitionLogMapperTest/TaskDefinitionMapperTest and bugfix
* codestyle
* codestyle
* Refactor process lineage
* Refactor process lineage
* codestyle
* codestyle
* Refactor process lineage
* Refactor process lineage
* add WorkFlowLineageMapperTest/WorkFlowLineageServiceTest
* add WorkFlowLineageMapperTest/WorkFlowLineageServiceTest
* add ProcessServiceTest
* 1
* 1
2021-03-24 15:14:22 +08:00
wen-hemin
e53367bdcc
[Feature][JsonSplit] task page, rename field "description" to "desc" ( #5137 )
...
* fix: task page, rename "description" to "desc"
* fix: The default value of field dependence is null.
* fix: frontend compile check
Co-authored-by: wen-hemin <wenhemin@apache.com>
2021-03-24 15:05:54 +08:00
Simon
21b7b77d50
[Feature][JsonSplit]fix modifyBy field ( #5112 )
...
* Modify Project and ProjectUser Mapper
* Modify Project and ProjectUser Mapper
* project_code is bigint(20)
* modify ERROR name
* modify saveProcessDefine, remove the duplicate code with createTaskAndRelation
* modify import/export processdefinition, add genProcessData
* fix ut and bug
* code style
* repalce project_id with code
* conflicts solve
* conflicts solve
* conflicts solve
* bugfix
* modify listResources mothod and remove getResourceIds mothod
* 1
* conflicts solve
* modify listResources mothod and remove getResourceIds mothod
* modify listResources mothod and remove getResourceIds mothod
* replace processDefinitionVersion with processDefinitionLog
* codestyle
* codestyle
* add mapper module ut
* codestyle
* fix ProcessInstanceMapperTest
* codestyle
* conflicts solve
* conflicts solve
* conflicts solve
* conflicts solve
* conflicts solve
* fix ProcessInstanceMapperTest
* fix ProjectMapperTest/ProjectUserMapperTest/ScheduleMapperTest
* fix ProjectMapperTest/ProjectUserMapperTest/ScheduleMapperTest
* fix TaskInstanceMapperTest
* add TaskDefinitionLogMapperTest/TaskDefinitionMapperTest and bugfix
* codestyle
* codestyle
* Refactor process lineage
* Refactor process lineage
* codestyle
* codestyle
* Refactor process lineage
* Refactor process lineage
* add WorkFlowLineageMapperTest/WorkFlowLineageServiceTest
* add WorkFlowLineageMapperTest/WorkFlowLineageServiceTest
* add ProcessServiceTest
* 1
* 1
* fix modifyBy field
* 1
* ut fix
2021-03-24 14:54:34 +08:00
JinyLeeChina
1eaf1f6234
[Feature][JsonSplit] fix taskId in processDefinitionJson ( #5136 )
...
* modify checkDAGRing and ProcessService method
* merge
* modify dagRing
* modify process instance for project home page
* fix save process bug
* codeStyle
* Fix logical bug in saving process definition
* codeSytle
* Fix bug in interface of queryProcessDefinitionList
* codeSytle
* Fix api bug"
* fix taskId in processDefinitionJson
Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-03-24 11:44:14 +08:00
wenjun
5d264c9f20
[Improvement][Api] Optimize query processInstanceList #5134 ( #5135 )
...
* Batch query user rather than several query
2021-03-23 23:48:13 +08:00
Shiwen Cheng
0d065c0712
[Feature-5062][*] Create/Edit/Delete Worker Group in Web UI ( #5113 )
...
* [Improvement][Server] Rename MASTER_PREFIX to MASTER_TYPE and WORKER_PREFIX to WORKER_TYPE
* [Feature][Api&Dao] Support api and dao of worker group
* [Feature][UI] Support to create, edit and delete worker group
* [Feature][API&UI] Support work groups from database and zookeeper to display together in worker group management
* [Feature][API&Dao] Support long ip list and check request parameters when saving worker group
* [Feature][*] Refactor ipList to addrList
* [Feature][Server] Support host manager to select worker host by worker group in database
* [Improvement][Server] Improve the config of master and worker
* [Improvement][*] Rename zkRegistered to systemDefault
* [Improvement][Test] Fix unit test and improve ui
* [Improvement][Server] Improve getServerMaps
Co-authored-by: dailidong <dailidong66@gmail.com>
2021-03-23 23:47:41 +08:00
Kirs
f109a758f8
[Bug][PGSQL] Delete the stopped workflow without deleting the correspon… ( #5066 )
...
* [Bug][Api] Delete the stopped workflow without deleting the corresponding sub process, the workflow instance name is not displayed in the sub process
* fix ut
* fix ut err
* sql style
* fix error
2021-03-23 11:26:11 +08:00
JinyLeeChina
d99934d0da
[Feature][JsonSplit] Fix api bug ( #5102 )
...
* modify checkDAGRing and ProcessService method
* merge
* modify dagRing
* modify process instance for project home page
* fix save process bug
* codeStyle
* Fix logical bug in saving process definition
* codeSytle
* Fix bug in interface of queryProcessDefinitionList
* codeSytle
* Fix api bug"
Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-03-23 10:44:25 +08:00
lenboo
010b49c933
merge from dev
2021-03-22 15:18:16 +08:00
zhuangchong
ba039dc252
[Improvement][dao] Resource features SQL slow query optimization. ( #4947 )
...
* Resourse features SQL slow query optimization.
* update resource mapper queryResourcePaging method.
* split resource and resource_uesr sql statement.
* determine the size of the collection.
* update process service class code style.
* update ResourceUserMapper java class code style.
* add resourceUserMapper in the test config file.
* solve code smell
* split paging query join statement.
* update ResourceUserMapper class.
* remove sql join statement.
* update resource test class code style.
* update ResourcesServiceTest test class.
* rerun ut.
* update user service test class.
* solve query resource list method code smell.
2021-03-20 12:50:26 +08:00
Shiwen Cheng
8ac72e80e6
[Feature-5087][SqlTask] Add a switch to send mail and print head logs in SqlTask ( #5088 )
...
* [Feature-5087][SqlTask] Add the switch to send mail and print head logs in SqlTask
* [Feature-5087][SqlTask] Add SqlParametersTest
2021-03-19 15:07:39 +08:00
guohaozhang
4a6e8b7afa
[Fix-#5103] fix resource rename cause can not update ( #5107 )
2021-03-19 14:51:12 +08:00
Shiwen Cheng
91c29e6ca3
[Improvement-5094][UI] Improve to add a main switch for refresh_in_switched_tab ( #5095 )
2021-03-19 11:58:33 +08:00
Shiwen Cheng
9008fa4b0c
[Fix-5089][UI] Fix code mirror cannot be displayed normally in sqoop task ( #5090 )
...
* [Improvement][UI] Rename from-mirror to form-mirror
* [Fix-5089][UI] Fix code mirror cannot be displayed normally in sqoop task
2021-03-19 11:57:46 +08:00
wenjun
086e716441
[Improvement][Dao] SQL optimization - check exist #5069 ( #5070 )
2021-03-18 22:49:12 +08:00
Shiwen Cheng
f94cfc620d
[Feature][Worker] Add a configuration item to set whether the tenant is automatically created on Worker ( #5007 )
2021-03-18 18:34:42 +08:00
wangxj3
29d42fd92d
[Feature-#3805][server-master] Task parameter transfer ( #5077 )
...
* fix out param format bug
Co-authored-by: wangxj <wangxj31>
2021-03-18 11:50:10 +08:00
Shiwen Cheng
5856a12855
[Improvement-4984][Worker] Refactor and Improve worker load balance ( #4996 )
...
* [Improvement][Remote] Improve unit tests
* [Improvement][Worker] Improve worker load balance
* [Improvement][Worker] Fix code smells
* [Improvement][Worker] Rename weight to hostWeight
2021-03-18 10:03:58 +08:00
zhuangchong
705715c313
[Improvement-5059][API] Verify that the new workflow definition version is the current version before removing it #5059 ( #5060 )
...
* this process definition version is used, not allowed to delete.
* update ProcessDefinitionVersionServiceTest test class.
* update check has associated process definition sql.
2021-03-18 09:37:17 +08:00
Ts686
6216817861
[Improvement-4971][alert|dao|service]Remove redundant PropertyUtils.class ( #5020 )
...
* [Improvement][alert|dao|service]
1. Remove redundant PropertyUtils.class (#4971 )
2. Remove useless import package.
* [Improvement][alert|dao|service]
1. Remove redundant PropertyUtils.class (#4971 )
2. Remove useless import package.
* [Improvement][alert|dao|service]
1. Remove redundant PropertyUtils.class (#4971 )
2. Remove useless import package.
* [Improvement][alert|dao|service]
1. Remove redundant PropertyUtils.class (#4971 )
2. Remove useless import package.
2021-03-17 15:28:07 +08:00
Shiwen Cheng
1cd62b4a5e
[Improvement][Datax] Re-assign DATAX_HOME to /opt/soft/datax instead of /opt/soft/datax/bin/datax.py ( #5019 )
2021-03-17 14:38:04 +08:00
