Commit Graph

586 Commits

Author SHA1 Message Date
KyoYang
676a952bcb
[Fix-#6854][dao] fix "ResourceMapper.xml -> queryResourcePaging" sql bug (#6907)
* fix complement data retry bug; issues: [Bug] [Master] A bug on task retry mechanism #6613

* [DS-#5293][chore]upgrade cron-utils version 5.0.5 -> 9.1.3

* [DS-#5293]update the licensing information of that dependency

* [DS-#5293]update the licensing information of that dependency

* update the licensing information of that dependency

* [DS-#5293]update the licensing information of that dependency

* [DS-#6854][fix] fix "ResourceMapper.xml -> queryResourcePaging" sql bug

* [DS-#6854][fix] fix "ResourceMapper.xml -> queryResourcePaging" sql bug

Co-authored-by: yangqiyu <yangqiyu@deepexi.com>
2021-12-15 19:18:00 +08:00
Zonglei Dong
7e0010de35
[Refactor] Rename class file, unified word spelling. (#7261) 2021-12-15 01:04:50 +08:00
Jiajie Zhong
04805818c2
Fix alert plugin instance filter (#7172) (#7210)
* Fix alert plugin instance filter (#7172)

alert plugin could not filter by given
pattern, cause api server do not handle
parameter searchVal, This patch add parameter
`searchVal` for alert plugin.

fix: #7209

* Add test

* Recover AlertPluginInstanceVO
2021-12-14 12:08:22 +08:00
wind
99b8ec6492
[Bug-7319][MasterServer] fix taskNode NPE when switch else branch is empty (#7320)
Co-authored-by: caishunfeng <534328519@qq.com>
2021-12-13 22:46:48 +08:00
kezhenxu94
2ff76c7e73
Fix some SQL issues and make docker-compose runnable (#7329) 2021-12-10 23:51:42 +08:00
calvin
ac67a11d44
fix a few issues in the sql file (#7327) 2021-12-10 19:50:55 +08:00
Kerwin
f1b85cf892
Fix the missing project information in the process instance timeout alarm (#7324) 2021-12-10 18:55:10 +08:00
wind
0f7e38ed3c
[Bug-7292][ApiServer] fix cache error when standalone (#7293) 2021-12-10 18:46:41 +08:00
ouyangyewei
f03b0e8c7b
to feature-7301 (#7308)
Co-authored-by: ouyangyewei <yewei.oyyw@alibaba-inc.com>
2021-12-10 14:40:06 +08:00
Kerwin
3fadfca085
fix in the process alarm message, projectid is 0 (#7298) 2021-12-10 12:25:35 +08:00
kezhenxu94
67cc260d52
Split the components into individual package, refactor configurations to use Spring properties (#7160)
* Split the components into individual package

A follow-up PR will be made to build dedicated Docker images for each
component, so that every component Docker image has minimal jars, which
is easy to maintain and good for security fixes.

* Split the components into individual package

A follow-up PR will be made to build dedicated Docker images for each
component, so that every component Docker image has minimal jars, which
is easy to maintain and good for security fixes.

* Split the components into individual package

A follow-up PR will be made to build dedicated Docker images for each
component, so that every component Docker image has minimal jars, which
is easy to maintain and good for security fixes.
2021-12-10 11:28:53 +08:00
Kerwin
4e6265df11
fix Property or field 'all' cannot be found (#7246) 2021-12-08 09:00:00 +08:00
Jiajie Zhong
29414a080f
Fix homepage user wrong statistics result for new user (#7242)
* Fix homepage user wrong statistics result for new user

For now, when we create a new user, it would see homepage
statistic info as admin user like, cause `projectCodeArray`
would not filter when we render SQL. So I add prefix conditions
to handle this situation.

Also, I do some minimal refactor here. including change function
name no quit fit

fix: #7182

* Fix UT
2021-12-07 18:50:06 +08:00
wind
a80a7b3fcf
[Feature-6963][MasterServer] unified cache manager (#7187)
* unified cache

* reduce db select

* checkstyle

Co-authored-by: caishunfeng <534328519@qq.com>
2021-12-07 15:55:22 +08:00
wind
9d65bf1452
fix postgre upgrade (#7230)
Co-authored-by: caishunfeng <534328519@qq.com>
2021-12-07 14:19:32 +08:00
Hua Jiang
4743e8f891
add two files (#7223) 2021-12-06 22:55:28 +08:00
Hua Jiang
705ba74f52
[Fix-7195] The file path of 2.0.1-schema used a wrong separator. (#7200) 2021-12-06 17:29:11 +08:00
ouyangyewei
3c8d1c6da9
to feature #7191 (#7194)
Co-authored-by: ouyangyewei <yewei.oyyw@alibaba-inc.com>
2021-12-06 07:17:33 +08:00
ouyangyewei
27ea43d5df
to feature #7110 (#7111)
Co-authored-by: ouyangyewei <yewei.oyyw@alibaba-inc.com>
2021-12-04 21:43:14 +08:00
zwZjut
af9ed35295
[Feature][MasterServer] add cache for workGroup and schedule #6987 (#7161)
* to #6987

* to #6987

* to #6987

* to #6987

* to #6987

* to #6987: @Param -> @Name

* to #6987: fix Sonar

* to #6987: fix Sonar

Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
2021-12-04 16:59:05 +08:00
BaoLiang
0e75bd8008
[BUG][SQL] fix the process list page load slowly. (#7166)
* fix #7601 move ddl to 2.0.1

* fix #7601 move ddl to 2.0.1
2021-12-04 15:53:18 +08:00
GaoTianDuo
ed9fca6c97
[Bug] [dolphinscheduler-api] The pages of project management, workflow instance and task instance are accessed slowly #7061 (#7139)
1、I found that the SQL behind this interface is very slow. The main table is large, but there is no index to use. By add the index, the all mode in the query plan can be changed into a more efficient ref mode. If this optimization still fails to meet the requirements, please contact me and I will continue to optimize。
2、modify index start_time_index in init script and upgrade scripts.
2021-12-04 13:13:30 +08:00
wind
ba2b2a67c2
[Feature-6988][MasterServer] add cache manager for workflow (#7090)
* add cache manager for workflow

* [DS-6988][MasterServer] add cache manager for workflow

* cache evict code optimization

* test

Co-authored-by: caishunfeng <534328519@qq.com>
2021-12-03 23:13:11 +08:00
zwZjut
2bf73603bb
[Feature][dolphinscheduler-api] access control of taskDefinition and taskInstance in project to #7081 (#7082)
* to #7081

* fix #7081

* to #7081

Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
2021-12-03 16:56:08 +08:00
zwZjut
8ebe060658
[Feature][dolphinscheduler-api] access control of processDefinition and processInstance in project #7073 (#7080)
* to #7073

* to #7073

* to #7073

Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
2021-12-03 16:55:20 +08:00
wangxj3
2b519f31e9
[Feature-#6422] task group queue (#6722)
* add task group

* modify task group

* pull dev

* add license header

* modify code style

* fix code style

* fix sql error

* fix error

* fix test

* fix test

* fix test

* fix test

* fix code style

* fix ut

* code style

* fix unit test

* test ut

* ut

* add unittest

* test ut

* modify back ut

* majorization code

* fix conflict

* fix ut

Co-authored-by: wangxj <wangxj31>
2021-12-03 14:02:31 +08:00
JinYong Li
b265297fc8
[Improvement][API] fix 6772 new api bug (#7098)
* add processDefinition releaseWorkflowAndSchedule

* add ProcessTaskRelationServiceImpl.moveTaskProcessRelation

* add dependency and subprocess judgement

* fix api bug

* fix codestyle
2021-12-02 14:06:39 +08:00
Hua Jiang
d7eb7453e5
[Fix-7070][UI] Remedy the issue with no saving the timeout's strategy into the database. (#7107) 2021-12-01 21:03:49 +08:00
kezhenxu94
2ed3c0c3a7
Recover unit tests that have not been run for some time (#7054) 2021-12-01 13:55:26 +08:00
Hua Jiang
a0955a078a
[Fix-7019][MasterServer] Fix the issue with failing to save and run the process that contains the switch task. (#7046) 2021-11-30 09:59:48 +08:00
zwZjut
b815da6158
to #7001: delete certain edge in processDefinition (#7038)
* to #7001: delete certain edge in processDefinition

* to #7001: delete certain edge in processDefinition

* to #7001: delete certain edge in processDefinition

* to #7001: delete certain edge in processDefinition

* to #7001: fix checkstyle

* to #7001: delete edge by projectCode,processDefinitionCode,preTaskCode and postTaskCode

* to #7001: add UT

Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
2021-11-29 20:51:58 +08:00
JinYong Li
c5bde33ac0
[Upgrade][install] upgrade compatibility exception (#7030)
* upgrade compatibility exception

* upgrade compatibility exception
2021-11-29 15:07:47 +08:00
kezhenxu94
ef0cb46661
Fix shell tools for database schema cannot run (#7003) 2021-11-26 11:10:38 +08:00
JinYong Li
5a04b8d49a
update schdule api (#6977) 2021-11-25 10:22:41 +08:00
zwZjut
1edd2bd0f6
to #6957 implement interface deleteUpstreamRelation deleteDownstreamRelation (#6966)
* to #6957

* to #6957

* to #6957

* to #6957: group by process definition code

* to #6957: fix sonar

* to #6957: fix sonar

* to #6957: add ut , add preTaskVersion

* to #6957: fix style

* to #6957: use org.apache.dolphinscheduler.spi.utils.StringUtils

Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
2021-11-24 17:12:47 +08:00
JinYong Li
e51a2a1642
[Improvement][API][num-8] add ProcessTaskRelationServiceImpl.createProcessTaskRelation (#6970)
* add ProcessTaskRelationServiceImpl.createProcessTaskRelation

* code style
2021-11-24 15:06:12 +08:00
zhang
abde2d7d28
update task_definition_log field typ (#6967)
* update task_definition_log field typ

update task_definition_log  field task_params type,modify text to longtext to be consistent with the main table

* update t_ds_task_definition_log field task_params

update t_ds_task_definition_log field task_params
2021-11-23 14:26:20 +08:00
Kerwin
38b14410ab
[Fix-6941][API] Fix the failure of querying details of alert group (#6942)
* Fix the failure of querying details of alert group

* code cleanup
2021-11-23 09:28:27 +08:00
LinKai
ad33d43498
[Improvement][Dao] SQL optimization - limit one piece of data #5061 (#6940)
* [Feature][unittest] Recover UT in AlertPluginManagerTest.java #6619

* use throws exception instead of try-catch to catch exceptions in AlertPluginManagerTest.java

* change the assert result from assertNull to assertNotNull

* [Improvement][Dao] SQL optimization - limit one piece of data #5069

* Improve the code to A rate

Co-authored-by: linkai <linkai@bigo.sg>
2021-11-22 20:27:29 +08:00
zwZjut
199a84aa05
[fix #6878] implement interface queryUpstreamRelation queryDownstreamRelation and add UT (#6888)
* fix: #6878, implement interface queryUpstreamRelation queryDownstreamRelation and add UT

* fix: #6878, implement interface queryUpstreamRelation queryDownstreamRelation and add UT

* fix: #6878, add license

* fix: #6878, code style

* fix: #6878, code style

* fix: #6878, fix Duplication

* fix: #6878

* fix: #6878 , fix style

* to #6878: fix style

* to #6878: fix style

* to #6878: add buildTaskDefinition to overwrite equals and hashCode

Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
2021-11-22 11:02:15 +08:00
JinYong Li
4bec792160
[Fix][Common] rewrite code generate,fix bit shift (#6914)
* rewrite code generate,fix bit shift

* fix ut

* add algorithm from licenses file

* add algorithm from licenses file

* add algorithm from licenses file

* add algorithm from licenses file

* add algorithm from licenses file

* fix ut
2021-11-19 16:04:05 +08:00
JinYong Li
fc6a5116c4
[Improvement][API][num-7] add task delete check (#6887)
* upgrade add sub_process

* add interface method

* add interface method

* fix ut

* add task delete check

* add task delete check
2021-11-18 10:51:29 +08:00
DingPengfei
842c3857c1
fix bug of extra 'and' in ResourceMapper.xml (#6876)
Co-authored-by: ding <dingpf@helowin>
2021-11-17 15:03:53 +08:00
OS
880c1d28ff
[Feature][master] change sub process id to code. (#6846)
* 6597 change sub process id to code.

* 6597 change sub process id to code.

* 6597 change sub process id to code.

* 6597 change sub process id to code.

* 6597 change sub process id to code.

* 6597 change sub process id to code.

* code style

* fix ut process

* fix ut process
2021-11-15 01:00:44 +08:00
JinYong Li
7f3b999e5c
upgrade add sub_process (#6845) 2021-11-15 00:11:29 +08:00
kezhenxu94
4114cb07f6
Add actuator and enable metrics, add Grafana configs (#6840) 2021-11-14 20:22:13 +08:00
mask
a998415afe
[Feature][datasource] Change DataSource Connection Pool from Druid to HiKariCP (#6490) (#6828)
* [Feature][datasource] Change DataSource Connection Pool from Druid to HiKariCP (#6490)

* fix pom dep scope

* filter alert

* filter alert

* filter alert

* fix checkstyle

* remove hikaricp version

* update pom

* add properties

* add properties

* rename properties

* fix e2e fail

* rename properties
2021-11-14 13:02:58 +08:00
Jiajie Zhong
6045728428
[ci] Add checkstyle prohibit rule System.out.println (#6778)
* [ci] Add checkstyle prohibit rule System.out.println

* Correct RegexpSingleline place
2021-11-13 16:47:29 +08:00
OS
d168a3cdf7
[feature][Install] 6597 add worker group when upgrading from version 1.x to 2.0 (#6806)
* feature 6597 add worker group. Upgrade dolphinscheduler version 1. X to version 2.0

* feature 6597 add worker group. Upgrade dolphinscheduler version 1. X to version 2.0

* code style

* code style

* code style

* code style

* update postgre sql

Co-authored-by: JinYong Li <42576980+JinyLeeChina@users.noreply.github.com>
2021-11-12 12:03:23 +08:00
JinYong Li
dfe4945b48
remove processDefinition resourceIds (#6813) 2021-11-12 11:39:16 +08:00