Commit Graph

5563 Commits

Author SHA1 Message Date
kezhenxu94
b3fe217e08
Fix E2E workflow (#7377) 2021-12-13 20:20:20 +08:00
Jiajie Zhong
1948030151
[python] Add task base database and procedure (#7279)
We add a new task procedure, and add parent class database
for both sql task and procedure task

fix: #6929
2021-12-13 19:54:24 +08:00
kezhenxu94
12b46dfaa6
Adjust Helm Chart templates in dev branch (#7353) 2021-12-12 23:17:50 +08:00
kezhenxu94
bacb7c1598
Fix typo in docker swarm (#7336) 2021-12-12 12:02:21 +08:00
kezhenxu94
2ff76c7e73
Fix some SQL issues and make docker-compose runnable (#7329) 2021-12-10 23:51:42 +08:00
calvin
ac67a11d44
fix a few issues in the sql file (#7327) 2021-12-10 19:50:55 +08:00
Kerwin
f1b85cf892
Fix the missing project information in the process instance timeout alarm (#7324) 2021-12-10 18:55:10 +08:00
wind
0f7e38ed3c
[Bug-7292][ApiServer] fix cache error when standalone (#7293) 2021-12-10 18:46:41 +08:00
Kerwin
939fa0c3ba
fix TaskResponseProcessor process NullPointerException (#7318) 2021-12-10 16:55:56 +08:00
ouyangyewei
f03b0e8c7b
to feature-7301 (#7308)
Co-authored-by: ouyangyewei <yewei.oyyw@alibaba-inc.com>
2021-12-10 14:40:06 +08:00
Kerwin
3fadfca085
fix in the process alarm message, projectid is 0 (#7298) 2021-12-10 12:25:35 +08:00
kezhenxu94
67cc260d52
Split the components into individual package, refactor configurations to use Spring properties (#7160)
* Split the components into individual package

A follow-up PR will be made to build dedicated Docker images for each
component, so that every component Docker image has minimal jars, which
is easy to maintain and good for security fixes.

* Split the components into individual package

A follow-up PR will be made to build dedicated Docker images for each
component, so that every component Docker image has minimal jars, which
is easy to maintain and good for security fixes.

* Split the components into individual package

A follow-up PR will be made to build dedicated Docker images for each
component, so that every component Docker image has minimal jars, which
is easy to maintain and good for security fixes.
2021-12-10 11:28:53 +08:00
mask
8d68cf48dd
[Fix-7277][datasource] Support Kerberos auto renewal (#7277) (#7278) 2021-12-09 09:27:57 +08:00
Jiajie Zhong
2a7d6b468f
[python] Refactor get object define communicate to gateway (#7272)
* Change class Base `to_dict` to `get_define` for more clearer
* Remove class TaskParams and sub class make code easy to
  understand and avoid task implement cycle dependence
* Remove class ObjectJsonBase in Task to reduce complexity

fix: #7271
2021-12-08 22:17:13 +08:00
wind
3a351da752
fix failover logic (#7275)
Co-authored-by: caishunfeng <534328519@qq.com>
2021-12-08 21:35:18 +08:00
百岁
f21eb8f9fe
[ISSUE 6444]Optimize task node helper document in workflow toolbar (#7226)
* [ISSUE 6444]Optimize task node helper document in workflow toolbar

* fix some review issue than submit by DS

* refact by add guard judgement when type is undefined

* move method helpUrlEnable and helpUrl nest in 'methods' body
2021-12-08 15:09:01 +08:00
songjianet
3ca2ee9ac8
[Bug][dolphinscheduler ui] When the edit time is repaired, the alarm group display. (#7262) 2021-12-08 14:24:53 +08:00
mask
deefc8e98a
[Fix-7235][datasource] unable to confige the multi-hosts (#7235) (#7253) 2021-12-08 09:21:54 +08:00
Kerwin
4e6265df11
fix Property or field 'all' cannot be found (#7246) 2021-12-08 09:00:00 +08:00
EdwardYang
6a144d69bf
[bug] fix task definition delete (#7244)
Co-authored-by: edward-yang <yangjianh210@gmail.com>
2021-12-07 20:20:29 +08:00
zwZjut
f7bdf174be
[Feature][dolphinscheduler-api] parse traceId in http header for Cross system delivery to #7237 (#7238)
* to #7237

* rerun test

Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
2021-12-07 20:17:45 +08:00
zwZjut
f577b5bd30
to #7228 (#7249)
Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
2021-12-07 20:05:57 +08:00
Jiajie Zhong
29414a080f
Fix homepage user wrong statistics result for new user (#7242)
* Fix homepage user wrong statistics result for new user

For now, when we create a new user, it would see homepage
statistic info as admin user like, cause `projectCodeArray`
would not filter when we render SQL. So I add prefix conditions
to handle this situation.

Also, I do some minimal refactor here. including change function
name no quit fit

fix: #7182

* Fix UT
2021-12-07 18:50:06 +08:00
Zonglei Dong
0b9792db76
[Refactor]Rename class file, DatasourceUtil to DataSourceUtils. (#7181)
* Rename class file, DatasourceUtil  to  DataSourceUtils.
2021-12-07 18:49:05 +08:00
wind
a80a7b3fcf
[Feature-6963][MasterServer] unified cache manager (#7187)
* unified cache

* reduce db select

* checkstyle

Co-authored-by: caishunfeng <534328519@qq.com>
2021-12-07 15:55:22 +08:00
JinYong Li
544b015f76
[Bug] [API] task definition release/delete bug (#7232)
* fix worker group display

* fix workflow query bug

* fix workflow query bug

* fix task definition release/delete bug
2021-12-07 15:26:28 +08:00
JinYong Li
2792c35d68
[Bug] [API] queryProcessDefinitionByCode bug (#7221) 2021-12-07 14:41:26 +08:00
wind
9d65bf1452
fix postgre upgrade (#7230)
Co-authored-by: caishunfeng <534328519@qq.com>
2021-12-07 14:19:32 +08:00
Hua Jiang
4743e8f891
add two files (#7223) 2021-12-06 22:55:28 +08:00
wind
ff9bc806ac
[Bug-7206] [MasterServer] fix process isntance always running when task timeout (#7207)
* fix timeout

* add task timeout map to avoid repeated timeout event

* split task check list for retry and timeout

Co-authored-by: caishunfeng <534328519@qq.com>
2021-12-06 18:20:56 +08:00
Hua Jiang
705ba74f52
[Fix-7195] The file path of 2.0.1-schema used a wrong separator. (#7200) 2021-12-06 17:29:11 +08:00
JinYong Li
c7b06b0079
fix worker group display (#7208) 2021-12-06 15:44:28 +08:00
Kerwin
36ebfe0984
[Implement][server]TaskProcessor code optimization (#7146)
* TaskProcessor code optimization

* fix code style

* update Collectors.toMap
2021-12-06 14:15:09 +08:00
Kerwin
8f77fbbeb8
[Fix-7154][API] Fix MultipartFile resource [file] cannot be resolved to absolute file… (#7155)
* fix MultipartFile resource [file] cannot be resolved to absolute file path

* update testCopyFile.
2021-12-06 14:13:53 +08:00
ouyangyewei
3c8d1c6da9
to feature #7191 (#7194)
Co-authored-by: ouyangyewei <yewei.oyyw@alibaba-inc.com>
2021-12-06 07:17:33 +08:00
zwZjut
05aef279c6
[Feature][dolphinscheduler-api] add queryLog and downloadTaskLog with projectCode #7153 (#7192)
* [Feature][dolphinscheduler-api] add queryLog and downloadTaskLog with projectCode #7153

* [Feature][dolphinscheduler-api] add queryLog and downloadTaskLog with projectCode #7153

* [Feature][dolphinscheduler-api] add queryLog and downloadTaskLog with projectCode #7153

Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
2021-12-05 23:57:46 +08:00
SbloodyS
d83735ab51
[Fix-7037][dolphincheduler-server] process instance can not be killed when complement data (#7140)
* fix bug 7037

* update processComplementData

Co-authored-by: SbloodyS <sbloodys@qq.com>
2021-12-05 15:03:52 +08:00
ouyangyewei
3361d763c4
to feature #7180 (#7185)
Co-authored-by: ouyangyewei <yewei.oyyw@alibaba-inc.com>
2021-12-05 14:13:23 +08:00
Manhua
2dc09c627f
[Fix-5551][common] Fix ExecutionStatus for hive-on-Tez application (#6289)
* add ENDED state

* Update HadoopUtils.java
2021-12-05 14:03:29 +08:00
SbloodyS
8d39bf14b1
[Fix-7093][dolphincheduler-server] fix dependent node stuck in running (#7116)
* fix bug 7093

* remove redundant else

Co-authored-by: SbloodyS <sbloodys@qq.com>
2021-12-05 13:52:48 +08:00
songjianet
eb07a2b1c9
[Fix][Warning instance manage] Expand the support of i18n. (#7184) 2021-12-05 11:23:50 +08:00
songjianet
2fa3157e6e
[Feature][Workflow] Main package. (#7177) 2021-12-04 21:54:59 +08:00
ouyangyewei
27ea43d5df
to feature #7110 (#7111)
Co-authored-by: ouyangyewei <yewei.oyyw@alibaba-inc.com>
2021-12-04 21:43:14 +08:00
Zonglei Dong
fa8ccd3855
Refactor code, add class import, keep ‘logger’ variable names uniform. (#7164) 2021-12-04 21:18:43 +08:00
ouyangyewei
db090685a8
to feature #7169 (#7170)
Co-authored-by: ouyangyewei <yewei.oyyw@alibaba-inc.com>
2021-12-04 19:24:09 +08:00
zwZjut
af9ed35295
[Feature][MasterServer] add cache for workGroup and schedule #6987 (#7161)
* to #6987

* to #6987

* to #6987

* to #6987

* to #6987

* to #6987: @Param -> @Name

* to #6987: fix Sonar

* to #6987: fix Sonar

Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
2021-12-04 16:59:05 +08:00
BaoLiang
0e75bd8008
[BUG][SQL] fix the process list page load slowly. (#7166)
* fix #7601 move ddl to 2.0.1

* fix #7601 move ddl to 2.0.1
2021-12-04 15:53:18 +08:00
GaoTianDuo
ed9fca6c97
[Bug] [dolphinscheduler-api] The pages of project management, workflow instance and task instance are accessed slowly #7061 (#7139)
1、I found that the SQL behind this interface is very slow. The main table is large, but there is no index to use. By add the index, the all mode in the query plan can be changed into a more efficient ref mode. If this optimization still fails to meet the requirements, please contact me and I will continue to optimize。
2、modify index start_time_index in init script and upgrade scripts.
2021-12-04 13:13:30 +08:00
wind
ba2b2a67c2
[Feature-6988][MasterServer] add cache manager for workflow (#7090)
* add cache manager for workflow

* [DS-6988][MasterServer] add cache manager for workflow

* cache evict code optimization

* test

Co-authored-by: caishunfeng <534328519@qq.com>
2021-12-03 23:13:11 +08:00
zwZjut
1f184440ca
[UT#7157]Fix Ut Run Error (#7158)
* to #7157

* to #7157

Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
2021-12-03 22:01:26 +08:00