Commit Graph

5624 Commits

Author SHA1 Message Date
KyoYang
94f6c5c27e
fix complement data retry bug; issues: [Bug] [Master] A bug on task retry mechanism #6613 (#6843)
Co-authored-by: yangqiyu <yangqiyu@deepexi.com>
2021-11-15 01:05:13 +08:00
OS
880c1d28ff
[Feature][master] change sub process id to code. (#6846)
* 6597 change sub process id to code.

* 6597 change sub process id to code.

* 6597 change sub process id to code.

* 6597 change sub process id to code.

* 6597 change sub process id to code.

* 6597 change sub process id to code.

* code style

* fix ut process

* fix ut process
2021-11-15 01:00:44 +08:00
JinYong Li
7f3b999e5c
upgrade add sub_process (#6845) 2021-11-15 00:11:29 +08:00
kezhenxu94
4114cb07f6
Add actuator and enable metrics, add Grafana configs (#6840) 2021-11-14 20:22:13 +08:00
zwZjut
4b848fb6d9
fix #6807: dolphinscheduler.zookeeper.env_vars - > dolphinscheduler.registry.env_vars (#6808)
Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
Co-authored-by: Kirs <acm_master@163.com>
2021-11-14 13:03:13 +08:00
mask
a998415afe
[Feature][datasource] Change DataSource Connection Pool from Druid to HiKariCP (#6490) (#6828)
* [Feature][datasource] Change DataSource Connection Pool from Druid to HiKariCP (#6490)

* fix pom dep scope

* filter alert

* filter alert

* filter alert

* fix checkstyle

* remove hikaricp version

* update pom

* add properties

* add properties

* rename properties

* fix e2e fail

* rename properties
2021-11-14 13:02:58 +08:00
GaoTianDuo
e3dc4016d2
[Bug] [readme] Error link to Docker and k8s in readme #6802 (#6833)
1、modify the error link
2021-11-14 13:01:23 +08:00
Jiajie Zhong
e76cf77040
[python] Add parameter schedule for process definition (#6664)
* [python] Add parameter schedule for process definition

* Rebase and fix some code style

* May schedule work on both string and datetime

* Fix flaky test

* Add comment about freeze time

* Add edge test for schedule_json with None schedule

* Fix test function name

* Fix rebase error
2021-11-13 17:21:40 +08:00
Assert
089f73ebe4
[Fix-6707][common] Fix complement data error because of global variables (#6759)
* [Fix-6707][common] Fix complement data error because of global variables

* [Fix-6707][common] Fix complement data error because of global variables

* [Fix-6707][common] Fix complement data error because of global variables

* [Fix-6707][common] Fix complement data error because of global variables

Co-authored-by: shangeyao <sgy960921>
Co-authored-by: OS <29528966+lenboo@users.noreply.github.com>
2021-11-13 17:06:15 +08:00
Jiajie Zhong
0f92a1f541
[community] Add version selection for issue bug selection (#6793)
closes: #6620

Co-authored-by: Kirs <acm_master@163.com>
Co-authored-by: Jiwei Guo <technoboy@apache.org>
2021-11-13 16:47:42 +08:00
Jiajie Zhong
6045728428
[ci] Add checkstyle prohibit rule System.out.println (#6778)
* [ci] Add checkstyle prohibit rule System.out.println

* Correct RegexpSingleline place
2021-11-13 16:47:29 +08:00
Jiajie Zhong
cd8205217a
[python] Fix process definition's user assigned not pass (#6753) 2021-11-13 16:47:08 +08:00
wangyizhi
e2d516f279
Optimize DAG page opening speed (#6830) 2021-11-12 15:48:55 +08:00
Jiajie Zhong
5453f1e349
[ci] Remove PR requests strict rebase (#6831) 2021-11-12 15:48:34 +08:00
wind
5741c758b3
[DS-6771][MasterServer] fix failover worker interrupt (#6801)
Co-authored-by: caishunfeng <534328519@qq.com>
Co-authored-by: Kirs <acm_master@163.com>
Co-authored-by: OS <29528966+lenboo@users.noreply.github.com>
2021-11-12 14:19:57 +08:00
OS
d168a3cdf7
[feature][Install] 6597 add worker group when upgrading from version 1.x to 2.0 (#6806)
* feature 6597 add worker group. Upgrade dolphinscheduler version 1. X to version 2.0

* feature 6597 add worker group. Upgrade dolphinscheduler version 1. X to version 2.0

* code style

* code style

* code style

* code style

* update postgre sql

Co-authored-by: JinYong Li <42576980+JinyLeeChina@users.noreply.github.com>
2021-11-12 12:03:23 +08:00
JinYong Li
dfe4945b48
remove processDefinition resourceIds (#6813) 2021-11-12 11:39:16 +08:00
mask
e36d18b588
[Fix-6792] resolve API and alert port conflicts (#6792) (#6815)
* [Fix-6792]Multiple ApplicationRunners on classpath make SpringApplication.run only run one (#6792)
2021-11-12 10:51:11 +08:00
JinYong Li
da2a044947
[Feature][API] add processDefinitionSimpleList api (#6814)
* add processDefinitionSimpleList api

* add processDefinitionSimpleList api
2021-11-11 23:45:41 +08:00
罗铭涛
975131e913
[Improvement#6655] [UI] remove registry monitor (#6789)
* [DS-6655][fix] remove zkmonitor
- delete zkmonitor ui

* [DS-6655][fix] remove zkmonitor
- delete zkmonitor ui
- Modify the actions.js comma

* [DS-6655][fix] remove zkmonitor
- remove all backend API

Co-authored-by: Kirs <acm_master@163.com>
2021-11-11 19:59:10 +08:00
Jiajie Zhong
c110488269
[ci] Remove GA path rule in build and unittest (#6797)
CI could not be run due to we add 9113817b to asf
configure and same file change will not trigger
build or unit test github actions run. This patch
try to skip github action path ignore rule temporarily

Maybe we should find more effective way to spearate py
, js, java CI later
2021-11-11 17:32:57 +08:00
zwZjut
088a5a4706
add default constructor (#6780)
Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
2021-11-11 12:11:43 +08:00
kezhenxu94
9113817b78
Require build and unit tests before merging (#6777) 2021-11-11 08:45:35 +08:00
kezhenxu94
11844415b0
Refactor datasource and task plugins and simplify its usage (#6767) 2021-11-10 18:29:17 +08:00
Jiajie Zhong
98f466e09e
Make sql file change synchronize (#6465)
closes: #6431
2021-11-10 14:50:05 +08:00
zhang
28f871e58b
[Fix-6764]modifying prompt messages and modify dolphinscheduler_mysql.sql dolphinscheduler version (#6765)
* [FIX:201106]fix issue#6657

* replacement version number

* update resource create error messag

Co-authored-by: 张柏强 <zhangbaiqiang@lvshou.com>
Co-authored-by: crane <zhangbq3306@163.com>
2021-11-09 23:34:13 +08:00
mask
0aea5ae11a
[feature-6071][plugin] add datasource plugin. (#6071) (#6361)
* [feature-6071][plugin] add datasource plugin. (#6071)

* remove the dependency of datasource in common and task-plugin module.

* the new datasource-api module provides data source connection acquisition.

* add sqlserver data source plug-in.

* add ds license

* fix check style

* update spi pom

* frx conflict resolution

* fix check style

* fix datasource ut

* fix code check style

* fix code check style

* fix datasource ut

* add pg hive mysql oracle clickhouse db2  datasource

* fix datasource package

* update datasource pom

* update datasource pom

* update datasource pom

* update datasource pom

* update pom

* update pom dep

* update pom dep

* fix license

* fix ut

* fix ut

* fix ut

* fix ut

* fix ut

* fix ut

* fix ut

* fix ut

* fix ut

* fix ut

* fix ut

* fix ut

* fix ut

* add oracle package

* modify comments

* conflict resolution

* rename log comment

* rename log comment
2021-11-09 20:20:07 +08:00
kezhenxu94
05bae0b226
Refactor alert plugins and simplify its usage (#6723) 2021-11-09 18:59:34 +08:00
kezhenxu94
02350ea37b
Fix datasource not work in tests and standalone server, and run all tests by default (#6743) 2021-11-09 16:57:26 +08:00
JinYong Li
be28d2c63b
[Bug][SnowFlakeUtils] fix snowFlake bug (#6744)
* fix snowFlake bug

* fix hash machine
2021-11-09 10:59:16 +08:00
GaoTianDuo
484153ef76
[Feature][Install-sql] supplement Postgre procedure sql file #6716 (#6741)
* [Feature][Install-sql] supplement Postgre procedure sql file #6716
1、add dolphinscheduler_ddl_post.sql

* [Feature][Install-sql] supplement Postgre procedure sql file #6716
1、modify  dolphinscheduler_ddl.sql

* [Feature][Install-sql] supplement Postgre procedure sql file #6716
1、modify Code annotation

* [Feature][Install-sql] supplement Postgre procedure sql file #6716
1、modify sql:ALTER TABLE "t_ds_process_definition" ADD CONSTRAINT "t_ds_process_definition_pkey" PRIMARY KEY ("id","code");

* [Feature][Install-sql] supplement Postgre procedure sql file #6716
1、delete code annotation
2021-11-09 10:00:08 +08:00
kezhenxu94
63602bf77c
Clean up the utility codes (#6732) 2021-11-08 23:22:07 +08:00
Wenjun Ruan
0f94577d26
Use nanoTime replace currentTimemill to avoid clock backwards (#6740) 2021-11-08 22:26:17 +08:00
wind
b6824b4741
[DS-6737][MasterServer] fix event handle twice (#6738)
Co-authored-by: caishunfeng <534328519@qq.com>
2021-11-08 16:54:58 +08:00
JinYong Li
bdea8d6ae4
[Upgrade][Install] fix upgrade 2.0 bug (#6734)
* add convert dependent/conditions

* fix upgrade 2.0 bug

* fix upgrade 2.0 bug
2021-11-08 11:57:38 +08:00
OS
24bd8a16e1
[Bug][Master] The task of dependent should use taskCode (#6730)
* fix #6682 The task of dependent should use taskCode

* fix #6682 The task of dependent should use taskCode
2021-11-07 23:20:35 +08:00
Wenjun Ruan
0afb780718
[Improvement-5807][Dao] Use h2 in UT (#6500)
* Use H2 in UT

* change scope

* Change profile name to h2

* Add datasource-h2.properties, datasource-postgresql.properties

* Fix UT

* Resolve conflict

* Remove startDatabase
2021-11-07 23:19:19 +08:00
wangxj3
d27c6ea8f0
fix bug (#6728)
Co-authored-by: wangxj <wangxj31>
2021-11-07 18:01:40 +08:00
kezhenxu94
5400abc74f
Refactor registry plugin and simplify its usage (#6712) 2021-11-07 15:18:10 +08:00
Yao WANG
73f20b7553
[Fix-#6641]fix npe when the LowerWeightHostManager can not get the workNodeInfo (#6656)
* change the return type of getHostWeight

Co-authored-by: ywang46 <ywang46@paypal.com>
2021-11-07 11:40:54 +08:00
wangyizhi
b226253eec
Replace taskName with taskCode (#6719) 2021-11-07 10:35:29 +08:00
小张
69f2d3b21b
[FIX:201106]fix issue#6657 (#6717)
Co-authored-by: 张柏强 <zhangbaiqiang@lvshou.com>
2021-11-06 19:27:00 +08:00
Yanbin Lin
a728c85a92
[Fix-6649][UI] Get project according to code (#6649) (#6650)
Co-authored-by: linyanbin <linyanbin@lizhi.fm>
2021-11-06 10:12:01 +08:00
JinYong Li
1144360257
add convert dependent/conditions (#6710) 2021-11-05 23:02:46 +08:00
myangle1120
b8d0f06ad8
[Feature-#6268][UI]Serial execte proces (#6267)
* serial

* del ip
2021-11-05 21:36:44 +08:00
ReonYu
d4d59b0627
[FIX 6701] [API] Fix paging error when worker group number more than 10 (#6704) 2021-11-05 20:23:20 +08:00
kezhenxu94
5e923971bf
Migrate http to https in maven address (#6698) 2021-11-05 20:17:27 +08:00
wind
72daba58cd
[DS-6694][Master] Reduce the refresh resource interval of LowerWeight (#6695)
* [DS-6694][Master] Reduce the refresh resource interval of LowerWeight

* add server node update event handle

Co-authored-by: caishunfeng <534328519@qq.com>
2021-11-05 19:10:36 +08:00
Jiajie Zhong
5855c936a4
[community] Add stale bot close issue/pr days (#6676)
* Add label `Waiting for reply` to bug/feature label
avoid stale bot close

* Remove bot label-on-issue

We do not need this anymore, cause when issue is created
it would auto add label `waiting for reply` for now

* Change issue/pr stale and close days.

After this PR merged, issue would be close in 37 days
and PR will be close in 127 days.
2021-11-05 18:35:41 +08:00
Jiajie Zhong
a269e3df76
Revert "Change the number of approving_review_count to 2 (#6456)" (#6706)
This reverts commit 4d69685b
2021-11-05 18:31:54 +08:00