Commit Graph

6743 Commits

Author SHA1 Message Date
JieguangZhou
3258438f6e
[Feature] [MLOps] support mlflow deploy with docker compose (#10217)
* [Feature] [MLOps] support mlflow deploy with docker compose

fix doc

Update docs/docs/en/guide/task/mlflow.md

fix doc

Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com>

revert cancel modification

fix ENV name and docker compose command

* fix doc image link

* fix testModelsDeployDockerCompose

* add docker compose container health check and fix mlflow bug

* update docker compose healthcheck timeout
2022-06-10 10:20:50 +08:00
WangJPLeo
9782fe4e7c
[Fix-10367] Permission check related key adjustment. (#10368)
* Permission check related key adjustment.

* resource create or upload fix.

* Analysis Permission fix.

* Ut coverage add.

* rerun

* source permission fix.

* ut coverage is achieved

* Reference permission package path modification

* k8s path bug fix.

* k8s path bug fix.

* Corrected controller method name and rerun.
2022-06-09 19:10:34 +08:00
xiangzihao
997f0d4bfd
[CI] Add some codeowner (#10385)
* add some codeowner

* add some codeowner
2022-06-09 17:06:27 +08:00
PJ Fanning
f90f0f8490
Upgrade log4j in E2E (#10359) 2022-06-09 16:09:46 +08:00
Hwting
5803d5c07c
[fix][deploy]fix run on kubernetes bug (#10379)
* [fix][kubernetes]fix run on kubernetes bug

Fix the problem that the "REGISTRY_ZOOKEEPER_CONNECT_STRING" variable zookeeper port number is "nil"

* [fix][kubernetes]fix run on kubernetes bug

Fix the problem that the "REGISTRY_ZOOKEEPER_CONNECT_STRING" variable zookeeper port number is "nil"
2022-06-09 15:57:20 +08:00
xiangzihao
bbba20d39a
change to common lang3 and fix checkstyle (#10380) 2022-06-09 15:56:42 +08:00
Wenjun Ruan
e21d7b1551
[Feature][metrics] Add master, worker metrics (#10326)
* Add mater metrics

* fix UT

* Add url to mysql profile

* Add worker metrics

* Update grafana config

* Add system metrics doc

* Add process failover counter

* Add metrics image

* Change jpg to png

* Add command insert metrics

* Fix UT

* Revert UT
2022-06-09 10:55:39 +08:00
Wenjun Ruan
3ecbee3885
Optimize MasterServer, add MasterRPCService (#10371)
* Optimize MasterServer, avoid NPE
2022-06-09 10:00:05 +08:00
xiangzihao
b1fb17119b
[Feature-6758][Task] Add limit resource usage for tasks base on cgroup (#10373) 2022-06-08 19:42:05 +08:00
chuxing
7e39e0d48d
[doc] Add switch task demo (#10247)
Co-authored-by: QuakeWang <45645138+QuakeWang@users.noreply.github.com>
2022-06-08 14:17:05 +08:00
songjianet
36bba3fcdf
[Docs] Upgrade license. (#10365) 2022-06-06 11:16:06 +08:00
retime123
67c4f2b259
[Feature][Monitoring] Added disk available capacity monitoring (#10287)
* Added disk available capacity monitoring

* resolve conflict
Added disk available capacity monitoring

* resolve conflict
Added disk available capacity monitoring

* resolve conflict
Added disk available capacity monitoring
2022-06-06 11:06:33 +08:00
Wenjun Ruan
516757cc74
[Fix] [Worker] Fix worker will hang if fails to start (#10342)
* Fix worker will hang if fails to start

* Add .run to ignore
2022-06-06 10:49:23 +08:00
He Zhao
da3c25dc67
[improve] Using create or replace function in sql task (#10170) 2022-06-06 10:33:24 +08:00
WangJPLeo
ffe809b2c5
[Fix-10338] Tenant does not belong to resource permission. (#10339)
* Tenant does not belong to resource permission.

* rerun

* rerun again

* resource create and upload fix.

* func constant clear.
2022-06-05 22:34:25 +08:00
Kirs
0c25d4a3ce
[Docs]Update Slack Invite Link (#10362) 2022-06-04 23:55:23 +08:00
Wenjun Ruan
2d3be6b36c
Add dolphinscheduler-scheduler module (#10360)
* Add dolphinscheduler-scheduler module
2022-06-04 16:39:33 +08:00
Wenjun Ruan
022e4886be
Remove quartz at WorkerServer (#10358)
* Remove quartz at WorkerServer

* move k8s and permission from dolphinscheduler-service to dolphinscheduler-api
2022-06-04 00:18:01 +08:00
PJ Fanning
d044e0479d
issue-10356: upgrade logback to fix cve (#10357) 2022-06-03 19:21:40 +08:00
PJ Fanning
b6350280e6
issues-10354: upgrade commons-io to fix CVE (#10355) 2022-06-03 17:17:52 +08:00
Wenjun Ruan
0bf4043d5f
Move datasource params to datasouce plugin module (#10353)
* Move datasource params to datasouce plugin module
2022-06-03 12:15:53 +08:00
Jiajie Zhong
12fdd9bbd9
[doc] Use related path for img (#10325)
Use related path in our docs for imgs, previous
can not use because website need the absolute path
from root directory, and after we merged
apache/dolphinscheduler-website#789 we have covert
function to do that

close: #9426
2022-06-02 23:28:53 +08:00
Wenjun Ruan
b02243c2f5
Add dolphinscheduler-task-all in api module (#10350) 2022-06-02 22:12:27 +08:00
guodong
96496f7255
[Fix-10318][ui] Fix EMR Task UI custom parameters Is not correct (#10334)
* Modify useCustomParams isSimple: true
2022-06-02 18:51:43 +08:00
xiangzihao
35155cde9a
[CI] Add code owner of .github (#10347)
* add code owner

* add code owner

* add code owner
2022-06-02 18:34:03 +08:00
songjianet
6134e0a628
[Build][UI] Upgrade dependencies. (#10346)
* [Build][UI] Upgrade dependencies.

* [Build][UI] Upgrade dependencies.

* [Build][UI] Upgrade dependencies.
2022-06-02 18:23:57 +08:00
xiangzihao
50e9f2916a
[CI] bumping actions/cache to v3 to fix ci error (#10345)
* bumping actions/cache to v3

* fix JSONUtilsTest.java unit test error
2022-06-02 17:26:52 +08:00
xiangzihao
d1c47c99ad
fix bug_10343 (#10344) 2022-06-02 16:13:51 +08:00
guodong
70dbcb2155
[Fix-10305][common] AWS config Constant definitions are repeated (#10311) 2022-06-02 13:56:36 +08:00
WangJPLeo
e57c2d1663
[Feature] Should not add refresh logic in the e2e case. (#10331)
* Should not add refresh logic in the e2e case.

* When the returned data from paging is empty, it should be an empty collection.

* resource query fix.

* e2e rerun

* timeZone ut fix

* udf resource check fix

* code deduplication

* rerun
2022-06-02 09:48:32 +08:00
songjianet
e8ad62bcd0
[Fix][UI] Fixed the issue that the status statistics on the project overview page were inconsistent. (#10328) 2022-06-01 21:50:43 +08:00
WangJPLeo
e3e39cbdea
[Fix] Privilege Reconstruction Data Query Fix & Resource Creation Post Oper… (#10313)
* Privilege Reconstruction Data Query Fix & Resource Creation Post Operations.

* Resource Retry

* e2e retry

* create project log add

* project list query log add

* clean project log

* delete delay

* delete delay

* remove post handle

* project e2e

* browser refresh

* browser refresh

* e2e fix

* e2e browser refresh

* rowk flow e2e fix

* mapper deduplication

* udf e2e

* e2e
2022-06-01 20:29:10 +08:00
QuakeWang
6828c9e015
[doc] Add how to use resource in shell task (#10308) 2022-06-01 17:28:27 +08:00
Eric Gao
442f93544e
[Bug][Api-Server]Fix bug when user without tenant uploads file in resource center (#10303) 2022-06-01 17:22:36 +08:00
aiwenmo
3f2a21293d
[Fix-10317] [ui] Fix the tips of startTimeout in the jupyter form (#10319)
Co-authored-by: wenmo <32723967+wenmo@users.noreply.github.com>
2022-06-01 10:19:36 +08:00
Jiajie Zhong
0ff34b8730
[ci] Fix mergeable for ddl change and remove label unexpect (#7099) 2022-06-01 09:32:50 +08:00
songjianet
8584aad490
[Style][UI] Format code. (#10314) 2022-05-31 21:01:36 +08:00
kezhenxu94
d80cf21456
Clean up unused dependencies and packaging issues (#9944) 2022-05-31 15:22:41 +08:00
xiangzihao
8c72552237
[Feature-10290] [API] Refactor org.apache.dolphinscheduler.api.controller.AccessTokenController#createToken api (#10301)
* add feature_10290

* fix dolphinscheduler-ui/pnpm-lock.yaml error

* fix dolphinscheduler-ui/pnpm-lock.yaml error

* fix incorrect dependency

* fix incorrect dependency and unit test

* fix unit test error

* add dolphinscheduler-api/logs to .gitignore

* add AccessTokenV2ControllerTest.java

* fix AccessTokenV2ControllerTest error
2022-05-31 14:01:25 +08:00
Wenjun Ruan
3a9e67325e
Fix database initialize late in standalone. (#10296) 2022-05-31 11:52:00 +08:00
JinYong Li
49979c658e
[Fix-8828] [Master] Assign tasks to worker optimization (#9919)
* fix 9584

* master recall

* fix ut

* update logger

* update delay queue

* fix ut

* remove sleep

Co-authored-by: 进勇 <lijinyong@cai-inc.com>
Co-authored-by: JinyLeeChina <jiny.li@foxmail.com>
2022-05-31 11:49:54 +08:00
Eric Gao
a8a5da367a
[Feature][Task Plugin] Add support for dynamic form for zeppelin task plugin (#9977) (#10269) 2022-05-31 11:19:41 +08:00
HuangWei
c5961aed25
[feat][task] Add OpenMLDB task plugin (#10198) 2022-05-31 11:00:21 +08:00
WangJPLeo
a4948f58e6
[Feature][Permission] Reconstruction of permissions of resource center and monitoring center. (#10307)
* Reconstruction of permissions of resource center and monitoring center.

* clear local logs.

* resource query fix
2022-05-31 08:14:45 +08:00
hstdream
53ab6f7b92
[Feature][API] Modify the permissions of project management, security center, data source center and data quality module. (#10289)
* Resource userName is null

* [Fix-10082]Resource userName is null and udf function manager add a username

* [Fix-10082]Resource userName is null and udf function manager add a username

* [Fix-10082]Resource userName is null and udf function manager add a username

* [Feature][API] Modify the permissions of project management, security center, data source center and data quality module.

* [Feature][API] Modify the permissions of project management, security center, data source center and data quality module.

* [Feature][API] Modify the permissions of project management, security center, data source center and data quality module.

Co-authored-by: houshitao <shitaohou@163.com>
2022-05-30 23:32:25 +08:00
Jiajie Zhong
a241135cbc
[chore] update by release version (#10299) 2022-05-30 23:09:01 +08:00
songjianet
2e6d3f5190
[Fix][UI] Display the result page when the DB page has no data. (#10298) 2022-05-30 19:28:36 +08:00
xiangzihao
12a09f6d5d
[doc] Fix api standard error (#10292)
* fix api standard error
* add patch method
2022-05-30 16:07:25 +08:00
Mr.An
8d26d4fe72
[doc] Update the development environment to build the software version (#10246) 2022-05-30 14:46:12 +08:00
Tq
592e433b1e
[Feature][Datasource]replace DataSourceController API with string JSON and cast to DTO (#10276)
* replace DataSourceController API with string JSON and cast to DTO

* add DataSourceProcessorManager and DataSourceProcessorProvider to datasource processor functions
2022-05-30 14:12:19 +08:00