Commit Graph

289 Commits

Author SHA1 Message Date
Eric Gao
16144a8858
[docs] Add links to metrics docs (#10583) 2022-06-24 17:27:40 +08:00
Stalary
b2df357814
[doc] Add run the task alone doc. (#10581) 2022-06-24 16:59:22 +08:00
xiangzihao
33cc68ed4a
[doc] Add doc about how to use healthcheck endpoints (#10592) 2022-06-24 16:56:26 +08:00
xiangzihao
b21e211d2c
[doc] Add docs about complement dependent mode (#10587) 2022-06-24 16:33:10 +08:00
xiangzihao
773ebc9e52
fix 10513 (#10564) 2022-06-23 13:44:22 +08:00
houshitao
1b1a567fec
[doc] Correct the complement add docs (#10521) 2022-06-23 12:02:28 +08:00
JieguangZhou
3918140e7e
[fix][doc] Add DVC task plugin doc link (#10529) 2022-06-23 10:01:49 +08:00
Eric Gao
0aba68f335
[Improvement][Metrics] Update some metrics names in grafana-demo dashboards (#10432) (#10552) 2022-06-22 23:05:54 +08:00
Eric Gao
cc06eaaf54
[Improvement][Metrics] Apply micrometer naming convention to metrics (#10477)
* Apply micrometer naming convention to worker metrics
* Apply micrometer naming convention all current metrics
* Fix remaining metrics names, update English docs and add Chinese docs
* Fix metrics names in grafana-demo dashboards
2022-06-21 14:27:06 +08:00
JieguangZhou
36e20cdfc8
[Feature][Task Plugin] Add DVC task plugin for MLops scenario (#10372) (#10407) 2022-06-21 13:13:46 +08:00
旺阳
6e4b2e6992
[Fix]Recovery Traffic Control Config (#10505)
* recovery traffic config

* run ci

* update
2022-06-20 18:43:48 +08:00
Eric Gao
64cee03fed
[Feature] Enable users to switch and install conda env in jupyter task (#10337) 2022-06-20 14:47:44 +08:00
xiangzihao
704e633659
[Doc] Add api test doc (#10499) 2022-06-19 13:55:23 +08:00
Eric Gao
4be3b877c3
[feature][task] Enable zeppelin schedule a whole zeppelin note (#10434) 2022-06-18 13:34:24 +08:00
guodong
2c227ab384
[Feature-10363][server] S3 Resource center supports bucket customization (#10433)
* [Feature-10363][server] S3 Resource center supports bucket customization

* Modify the configuration. Sort out HDFS and S3 configurations to distinguish them
* Modify the configuration comment to clarify the configuration meaning
* Modify official documents to add missing configuration descriptions and clarify configuration meanings and usage
* Modify the configuration file in e2E to the correct definition, dolphinscheduler-e2e/dolphinscheduler-e2e-case/src/test/resources/docker/file-manage/common.properties
* Modified code to support bucket customization
* Check that the bucket on personal permission, Otherwise, the startup fails
* S3 increase support resource. Storage. Upload. Base. The base path path
* Change S3Utils implementation style to be consistent with HadoopUtils

* [Feature-10363][server] S3 Resource center supports bucket customization

* Modify code review comments
2022-06-18 10:20:01 +08:00
liubo1990
b8d5e78994
[Doc] Add how to set the alarm mailbox in faq (#10401) 2022-06-17 21:16:38 +08:00
旺阳
6515c66208
[Improve] Add LDAP user not exsitst action config (#10451) 2022-06-17 14:18:05 +08:00
qianli2022
ff065d8e5b
[Feature][Improvement] Support multi cluster environments - k8s type (#10096)
* service code

* [Feature][UI] Add front-end for cluster manage

* fix e2e

* remove comment on cluster controller

* doc

* img

* setting e2e.yaml

* test

* rerun e2e

* fix bug from comment

* Update index.tsx

use Nspace instead of css.

* Update index.tsx

Remove the style.

* Delete index.module.scss

Remove the useless file.

Co-authored-by: qianl4 <qianl4@cicso.com>
Co-authored-by: William Tong <weitong@cisco.com>
Co-authored-by: Amy0104 <97265214+Amy0104@users.noreply.github.com>
2022-06-15 13:39:20 +08:00
Jiajie Zhong
b655aa9b1f
[doc] Correct resource in HDFS HA cluster (#10445) 2022-06-15 11:08:18 +08:00
旺阳
0efcd5c631
[Fix-10425]Recovery LDAP Config (#10429)
* recovery ladp code

* add ldap config in doc
2022-06-15 09:52:37 +08:00
阿福Chris
8205444e0a
[DolphinScheduler] Fix wrong class name in doc (#10438) 2022-06-14 19:53:33 +08:00
hstdream
f6fea06f10
[Improve] Enhance complement function transformation (#10376) 2022-06-12 12:35:08 +08:00
JieguangZhou
3258438f6e
[Feature] [MLOps] support mlflow deploy with docker compose (#10217)
* [Feature] [MLOps] support mlflow deploy with docker compose

fix doc

Update docs/docs/en/guide/task/mlflow.md

fix doc

Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com>

revert cancel modification

fix ENV name and docker compose command

* fix doc image link

* fix testModelsDeployDockerCompose

* add docker compose container health check and fix mlflow bug

* update docker compose healthcheck timeout
2022-06-10 10:20:50 +08:00
Wenjun Ruan
e21d7b1551
[Feature][metrics] Add master, worker metrics (#10326)
* Add mater metrics

* fix UT

* Add url to mysql profile

* Add worker metrics

* Update grafana config

* Add system metrics doc

* Add process failover counter

* Add metrics image

* Change jpg to png

* Add command insert metrics

* Fix UT

* Revert UT
2022-06-09 10:55:39 +08:00
xiangzihao
b1fb17119b
[Feature-6758][Task] Add limit resource usage for tasks base on cgroup (#10373) 2022-06-08 19:42:05 +08:00
chuxing
7e39e0d48d
[doc] Add switch task demo (#10247)
Co-authored-by: QuakeWang <45645138+QuakeWang@users.noreply.github.com>
2022-06-08 14:17:05 +08:00
He Zhao
da3c25dc67
[improve] Using create or replace function in sql task (#10170) 2022-06-06 10:33:24 +08:00
Kirs
0c25d4a3ce
[Docs]Update Slack Invite Link (#10362) 2022-06-04 23:55:23 +08:00
Jiajie Zhong
12fdd9bbd9
[doc] Use related path for img (#10325)
Use related path in our docs for imgs, previous
can not use because website need the absolute path
from root directory, and after we merged
apache/dolphinscheduler-website#789 we have covert
function to do that

close: #9426
2022-06-02 23:28:53 +08:00
xiangzihao
d1c47c99ad
fix bug_10343 (#10344) 2022-06-02 16:13:51 +08:00
QuakeWang
6828c9e015
[doc] Add how to use resource in shell task (#10308) 2022-06-01 17:28:27 +08:00
Eric Gao
442f93544e
[Bug][Api-Server]Fix bug when user without tenant uploads file in resource center (#10303) 2022-06-01 17:22:36 +08:00
Eric Gao
a8a5da367a
[Feature][Task Plugin] Add support for dynamic form for zeppelin task plugin (#9977) (#10269) 2022-05-31 11:19:41 +08:00
HuangWei
c5961aed25
[feat][task] Add OpenMLDB task plugin (#10198) 2022-05-31 11:00:21 +08:00
Jiajie Zhong
a241135cbc
[chore] update by release version (#10299) 2022-05-30 23:09:01 +08:00
xiangzihao
12a09f6d5d
[doc] Fix api standard error (#10292)
* fix api standard error
* add patch method
2022-05-30 16:07:25 +08:00
Mr.An
8d26d4fe72
[doc] Update the development environment to build the software version (#10246) 2022-05-30 14:46:12 +08:00
Yiming Guo
2294160cdb
[docs] Added local file configuration guide for resource center (#10264)
* Added Local File Resource Configuration Guide to the document.
* Removed contents with windows features in the documents and improved expression.
* Specify `the user who deploy dolphinscheduler have read and write permissions` in en and zh docs.

Co-authored-by: xiangzihao <460888207@qq.com>
2022-05-30 12:07:29 +08:00
QuakeWang
cd36401a6a
[doc] Refactor context parameter docment (#10227) 2022-05-30 11:44:10 +08:00
QuakeWang
58b1e1da1e
[doc] Upgrade screenshot to latest version in dev document (#10242)
* fix issue-9898
* delete no use pictures
2022-05-30 11:04:42 +08:00
lugela
a0771541e5
[Fix-10181] Fix the logic of judging that the tenant does not exist (#10185)
* [Fix-10181] Fix the logic of judging that the tenant does not exist

Use the linux command as id to get the user information that exists in /etc/passwd file and the cached sssd user.
for example:
id test
1. exist in /etc/passwd file or ldap :  uid=1030(test) gid=1030(test) groups=1030(test)
2. no exist  in /etc/passwd file and ldap: id: test: no such user

Temporarily unable to test the system for windows and mac

* [Fix-10181] Fix the logic of judging that the tenant does not exist

Use the linux command as id to get the user information that exists in /etc/passwd file and the cached sssd user.
for example:
id test
1. exist in /etc/passwd file or ldap :  uid=1030(test) gid=1030(test) groups=1030(test)
2. no exist  in /etc/passwd file and ldap: id: test: no such user

Temporarily unable to test the system for windows and mac

* [Fix-10181] Fix the logic of judging that the tenant does not exist

Use the linux command as id to get the user information that exists in /etc/passwd file and the cached sssd user.
for example:
id test
1. exist in /etc/passwd file or ldap :  uid=1030(test) gid=1030(test) groups=1030(test)
2. no exist  in /etc/passwd file and ldap: id: test: no such user

Temporarily unable to test the system for windows and mac

* [Fix-10181] Fix the logic of judging that the tenant does not exist

The configuration item adds 'tenant-distributed-user' in worker application.yaml to make it suitable for distributed users. If it is false, the original logic remains unchanged.

At present, considering that it is a distributed user, it should not be allowed to create users in linux

Use the linux command as id to get the user information that exists in /etc/passwd file and the cached sssd user.
for example:
id test
1. exist in /etc/passwd file or ldap :  uid=1030(test) gid=1030(test) groups=1030(test)
2. no exist  in /etc/passwd file and ldap: id: test: no such user

Temporarily unable to test the system for windows and mac

* [Fix-10181] Fix the logic of judging that the tenant does not exist

Add test method

The configuration item adds 'tenant-distributed-user' in worker application.yaml to make it suitable for distributed users. If it is false, the original logic remains unchanged.

At present, considering that it is a distributed user, it should not be allowed to create users in linux

Use the linux command as id to get the user information that exists in /etc/passwd file and the cached sssd user.
for example:
id test
1. exist in /etc/passwd file or ldap :  uid=1030(test) gid=1030(test) groups=1030(test)
2. no exist  in /etc/passwd file and ldap: id: test: no such user

Temporarily unable to test the system for windows and mac

* [Fix-10181] Fix the logic of judging that the tenant does not exist

Add parameter description to configuration.md

Add test method

The configuration item adds 'tenant-distributed-user' in worker application.yaml to make it suitable for distributed users. If it is false, the original logic remains unchanged.

At present, considering that it is a distributed user, it should not be allowed to create users in linux

Use the linux command as id to get the user information that exists in /etc/passwd file and the cached sssd user.
for example:
id test
1. exist in /etc/passwd file or ldap :  uid=1030(test) gid=1030(test) groups=1030(test)
2. no exist  in /etc/passwd file and ldap: id: test: no such user

Temporarily unable to test the system for windows and mac

* [Fix-10181] Fix the logic of judging that the tenant does not exist

Add parameter description to configuration.md

Add test method

The configuration item adds 'tenant-distributed-user' in worker application.yaml to make it suitable for distributed users. If it is false, the original logic remains unchanged.

At present, considering that it is a distributed user, it should not be allowed to create users in linux

Use the linux command as id to get the user information that exists in /etc/passwd file and the cached sssd user.
for example:
id test
1. exist in /etc/passwd file or ldap :  uid=1030(test) gid=1030(test) groups=1030(test)
2. no exist  in /etc/passwd file and ldap: id: test: no such user

Temporarily unable to test the system for windows and mac

* [Fix-10181] Fix the logic of judging that the tenant does not exist

Add parameter description to configuration.md

Add test method

The configuration item adds 'tenant-distributed-user' in worker application.yaml to make it suitable for distributed users. If it is false, the original logic remains unchanged.

At present, considering that it is a distributed user, it should not be allowed to create users in linux

Use the linux command as id to get the user information that exists in /etc/passwd file and the cached sssd user.
for example:
id test
1. exist in /etc/passwd file or ldap :  uid=1030(test) gid=1030(test) groups=1030(test)
2. no exist  in /etc/passwd file and ldap: id: test: no such user

Temporarily unable to test the system for windows and mac

Co-authored-by: ouyangl <ouyangl@tebon.com.cn>
2022-05-26 14:58:07 +08:00
Jiajie Zhong
e3579dd13d
[doc] Correct all setup documents (#10074)
* Correct the login page URL
* change version and database configuration
* change setup and upgrade
2022-05-26 14:53:47 +08:00
Jiajie Zhong
6d9bc0743b
[doc] Add setValue syntax in local param (#10245)
ref: #9576
same as: #9671
2022-05-25 21:53:41 +08:00
旺阳
aba5f8a40e
[improve] Change Mysql Driver (#10220) 2022-05-25 14:09:15 +08:00
旺阳
87d9ec1dee
[doc] Delete SPRING_DATASOURCE_DRIVER_CLASS_NAME in doc (#10228) 2022-05-25 11:45:16 +08:00
QuakeWang
5998c73aac
[doc] Refactor local parameter docment (#10119) 2022-05-24 17:16:08 +08:00
旺阳
c12848a4ff
[doc] Update kubernetes' FAQ (#10221) 2022-05-24 17:00:57 +08:00
Jiajie Zhong
f4b7754952
[doc] Update task DataX document (#10218) 2022-05-24 13:45:26 +08:00
JieguangZhou
54ab2651bd
[Feature][MLops] Add MLflow Projects (custom project) component (#10175)
* Add MLflow Projects (custom project) component

* fix something

* update document

* update document

* Supplementary test cases
2022-05-24 09:42:46 +08:00
Mr.An
31cb485312
[doc] Update delete workflow description (#10203) 2022-05-23 20:59:19 +08:00