* [Improvement-11550] [Doc]Document content update
* Update pictures
* fix
* Update Process word to Workflow
Co-authored-by: David <davidzollo365@gmail.com>
In the process of realizing datax synchronization, use custom json for better connection to enable kerberos-authenticated clusters, such as using plug-ins such as hdfs and hbase
* Add waiting strategy to support master/worker can recover from registry lost
* throw exception when zookeeper registry start failed due to interrupted
* [Doc][Resources] Instruct users to use local storage if they have remote storage mounted to local (#11427)
* Remove dead link in pyds README
* Add hyperlinks for docs
Co-authored-by: Jiajie Zhong <zhongjiajie955@hotmail.com>
* Using sphinx-multiversion to support multiple version
of python api document
* change python api sidebar link
* fix whitelist_externals deprecated and add git
close: #8227
This pull request update the FAQ doc (faq.md) by adding how to obtain the IP address
from the specified network card, which is part of relevant information created when
the master service and worker service are registered with zookeeper.
* Our community do not have official support for stack overflow any more
* Change the order side for Slack and mailing list, to avoid some users
just want to ask question but join mailing list, and then unsubscribe
found there are too many emails in mailing list
* [Improvement][Metrics] Add metrics for alert server (#11131)
* Update related docs of metrics
* Add grafana demo dashboards for alert server metrics
* Refactor metric classes with UtilityClass annotation
* Refactor meter names in camelCase for checkstyle
* [Improvement][Metrics] Switch to use tags to indicate task / workflow execution status for metrics (#10867)
* Update docs and grafana demo dashboards
* [Feature][Task Plugin] Increase zeppelin task stability in production (#10584)
* Add front-end and update docs for the production mode of zeppelin task
* Fix minor front-end bug of zeppelin task plugin
* Refactor ZeppelinParameters with lombok
* Fix formatting
* Replace @Data with @Getter, @Setter, @ToString to avoid decrease in test coverage
* [Feature][Metrics] Add resource download related metrics for workers (#9324)
* [Feature][Metrics] Fix bugs and add grafana demos for worker resource download metrics (#9324)
* [Feature][Metrics] Add docs to resource related metrics (#9324)
* [Feature][Metrics] Use tags to indicate status in metrics (#9324)
* [Feature][Metrics] Fix demos, docs and remove redundant code (#9324)
* [Feature][Metrics] Remove .pnpm-debug.log (#9324)
* [Feature][Metrics] Fix style check (#9324)
* [Feature][Metrics] Replace KB with bytes for the unit of resource file size in metrics (#9324)
* [Feature][Metrics] Make code neat (#9324)
* Add the ProgramType parameter to distinguish task types
* EmrAddStepsTask supports Add-Steps
* UI supports Add-Steps
* EmrTask modify the name of the class to EmrJobFlowTask
* add ERM Task abstract base class AbstractEmrTask
* add testcase for EmrAddStepsTask
* Modifying help Documents
* Change sh to bash
Change the script execution command sh to bash
* Change sh to bash
Change the script execution command sh to bash
* Change sh to bash
Change the script execution command sh to bash
* Change sh to bash
Change the script execution command sh to bash
* Change sh to bash
Co-authored-by: luke <luke.yan@china-entercom.net>
* Apply micrometer naming convention to worker metrics
* Apply micrometer naming convention all current metrics
* Fix remaining metrics names, update English docs and add Chinese docs
* Fix metrics names in grafana-demo dashboards
* [Feature-10363][server] S3 Resource center supports bucket customization
* Modify the configuration. Sort out HDFS and S3 configurations to distinguish them
* Modify the configuration comment to clarify the configuration meaning
* Modify official documents to add missing configuration descriptions and clarify configuration meanings and usage
* Modify the configuration file in e2E to the correct definition, dolphinscheduler-e2e/dolphinscheduler-e2e-case/src/test/resources/docker/file-manage/common.properties
* Modified code to support bucket customization
* Check that the bucket on personal permission, Otherwise, the startup fails
* S3 increase support resource. Storage. Upload. Base. The base path path
* Change S3Utils implementation style to be consistent with HadoopUtils
* [Feature-10363][server] S3 Resource center supports bucket customization
* Modify code review comments
Use related path in our docs for imgs, previous
can not use because website need the absolute path
from root directory, and after we merged
apache/dolphinscheduler-website#789 we have covert
function to do that
close: #9426
* Added Local File Resource Configuration Guide to the document.
* Removed contents with windows features in the documents and improved expression.
* Specify `the user who deploy dolphinscheduler have read and write permissions` in en and zh docs.
Co-authored-by: xiangzihao <460888207@qq.com>
* [Fix-10181] Fix the logic of judging that the tenant does not exist
Use the linux command as id to get the user information that exists in /etc/passwd file and the cached sssd user.
for example:
id test
1. exist in /etc/passwd file or ldap : uid=1030(test) gid=1030(test) groups=1030(test)
2. no exist in /etc/passwd file and ldap: id: test: no such user
Temporarily unable to test the system for windows and mac
* [Fix-10181] Fix the logic of judging that the tenant does not exist
Use the linux command as id to get the user information that exists in /etc/passwd file and the cached sssd user.
for example:
id test
1. exist in /etc/passwd file or ldap : uid=1030(test) gid=1030(test) groups=1030(test)
2. no exist in /etc/passwd file and ldap: id: test: no such user
Temporarily unable to test the system for windows and mac
* [Fix-10181] Fix the logic of judging that the tenant does not exist
Use the linux command as id to get the user information that exists in /etc/passwd file and the cached sssd user.
for example:
id test
1. exist in /etc/passwd file or ldap : uid=1030(test) gid=1030(test) groups=1030(test)
2. no exist in /etc/passwd file and ldap: id: test: no such user
Temporarily unable to test the system for windows and mac
* [Fix-10181] Fix the logic of judging that the tenant does not exist
The configuration item adds 'tenant-distributed-user' in worker application.yaml to make it suitable for distributed users. If it is false, the original logic remains unchanged.
At present, considering that it is a distributed user, it should not be allowed to create users in linux
Use the linux command as id to get the user information that exists in /etc/passwd file and the cached sssd user.
for example:
id test
1. exist in /etc/passwd file or ldap : uid=1030(test) gid=1030(test) groups=1030(test)
2. no exist in /etc/passwd file and ldap: id: test: no such user
Temporarily unable to test the system for windows and mac
* [Fix-10181] Fix the logic of judging that the tenant does not exist
Add test method
The configuration item adds 'tenant-distributed-user' in worker application.yaml to make it suitable for distributed users. If it is false, the original logic remains unchanged.
At present, considering that it is a distributed user, it should not be allowed to create users in linux
Use the linux command as id to get the user information that exists in /etc/passwd file and the cached sssd user.
for example:
id test
1. exist in /etc/passwd file or ldap : uid=1030(test) gid=1030(test) groups=1030(test)
2. no exist in /etc/passwd file and ldap: id: test: no such user
Temporarily unable to test the system for windows and mac
* [Fix-10181] Fix the logic of judging that the tenant does not exist
Add parameter description to configuration.md
Add test method
The configuration item adds 'tenant-distributed-user' in worker application.yaml to make it suitable for distributed users. If it is false, the original logic remains unchanged.
At present, considering that it is a distributed user, it should not be allowed to create users in linux
Use the linux command as id to get the user information that exists in /etc/passwd file and the cached sssd user.
for example:
id test
1. exist in /etc/passwd file or ldap : uid=1030(test) gid=1030(test) groups=1030(test)
2. no exist in /etc/passwd file and ldap: id: test: no such user
Temporarily unable to test the system for windows and mac
* [Fix-10181] Fix the logic of judging that the tenant does not exist
Add parameter description to configuration.md
Add test method
The configuration item adds 'tenant-distributed-user' in worker application.yaml to make it suitable for distributed users. If it is false, the original logic remains unchanged.
At present, considering that it is a distributed user, it should not be allowed to create users in linux
Use the linux command as id to get the user information that exists in /etc/passwd file and the cached sssd user.
for example:
id test
1. exist in /etc/passwd file or ldap : uid=1030(test) gid=1030(test) groups=1030(test)
2. no exist in /etc/passwd file and ldap: id: test: no such user
Temporarily unable to test the system for windows and mac
* [Fix-10181] Fix the logic of judging that the tenant does not exist
Add parameter description to configuration.md
Add test method
The configuration item adds 'tenant-distributed-user' in worker application.yaml to make it suitable for distributed users. If it is false, the original logic remains unchanged.
At present, considering that it is a distributed user, it should not be allowed to create users in linux
Use the linux command as id to get the user information that exists in /etc/passwd file and the cached sssd user.
for example:
id test
1. exist in /etc/passwd file or ldap : uid=1030(test) gid=1030(test) groups=1030(test)
2. no exist in /etc/passwd file and ldap: id: test: no such user
Temporarily unable to test the system for windows and mac
Co-authored-by: ouyangl <ouyangl@tebon.com.cn>
This patch change the structure and make it more sense
* Remove dev doc in tab Docs, and for now you could only see it in
history-version page
* Move PyDolphinScheduler from tab DOCS to sidebar and combine with open
API
* Move installation after Functions
* Move data quality after HOW-TOS
* [fix] Enhance name pre checker in resource center
Add file name and directory checker to avoid directory traversal
* add some missing change and change docs
* change var name in directoryTraversal
* Fix ci
