* Fixed this bug "multiple threads creating folders will result in multiple identical folders" #3487
* 1. Improvement `DataSourceServiceTest` #3558
2. Get rid of some useless arguments to functions in the `DataSourceService` class
* The rollback `ResourceService.java`
* The rollback `ResourceService.java`
* The rollback `ResourceService.java`
* 1.Modify code styles.
2.Make improvement according to Yichao Yang's advice. Thank you very much for guidance.@Yichao Yang
* Modify code styles.
* Circumventing the checkConnection problem
* Modify code styles.
* * add presto datasource support
update .gitigonre to igonre some files
* * use another presto driver
* * add LICENSE files about presto-jdbc
* * just for test sonar
Co-authored-by: rockxsj <junjun@mgtv.com>
* fix bug #3165 get resource.storage.type value toUpperCase
* fix bug #3176 optimize Gets the value of this property “resource.storage.type”
* fix When you request the URL through applicationID to get the application status, you cannot get it if Kerberos authentication is enabled
* fix When you request the URL through applicationID to get the application status, you cannot get it if Kerberos authentication is enabled
* fix When you request the URL through applicationID to get the application status, you cannot get it if Kerberos authentication is enabled
* fix miss
* add KerberosHttpClient fix kerberos bug
* fix map init
* add juint test
* Extraction of public methods
* Extraction of public methods
* Fix code quality
* Fix code quality
* Fix code quality
* Fix code quality
* Fix code quality
* Fix code quality
* Fix code quality
* rebuild test
* rebuild test
* check style
* check style
* check style
* revert UT POM
* Kerberos judgment goes up to hadoopUtils
* fix merge
* Remove connection pool shutdown
Co-authored-by: dailidong <dailidong66@gmail.com>
* [Feature-3134]Refactor to use a shared singleton Curator Zookeeper Client
* autowire CuratorZookeeperClient to ZookeeperOperator
* [Improvement] Add the issue specifications reference (#3221)
* log zookeeper address when the connect state change
* resume the operation of add connect state listener in MasterRegistry
Co-authored-by: Yichao Yang <1048262223@qq.com>
* fix the bug #2923 (Hive JDBC connection parameter ignored) , the hive jdbc connection url ;
* fix the bug index out of bound and add some test case
* add the Licensed
* fix the checkstyle and import StringBuilder
* put HiveDataSourceTest.java to root pom maven-surefire-plugin to avoiding the sonar 0.0% Coverage.
* Update HiveDataSource.java
* fix the bug #3240 [worker] Cannot find a (Map) Key deserializer for type
* [DS-3245][fix] change toString use JsonString
- change the method of UdfFunc.toString use JsonString
This closes#3245
* [DS-3245][fix] change toString use JsonString
- fix the junit test for full
This closes#3245
* [DS-3245][fix] change toString use JsonString
- fix the sonarcloud merge error, Swap these 2 arguments so they are in the correct order: expected value, actual value.
This closes#3245
* [DS-3245][fix] move the Class UdfFuncDeserializer to UdfFunc.java
move the class UdfFuncDeserializer.java to UdfFunc.java
This closes#3245
* [DS-3245][fix] move the Class UdfFuncDeserializer to UdfFunc.java
add blank line
This closes#3245
* [DS-3245][fix] add the unit test of UdfFunc
add the unit test of UdfFunc.java
This closes#3245
* [DS-3245][fix] add the unit test of UdfFunc
remove the UdfFuncTest.java from maven-surefire-plugin list
This closes#3245
* [DS-3245][fix] add the unit test of UdfFunc
add the UdfFuncTest to the root pom maven-surefire-plugin
This closes#3245
Co-authored-by: dailidong <dailidong66@gmail.com>
* [Feature-2815][server] One worker can belong to different workergroups
* Feature: Add test cases
* Update MonitorService.java
* Update MonitorService.java
Co-authored-by: dailidong <dailidong66@gmail.com>
* merge from 1.3.1-release
* [bugfix] remove Conflicting configuration sudo and user (#3038)
Co-authored-by: zhangchunyang1024 <zhangchunyang0123@gamil.com>
* remove some standard notices in order to cut down the NOTICE file. (#3101)
* #3084 remove some standard notices in order to cut down the NOTICE file.
* update exclude element sql/soft_version of rat plugin
* update soft_version
* merge from 1.3.1-release
* fix 3112:download udf resource need find the tenant code of resource owner (#3114)
* fix 3112:download udf resource need find the tenant code of resource owner
* fix 2975:download resource need find the tenant code of resource owner
* remove useless code
* merge from 1.3.1-release
* remove some notes (#3127)
Co-authored-by: lenboo <baoliang@analysys.com.cn>
* fix Permission denied if not found dolphinscheder-env.sh on 1.3.1 (#3128)
* fix Permission denied if not found dolphinscheder-env.sh
* update some comments on method
* add PropertyPlaceholderHelper on license (#3129)
* update license header location
* add PropertyPlaceholderHelper on license
* merge from 1.3.1-release
* change from 1.3.0 to 1.3.2
* add Set import
Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: zhangchunyang <zhangchunyang0123@gmail.com>
Co-authored-by: zhangchunyang1024 <zhangchunyang0123@gamil.com>
Co-authored-by: lgcareer <18610854716@163.com>
Co-authored-by: lenboo <baoliang@analysys.com.cn>
* fix the bug #2923 (Hive JDBC connection parameter ignored) , the hive jdbc connection url ;
* fix the bug index out of bound and add some test case
* add the Licensed
* fix the checkstyle and import StringBuilder
* put HiveDataSourceTest.java to root pom maven-surefire-plugin to avoiding the sonar 0.0% Coverage.
* Update HiveDataSource.java
Co-authored-by: dailidong <dailidong66@gmail.com>
2. remove getMapper method,replace with toList()
3. rewrite toList method,support constructCollectionType.
4. parseObject(resp, Map.class) replace to toMap() method
5. rename JacksonSerialize to JsonSerializer
rewrite createArrayNode,createObjectNode method will Affect too many lines ，i will fix it in the next pr.
* fix bug: send task result to master until success.
* fix bug: send task result to master until success.
* add sleep
* set null if send result failed
* set null if send result failed
* change max length of username to 39
* test for gc
* change version to 1.3.0
* check and add some license files
* check and add some license files
* revert
Co-authored-by: baoliang <baoliang@analysys.com.cn>
* add LoggerServerTest UT
* add LoggerServerTest UT
* add LoggerServerTest UT
add RemoveTaskLogRequestCommandTest UT
add RemoveTaskLogResponseCommandTest
Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
* Add a script to check the license
### Motivation
Check licenses in the distribution package
### Modification
1. Add a script to check licenses, and list known licenses.
1. Remove unused dep license, lombok.
1. Remove unnecessary ignored patterns in .gitignore.
1. Add missing config items to apache-rat plugin, which just checked a small part of files before.
1. Add check to GitHub Actions process
### Result
1. Newly-added dependencies should be checked and confirmed.
1. Closes#1578
* Separate rat and dependency check
* Tee dependencies to stdout
* Add more debug messages
* Check light-weight task first
* dispatch task fail will set task status failed
* 1，no worker condition , master will while ture wait for worker startup
2，worker response task status sync wait for result
* 1，no worker condition , master will while ture wait for worker startup
2，worker response task status sync wait for result
* 1，no worker condition , master will while ture wait for worker startup
2，worker response task status sync wait for result
* 1，no worker condition , master will while ture wait for worker startup
2，worker response task status sync wait for result
* 1，no worker condition , master will while ture wait for worker startup
2，worker response task status sync wait for result
* 1，no worker condition , master will while ture wait for worker startup
2，worker response task status sync wait for result
* 1，no worker condition , master will while ture wait for worker startup
2，worker response task status sync wait for result
* 1，no worker condition , master will while ture wait for worker startup
2，worker response task status sync wait for result
* 1，no worker condition , master will while ture wait for worker startup
2，worker response task status sync wait for result
* 1，no worker condition , master will while ture wait for worker startup
2，worker response task status sync wait for result
* 1，no worker condition , master will while ture wait for worker startup
2，worker response task status sync wait for result
* 1，task status statistics and process status statistics bug fix (#2357)
2，worker group bug fix
* 1，task status statistics and process status statistics bug fix (#2357)
2，worker group bug fix
* 1，task status statistics and process status statistics bug fix (#2357)
2，worker group bug fix
* 1，task status statistics and process status statistics bug fix (#2357)
2，worker group bug fix
* send mail error, #2466 bug fix
* send mail error, #2466 bug fix
* send mail error, #2466 bug fix
* send mail error, #2466 bug fix
* #2486 bug fix
* host and workergroup compatible
* EnterpriseWeChatUtils modify
* EnterpriseWeChatUtils modify
* EnterpriseWeChatUtils modify
* #2499 bug fix
* add comment
* revert comment
* revert comment
* #2499 buf fix
* #2499 bug fix
* #2499 bug fix
* #2499 bug fix
* #2499 bug fix
* #2499 bug fix
* #2499 bug fix
* no valid worker group，master can kill task directly
* no valid worker group，master can kill task directly
* no valid worker group，master can kill task directly
* no valid worker group，master can kill task directly
* no valid worker group，master can kill task directly
* no valid worker group，master can kill task directly
* no valid worker group，master can kill task directly
* no valid worker group，master can kill task directly
* no valid worker group，master can kill task directly
Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
* solve code spell problem
* remove controller test
* add controller test
* sovle spell problem and sovle conflict
* sovle spell problem and sovle conflict
* sovle spell problem and add test
Co-authored-by: sunchaohe <sunzhaohe@linklogis.com>
* api server exception management and code optimization (#397)
* add unit test for exception handler.
* fix some style and naming issues
Co-authored-by: dailidong <dailidong66@gmail.com>
* Remove .helmignore file
* Exclude .mvn/jvm.config and .mvn/wrapper/maven-wrapper.properties
When running rat, exclude .mvn/jvm.config and .mvn/wrapper/maven-wrapper.properties
* refactor ConnectionFactory to static inner holder Singleton
* remove redundant import
* make getSqlSessionFactory() method private
* fix sonar issue
* remove @Ignore of MailUtilsTest
* add MailUtilsTest to pom.xml to test
* fix MailUtilsTest path error in pom.xml
* modify test method name
* add assert and logger to unit test
* add log
* add log
* add AlertDaoTest
* move AlertDaoTest to new module
* modify test in pom.xml
* remove unnecessary log
* Update quick-start.md
* 更新demo的地址
* add rpm plugin
* delete the git ignore files
* 测试git ignore
* 测试ignore
* optimize the .gitignore file
* add rpm package, delete unuse copy jar action in assembly
* delete docs dir
* add rpm plugin support
* add rpm plugin support
* add rpm plugin support
* dolphinscheduler-common have not bin dir
* dolphinscheduler-common have not bin dir
* delete unuse config about assembly
* add defineSatement
* add rpm plugin support
* add rpm plugin support
* add rpm plugin support
* add rpm support
* update install dirname , make front and backend together
* update rpm name
* update rpm name
* update rpm config
* add jars excludes
* add jars excludes
* add rpm plugion
* add rpm plugion
* add rpm plugion
* add auto create /opt/soft
* The install dir can not be fixed to apache-dolphinscheduler-incubating-1.2.1 , its version is 1.2.2
* install from rpm will create a link
* the ui install thrown rpm only support jetty not nginx
* the ui install thrown rpm only support jetty not nginx
* sql目录里加上soft_version文件
* env dir need in conf dir
* remove no use module
* add module
* updaate
* rpmbuild add dolphinscheduler-service module
* Fix javadoc and DataxTaskTest UT error
* Fix UT error
* LoggerServiceTest UT error
* WorkerGroupServiceTest UT error
Modify Host.of()
* Add address NPE check
* Trigger actions rerun
* Try to solve maven VM crash
* Try to solve maven VM crash 1
* Try to solve maven VM crash 2
* Try to solve maven VM crash 3
* Adapting partial code(file name start with F) to the sonar cloud rule
* add more unit test
* add License
* add includes configuration to maven-surefire-plugin
* fix getResourceFilesList incorrect logic
* Support worker server to run bat script
1. Reimplement ProcessImpl.java, ProcessEnvironment.java and ProcessBuilder.java for Windows
2. Modify shell task code for windows
3. Add ASF License
* Add Unit Test
* 1，remove dolphinscheduler-rpc module 2，add dolphinscheduler-remote module 3，add dolphinscheduler-service module 4，refactor LoggerServer module (#1925)
* 1，remove dolphinscheduler-rpc module
2，add dolphinscheduler-remote module
3，add dolphinscheduler-service module
4，refactor LoggerServer module
* ProcessUtils modify
* Refactor architecture (#1926)
* move version to parent pom
* move version properties to parent pom for easy management
* remove freemarker dependency
* delete CombinedApplicationServer
* #1871 correct spelling
* #1873 some updates for TaskQueueZkImpl
* #1875 remove unused properties in pom
* #1878
1. remove tomcat dependency
2. remove combined_logback.xml in api module
3. format pom.xml for not aligning
* #1885 fix api server startup failure
1. add jsp-2.1 dependency
2. remove jasper-runtime dependency
* add stringutils ut (#1921)
* add stringutils ut
* Newfeature for #1675. (#1908)
Continue to finish the rest works, add the cache feature for dependence,mr,python,sub_process,procedure and shell.
* Add modify user name for process definition (#1919)
* class overrides equals() and should therefore also override hashCode()
* #1862 add modify user in process difinition list
* #1862 add pg-1.2.2 ddl.sql
* modify ScriptRunnerTest
* add updateProessDifinition UT
* modify updateProcessDifinition UT
* modify updateProcessDifinition UT
* modify mysql 1.2.2 ddl.sql&dml.sql
* add scope test to mysql in pom
* modify pg-1.2.2 ddl.sql
* refactor module
* updates
Co-authored-by: khadgarmage <khadgar.mage@outlook.com>
Co-authored-by: zhukai <boness@qq.com>
Co-authored-by: Yelli <amarantine@my.com>
* dolphinscheduler-common remove spring (#1931)
* dolphinscheduler-common remove spring
* dolphinscheduler-common remove spring
* dolphinscheduler-common remove spring
* dolphinscheduler-common remove spring
* dolphinscheduler-common remove spring
* SpringApplicationContext class title add license (#1932)
* dolphinscheduler-common remove spring
* dolphinscheduler-common remove spring
* dolphinscheduler-common remove spring
* dolphinscheduler-common remove spring
* dolphinscheduler-common remove spring
* dolphinscheduler-common remove spring
* add license (#1934)
* dolphinscheduler-common remove spring
* dolphinscheduler-common remove spring
* dolphinscheduler-common remove spring
* dolphinscheduler-common remove spring
* dolphinscheduler-common remove spring
* dolphinscheduler-common remove spring
* Refactor architecture (#1936)
* move datasource classes to dao module
* fix send4LetterWord bug
* LoggerServiceTest remove ProcessDao (#1944)
* dolphinscheduler-common remove spring
* dolphinscheduler-common remove spring
* dolphinscheduler-common remove spring
* dolphinscheduler-common remove spring
* dolphinscheduler-common remove spring
* dolphinscheduler-common remove spring
* LoggerServiceTest remove ProcessDao
* exclude jasper-compiler in case of runtime conflict (#1938)
* move datasource classes to dao module
* fix send4LetterWord bug
* exclude jasper-compiler in case of runtime conflict
* DataAnaylysisServiceTest and ProcessDefinitionService modify
* remote module add comment
* OSUtilsTest modify
* add finally block to close channel (#1951)
* move datasource classes to dao module
* fix send4LetterWord bug
* exclude jasper-compiler in case of runtime conflict
* add finally block to close channel
Co-authored-by: Tboy <technoboy@yeah.net>
Co-authored-by: khadgarmage <khadgar.mage@outlook.com>
Co-authored-by: zhukai <boness@qq.com>
Co-authored-by: Yelli <amarantine@my.com>
* class overrides equals() and should therefore also override hashCode()
* #1862 add modify user in process difinition list
* #1862 add pg-1.2.2 ddl.sql
* modify ScriptRunnerTest
* add updateProessDifinition UT
* modify updateProcessDifinition UT
* modify updateProcessDifinition UT
* modify mysql 1.2.2 ddl.sql&dml.sql
* add scope test to mysql in pom
* modify pg-1.2.2 ddl.sql
* feature：data synchronization function
* feature：data synchronization function
* feature：data synchronization function
* fix：add license
* fix：resource close bug
* fix：remove class author
* fix：Improve test coverage
* fix：add UT
* fix：add UT
* fix：data sync node renamed to datax
* fix：add UT
* fix: UT bug
* fix：Optimize variable name
* fix：pom
* fix：Variable name error
* fix：optimize the code
* move version to parent pom
* move version properties to parent pom for easy management
* remove freemarker dependency
* delete CombinedApplicationServer
* #1871 correct spelling
* #1873 some updates for TaskQueueZkImpl
* #1875 remove unused properties in pom
* #1878
1. remove tomcat dependency
2. remove combined_logback.xml in api module
3. format pom.xml for not aligning
* #1885 fix api server startup failure
1. add jsp-2.1 dependency
2. remove jasper-runtime dependency
* move version to parent pom
* move version properties to parent pom for easy management
* remove freemarker dependency
* delete CombinedApplicationServer
* #1871 correct spelling
* #1873 some updates for TaskQueueZkImpl
* #1875 remove unused properties in pom
* #1878
1. remove tomcat dependency
2. remove combined_logback.xml in api module
3. format pom.xml for not aligning
* Supplementary data by schedule
* fix sonar check bug
* fix code duplicated blocks
* ut
* loop by day
* MasterExecThread test
* test add licene
Co-authored-by: dailidong <dailidong66@gmail.com>
* remove commons-lang3 dependancy
* remove commons-lang3 dependancy
* remove unused import
* all the submodule should not contain version, so move version to parent,
1.Add FlinkArgsUtilsTest
2.Optimize FlinkArgsUtils and fix bug.
use StringUtils.isNotEmpty(queue) instead of StringUtils.isNotEmpty(others)
3.Add the UT path in pom
