Commit Graph

199 Commits

Author SHA1 Message Date
ruson
2bb55e4993 add repace time when run history job and batch complement job ,not current time (#2196)
* add release notes in 1.2.1 (#1654)

* fix Monitor bug (#1656)

* fix Monitor bug

* fix zk monitor bug

* fix api url (#1673)

* get root path from zookeeper config

* Fixed DAG zoom in and zoom out nodes separated from arrows #1679 (#1685)

* fix api url

* Fixed DAG zoom in and zoom out nodes separated from arrows #1679

* check license and update (#1725)

* Fixed tasks_queue and tasks_kill did not exist in zookeeper #1696 (#1734)

Co-authored-by: elonlo <gamelifedong@gmail.com>

* upgrade jackson from 2.9.8 to 2.10.1 (#1767)

* add out put log when master/worker server start (#1769)

* merge hadoop.properties into common.properties

* merge hadoop,zookeeper.properties into common.properties
remove combined.properties/master.properties/worker.properties

* change db user/pwd to test/test

* rename .dolphinscheduler_en.sh to dolphinscheduler_env.sh
remove some unused in install.sh

* add out put log when master/worker server start...

* add start log when servers start

* add check download resource permission in order to fix issues 1770 (#1788)

* merge dev branch front-end code (#1786)

* fix #1775 bug,delete process definition when process instance is running (#1790)

* fix #1775 bug,delete process definition when process instance is running

* revert CONTRIBUTING.md

* fix udfs assignment and task instance page jump #1789 (#1791)

* merge dev branch front-end code

* fix udfs assignment

* Fix task instance page jump

* fix udfs assignment and task instance page jump #1789

* update method checkDownloadPermission in order to fix issues 1770 (#1794)

* add check download resource permission in order to fix issues 1770

* update method checkDownloadPermission in order to fix issues 1770

* update method listUnauthorizedResource in order to fix issues 1770

* update method listUnauthorizedResource in order to fix issues 1770 (#1797)

* if login user is admin,it will has permission to query all udf function (#1799)

* Fixed space and icon display issues before and after the input box (#1798)

* merge dev branch front-end code

* fix udfs assignment

* Fix task instance page jump

* fix udfs assignment and task instance page jump #1789

* Fixed space and icon display issues before and after the input box

* add license

* add license

* donot submit task to queue if sub process (#1793)

* dockerfile modify (#1800)

* fix #1775 bug,delete process definition when process instance is running

* revert CONTRIBUTING.md

* dockerfile modify

* dockerfile modify

* dockerfile modify

* Fix data echo, style and popup cannot be closed (#1801)

* merge dev branch front-end code

* fix udfs assignment

* Fix task instance page jump

* fix udfs assignment and task instance page jump #1789

* Fixed space and icon display issues before and after the input box

* add license

* add license

* Fix data echo, style and popup cannot be closed

* Fix page number loading issue and dag not getting value(#1810) (#1815)

* merge dev branch front-end code

* fix udfs assignment

* Fix task instance page jump

* fix udfs assignment and task instance page jump #1789

* Fixed space and icon display issues before and after the input box

* add license

* add license

* Fix data echo, style and popup cannot be closed

* Fix page number loading issue and dag not getting value

* [Fix issue #1770]check udf and data source in order to fix issue 1770 (#1817)

* if login user is admin,it will has permission to query all udf function

* check udf and data source in order to fix issue 1770

* check udf and data source in order to fix issue 1770

* check udf and data source in order to fix issue 1770

* check udf and data source in order to fix issue 1770

* check udf and data source in order to fix issue 1770

* revert common.properties

* update the test method name

* remove useless code and import in unit test

* refactor has permission and check permission

* #1813 remove "_001" after the master/server register path in zookeepe (#1824)

* donot submit task to queue if sub process

* [feature] #1813 remove "_001" after the master/server register path in zookeeper (#1820)

* change master/worker register path.

* remove "_" from register path.

* remove install.sh server.servlet.context-path replace (#1823)

* fix #1775 bug,delete process definition when process instance is running

* revert CONTRIBUTING.md

* dockerfile modify

* dockerfile modify

* dockerfile modify

* remove install.sh server.servlet.context-path replace

* Fix DAG add dependent project value exception # 1816 and keep requesting task interface (#1827)

* merge dev branch front-end code

* fix udfs assignment

* Fix task instance page jump

* fix udfs assignment and task instance page jump #1789

* Fixed space and icon display issues before and after the input box

* add license

* add license

* Fix data echo, style and popup cannot be closed

* Fix page number loading issue and dag not getting value

* Fix DAG add dependent project value exception # 1816 and keep requesting task interface

* rpm package add UI (#1846)

* [fix #1828] when the executor of process instance is not the owner of udf resouce, the path of the read resource file is incorrect (#1847)

* fix issue 1828:get the udf resource path error when create udf function

* update grantResources

* first verify whether udf resource is bound by udf function

* update grantResources

* update testListAuthorizedUdfFunc

* update getUserInfo in order to run success

* add LoggerServer RPC PORT modified. #1848 (#1849)

* fix #1775 bug,delete process definition when process instance is running

* revert CONTRIBUTING.md

* dockerfile modify

* dockerfile modify

* dockerfile modify

* remove install.sh server.servlet.context-path replace

* add LoggerServer RPC PORT modified. #1848

* LoggerService Logger RPC PORT get Error modify . #1848 (#1857)

* fix #1775 bug,delete process definition when process instance is running

* revert CONTRIBUTING.md

* dockerfile modify

* dockerfile modify

* dockerfile modify

* remove install.sh server.servlet.context-path replace

* add LoggerServer RPC PORT modified. #1848

* LoggerService Logger RPC PORT get Error modify . #1848

* Fix UDF function list delete data without refresh and page data display widened #1851 (#1852)

* merge dev branch front-end code

* fix udfs assignment

* Fix task instance page jump

* fix udfs assignment and task instance page jump #1789

* Fixed space and icon display issues before and after the input box

* add license

* add license

* Fix data echo, style and popup cannot be closed

* Fix page number loading issue and dag not getting value

* Fix DAG add dependent project value exception # 1816 and keep requesting task interface

* Fix UDF function list delete data without refresh and page data display widened

* [Fix #1828]check whether has permission to download udf file or delete udf function (#1858)

* fix issue 1828:get the udf resource path error when create udf function

* update grantResources

* first verify whether udf resource is bound by udf function

* update grantResources

* update testListAuthorizedUdfFunc

* update getUserInfo in order to run success

* check whether has permission to download udf file or delete udf file

* update listAuthorizedResourceById in ResourceMapper.xml

* add log, don't swallow exception info (#1877)

* Added hints in Resource Center (#1891)

* update jackson version from 2.9.8 to 2.10.1 (#1892)

* update release notes and modify the plugin of rat (#1899)

* modify how to build (#1902)

* update release notes and modify the plugin of rat

* modify how to build

* [maven-release-plugin] prepare release 1.2.1

* [maven-release-plugin] prepare for next development iteration

* get authorized udf resources need exclude all created by self (#1905)

* get authorized udf resources need exclude all created by self

* reset pom version to 1.2.1-SNAPSHOT

* Update ReleaseNotes.md

add detail info

* Update ReleaseNotes.md

* update notice year form 2019 to 2019-2020 (#1907)

* [maven-release-plugin] prepare release 1.2.1

* [maven-release-plugin] prepare for next development iteration

* 当手动重跑历史活着补数的时候对shell和sql脚本中的[YYYYmmddd...]变量赋值与传递的日期,而不是当前日期

Co-authored-by: lgcareer <18610854716@163.com>
Co-authored-by: bao liang <29528966+lenboo@users.noreply.github.com>
Co-authored-by: break60 <790061044@qq.com>
Co-authored-by: Tboy <guo.jiwei@immomo.com>
Co-authored-by: elonlo <gamelifedong@gmail.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>
Co-authored-by: zhangchunyang <18910529250@163.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: 谷雨 <lixun@guyudeMacBook-Pro.local>
2020-06-29 17:36:28 +08:00
t1mon
9c754378f3 Optimize PropertyUtils instantiation. (#2199)
* Optimize PropertyUtils instantiation.

* Fix info error.
2020-06-29 17:36:28 +08:00
tswstarplanet
585f833549 change static field to instance field (#2190) 2020-06-29 17:36:28 +08:00
tswstarplanet
77c4bcb96e Cache HadoopUtils instance with specific days expire time (#2181)
* Cache HadoopUtils instance with 7 days expire time

* solve sonar issue

* add kerberos expire time config

* move KERBEROS_EXPIRE_TIME to Constants.java
2020-06-29 17:36:28 +08:00
gabry.wu
2f4fda7cdb Adapting partial code(file name start with M #1) to the sonar cloud rule (#2173)
* Adapting partial code(file name start with M #1) to the sonar cloud rule

* remove unused import
2020-06-29 17:36:28 +08:00
gabry.wu
7f7f50a828 Adapting partial code(file name start with L) to the sonar cloud rule (#2171) 2020-06-29 17:36:28 +08:00
gabry.wu
bdfb9d6a50 Adapting partial code(file name start with J) to the sonar cloud rule (#2165)
* Adapting partial code(file name start with J) to the sonar cloud rule

* fix unit failure
2020-06-29 17:36:27 +08:00
Rubik-W
8b61afc9be The master and worker server exit exception #2163 (#2176)
* fix: #2163

* fix: format
2020-06-29 17:36:27 +08:00
gabry.wu
610c95a470 use static method in static class JSON (#2156) 2020-06-29 17:36:27 +08:00
gabry.wu
33e86a97bf Adapting partial code(file name start with I) to the sonar cloud rule (#2155) 2020-06-29 17:36:27 +08:00
gabry.wu
156533ad5e Adapting partial code(file name start with H) to the sonar cloud rule (#2145) 2020-06-29 17:36:22 +08:00
tswstarplanet
6b6584ac8e use logger to print exception (#2151) 2020-06-29 17:36:22 +08:00
gabry.wu
493499f118 Adapting partial code(file name start with F) to the sonar cloud rule (#2045)
* Adapting partial code(file name start with F) to the sonar cloud rule

* add more unit test

* add License

* add includes configuration to maven-surefire-plugin

* fix getResourceFilesList incorrect logic
2020-06-29 17:36:22 +08:00
gabry.wu
0209cb7431 move UT class to reasonable package (#2116) 2020-06-29 17:36:13 +08:00
gabry.wu
fdc9e0090b remove incorrect or unnecessary URL tag (#2099) 2020-06-29 17:36:13 +08:00
dailidong
1f92b4c4db
Optimize ParameterUtils.curingGlobalParams() execution efficiency (#2090)
* Optimize ParameterUtils.curingGlobalParams() execution efficiency

* Remove excess null check
2020-03-05 23:21:10 +08:00
qiaozhanwei
3e5a939913
delete unused import (#2066) 2020-03-05 11:07:44 +08:00
bao liang
0df7c6719d
[New Feature] add conditions task #205 (#2003)
* add funtion conditions task

* update conditions tasks

* update conditions for ui

* update conditions

* update

* revert

* update
2020-02-28 17:08:34 +08:00
gabry.wu
5ca4862ea9
remove String.format for org.slf4j.Logger (#2001) 2020-02-25 14:19:26 +08:00
孙朝和
3d6eee5d80
#203 add sqoop task (#1974)
* add sqoop task

* add sqoop task

* add sqoop task test

* add sqoop task test

* add sqoop task test

* modify add sqoop task test

* modify sqoop task test

* modify sqoop task test

* modify sqoop task test

* modify sqoop task test 2

* modify sqoop task test 3

* modify sqoop task test 3

* modify sqoop task test 3

* modify sqoop task test 4
2020-02-24 10:30:17 +08:00
dailidong
24f22974a2
solve jar conflict, delete servlet-api 2.5 jar (#1966)
* update README about DolphinScheduler

* Update issue templates

* update

* regularize api pom xml
update rpc maven compile to 1.8

* regularize api pom xml

* change commons.lang3.StringUtils to common.utils.StringUtils

* update pom.xml

* update

* correct equals method

* jasper-runtime is needed when api server start

* jasper-runtime jar is needed when api server start

* combine logback config of master/worker/alert/api server to one logback.xml

* remove tomcat runtime jar

* add UT

* add license

* remove jasper-runtime jar, not need anymore

* sovle jar conflict, remove servlet-api 2.5 jar

* remove servlet 2.5

Co-authored-by: DS <escheduler@outlook.com>
2020-02-17 10:16:40 +08:00
liwenhe1993
d674eaba57
Support DS to create user and group in windows environment (#1953)
* Support DS to create user and group in windows environment

* Add unit test
2020-02-16 09:52:47 +08:00
gabry.wu
99ac739851
rename vaild to valid (#1961) 2020-02-15 18:55:16 +08:00
qiaozhanwei
c1ee133392
Refactor Architecture Basic modification #1658 (#1946)
* 1,remove dolphinscheduler-rpc module  2,add dolphinscheduler-remote module 3,add dolphinscheduler-service module 4,refactor LoggerServer module (#1925)

* 1,remove dolphinscheduler-rpc module
2,add dolphinscheduler-remote module
3,add dolphinscheduler-service module
4,refactor LoggerServer module

* ProcessUtils modify

* Refactor architecture (#1926)

* move version to parent pom

* move version properties to parent pom for easy management

* remove freemarker dependency

* delete CombinedApplicationServer

* #1871 correct spelling

* #1873 some updates for TaskQueueZkImpl

* #1875 remove unused properties in pom

* #1878
1. remove tomcat dependency
2. remove combined_logback.xml in api module
3. format pom.xml for not aligning

* #1885 fix api server startup failure
1. add jsp-2.1 dependency
2. remove jasper-runtime dependency

* add stringutils ut (#1921)

* add stringutils ut

* Newfeature for #1675. (#1908)

Continue to finish the rest works, add the cache feature for dependence,mr,python,sub_process,procedure and shell.

* Add modify user name for process definition (#1919)

* class overrides equals() and should therefore also override hashCode()

* #1862 add modify user in process difinition list

* #1862 add pg-1.2.2 ddl.sql

* modify ScriptRunnerTest

* add updateProessDifinition UT

* modify updateProcessDifinition UT

* modify updateProcessDifinition UT

* modify mysql 1.2.2 ddl.sql&dml.sql

* add scope test to mysql in pom

* modify pg-1.2.2 ddl.sql

* refactor module

* updates

Co-authored-by: khadgarmage <khadgar.mage@outlook.com>
Co-authored-by: zhukai <boness@qq.com>
Co-authored-by: Yelli <amarantine@my.com>

* dolphinscheduler-common remove spring (#1931)

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* SpringApplicationContext class title add license (#1932)

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* add license (#1934)

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* Refactor architecture (#1936)

* move datasource classes to dao module

* fix send4LetterWord bug

* LoggerServiceTest remove ProcessDao (#1944)

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* LoggerServiceTest remove ProcessDao

* exclude jasper-compiler in case of runtime conflict (#1938)

* move datasource classes to dao module

* fix send4LetterWord bug

* exclude jasper-compiler in case of runtime conflict

* DataAnaylysisServiceTest and ProcessDefinitionService modify

* remote module add comment

* OSUtilsTest modify

* add finally block to close channel (#1951)

* move datasource classes to dao module

* fix send4LetterWord bug

* exclude jasper-compiler in case of runtime conflict

* add finally block to close channel

Co-authored-by: Tboy <technoboy@yeah.net>
Co-authored-by: khadgarmage <khadgar.mage@outlook.com>
Co-authored-by: zhukai <boness@qq.com>
Co-authored-by: Yelli <amarantine@my.com>
2020-02-14 22:00:20 +08:00
gabry.wu
44b76468a8
Adapting partial code(file name start with C) to the sonar cloud rules (#1918)
* Adapting partial code(file name start with C) to the sonar cloud rules

* add some UT

* add some UT(rollback JSONUtils.checkJsonVaild)

* add some UT(fix OSUtils.exeCmd UT)
2020-02-13 12:42:08 +08:00
samz406
95d62133a4
Merge pull request #1928 from khadgarmage/ut-resinfo
add ResInfoTest.java ut
2020-02-11 09:49:08 +08:00
xiaochun.liu
0ed6156b22 swap param 2020-02-11 09:26:43 +08:00
xiaochun.liu
980b5ae5ea swap param 2020-02-11 09:21:59 +08:00
xiaochun.liu
17162341c3 add ResInfoTest.java ut 2020-02-11 00:18:06 +08:00
Yelli
d937a6ae94 Add modify user name for process definition (#1919)
* class overrides equals() and should therefore also override hashCode()

* #1862 add modify user in process difinition list

* #1862 add pg-1.2.2 ddl.sql

* modify ScriptRunnerTest

* add updateProessDifinition UT

* modify updateProcessDifinition UT

* modify updateProcessDifinition UT

* modify mysql 1.2.2 ddl.sql&dml.sql

* add scope test to mysql in pom

* modify pg-1.2.2 ddl.sql
2020-02-10 16:37:51 +08:00
khadgarmage
c447bb489f add stringutils ut (#1921)
* add stringutils ut
2020-02-10 13:52:32 +08:00
gabry.wu
b77dddfeef Adapting partial code(file name start with B) to the sonar cloud rules (#1915)
* Adapting partial code(file name start with B) to the sonar cloud rules

* add @Ignore to BaseTaskQueueTest class
2020-02-09 17:56:55 +08:00
xiaochun.liu
2f33a9e2ab delete dup code 2020-02-09 10:20:18 +08:00
xiaochun.liu
bd10b78e29 delete dup code 2020-02-09 10:09:31 +08:00
xiaochun.liu
3364a8c8a9 add script runner ut 2020-02-08 21:22:20 +08:00
Jave-Chen
b3b075d915
Fix bug: Use try-with-resources or close this "Statement" in a "finally" clause. (#1702)
* #1701  Fix bug: Use try-with-resources or close this "Statement" in a "finally" clause.

* fix some indent

* refix code smell
2020-02-06 19:32:31 +08:00
dailidong
c63cd3e237
combine logback config of master/worker/alert/api server to one (#1898)
* combine logback config of master/worker/alert/api server to one logback.xml
2020-02-06 11:08:06 +08:00
魔方不在手
f942e5ced3
[feature] data synchronization function (#1881)
* feature:data synchronization function

* feature:data synchronization function

* feature:data synchronization function

* fix:add license

* fix:resource close bug

* fix:remove class author

* fix:Improve test coverage

* fix:add UT

* fix:add UT

* fix:data sync node renamed to datax

* fix:add UT

* fix: UT bug

* fix:Optimize variable name

* fix:pom

* fix:Variable name error

* fix:optimize the code
2020-02-06 09:29:23 +08:00
Yelli
5169689c35
class overrides equals() and should therefore also override hashCode() (#1882) 2020-02-03 10:24:24 +08:00
Tboy
38eaaa98e6 some updates for TaskQueueZkImpl (#1874)
small changes, no need more people review, I will merge.
2020-01-27 21:09:11 +08:00
gabry.wu
0d5c6e4e7c [Feature] set acquireTriggersWithinLock true (#1866)
issue: https://github.com/apache/incubator-dolphinscheduler/issues/1863
2020-01-22 21:09:12 +08:00
liwenhe1993
30db5c1467 Api module relp server module/1756 (#1766)
* Remove server module on api module

1. Remove server module and add dao module.
2. Move quartz in server module to dao module.
3. Update config and class.


to resolve conflicts,I deleted CombinedApplicationServer.java

Co-authored-by: dailidong <dailidong66@gmail.com>
2020-01-20 16:26:23 +08:00
Jave-Chen
16913bacd7 fix return, obj error (#1855) 2020-01-17 21:55:50 +08:00
Jave-Chen
c522ea7ebb Fix bug: NullPointerException etc. (#1721)
* #1720 Fix bug: NullPointerException etc.

* fix code smell

* Update DateInterval.java

refactor equals method

Co-authored-by: dailidong <dailidong66@gmail.com>
2020-01-17 16:55:01 +08:00
lgcareer
779672ec2a [Fix issue #1770]check udf and data source in order to fix issue 1770 (#1826)
* check udf and data source in order to fix issue 1770

* check udf and data source in order to fix issue 1770

* update testListAuthorizedUdfFunc
2020-01-15 15:07:00 +08:00
ZHAOLIPAN
07f96375b3 replace strings with constants already exist. (#1825) 2020-01-15 14:00:28 +08:00
bao liang
0b272f9ca6 [feature] #1813 remove "_001" after the master/server register path in zookeeper (#1820)
* change master/worker register path.

* remove "_" from register path.
2020-01-14 16:27:10 +08:00
ZHAOLIPAN
68f736953b fix typos (#1821) 2020-01-14 16:15:34 +08:00
DK.Pino
ea0caef88f Regularize quartz.properties conf file (#1783)
* remove lombok dependency(last)

* regularize quartz.properties conf file
2020-01-09 16:53:03 +08:00
Yelli
2ec9943f64 fix bug: zk hasTask method NPE (#1785) 2020-01-09 14:50:15 +08:00
gabry.wu
88b022cb57 ITaskQueue should have "hasTask" function (#1744)
* issue https://github.com/apache/incubator-dolphinscheduler/issues/1742

* remove unnecessary catch

* fix exception type

* fix bad code smell

* remove log exceptions

* catch exception
2020-01-09 10:41:48 +08:00
bao liang
e0e0696b4f add out put log when master/worker server start (#1769)
* merge hadoop.properties into common.properties

* merge hadoop,zookeeper.properties into common.properties
remove combined.properties/master.properties/worker.properties

* change db user/pwd to test/test

* rename .dolphinscheduler_en.sh to dolphinscheduler_env.sh
remove some unused in install.sh

* add out put log when master/worker server start...

* add start log when servers start
2020-01-09 10:36:15 +08:00
Yelli
963ed4a8c6 ds.server & ds.common.zk regularize log print (#1754) 2020-01-08 10:00:34 +08:00
xingchun-chen
dad643dbb5 commom/Preconditions Util Test (#1727)
* common/parameterUtils unit test
2020-01-06 15:51:55 +08:00
Tboy
4d1abb3fc7 remove commons-lang3 dependancy (#1709)
* remove commons-lang3 dependancy

* remove commons-lang3 dependancy

* remove unused import
2020-01-04 18:01:10 +08:00
khadgarmage
5f145efa6b DependentUtils.java ut (#1703)
* delete bytes file

* DependentUtilsTest.java ut
2020-01-04 17:58:29 +08:00
Yelli
7ecd38f7d6 fix sonar bug: change condition & not enough arguments (#1692)
* modify FileUtils.readFile2Str

* #1300 Add right alignment function in sql email content

* cancel formatted for alert_mail_template.ftl

* #747 sql task password Log desensitization

* cancel mail_temple

* edit ExcelUtils

* modify test method name

* #747 sql task password Log desensitization

* #1544 workflow import

* Constants add DATASOURCE_PASSWORD_REGEX

* #747 sql task password Log desensitization

* deal with import project have sub process

* modify export process addTaskNodeParam method name

* add testAddTaskNodeSpecialParam UT

* add ProcessDefinitionServiceTest-ut to pom

* add testImportSubProcess in ProcessDefinitionServiceTest

* add testImportSubProcess in ProcessDefinitionServiceTest

* add testImportProcessDefinition

* fix sonar bug: not enough arguments

* fix sonar bug: change condition & not enough arguments
2020-01-04 11:32:07 +08:00
Tboy
0a3954107e
Merge pull request #1699 from lgcareer/dev-taskqueue
get root path from zookeeper config
2020-01-03 19:30:51 +08:00
Tboy
672a48255a
Merge pull request #1700 from fordeal-smalldata/fix-task-zknode-miss
Fixed tasks_queue and tasks_kill did not exist in zookeeper #1696
2020-01-03 19:30:21 +08:00
elonlo
cae656c1bc uniform naming 2020-01-03 19:26:06 +08:00
Tboy
884237487c remove commons-lang3 dependancy, using common-module instead for server module (#1698)
* remove commons-lang3 dependancy

* remove commons-lang3 and commons-lang dependancy

* remove commons-collections4/commons-logging/commons-io dependancy

* tmp add commons-lang3 for compile

* remove commons-lang3 dependancy, using common-module instead

* remove common StringUtils instead of lang3

* add tmp lang3 for compile
2020-01-03 19:23:05 +08:00
elonlo
c21d25edf2 Fixed tasks_queue and tasks_kill did not exist in zookeeper #1696 2020-01-03 18:37:39 +08:00
lgcareer
8090f14504 get root path from zookeeper config 2020-01-03 18:05:00 +08:00
Tboy
0e66c71f38 remove commons-collections4/commons-logging/commons-io dependancy for API module (#1693)
* remove commons-lang3 dependancy

* remove commons-lang3 and commons-lang dependancy

* remove commons-collections4/commons-logging/commons-io dependancy

* tmp add commons-lang3 for compile
2020-01-03 16:10:18 +08:00
Yelli
216154dc06 fix sonar bug: null check & clean throw exception (#1690)
* fix sonar bug: null check & clean throw exception

* fix sonar bug: null check & clean throw exception
2020-01-03 11:46:51 +08:00
Yelli
a80e90276f fix sonar bug: not enough arguments (#1687)
* modify FileUtils.readFile2Str

* #1300 Add right alignment function in sql email content

* cancel formatted for alert_mail_template.ftl

* #747 sql task password Log desensitization

* cancel mail_temple

* edit ExcelUtils

* modify test method name

* #747 sql task password Log desensitization

* #1544 workflow import

* Constants add DATASOURCE_PASSWORD_REGEX

* #747 sql task password Log desensitization

* deal with import project have sub process

* modify export process addTaskNodeParam method name

* add testAddTaskNodeSpecialParam UT

* add ProcessDefinitionServiceTest-ut to pom

* add testImportSubProcess in ProcessDefinitionServiceTest

* add testImportSubProcess in ProcessDefinitionServiceTest

* add testImportProcessDefinition

* fix sonar bug: not enough arguments
2020-01-03 11:24:01 +08:00
Tboy
a9cd31eca6 remove commons-lang3 and commons-lang dependancy for Dao module (#1663)
* remove commons-lang3 dependancy

* remove commons-lang3 and commons-lang dependancy
2020-01-03 11:14:19 +08:00
DK.Pino
0bfb0e4b16 remove lombok depend in common dodule (#1677) 2020-01-02 17:14:34 +08:00
khadgarmage
bb414b0794 delete bytes file (#1669) 2020-01-02 10:15:37 +08:00
khadgarmage
bc8e06bb08
SchemaUtils.java TaskParametersUtils.java unit test (#1660)
* schemautils test and TaskParametersUtils test
2019-12-31 22:51:56 +08:00
Tboy
677e9cfd33 remove commons-lang3 dependancy (#1655) 2019-12-31 16:03:08 +08:00
AmoryWang
e1d39e1e8e add UT for common thread package (#1628)
* add common thread ut

* Update ThreadUtilsTest.java

add license header

Co-authored-by: dailidong <dailidong66@gmail.com>
2019-12-30 19:42:48 +08:00
bao liang
d16500e8be [Feature] merge some configurations #1635 (#1636)
* merge hadoop.properties into common.properties

* merge hadoop,zookeeper.properties into common.properties
remove combined.properties/master.properties/worker.properties

* change db user/pwd to test/test
2019-12-30 18:14:31 +08:00
Tboy
5e935d579e refactor AbstractZKClient (#1627)
* we should insert alert DB once , and trigger this type of alert 3 times

* refactor AbstractZKClient
2019-12-30 17:59:06 +08:00
Tboy
dce7441bea
Merge pull request #1611 from wenhemin/fixbug-20191227
fix:In a cluster environment, verify that nodes have matching bug. #{1613}
2019-12-28 20:01:48 +08:00
DK.Pino
b6cca46d8f remove lombok annocation (#1603) 2019-12-27 17:04:25 +08:00
DK.Pino
63f396b61f refactor zk tree cache (#1577)
* refactor zk tree cache

* refactor zk tree cache
2019-12-27 17:03:10 +08:00
Tboy
8b2b5ba684 refactor TaskQueueZkImpl (#1591)
* fix #1515

* sleep when resource in not satisfy. fix #1522

* add sleep 1s for no command

* fix MasterBaseTaskExecThread submit method bug

* updates

* add log

* delete lombok

* remove duplicate code

* refactor TaskQueueZkImpl

* ignore First , we have to rewrite

* updates
2019-12-27 17:00:06 +08:00
DK.Pino
aadf0aae46 remove lombok annocation (#1600) 2019-12-27 16:59:47 +08:00
温合民
3c6780c3cd fix:In a cluster environment, verify that nodes have matching bug. 2019-12-27 16:53:51 +08:00
DK.Pino
c17fe38a6c remove lombok annocation (#1605) 2019-12-27 16:41:34 +08:00
loushang
110de68fcc fix Stirng format bug 2019-12-27 12:05:34 +08:00
xingchun-chen
90730133ac common/ParameterUtils Utils Test (#1583)
* common/parameterUtils unit test
2019-12-26 20:09:21 +08:00
Yelli
eefb71855d #747 Worker Log desensitization(日志脱敏) (#1568)
* modify FileUtils.readFile2Str

* #1300 Add right alignment function in sql email content

* cancel formatted for alert_mail_template.ftl

* #747 sql task password Log desensitization

* cancel mail_temple

* edit ExcelUtils

* modify test method name

* #747 sql task password Log desensitization

* Constants add DATASOURCE_PASSWORD_REGEX
2019-12-26 19:56:15 +08:00
qiaozhanwei
18e8703b70 remove kazoo (#1575)
* remove LogViewServiceGrpc.java file and pom modify

* remove kazoo

* remove kazoo

* remove kazoo

* remove common monitor package

* add license

* remove kazoo modify

* remove kazoo modify

* remove kazoo modify

* remove kazoo modify

* remove kazoo modify

* remove kazoo modify

* install.sh remove python kazoo

* add system param whether repeat running

* remove kazoo modify

* BusinessTimeUtils remove whther repeat running inner param
2019-12-26 11:41:58 +08:00
Tboy
3e4c221463 delete lombok (#1567)
* fix #1515

* sleep when resource in not satisfy. fix #1522

* add sleep 1s for no command

* fix MasterBaseTaskExecThread submit method bug

* updates

* add log

* delete lombok
2019-12-25 12:25:40 +08:00
DK.Pino
4bbf31f807 refactor zk client and zk config module (#1537)
* refactor zk client & zk config module

* refactor zk client & zk config module

* add license
2019-12-25 10:43:37 +08:00
lipan.zhao
d7fc79c8fb fix typos 2019-12-24 19:58:56 +08:00
lipan.zhao
88a7235ee2 fix 'directory' spell bug 2019-12-24 17:40:05 +08:00
Yelli
d52a83bfcf modify FileUtils.readFile2Str (#1535) 2019-12-21 10:48:40 +08:00
khadgarmage
7dcd8b26f7
unit test for common utils (#1520)
* httputil osutil encrytiontutil ut
2019-12-19 21:54:37 +08:00
samz406
6de55c3cea Modify common queue TaskQueueZKImplTest.java unit test (#1505)
* misspell  words

* modify common queue TaskQueueZKImplTest.java unit test

* extends  BaseTaskQueueTest get zkServer

* modify zk config
2019-12-18 13:57:47 +08:00
Tboy
c2f4b65a86
Merge branch 'dev' into dev 2019-12-18 10:52:43 +08:00
dk.technoboy
85b8324ee3 add BaseTaskQueueTest for UT 2019-12-18 10:49:55 +08:00
bao liang
21cb38d1dd fix issue:1477 some tasks would be running all the time when db delayed(#1477) (#1501)
* fix issue:1477 some tasks would be running all the time when db delayed

* fix issue:1477 some tasks would be running all the time when db delayed

* fix issue:1477 some tasks would be running all the time when db delayed
2019-12-18 10:10:19 +08:00
khadgarmage
039f02a624
Add common utils CollectionUtils.java DateUtils.java unit test (#1496)
* dateutil test

* pom.xml
2019-12-17 19:52:47 +08:00
Tboy
93188b7da1 add ZkServer for UT (#1499)
* add ZkServer for UT

* Add FileUtilsTest.java , the unit test for FileUtils (#1493) (#1)

* updates for reference ZkServer
2019-12-17 16:13:07 +08:00
Yelli
4d8d74ac18 #839 enhancement : add Spark Task Component can switch Spark Version (#1494)
* add Spark Version in Spark Component

add Spark Version in Spark Component

* add license for SparkVersion.class

add license

* 1 add spark task UT
2 add spark version param check

* add assert check for sparkTaskTest
2019-12-17 11:33:11 +08:00
dk.technoboy
d88c9219e4 updates for reference ZkServer 2019-12-17 11:06:02 +08:00
dk.technoboy
2f3d270469 add ZkServer for UT 2019-12-17 10:46:31 +08:00