Commit Graph

2087 Commits

Author SHA1 Message Date
zhukai
3186545da1
1. add input cache for datax task (#2034)
2. add cache for codemirror component, include python, shell and sql task.
2020-03-04 09:28:23 +08:00
break60
3277b90647
Change DOM label (#2064)
* Change DOM label

* Change name to lowercase
2020-03-03 14:38:21 +08:00
Tboy
fbf05de39b
delete duplicate startLocalZkServer (#2062) 2020-03-03 12:27:07 +08:00
Jave-Chen
46f46ed795
Merge pull request #2055 from Jave-Chen/bcai-dev-fix-bug-20200302-2051
remove submodules for checkout v2
2020-03-02 15:39:23 +08:00
bcai
8b894de3d2 remove submodules for checkout v2 2020-03-02 15:18:35 +08:00
samz406
7577fab411
Merge pull request #2053 from Jave-Chen/bcai-dev-fix-bug-20200302-2051
Fix github action rerun failed
2020-03-02 15:12:21 +08:00
bcai
3d4acbc012 fix github action rerun failed #2052 2020-03-02 15:02:17 +08:00
samz406
099fc640f9
Merge pull request #2050 from khadgarmage/ci-action-yml
modify action build
2020-03-02 11:15:49 +08:00
xiaochun.liu
c9c4402cb6 modify action build 2020-03-02 10:55:40 +08:00
Rubik
1ea48375f4
Fixbug i18n #2008 (#2035)
* fix:add chinese message

* fix:add chinese message

* fix:'getLocale()' can return not null

* fix:optimize message content
2020-03-02 10:29:42 +08:00
DS
c8adf824e2
Update README.md (#2048) 2020-03-01 20:48:38 +08:00
DS
8b2a224716
image about monitor screenshot is wrong (#2047)
image about monitor screenshot is wrong
2020-03-01 20:43:09 +08:00
DS
415d087dfc
Update README.md (#2046) 2020-03-01 20:42:45 +08:00
bao liang
0df7c6719d
[New Feature] add conditions task #205 (#2003)
* add funtion conditions task

* update conditions tasks

* update conditions for ui

* update conditions

* update

* revert

* update
2020-02-28 17:08:34 +08:00
Yelli
84409b57ee
fix taskinstance.isSubprocess misjudgment (#2007)
* add processInstanceService UT

* fix taskInstance.isSubProcess
add processInstanceService UT

* add taskInstance.isSubProcess UT

* modify taskInstance.isSubProcess UT

* add license

* remove author&date
2020-02-28 16:15:21 +08:00
xingchun-chen
f407adcb71
Add e2e to create workflow case   (#2027)
* add e2e

* add ui-test

* add e2e license and notice

* add e2e license

* add license

* add ui-test

* add ui-test

* add ui-test

* e2e delete dolphinscheduler parent project dependency

* add ui-test

* dolphinscheduler_dist delete e2e license/notice

* dolphinscheduler_dist delete e2e license/notice

* dolphinscheduler_dist delete e2e license/notice

* dolphinscheduler_dist delete e2e license/notice

* pom.xml delete e2e dependency

* add e2e create workflow

* Merge remote-tracking branch 'upstream/dev' into dev

# Conflicts:
#	e2e/src/test/java/org/apache/dolphinscheduler/base/BaseDriver.java

* Merge remote-tracking branch 'upstream/dev' into dev

# Conflicts:
#	e2e/src/test/java/org/apache/dolphinscheduler/base/BaseDriver.java

* Merge remote-tracking branch 'upstream/dev' into dev

# Conflicts:
#	e2e/src/test/java/org/apache/dolphinscheduler/base/BaseDriver.java

* Merge remote-tracking branch 'upstream/dev' into dev

# Conflicts:
#	e2e/src/test/java/org/apache/dolphinscheduler/base/BaseDriver.java

* modify workflow case

Co-authored-by: chenxingchun <50446296+chenxingchun@users.noreply.github.com>
2020-02-27 11:43:09 +08:00
khadgarmage
ee3c0aedc8
docker and testcase config for e2e test (#2028)
* github action for e2e test

* github action for e2e test

* github action for e2e test

* chromedriver

* e2e

* driver option

* t

* args bug

* check

* restore docker
2020-02-27 09:51:25 +08:00
dailidong
6b62501301
correct repeat images in README (#2017)
some images not right,correct
2020-02-26 13:03:01 +08:00
xingchun-chen
1cadb1a251
Add e2e automated testing (#2005)
* add ui-test

* add e2e license and notice

* e2e delete dolphinscheduler parent project dependency

* pom.xml delete e2e dependency

Co-authored-by: chenxingchun <50446296+chenxingchun@users.noreply.github.com>
2020-02-26 11:08:55 +08:00
gabry.wu
5ca4862ea9
remove String.format for org.slf4j.Logger (#2001) 2020-02-25 14:19:26 +08:00
liwenhe1993
205b302634
Modify dockerfile build script and add a new README_zh_CN.md file for dockerfile (#2014)
* Support DS to create user and group in windows environment

* Add unit test

* delete combined-server config in dolphinscheduler-daemon.sh file

* refactor dockerfile

* modify dockerfile

* modify dockerfile

* 1. Modify dockerfile build script
2. Add something about how to build a docker image
3. Add a new README_zh_CN.md file for dockerfile
2020-02-25 14:17:59 +08:00
dailidong
1ce6f35625
Update README_zh_CN.md (#2011)
update image url
2020-02-25 14:01:31 +08:00
dailidong
b1624fde71
update English Readme about image update (#2013)
update English Readme about image update
2020-02-25 14:00:26 +08:00
xingchun-chen
df0034ebae
Merge pull request #2009 from khadgarmage/docker-compose
Support docker deploy for e2e test
2020-02-25 10:27:44 +08:00
zhukai
5f3d88e4c2
Fix bug #1986. (#1987)
When in the projects define page, click the toolbar's button(task type) will cause the task details window switch to new one.

For now, the task details window will not switch.
2020-02-25 09:47:51 +08:00
xiaochun.liu
e2cc538c6e docker github action 2020-02-24 20:59:19 +08:00
xiaochun.liu
8a362b5464 docker github action 2020-02-24 20:44:44 +08:00
xiaochun.liu
5aebdc8b26 docker github action 2020-02-24 20:43:31 +08:00
xiaochun.liu
7a11d39f10 docker for e2e 2020-02-24 20:11:27 +08:00
孙朝和
3d6eee5d80
#203 add sqoop task (#1974)
* add sqoop task

* add sqoop task

* add sqoop task test

* add sqoop task test

* add sqoop task test

* modify add sqoop task test

* modify sqoop task test

* modify sqoop task test

* modify sqoop task test

* modify sqoop task test 2

* modify sqoop task test 3

* modify sqoop task test 3

* modify sqoop task test 3

* modify sqoop task test 4
2020-02-24 10:30:17 +08:00
DK.Pino
5ecd3b30b9
fix bug #1996 (#1998) 2020-02-23 16:59:24 +08:00
zhangchunyang
b4f2e5f421
Add ambari plugin #1469 (#1970)
* ambari plugin

* add readme.pdf

* ambari plugin

* add readme.md

* add readme.pdf

* 1.add license 2. update master cardinality 3. nesessary paramaters default empty 4. update alert message
2020-02-22 18:28:02 +08:00
liwenhe1993
34e0a663ed
Refactor dockerfile (#1978)
* Support DS to create user and group in windows environment

* Add unit test

* delete combined-server config in dolphinscheduler-daemon.sh file

* refactor dockerfile

* modify dockerfile

* modify dockerfile
2020-02-20 20:49:38 +08:00
Yelli
619fb30c50
#1310 add executor user to the process instance & task instance pages (#1973)
* add executor name in processDefinitionInstance & taskInstance

* modify processInstanceTest

* modify processInstanceTest

* #1310 add executor user to the process instance & task instance pages

* add processInstanceUT & taskInstanceUT

* add processInstanceUT & taskInstanceUT

* modify processInstanceTest & taskInstanceTest

* Remove duplicate code

* add userServiceUT
2020-02-20 13:25:41 +08:00
Tboy
5cc2ef5ce6
add scanFutureTable and testcase (#1971) 2020-02-17 22:07:19 +08:00
zhukai
afd5c75cd2
Fix the bug #1968. (#1969)
When create a new task or edit an exist task.The task details window will be open.
Then if delete the task node or click the DataSource menu. And the task details window still open.

The task details window should be close.
2020-02-17 19:09:10 +08:00
liwenhe1993
d7d7e95498
Remove combined-server config in dolphinscheduler-daemon.sh file (#1967)
* Support DS to create user and group in windows environment

* Add unit test

* delete combined-server config in dolphinscheduler-daemon.sh file
2020-02-17 19:07:25 +08:00
dailidong
24f22974a2
solve jar conflict, delete servlet-api 2.5 jar (#1966)
* update README about DolphinScheduler

* Update issue templates

* update

* regularize api pom xml
update rpc maven compile to 1.8

* regularize api pom xml

* change commons.lang3.StringUtils to common.utils.StringUtils

* update pom.xml

* update

* correct equals method

* jasper-runtime is needed when api server start

* jasper-runtime jar is needed when api server start

* combine logback config of master/worker/alert/api server to one logback.xml

* remove tomcat runtime jar

* add UT

* add license

* remove jasper-runtime jar, not need anymore

* sovle jar conflict, remove servlet-api 2.5 jar

* remove servlet 2.5

Co-authored-by: DS <escheduler@outlook.com>
2020-02-17 10:16:40 +08:00
qiaozhanwei
9b75bebf1b
1,add sendAsync method 2,refactor LoggerClient (#1965)
* 1,remove dolphinscheduler-rpc module  2,add dolphinscheduler-remote module 3,add dolphinscheduler-service module 4,refactor LoggerServer module (#1925)

* 1,remove dolphinscheduler-rpc module
2,add dolphinscheduler-remote module
3,add dolphinscheduler-service module
4,refactor LoggerServer module

* ProcessUtils modify

* Refactor architecture (#1926)

* move version to parent pom

* move version properties to parent pom for easy management

* remove freemarker dependency

* delete CombinedApplicationServer

* #1871 correct spelling

* #1873 some updates for TaskQueueZkImpl

* #1875 remove unused properties in pom

* #1878
1. remove tomcat dependency
2. remove combined_logback.xml in api module
3. format pom.xml for not aligning

* #1885 fix api server startup failure
1. add jsp-2.1 dependency
2. remove jasper-runtime dependency

* add stringutils ut (#1921)

* add stringutils ut

* Newfeature for #1675. (#1908)

Continue to finish the rest works, add the cache feature for dependence,mr,python,sub_process,procedure and shell.

* Add modify user name for process definition (#1919)

* class overrides equals() and should therefore also override hashCode()

* #1862 add modify user in process difinition list

* #1862 add pg-1.2.2 ddl.sql

* modify ScriptRunnerTest

* add updateProessDifinition UT

* modify updateProcessDifinition UT

* modify updateProcessDifinition UT

* modify mysql 1.2.2 ddl.sql&dml.sql

* add scope test to mysql in pom

* modify pg-1.2.2 ddl.sql

* refactor module

* updates

Co-authored-by: khadgarmage <khadgar.mage@outlook.com>
Co-authored-by: zhukai <boness@qq.com>
Co-authored-by: Yelli <amarantine@my.com>

* dolphinscheduler-common remove spring (#1931)

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* SpringApplicationContext class title add license (#1932)

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* add license (#1934)

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* move datasource classes to dao module

* fix send4LetterWord bug

* Refactor architecture (#1936)

* move datasource classes to dao module

* fix send4LetterWord bug

* exclude jasper-compiler in case of runtime conflict

* LoggerServiceTest remove ProcessDao (#1944)

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* LoggerServiceTest remove ProcessDao

* exclude jasper-compiler in case of runtime conflict (#1938)

* move datasource classes to dao module

* fix send4LetterWord bug

* exclude jasper-compiler in case of runtime conflict

* DataAnaylysisServiceTest and ProcessDefinitionService modify

* remote module add comment

* OSUtilsTest modify

* add finally block to close channel

* add finally block to close channel (#1951)

* move datasource classes to dao module

* fix send4LetterWord bug

* exclude jasper-compiler in case of runtime conflict

* add finally block to close channel

* refactor log client service

* add sendSync method

* 1,quartz.properties add conf category
2,dolphinscheduler-daemon.sh modify

* dolphinscheduler-binary.xml modify

* add sendAsync method (#1962)

* add sendAsync method

* add sendAsync method

* 1,add sendAsync method
2,refactor LoggerClient

* LogClientService modify

Co-authored-by: Tboy <technoboy@yeah.net>
Co-authored-by: khadgarmage <khadgar.mage@outlook.com>
Co-authored-by: zhukai <boness@qq.com>
Co-authored-by: Yelli <amarantine@my.com>
2020-02-16 19:12:01 +08:00
liwenhe1993
d674eaba57
Support DS to create user and group in windows environment (#1953)
* Support DS to create user and group in windows environment

* Add unit test
2020-02-16 09:52:47 +08:00
qiaozhanwei
c6f252451b
1,quartz.properties add conf category 2,dolphinscheduler-daemon.sh modify (#1960)
* 1,remove dolphinscheduler-rpc module  2,add dolphinscheduler-remote module 3,add dolphinscheduler-service module 4,refactor LoggerServer module (#1925)

* 1,remove dolphinscheduler-rpc module
2,add dolphinscheduler-remote module
3,add dolphinscheduler-service module
4,refactor LoggerServer module

* ProcessUtils modify

* Refactor architecture (#1926)

* move version to parent pom

* move version properties to parent pom for easy management

* remove freemarker dependency

* delete CombinedApplicationServer

* #1871 correct spelling

* #1873 some updates for TaskQueueZkImpl

* #1875 remove unused properties in pom

* #1878
1. remove tomcat dependency
2. remove combined_logback.xml in api module
3. format pom.xml for not aligning

* #1885 fix api server startup failure
1. add jsp-2.1 dependency
2. remove jasper-runtime dependency

* add stringutils ut (#1921)

* add stringutils ut

* Newfeature for #1675. (#1908)

Continue to finish the rest works, add the cache feature for dependence,mr,python,sub_process,procedure and shell.

* Add modify user name for process definition (#1919)

* class overrides equals() and should therefore also override hashCode()

* #1862 add modify user in process difinition list

* #1862 add pg-1.2.2 ddl.sql

* modify ScriptRunnerTest

* add updateProessDifinition UT

* modify updateProcessDifinition UT

* modify updateProcessDifinition UT

* modify mysql 1.2.2 ddl.sql&dml.sql

* add scope test to mysql in pom

* modify pg-1.2.2 ddl.sql

* refactor module

* updates

Co-authored-by: khadgarmage <khadgar.mage@outlook.com>
Co-authored-by: zhukai <boness@qq.com>
Co-authored-by: Yelli <amarantine@my.com>

* dolphinscheduler-common remove spring (#1931)

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* SpringApplicationContext class title add license (#1932)

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* add license (#1934)

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* Refactor architecture (#1936)

* move datasource classes to dao module

* fix send4LetterWord bug

* LoggerServiceTest remove ProcessDao (#1944)

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* LoggerServiceTest remove ProcessDao

* exclude jasper-compiler in case of runtime conflict (#1938)

* move datasource classes to dao module

* fix send4LetterWord bug

* exclude jasper-compiler in case of runtime conflict

* DataAnaylysisServiceTest and ProcessDefinitionService modify

* remote module add comment

* OSUtilsTest modify

* add finally block to close channel (#1951)

* move datasource classes to dao module

* fix send4LetterWord bug

* exclude jasper-compiler in case of runtime conflict

* add finally block to close channel

* 1,quartz.properties add conf category
2,dolphinscheduler-daemon.sh modify

* dolphinscheduler-binary.xml modify

Co-authored-by: Tboy <technoboy@yeah.net>
Co-authored-by: khadgarmage <khadgar.mage@outlook.com>
Co-authored-by: zhukai <boness@qq.com>
Co-authored-by: Yelli <amarantine@my.com>
2020-02-15 19:58:45 +08:00
gabry.wu
99ac739851
rename vaild to valid (#1961) 2020-02-15 18:55:16 +08:00
Jave-Chen
90df148497
Merge pull request #1948 from Yeleights/dev-process
Add process null check.
2020-02-15 16:37:41 +08:00
Yeleights
87cb33b233 deal with conflict 2020-02-14 22:46:45 +08:00
qiaozhanwei
c1ee133392
Refactor Architecture Basic modification #1658 (#1946)
* 1,remove dolphinscheduler-rpc module  2,add dolphinscheduler-remote module 3,add dolphinscheduler-service module 4,refactor LoggerServer module (#1925)

* 1,remove dolphinscheduler-rpc module
2,add dolphinscheduler-remote module
3,add dolphinscheduler-service module
4,refactor LoggerServer module

* ProcessUtils modify

* Refactor architecture (#1926)

* move version to parent pom

* move version properties to parent pom for easy management

* remove freemarker dependency

* delete CombinedApplicationServer

* #1871 correct spelling

* #1873 some updates for TaskQueueZkImpl

* #1875 remove unused properties in pom

* #1878
1. remove tomcat dependency
2. remove combined_logback.xml in api module
3. format pom.xml for not aligning

* #1885 fix api server startup failure
1. add jsp-2.1 dependency
2. remove jasper-runtime dependency

* add stringutils ut (#1921)

* add stringutils ut

* Newfeature for #1675. (#1908)

Continue to finish the rest works, add the cache feature for dependence,mr,python,sub_process,procedure and shell.

* Add modify user name for process definition (#1919)

* class overrides equals() and should therefore also override hashCode()

* #1862 add modify user in process difinition list

* #1862 add pg-1.2.2 ddl.sql

* modify ScriptRunnerTest

* add updateProessDifinition UT

* modify updateProcessDifinition UT

* modify updateProcessDifinition UT

* modify mysql 1.2.2 ddl.sql&dml.sql

* add scope test to mysql in pom

* modify pg-1.2.2 ddl.sql

* refactor module

* updates

Co-authored-by: khadgarmage <khadgar.mage@outlook.com>
Co-authored-by: zhukai <boness@qq.com>
Co-authored-by: Yelli <amarantine@my.com>

* dolphinscheduler-common remove spring (#1931)

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* SpringApplicationContext class title add license (#1932)

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* add license (#1934)

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* Refactor architecture (#1936)

* move datasource classes to dao module

* fix send4LetterWord bug

* LoggerServiceTest remove ProcessDao (#1944)

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* dolphinscheduler-common remove spring

* LoggerServiceTest remove ProcessDao

* exclude jasper-compiler in case of runtime conflict (#1938)

* move datasource classes to dao module

* fix send4LetterWord bug

* exclude jasper-compiler in case of runtime conflict

* DataAnaylysisServiceTest and ProcessDefinitionService modify

* remote module add comment

* OSUtilsTest modify

* add finally block to close channel (#1951)

* move datasource classes to dao module

* fix send4LetterWord bug

* exclude jasper-compiler in case of runtime conflict

* add finally block to close channel

Co-authored-by: Tboy <technoboy@yeah.net>
Co-authored-by: khadgarmage <khadgar.mage@outlook.com>
Co-authored-by: zhukai <boness@qq.com>
Co-authored-by: Yelli <amarantine@my.com>
2020-02-14 22:00:20 +08:00
Jave-Chen
cdd53a7673
Fix socket bug: read before sock connect. Add UT for FourLetterWordMain. (#1954)
* Fix socket bug: read before sock connect. Add UT for FourLetterWordMain.

* Fix code smell

   Remove  @Ignore and  @Test from testCmd().

* correct comment
2020-02-14 10:39:02 +08:00
Yeleights
77023aee1b add DagHelperTest to pom 2020-02-13 16:18:26 +08:00
Yeleights
6c90575a39 add process null check
Remove duplicate code in processDefinitionService
add processDefinitionService UT
2020-02-13 15:39:30 +08:00
gabry.wu
44b76468a8
Adapting partial code(file name start with C) to the sonar cloud rules (#1918)
* Adapting partial code(file name start with C) to the sonar cloud rules

* add some UT

* add some UT(rollback JSONUtils.checkJsonVaild)

* add some UT(fix OSUtils.exeCmd UT)
2020-02-13 12:42:08 +08:00
Yelli
768b5d9dfd
add state null check (#1939)
add process definition UT
2020-02-12 15:05:21 +08:00