Commit Graph

2147 Commits

Author SHA1 Message Date
John Liu
0714971453
Optimizing code (#2240)
* updated

* optimized the code

* Delete ParameterUtils.java

* rollback source code

* rollback source code

* rollback source code

Co-authored-by: dailidong <dailidong66@gmail.com>
2020-03-19 19:26:03 +08:00
Simon
00ff88ef1c
support custom datax configuration (#2224)
* fix #1441

* support custom datax config

* support datax custom config

* support datax custom config

* support datax custom config

Co-authored-by: 张世鸣 <zhangshiming@cvte.com>
2020-03-19 18:56:58 +08:00
tswstarplanet
3c5227ac0f
Refactor ConnectionFactory to static inner holder Singleton (#2204)
* refactor ConnectionFactory to static inner holder Singleton

* remove redundant import

* make getSqlSessionFactory() method private

* fix sonar issue

* remove @Ignore of MailUtilsTest

* add MailUtilsTest to pom.xml to test

* fix MailUtilsTest path error in pom.xml

* modify test method name

* add assert and logger to unit test

* add log

* add log

* add AlertDaoTest

* move AlertDaoTest to new module

* modify test in pom.xml

* remove unnecessary log
2020-03-18 23:48:19 +08:00
ruson
880e995654
add repace time when run history job and batch complement job ,not current time (#2196)
* add release notes in 1.2.1 (#1654)

* fix Monitor bug (#1656)

* fix Monitor bug

* fix zk monitor bug

* fix api url (#1673)

* get root path from zookeeper config

* Fixed DAG zoom in and zoom out nodes separated from arrows #1679 (#1685)

* fix api url

* Fixed DAG zoom in and zoom out nodes separated from arrows #1679

* check license and update (#1725)

* Fixed tasks_queue and tasks_kill did not exist in zookeeper #1696 (#1734)

Co-authored-by: elonlo <gamelifedong@gmail.com>

* upgrade jackson from 2.9.8 to 2.10.1 (#1767)

* add out put log when master/worker server start (#1769)

* merge hadoop.properties into common.properties

* merge hadoop,zookeeper.properties into common.properties
remove combined.properties/master.properties/worker.properties

* change db user/pwd to test/test

* rename .dolphinscheduler_en.sh to dolphinscheduler_env.sh
remove some unused in install.sh

* add out put log when master/worker server start...

* add start log when servers start

* add check download resource permission in order to fix issues 1770 (#1788)

* merge dev branch front-end code (#1786)

* fix #1775 bug,delete process definition when process instance is running (#1790)

* fix #1775 bug,delete process definition when process instance is running

* revert CONTRIBUTING.md

* fix udfs assignment and task instance page jump #1789 (#1791)

* merge dev branch front-end code

* fix udfs assignment

* Fix task instance page jump

* fix udfs assignment and task instance page jump #1789

* update method checkDownloadPermission in order to fix issues 1770 (#1794)

* add check download resource permission in order to fix issues 1770

* update method checkDownloadPermission in order to fix issues 1770

* update method listUnauthorizedResource in order to fix issues 1770

* update method listUnauthorizedResource in order to fix issues 1770 (#1797)

* if login user is admin,it will has permission to query all udf function (#1799)

* Fixed space and icon display issues before and after the input box (#1798)

* merge dev branch front-end code

* fix udfs assignment

* Fix task instance page jump

* fix udfs assignment and task instance page jump #1789

* Fixed space and icon display issues before and after the input box

* add license

* add license

* donot submit task to queue if sub process (#1793)

* dockerfile modify (#1800)

* fix #1775 bug,delete process definition when process instance is running

* revert CONTRIBUTING.md

* dockerfile modify

* dockerfile modify

* dockerfile modify

* Fix data echo, style and popup cannot be closed (#1801)

* merge dev branch front-end code

* fix udfs assignment

* Fix task instance page jump

* fix udfs assignment and task instance page jump #1789

* Fixed space and icon display issues before and after the input box

* add license

* add license

* Fix data echo, style and popup cannot be closed

* Fix page number loading issue and dag not getting value(#1810) (#1815)

* merge dev branch front-end code

* fix udfs assignment

* Fix task instance page jump

* fix udfs assignment and task instance page jump #1789

* Fixed space and icon display issues before and after the input box

* add license

* add license

* Fix data echo, style and popup cannot be closed

* Fix page number loading issue and dag not getting value

* [Fix issue #1770]check udf and data source in order to fix issue 1770 (#1817)

* if login user is admin,it will has permission to query all udf function

* check udf and data source in order to fix issue 1770

* check udf and data source in order to fix issue 1770

* check udf and data source in order to fix issue 1770

* check udf and data source in order to fix issue 1770

* check udf and data source in order to fix issue 1770

* revert common.properties

* update the test method name

* remove useless code and import in unit test

* refactor has permission and check permission

* #1813 remove "_001" after the master/server register path in zookeepe (#1824)

* donot submit task to queue if sub process

* [feature] #1813 remove "_001" after the master/server register path in zookeeper (#1820)

* change master/worker register path.

* remove "_" from register path.

* remove install.sh server.servlet.context-path replace (#1823)

* fix #1775 bug,delete process definition when process instance is running

* revert CONTRIBUTING.md

* dockerfile modify

* dockerfile modify

* dockerfile modify

* remove install.sh server.servlet.context-path replace

* Fix DAG add dependent project value exception # 1816 and keep requesting task interface (#1827)

* merge dev branch front-end code

* fix udfs assignment

* Fix task instance page jump

* fix udfs assignment and task instance page jump #1789

* Fixed space and icon display issues before and after the input box

* add license

* add license

* Fix data echo, style and popup cannot be closed

* Fix page number loading issue and dag not getting value

* Fix DAG add dependent project value exception # 1816 and keep requesting task interface

* rpm package add UI (#1846)

* [fix #1828] when the executor of process instance is not the owner of udf resouce, the path of the read resource file is incorrect (#1847)

* fix issue 1828:get the udf resource path error when create udf function

* update grantResources

* first verify whether udf resource is bound by udf function

* update grantResources

* update testListAuthorizedUdfFunc

* update getUserInfo in order to run success

* add LoggerServer RPC PORT modified. #1848 (#1849)

* fix #1775 bug,delete process definition when process instance is running

* revert CONTRIBUTING.md

* dockerfile modify

* dockerfile modify

* dockerfile modify

* remove install.sh server.servlet.context-path replace

* add LoggerServer RPC PORT modified. #1848

* LoggerService Logger RPC PORT get Error modify . #1848 (#1857)

* fix #1775 bug,delete process definition when process instance is running

* revert CONTRIBUTING.md

* dockerfile modify

* dockerfile modify

* dockerfile modify

* remove install.sh server.servlet.context-path replace

* add LoggerServer RPC PORT modified. #1848

* LoggerService Logger RPC PORT get Error modify . #1848

* Fix UDF function list delete data without refresh and page data display widened #1851 (#1852)

* merge dev branch front-end code

* fix udfs assignment

* Fix task instance page jump

* fix udfs assignment and task instance page jump #1789

* Fixed space and icon display issues before and after the input box

* add license

* add license

* Fix data echo, style and popup cannot be closed

* Fix page number loading issue and dag not getting value

* Fix DAG add dependent project value exception # 1816 and keep requesting task interface

* Fix UDF function list delete data without refresh and page data display widened

* [Fix #1828]check whether has permission to download udf file or delete udf function (#1858)

* fix issue 1828:get the udf resource path error when create udf function

* update grantResources

* first verify whether udf resource is bound by udf function

* update grantResources

* update testListAuthorizedUdfFunc

* update getUserInfo in order to run success

* check whether has permission to download udf file or delete udf file

* update listAuthorizedResourceById in ResourceMapper.xml

* add log, don't swallow exception info (#1877)

* Added hints in Resource Center (#1891)

* update jackson version from 2.9.8 to 2.10.1 (#1892)

* update release notes and modify the plugin of rat (#1899)

* modify how to build (#1902)

* update release notes and modify the plugin of rat

* modify how to build

* [maven-release-plugin] prepare release 1.2.1

* [maven-release-plugin] prepare for next development iteration

* get authorized udf resources need exclude all created by self (#1905)

* get authorized udf resources need exclude all created by self

* reset pom version to 1.2.1-SNAPSHOT

* Update ReleaseNotes.md

add detail info

* Update ReleaseNotes.md

* update notice year form 2019 to 2019-2020 (#1907)

* [maven-release-plugin] prepare release 1.2.1

* [maven-release-plugin] prepare for next development iteration

* 当手动重跑历史活着补数的时候对shell和sql脚本中的[YYYYmmddd...]变量赋值与传递的日期,而不是当前日期

Co-authored-by: lgcareer <18610854716@163.com>
Co-authored-by: bao liang <29528966+lenboo@users.noreply.github.com>
Co-authored-by: break60 <790061044@qq.com>
Co-authored-by: Tboy <guo.jiwei@immomo.com>
Co-authored-by: elonlo <gamelifedong@gmail.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>
Co-authored-by: zhangchunyang <18910529250@163.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: 谷雨 <lixun@guyudeMacBook-Pro.local>
2020-03-18 23:26:07 +08:00
gabry.wu
0e5fb86bef
Adapting partial code(file name start with N) to the sonar cloud rule (#2215) 2020-03-18 17:02:57 +08:00
xingchun-chen
7789e05c20
add LoginTest license (#2217)
Co-authored-by: chenxingchun <438044805@qq.com>
2020-03-18 10:03:03 +08:00
xingchun-chen
c82758c505
e2e add the case of deleting user and tenant   (#2210)
* add e2e
2020-03-17 17:57:12 +08:00
t1mon
7dd126da71
Optimize PropertyUtils instantiation. (#2199)
* Optimize PropertyUtils instantiation.

* Fix info error.
2020-03-17 15:01:42 +08:00
gabry.wu
c5e1c55c00
Adapting partial code(file name start with M #2) to the sonar cloud rule (#2179)
* Adapting partial code(file name start with M #2) to the sonar cloud rule

* remove unused import
2020-03-17 14:58:16 +08:00
Rubik-W
69633a29c6
fix: Api server startup fail (#2198) 2020-03-17 11:10:55 +08:00
tswstarplanet
f28ba55014
fix ResultSet not close and reformat code (#2183)
* fix ResultSet not close

* fix ResultSet not close

* extract code of closing resource to a method

* remove redundant if condition

* modify e2e timeout
2020-03-16 18:56:36 +08:00
tswstarplanet
a3767642a5
change static field to instance field (#2190) 2020-03-16 18:06:13 +08:00
t1mon
f6f29f4abb
Replace Charset.forName("UTF-8") with StandardCharsets.UTF_8. (#2193) 2020-03-16 17:10:55 +08:00
gabry.wu
1658176ef5
Adapting partial code(file name start with P #1) to the sonar cloud rule (#2171) (#2187) 2020-03-15 21:25:52 +08:00
tswstarplanet
e0fc3171da
Cache HadoopUtils instance with specific days expire time (#2181)
* Cache HadoopUtils instance with 7 days expire time

* solve sonar issue

* add kerberos expire time config

* move KERBEROS_EXPIRE_TIME to Constants.java
2020-03-15 21:14:43 +08:00
gabry.wu
fcc57320d6
Adapting partial code(file name start with M #1) to the sonar cloud rule (#2173)
* Adapting partial code(file name start with M #1) to the sonar cloud rule

* remove unused import
2020-03-15 19:05:33 +08:00
gabry.wu
61ebbf2946
Adapting partial code(file name start with L) to the sonar cloud rule (#2171) 2020-03-14 19:22:24 +08:00
Rubik-W
e0389aebf6
fix: NullPointException in the case of task queuing (#2175) 2020-03-14 19:20:46 +08:00
gabry.wu
a076ae4c36
Adapting partial code(file name start with J) to the sonar cloud rule (#2165)
* Adapting partial code(file name start with J) to the sonar cloud rule

* fix unit failure
2020-03-14 18:12:14 +08:00
Rubik-W
80c6ce5711
The master and worker server exit exception #2163 (#2176)
* fix: #2163

* fix: format
2020-03-14 18:10:27 +08:00
DS
2e63dc9c96
Delete .asf.yaml (#2168) 2020-03-14 08:47:23 +08:00
DS
e98c6f3008
Update .asf.yaml (#2167) 2020-03-14 08:42:55 +08:00
DS
205af16d55
Create .asf.yaml (#2166) 2020-03-14 08:30:41 +08:00
gabry.wu
daad5efbab
use static method in static class JSON (#2156) 2020-03-13 23:53:22 +08:00
liwenhe1993
dd2c50a10d
Toolbar of the DAG has not prompt (#2161)
* Toolbar of the DAG has not prompt

* Remove duplication code
2020-03-13 23:38:38 +08:00
gaojun2048
a0088c1137
fix rpmbuild and DS Ambari plugin bug (#2144)
* Update quick-start.md

* 更新demo的地址

* add rpm plugin

* delete the git ignore files

* 测试git ignore

* 测试ignore

* optimize the .gitignore file

* add rpm package, delete unuse copy jar action in assembly

* delete docs dir

* add rpm plugin support

* add rpm plugin support

* add rpm plugin support

* dolphinscheduler-common have not bin dir

* dolphinscheduler-common have not bin dir

* delete unuse config about assembly

* add defineSatement

* add rpm plugin support

* add rpm plugin support

* add rpm plugin support

* add rpm support

* update install dirname , make front and backend together

* update rpm name

* update rpm name

* update rpm config

* add jars excludes

* add jars excludes

* add rpm plugion

* add rpm plugion

* add rpm plugion

* add auto create /opt/soft

* The install dir can not be fixed to apache-dolphinscheduler-incubating-1.2.1 , its version is 1.2.2

* install from rpm will create a link

* the ui install thrown rpm only support jetty not nginx

* the ui install thrown rpm only support jetty not nginx

* sql目录里加上soft_version文件

* env dir need in conf dir

* remove no use module

* add module

* updaate

* rpmbuild add dolphinscheduler-service module
2020-03-13 13:04:33 +08:00
gabry.wu
93aa160698
Adapting partial code(file name start with I) to the sonar cloud rule (#2155) 2020-03-13 13:02:51 +08:00
dailidong
455d8c7a7c
Delete .asf.yaml (#2158) 2020-03-13 11:13:17 +08:00
Yelli
91ecc549a1
fix bug: data too long for column 'app_link' (#2146)
* fix bug: data to long for app_link field

* change app_link field length in 1.2.2 ddl

* change app_link field type to text
2020-03-12 21:52:48 +08:00
zhukai
68e4e5c4e0
add input cache for sqoop and conditions tasks (#2118)
Co-authored-by: zhukai <zhukai@ygsoft.com>
2020-03-12 18:08:49 +08:00
gabry.wu
7473d5e928
Adapting partial code(file name start with H) to the sonar cloud rule (#2145) 2020-03-12 18:06:10 +08:00
Rubik-W
9fda4da70e
fix: The toolbar buttons of Dag disable bug (#2124) 2020-03-12 18:00:33 +08:00
break60
712a1bc126
Limit customization file content to no more than 3000 lines(#2128) (#2141)
* Change DOM label

* Change name to lowercase

* Limit customization file content to no more than 3000 lines
2020-03-12 17:59:16 +08:00
t1mon
46e30b1e50
Add mail server configuration description information (#2143)
* Add mail server configuration description information

* Remove mail server configuration note in alert.properties
2020-03-12 17:50:37 +08:00
tswstarplanet
afb907b4b0
fix unequal number of arguments and placeholder (#2149) 2020-03-12 17:41:10 +08:00
tswstarplanet
e3065dbfcc
fix field access modifier (#2150) 2020-03-12 17:39:25 +08:00
tswstarplanet
e27ba3fd97
use logger to print exception (#2151) 2020-03-12 17:38:38 +08:00
liwenhe1993
ebfacbfcfa
docker run failed when docker command was docker run -d (#2125)
* docker run failed when docker command was `docker run -d`

1. add tini
2. replace `tee` command by `tail` command in startup.sh
3. api-server need to support zookeeper

* modify `check.sh` in dockerfile

* add init directories and log files
2020-03-12 10:11:07 +08:00
Jave-Chen
b03e0a02c5
fix wrong sonar analysis with checkout V2. (#2148) 2020-03-12 10:09:13 +08:00
AhahaGe
1db4ac6b63
edit README.md modify word from expand to extend (#2138)
Co-authored-by: guirong.ggr <guirong.ggr@dtwave-inc.com>
2020-03-10 22:28:40 +08:00
AhahaGe
ce17531024
add .asf.yaml file (#2136)
Co-authored-by: guirong.ggr <guirong.ggr@dtwave-inc.com>
2020-03-10 22:00:40 +08:00
gabry.wu
ebf7a980d2
Adapting partial code(file name start with F) to the sonar cloud rule (#2045)
* Adapting partial code(file name start with F) to the sonar cloud rule

* add more unit test

* add License

* add includes configuration to maven-surefire-plugin

* fix getResourceFilesList incorrect logic
2020-03-10 18:59:45 +08:00
tison
9224b49b58
access field handle of FileDescriptor in ProcessImplForWin32 by reflection for portability (#2113)
* access field handle of FileDescriptor in ProcessImplForWin32 by reflection for portability

Current implementation relies on `sun.misc.JavaIOFileDescriptorAccess`
which is only accessible on oraclejdk8.

Basically the demand is getting & setting `handle` field of
`FileDescriptor`, so we can directly do that with reflection.

Though, I suspect the necessity we introduce ProcessImplForWin32. Maybe
we could have a better way to support worker server to run bat script.

* harden initialization of ProcessImplForWin32

* ignore ShellTaskTest#testHandleForWindows outside Windows
2020-03-09 19:06:41 +08:00
gabry.wu
450a1f56fc
Adapting partial code(file name start with E) to the sonar cloud rule (#1999)
* Adapting partial code(file name start with E) to the sonar cloud rule

* remove isEmpty invoke

* resolve conflicts
2020-03-08 21:28:49 +08:00
gabry.wu
ad381fcb71
move UT class to reasonable package (#2116) 2020-03-08 21:22:07 +08:00
Rubik-W
275df85250
fix: remove redundant import class (#2110) 2020-03-08 16:00:47 +08:00
tswstarplanet
b2268b1af9
use stringbuilder to append string (#2108) 2020-03-08 15:57:06 +08:00
liwenhe1993
89b42db3b4
Add docker internal server check script (#2092)
* add environment variables add checkpoint.sh file to docker image

1. add `POSTGRESQL_DATABASE` environment variable
2. add `DOLPHINSCHEDULER_DATA_BASEDIR_PATH` environment variable
3. add `DOLPHINSCHEDULER_DATA_DOWNLOAD_BASEDIR_PATH` environment variable
4. add `DOLPHINSCHEDULER_PROCESS_EXEC_BASEPATH` environment variable
5. add checkpoint.sh file to check server process
6. modify `README.md` and `README_zh_CN.md`

* add environment variables add checkpoint.sh file to docker image

1. add `POSTGRESQL_DATABASE` environment variable
2. add `DOLPHINSCHEDULER_DATA_BASEDIR_PATH` environment variable
3. add `DOLPHINSCHEDULER_DATA_DOWNLOAD_BASEDIR_PATH` environment variable
4. add `DOLPHINSCHEDULER_PROCESS_EXEC_BASEPATH` environment variable
5. add checkpoint.sh file to check server process
6. modify `README.md` and `README_zh_CN.md`

* add logger port and modify dolphinscheduler_env to dolphinscheduler_env.sh
2020-03-08 15:56:24 +08:00
gabry.wu
0e1dd8d7d9
remove incorrect or unnecessary URL tag (#2099) 2020-03-07 18:07:53 +08:00
dailidong
4bb97dfb8b
fix actions/checkout@v2 problem (#2095) 2020-03-05 23:53:12 +08:00