Commit Graph

2381 Commits

Author SHA1 Message Date
break60
e6527c7a92 Corrections to historical jar packages and resource data 2020-06-29 17:36:41 +08:00
break60
b7851639eb Corrections to historical jar packages and resource data 2020-06-29 17:36:41 +08:00
lgcareer
54f04a378e init full_name in dml of version 1.2.2 (#2341)
Co-authored-by: dailidong <dailidong66@gmail.com>
2020-06-29 17:36:41 +08:00
Rubik-W
d8ccab6a49 fix:mail send fail #2345 (#2346)
Co-authored-by: dailidong <dailidong66@gmail.com>
2020-06-29 17:36:41 +08:00
BoYiZhang
7fa0c0f79a Remove invalid code (#2342)
Co-authored-by: zhanglong <zhanglong@ysstech.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
2020-06-29 17:36:41 +08:00
break60
18d7c66bec Supplemental licenses and repair resources cannot be renamed 2020-06-29 17:36:39 +08:00
break60
3d72643d4a Modify ans-ui version and timing management style 2020-06-29 17:36:38 +08:00
dailidong
db3c6ce861 Delete ReleaseNotes.md (#2338) 2020-06-29 17:36:38 +08:00
Rubik-W
9b6ab81e60 fix #2334 (#2335)
Co-authored-by: dailidong <dailidong66@gmail.com>
2020-06-29 17:36:38 +08:00
songgg
d8c7a44e82 some annotation optimizations (#2333)
Co-authored-by: songqh <songquanhe@foxmail.com>
2020-06-29 17:36:38 +08:00
liwenhe1993
c814a30316 Remove .helmignore file (#2328) 2020-06-29 17:36:37 +08:00
liwenhe1993
053dd3263e Support kubernetes deployment (#2153)
* Support kubernetes deployment

* Support kubernetes deployment
2020-06-29 17:36:37 +08:00
t1mon
1d9bc0cd74 Fix SqlTask kerberos load scope unreasonable. #2178 (#2321)
* Optimize PropertyUtils instantiation.

* Fix info error.

* Fix SqlTask kerberos load scope unreasonable. #2178
2020-06-29 17:36:37 +08:00
Rubik-W
668b86f12b fix:Optimize content returned by interface (#2294) 2020-06-29 17:36:37 +08:00
lgcareer
82d6dc9e52 new feature for #404 add resource tree function (#2323)
* add create resource directory

* add create resource directory

* update the resource test

* add upgrade sql in version 1.2.2

* Adding request parameter id to update queryResourceListPaging

* set isDirectory value is false default

* add full name to update updateResource

* remove request parameter isDirectory to update createResource method

* update queryResourceListPaging with change get to post

* update updateResource method with remove fullName

* File management list modification (#1976)

* add resource component

* add resource tree visitor

* return json string

* update queryResourceList

* upload file need fullName

* add method rootNode

* Shell task resources and authorization resources (#1989)

* File management list modification

* Shell task resources and authorization resources

* download resource when execute task

* download resource when execute task

* update authorization type

* download resource when execute task

* Spark task resource changes (#1990)

* File management list modification

* Shell task resources and authorization resources

* Spark task resource changes

* download resource when execute task

* update udf function service

* add resource type in ResourceComponent

* UDF resource tree and change DAG style (#2019)

* File management list modification

* Shell task resources and authorization resources

* Spark task resource changes

* UDF resource tree and change DAG style

* add deleteIds method in ResourceMapper and ResourceMapperTest

* Add comments on class and method

* add queryResourceByName method in controller

* update verify-name with change name to full name

* update queryResource with add parameter pid

* update queryResource with add parameter pid

* add resource ids in process definition and delete resource need judge whether it is used by any process definition

* Breadcrumb development (#2033)

* File management list modification

* Shell task resources and authorization resources

* Spark task resource changes

* UDF resource tree and change DAG style

* Breadcrumb development

* Breadcrumb development

* Resource tree bug fix (#2040)

* File management list modification

* Shell task resources and authorization resources

* Spark task resource changes

* UDF resource tree and change DAG style

* Breadcrumb development

* Breadcrumb development

* Resource tree bug fix

* update resource service test

* Fix github action rerun failed

* add status of PARENT_RESOURCE_NOT_EXIST

* Fix github action rerun failed (#2067)

* update resource service test

* Fix github action rerun failed

* add status of PARENT_RESOURCE_NOT_EXIST

* Change crumb position

* Change crumb position (#2068)

* build resource process definition map

* UDF changed to multiple choice

* UDF changed to multiple choice (#2077)

* Change crumb position

* UDF changed to multiple choice

* build resource process definition map (#2076)

* update resource service test

* Fix github action rerun failed

* add status of PARENT_RESOURCE_NOT_EXIST

* build resource process definition map

* update resource name also need update all the children full name

* need add queryResource

* update resource name also need update all the children full name (#2096)

* update resource service test

* Fix github action rerun failed

* add status of PARENT_RESOURCE_NOT_EXIST

* build resource process definition map

* update resource name also need update all the children full name

* need add queryResource

* Limit customization file content to no more than 3000 lines

* Limit customization file content to no more than 3000 lines(#2128)

* Limit customization file content to no more than 3000 lines(#2128) (#2140)

* Change crumb position

* UDF changed to multiple choice

* Limit customization file content to no more than 3000 lines

* Limit customization file content to no more than 3000 lines(#2128)

* add queryResourceJarList

* add queryResourceJarList

* add queryResourceJarList

* add queryResourceJarList (#2192)

* update resource service test

* Fix github action rerun failed

* add status of PARENT_RESOURCE_NOT_EXIST

* build resource process definition map

* update resource name also need update all the children full name

* need add queryResource

* add queryResourceJarList

* add queryResourceJarList

* add queryResourceJarList

* Modify the main jar package

* Modify the main jar package (#2200)

* Change crumb position

* UDF changed to multiple choice

* Limit customization file content to no more than 3000 lines

* Limit customization file content to no more than 3000 lines(#2128)

* Modify the main jar package

* add resource filter in order to get filtered resource

* add comments of resource filter

* update list children by resource

* choose main jar with resource tree (#2220)

* update resource service test

* Fix github action rerun failed

* add status of PARENT_RESOURCE_NOT_EXIST

* build resource process definition map

* update resource name also need update all the children full name

* need add queryResource

* add queryResourceJarList

* add queryResourceJarList

* add queryResourceJarList

* add resource filter in order to get filtered resource

* add comments of resource filter

* update list children by resource

* Return null if query resource list is empty

* update queryResource method change parameter pid to id

* getResouDelete checksum and modify parameter namerceId

* revert .env

* remove parameter pid

* Delete request interface

* go back to the last page

* jar interface call

* Fix issue #2234 and #2228

* change resource name with full name

* Fix issue #2234 and #2228 (#2246)

* update resource service test

* Fix github action rerun failed

* add status of PARENT_RESOURCE_NOT_EXIST

* build resource process definition map

* update resource name also need update all the children full name

* need add queryResource

* add queryResourceJarList

* add queryResourceJarList

* add queryResourceJarList

* add resource filter in order to get filtered resource

* add comments of resource filter

* update list children by resource

* Return null if query resource list is empty

* update queryResource method change parameter pid to id

* revert .env

* remove parameter pid

* Fix issue #2234 and #2228

* change resource name with full name

* Fix list query value error

* remove unauth-file with authorize-resource-tree

* Repair data cannot be echoed

* Repair data cannot be echoed

* execute mr and spark task need query resource name before

* Authorized resource interface replacement

* Authorized resource interface replacement

* Filter UDF resources

* Change parameters

* need query all authorized directory children when create task

* Change normalize.scss import method and animation.scss license modification

* Delete file list update processing

* It's fixed that resource not deleted in hdfs when delete it.

* add tooltips

* add tooltips (#2310)

* Echo workflow name and modify udf management name

* [new feature]add resource tree function

* revert front code in order to be same as dev branch

* revert front code in order to be same as dev branch

* revert common.properties and application.properties

* add super method

* update flink parameter test

* update flink parameter and unit test

* update resource service test

* If resource list is empty,need init it

* update flink parameter test

Co-authored-by: break60 <790061044@qq.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>
2020-06-29 17:36:36 +08:00
gabry.wu
f9c91b8cbe Adapting partial code(file name start with S #2) to the sonar cloud rule (#2270) 2020-06-29 17:36:36 +08:00
break60
4a8e148b42 Resource tree code merge 2020-06-29 17:36:34 +08:00
孙朝和
588c003860 resolve shell task build command nullpointexception (#2267)
Co-authored-by: sunchaohe <sunzhaohe@linklogis.com>
2020-06-29 17:36:34 +08:00
gabry.wu
30b98b89a2 Adapting partial code(file name start with T) to the sonar cloud rule (#2271) 2020-06-29 17:36:33 +08:00
tswstarplanet
0c5ff32797 Decouple DruidDataSource in ConnectionFactory and cache DataSource instance (#2232)
* Decouple DruidDataSource in ConnectionFactory and cache DataSource
instance

* init dataSource first

* fix code smell

* add unit test

* add unit test

* add unit test

* add unit test

* fix unit test

* delete useless logger
2020-06-29 17:36:33 +08:00
gabry.wu
a1f7614476 Adapting partial code(file name start with W) to the sonar cloud rule (#2287) 2020-06-29 17:36:33 +08:00
WeiYe
6cca69369b Update test code logger class name (#2303) 2020-06-29 17:36:33 +08:00
xingchun-chen
e496206a88 e2e add project and workflow case (#2309)
* add LoginTest license

* Delete useless packages

* e2e add project and workflow case

* e2e add project and workflow case

* e2e add project and workflow case

* e2e add project and workflow case

* e2e add project and workflow case

* e2e add project and workflow case

* e2e add project and workflow case

* e2e add project and workflow case

* e2e add project and workflow case

* e2e add project and workflow case

* e2e add project and workflow case

* e2e add project and workflow case

Co-authored-by: chenxingchun <438044805@qq.com>
2020-06-29 17:36:32 +08:00
break60
cd529267f5 Change normalize.scss import method and animation.scss license modification 2020-06-29 17:36:31 +08:00
khadgarmage
e37aecdbc7 add ds logs and server status check with e2e test (#2277) 2020-06-29 17:36:31 +08:00
songquanhe-gitstudy
fc18d4a216 fix: No operations allowed after statement closed when running sql task (#2265)
* sqlTask failed to run

* get correct attribute value and logger content

Co-authored-by: songqh <songquanhe@foxmail.com>
2020-06-29 17:36:30 +08:00
khadgarmage
5d0f0e9490 Fix checkoutv2 push trigger on dev (#2263)
* push trigger on dev
2020-06-29 17:36:29 +08:00
break60
e31f38a189 datax add custom 2020-06-29 17:36:29 +08:00
khadgarmage
416e0417ad license and checkoutv2 fix (#2261) 2020-06-29 17:36:29 +08:00
gabry.wu
afa9a6467a Adapting partial code(file name start with Q) to the sonar cloud rule (#2242) 2020-06-29 17:36:29 +08:00
Rubik-W
1acbc0dbfe fix:Depend check bug (#2229)
* fix:#1787

* fix:#1787
2020-06-29 17:36:29 +08:00
gabry.wu
e1771de45a Adapting partial code(file name start with R) to the sonar cloud rule (#2243) 2020-06-29 17:36:29 +08:00
jin
24fa31939b feat:optimizing code (#2245)
* feat:optimizing code

* add licence and  format code
2020-06-29 17:36:29 +08:00
tswstarplanet
bc65253fd1 fix blocker problem (#2244)
* fix blocker problem

* use error level to log exception
2020-06-29 17:36:29 +08:00
John Liu
4898447236 Optimizing code (#2240)
* updated

* optimized the code

* Delete ParameterUtils.java

* rollback source code

* rollback source code

* rollback source code

Co-authored-by: dailidong <dailidong66@gmail.com>
2020-06-29 17:36:29 +08:00
Simon
5c30205574 support custom datax configuration (#2224)
* fix #1441

* support custom datax config

* support datax custom config

* support datax custom config

* support datax custom config

Co-authored-by: 张世鸣 <zhangshiming@cvte.com>
2020-06-29 17:36:29 +08:00
tswstarplanet
5a74b8bbf5 Refactor ConnectionFactory to static inner holder Singleton (#2204)
* refactor ConnectionFactory to static inner holder Singleton

* remove redundant import

* make getSqlSessionFactory() method private

* fix sonar issue

* remove @Ignore of MailUtilsTest

* add MailUtilsTest to pom.xml to test

* fix MailUtilsTest path error in pom.xml

* modify test method name

* add assert and logger to unit test

* add log

* add log

* add AlertDaoTest

* move AlertDaoTest to new module

* modify test in pom.xml

* remove unnecessary log
2020-06-29 17:36:29 +08:00
ruson
2bb55e4993 add repace time when run history job and batch complement job ,not current time (#2196)
* add release notes in 1.2.1 (#1654)

* fix Monitor bug (#1656)

* fix Monitor bug

* fix zk monitor bug

* fix api url (#1673)

* get root path from zookeeper config

* Fixed DAG zoom in and zoom out nodes separated from arrows #1679 (#1685)

* fix api url

* Fixed DAG zoom in and zoom out nodes separated from arrows #1679

* check license and update (#1725)

* Fixed tasks_queue and tasks_kill did not exist in zookeeper #1696 (#1734)

Co-authored-by: elonlo <gamelifedong@gmail.com>

* upgrade jackson from 2.9.8 to 2.10.1 (#1767)

* add out put log when master/worker server start (#1769)

* merge hadoop.properties into common.properties

* merge hadoop,zookeeper.properties into common.properties
remove combined.properties/master.properties/worker.properties

* change db user/pwd to test/test

* rename .dolphinscheduler_en.sh to dolphinscheduler_env.sh
remove some unused in install.sh

* add out put log when master/worker server start...

* add start log when servers start

* add check download resource permission in order to fix issues 1770 (#1788)

* merge dev branch front-end code (#1786)

* fix #1775 bug,delete process definition when process instance is running (#1790)

* fix #1775 bug,delete process definition when process instance is running

* revert CONTRIBUTING.md

* fix udfs assignment and task instance page jump #1789 (#1791)

* merge dev branch front-end code

* fix udfs assignment

* Fix task instance page jump

* fix udfs assignment and task instance page jump #1789

* update method checkDownloadPermission in order to fix issues 1770 (#1794)

* add check download resource permission in order to fix issues 1770

* update method checkDownloadPermission in order to fix issues 1770

* update method listUnauthorizedResource in order to fix issues 1770

* update method listUnauthorizedResource in order to fix issues 1770 (#1797)

* if login user is admin,it will has permission to query all udf function (#1799)

* Fixed space and icon display issues before and after the input box (#1798)

* merge dev branch front-end code

* fix udfs assignment

* Fix task instance page jump

* fix udfs assignment and task instance page jump #1789

* Fixed space and icon display issues before and after the input box

* add license

* add license

* donot submit task to queue if sub process (#1793)

* dockerfile modify (#1800)

* fix #1775 bug,delete process definition when process instance is running

* revert CONTRIBUTING.md

* dockerfile modify

* dockerfile modify

* dockerfile modify

* Fix data echo, style and popup cannot be closed (#1801)

* merge dev branch front-end code

* fix udfs assignment

* Fix task instance page jump

* fix udfs assignment and task instance page jump #1789

* Fixed space and icon display issues before and after the input box

* add license

* add license

* Fix data echo, style and popup cannot be closed

* Fix page number loading issue and dag not getting value(#1810) (#1815)

* merge dev branch front-end code

* fix udfs assignment

* Fix task instance page jump

* fix udfs assignment and task instance page jump #1789

* Fixed space and icon display issues before and after the input box

* add license

* add license

* Fix data echo, style and popup cannot be closed

* Fix page number loading issue and dag not getting value

* [Fix issue #1770]check udf and data source in order to fix issue 1770 (#1817)

* if login user is admin,it will has permission to query all udf function

* check udf and data source in order to fix issue 1770

* check udf and data source in order to fix issue 1770

* check udf and data source in order to fix issue 1770

* check udf and data source in order to fix issue 1770

* check udf and data source in order to fix issue 1770

* revert common.properties

* update the test method name

* remove useless code and import in unit test

* refactor has permission and check permission

* #1813 remove "_001" after the master/server register path in zookeepe (#1824)

* donot submit task to queue if sub process

* [feature] #1813 remove "_001" after the master/server register path in zookeeper (#1820)

* change master/worker register path.

* remove "_" from register path.

* remove install.sh server.servlet.context-path replace (#1823)

* fix #1775 bug,delete process definition when process instance is running

* revert CONTRIBUTING.md

* dockerfile modify

* dockerfile modify

* dockerfile modify

* remove install.sh server.servlet.context-path replace

* Fix DAG add dependent project value exception # 1816 and keep requesting task interface (#1827)

* merge dev branch front-end code

* fix udfs assignment

* Fix task instance page jump

* fix udfs assignment and task instance page jump #1789

* Fixed space and icon display issues before and after the input box

* add license

* add license

* Fix data echo, style and popup cannot be closed

* Fix page number loading issue and dag not getting value

* Fix DAG add dependent project value exception # 1816 and keep requesting task interface

* rpm package add UI (#1846)

* [fix #1828] when the executor of process instance is not the owner of udf resouce, the path of the read resource file is incorrect (#1847)

* fix issue 1828:get the udf resource path error when create udf function

* update grantResources

* first verify whether udf resource is bound by udf function

* update grantResources

* update testListAuthorizedUdfFunc

* update getUserInfo in order to run success

* add LoggerServer RPC PORT modified. #1848 (#1849)

* fix #1775 bug,delete process definition when process instance is running

* revert CONTRIBUTING.md

* dockerfile modify

* dockerfile modify

* dockerfile modify

* remove install.sh server.servlet.context-path replace

* add LoggerServer RPC PORT modified. #1848

* LoggerService Logger RPC PORT get Error modify . #1848 (#1857)

* fix #1775 bug,delete process definition when process instance is running

* revert CONTRIBUTING.md

* dockerfile modify

* dockerfile modify

* dockerfile modify

* remove install.sh server.servlet.context-path replace

* add LoggerServer RPC PORT modified. #1848

* LoggerService Logger RPC PORT get Error modify . #1848

* Fix UDF function list delete data without refresh and page data display widened #1851 (#1852)

* merge dev branch front-end code

* fix udfs assignment

* Fix task instance page jump

* fix udfs assignment and task instance page jump #1789

* Fixed space and icon display issues before and after the input box

* add license

* add license

* Fix data echo, style and popup cannot be closed

* Fix page number loading issue and dag not getting value

* Fix DAG add dependent project value exception # 1816 and keep requesting task interface

* Fix UDF function list delete data without refresh and page data display widened

* [Fix #1828]check whether has permission to download udf file or delete udf function (#1858)

* fix issue 1828:get the udf resource path error when create udf function

* update grantResources

* first verify whether udf resource is bound by udf function

* update grantResources

* update testListAuthorizedUdfFunc

* update getUserInfo in order to run success

* check whether has permission to download udf file or delete udf file

* update listAuthorizedResourceById in ResourceMapper.xml

* add log, don't swallow exception info (#1877)

* Added hints in Resource Center (#1891)

* update jackson version from 2.9.8 to 2.10.1 (#1892)

* update release notes and modify the plugin of rat (#1899)

* modify how to build (#1902)

* update release notes and modify the plugin of rat

* modify how to build

* [maven-release-plugin] prepare release 1.2.1

* [maven-release-plugin] prepare for next development iteration

* get authorized udf resources need exclude all created by self (#1905)

* get authorized udf resources need exclude all created by self

* reset pom version to 1.2.1-SNAPSHOT

* Update ReleaseNotes.md

add detail info

* Update ReleaseNotes.md

* update notice year form 2019 to 2019-2020 (#1907)

* [maven-release-plugin] prepare release 1.2.1

* [maven-release-plugin] prepare for next development iteration

* 当手动重跑历史活着补数的时候对shell和sql脚本中的[YYYYmmddd...]变量赋值与传递的日期,而不是当前日期

Co-authored-by: lgcareer <18610854716@163.com>
Co-authored-by: bao liang <29528966+lenboo@users.noreply.github.com>
Co-authored-by: break60 <790061044@qq.com>
Co-authored-by: Tboy <guo.jiwei@immomo.com>
Co-authored-by: elonlo <gamelifedong@gmail.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>
Co-authored-by: zhangchunyang <18910529250@163.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: 谷雨 <lixun@guyudeMacBook-Pro.local>
2020-06-29 17:36:28 +08:00
gabry.wu
7ac6ad5491 Adapting partial code(file name start with N) to the sonar cloud rule (#2215) 2020-06-29 17:36:28 +08:00
break60
280a240ff5 datax 2020-06-29 17:36:28 +08:00
break60
55e232db84 dd branch flow node verification 2020-06-29 17:36:28 +08:00
xingchun-chen
98f3201a4d add LoginTest license (#2217)
Co-authored-by: chenxingchun <438044805@qq.com>
2020-06-29 17:36:28 +08:00
xingchun-chen
7c54496f5a e2e add the case of deleting user and tenant   (#2210)
* add e2e
2020-06-29 17:36:28 +08:00
t1mon
9c754378f3 Optimize PropertyUtils instantiation. (#2199)
* Optimize PropertyUtils instantiation.

* Fix info error.
2020-06-29 17:36:28 +08:00
gabry.wu
015a8ca269 Adapting partial code(file name start with M #2) to the sonar cloud rule (#2179)
* Adapting partial code(file name start with M #2) to the sonar cloud rule

* remove unused import
2020-06-29 17:36:28 +08:00
Rubik-W
9fe04524ae fix: Api server startup fail (#2198) 2020-06-29 17:36:28 +08:00
tswstarplanet
1d4cad7913 fix ResultSet not close and reformat code (#2183)
* fix ResultSet not close

* fix ResultSet not close

* extract code of closing resource to a method

* remove redundant if condition

* modify e2e timeout
2020-06-29 17:36:28 +08:00
tswstarplanet
585f833549 change static field to instance field (#2190) 2020-06-29 17:36:28 +08:00
t1mon
55cc1c7c5b Replace Charset.forName("UTF-8") with StandardCharsets.UTF_8. (#2193) 2020-06-29 17:36:28 +08:00
tswstarplanet
77c4bcb96e Cache HadoopUtils instance with specific days expire time (#2181)
* Cache HadoopUtils instance with 7 days expire time

* solve sonar issue

* add kerberos expire time config

* move KERBEROS_EXPIRE_TIME to Constants.java
2020-06-29 17:36:28 +08:00