Commit Graph

269 Commits

Author SHA1 Message Date
CalvinKirs
38e7494f78
replace CollectionUtils deprecated method (#3027)
* replace deprecated method

* replace deprecated method

* replace deprecated method

* replace deprecated method

* replace deprecated method

* messy

* messy

* messy

* add missing license

Co-authored-by: gabry.wu <gabrywu@apache.org>
Co-authored-by: dailidong <dailidong66@gmail.com>
2020-07-13 19:42:33 +08:00
CalvinKirs
6f9970b189
[Feature-2574][Server]Specify Network Interface (#3186)
* [Feature-2574][Server]Specify Network Interface
This closes #2574

* code smell

* code smell

* move DOLPHIN_SCHEDULER_PREFERRED_NETWORK_INTERFACE to Constants.java
2020-07-13 18:51:38 +08:00
Yichao Yang
b4af3fd176
[Feature-2815][server] One worker can belong to different workergroups (#2934)
* [Feature-2815][server] One worker can belong to different workergroups

* Feature: Add test cases

* Update MonitorService.java

* Update MonitorService.java

Co-authored-by: dailidong <dailidong66@gmail.com>
2020-07-13 17:25:13 +08:00
bao liang
657ea4752f
merge from 1.3.1-release (#3161)
* merge from 1.3.1-release

* [bugfix] remove Conflicting configuration sudo and user (#3038)

Co-authored-by: zhangchunyang1024 <zhangchunyang0123@gamil.com>

* remove some standard notices in order to cut down the NOTICE file. (#3101)

* #3084 remove some standard notices in order to cut down the NOTICE file.

* update exclude element sql/soft_version of rat plugin

* update soft_version

* merge from 1.3.1-release

* fix 3112:download udf resource need find the tenant code of resource owner (#3114)

* fix 3112:download udf resource need find the tenant code of resource owner

* fix 2975:download resource need find the tenant code of resource owner

* remove useless code

* merge from 1.3.1-release

* remove some notes (#3127)

Co-authored-by: lenboo <baoliang@analysys.com.cn>

* fix Permission denied if not found dolphinscheder-env.sh on 1.3.1 (#3128)

* fix Permission denied if not found dolphinscheder-env.sh

* update some comments on method

* add PropertyPlaceholderHelper on license (#3129)

* update license header location

* add PropertyPlaceholderHelper on license

* merge from 1.3.1-release

* change from 1.3.0  to 1.3.2

* add Set import

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: zhangchunyang <zhangchunyang0123@gmail.com>
Co-authored-by: zhangchunyang1024 <zhangchunyang0123@gamil.com>
Co-authored-by: lgcareer <18610854716@163.com>
Co-authored-by: lenboo <baoliang@analysys.com.cn>
2020-07-13 16:35:42 +08:00
muzhongjiang
c49626b877
Word spelling modification,Comment modification,Log level modification (#3180)
* add state

* fixed bug "jackson enum conversion  : InvalidFormatException"

* Word spelling modification
Comment modification
Word spelling modification,Comment modification,Log level modification

* Update EmailManager.java

* Update FlinkParameters.java

* Update SqlTask.java

Co-authored-by: mzjnumber1@163.com <mzjnumber1@163.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
2020-07-12 22:20:14 +08:00
rockxsj
dcdd7dedd0
Modify the AWS S3 request encryption method to V4. (#3182)
Co-authored-by: rockxsj <junjun@mgtv.com>
2020-07-11 08:56:38 +08:00
felix-thinkingdata
1e7582e910
#3176 optimize Gets the value of this property “resource.storage.type”, Comparison with enumerated types (#3178)
* fix bug #3165   get resource.storage.type value   toUpperCase

* fix bug #3176  optimize Gets the value of this property “resource.storage.type”

Co-authored-by: dailidong <dailidong66@gmail.com>
2020-07-10 15:21:42 +08:00
felix-thinkingdata
a23a3e2d12
fix bug #3165 get resource.storage.type value toUpperCase (#3166)
Co-authored-by: dailidong <dailidong66@gmail.com>
2020-07-09 18:28:04 +08:00
liukunyuan
0692dca4e6
update CommonUtils.java (#3137)
Co-authored-by: dailidong <dailidong66@gmail.com>
2020-07-05 21:50:42 +08:00
t1mon
83589c7b7c
Optimize kerberos tgt expire time in HadoopUtils. (#2700)
* Optimize PropertyUtils instantiation.

* Fix info error.

* Optimize kerberos tgt expire time.

Co-authored-by: dailidong <dailidong66@gmail.com>
2020-07-04 18:22:30 +08:00
CalvinKirs
d1282fdd96
In the case of multiple network cards, a valid host is returned (#2924)
* In the case of multiple network cards, a valid host is returned

* In the case of multiple network cards, a valid host is returned

* In the case of multiple network cards, a valid host is returned

* In the case of multiple network cards, a valid host is returned

* fix LOCAL_ADDRESS not assigned  value

* fix LOCAL_ADDRESS not assigned  value

* fix LOCAL_ADDRESS not assigned  value

* fix lock error

* fix lock error

* end

* end

* end

* end

* code smell

* code smell

* code smell

* code smell

* code smell

* code smell

* code smell

* lowering

* messy

* small change

* compile error

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: gabry.wu <gabrywu@apache.org>
2020-07-02 19:16:37 +08:00
muzhongjiang
a9eb7b18b0
[bugfix] fixed "jackson enum conversion : InvalidFormatException" (#3078)
* add state

* fixed bug "jackson enum conversion  : InvalidFormatException"

Co-authored-by: mzjnumber1@163.com <mzjnumber1@163.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
2020-07-02 18:07:57 +08:00
lenboo
fd76f172c1 Merge remote-tracking branch 'upstream/dev' into dev-merge
# Conflicts:
#	dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/runner/TaskExecuteThread.java
2020-06-30 11:17:14 +08:00
Yichao Yang
38e485373d
[Feature-2925][server] Init TaskLogger in TaskExecuteProcessor (#2925) (#2965)
* [Feature-2925][common] Add exitVal judge in OSUtils.exeCmd (#2925)

* optimize the logger utils
2020-06-28 17:43:13 +08:00
lenboo
ea1c653e08 merge from dev 2020-06-22 10:21:15 +08:00
CalvinKirs
518d03b6a6
Simplify some methods for easy understanding (#2920)
Co-authored-by: gabry.wu <8545796+gabrywu@users.noreply.github.com>
2020-06-21 15:48:35 +08:00
lenboo
73cae7f046 merge from dev-1.3.0 2020-06-19 15:19:32 +08:00
dailidong
620ecffa72
fix singe resource manager will NPE (#3013)
* fix worker group config no effect

* remove codehaus janino jar
the license about janino maybe not compatiable with Apache v2

* Merge remote-tracking branch 'upstream/dev-1.3.0' into dev-1.3.0

# Conflicts:
#	dolphinscheduler-server/src/main/resources/config/install_config.conf

* datasource config

* Update datasource.properties

* fix RunConfig bug

* remove param monitor server state

* fix table T_DS_ALERT

* update h2 database

* fix #2910 master server will show exception for some time when it restart

* fix oom when no master is active

* fix worker oom when master server restart

* fix oom

* fix

* add UT

* fix worker group config no effect

* fix singe resource manager will NPE

* Update HadoopUtilsTest.java
2020-06-19 14:45:48 +08:00
lenboo
714ae33e40 merge from dev 2020-06-19 14:20:24 +08:00
Yichao Yang
0744d4070c
[Hotfix-2922][common] Fix Jackson UDde error (#2922) (#2926)
Co-authored-by: dailidong <dailidong66@gmail.com>
2020-06-17 11:33:41 +08:00
Yichao Yang
0e7403e6a8
[Feature-2927][common] Fix NEF when get info from OSUtils (#2927) (#2964)
* [Feature-2927][common] Fix NEF when get info from OSUtils (#2927)

* Update OSUtils.java

Change java doc

Co-authored-by: gabry.wu <8545796+gabrywu@users.noreply.github.com>
2020-06-14 11:17:12 +08:00
simon824
3fa4dd702b Resolved the conflict 2020-06-13 09:47:18 +08:00
Simon
eebd0c73bb
Merge pull request #3 from apache/dev
merge
2020-06-13 09:26:40 +08:00
BoYiZhang
f56ae9e64a
add http task timeout & test unit (#2955)
* add http task timeout & test unit

* add http task timeout & test unit

Co-authored-by: zhanglong <zhanglong@ysstech.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
2020-06-12 18:52:19 +08:00
lijufeng2016
284c50f66c
Compatible with flink1.10 or newer (#2952)
* 兼容flink1.10以上版本

* fix null point bug

Co-authored-by: 李巨丰 <lijf@2345.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
2020-06-12 18:23:28 +08:00
simon824
1b7d8ee236 fix the bad code smell 2020-06-12 12:35:18 +08:00
simon824
24120a4641 fix the bad code smell 2020-06-12 11:27:42 +08:00
simon824
cfecfb799b 1. change windows new line 'CR' to linux new line 'LF'
2. solve conflict
2020-06-12 09:59:40 +08:00
Simon
0b11275880
Merge pull request #2 from apache/dev
merge
2020-06-12 09:22:58 +08:00
lenboo
1904a7022a change 1.3.0 to 1.3.0-snapshot 2020-06-10 11:42:15 +08:00
simon824
fad09075f9 remove fastjson maven dependence 2020-06-10 09:16:28 +08:00
eights
5d83d797d3 modify sqoop task UT 2020-06-10 09:11:17 +08:00
eights
c8f28ab2ba sqoop task optimization 2020-06-09 03:16:35 -07:00
simon824
8cabb7a907 1. specified jackson Date format.
2. remove getMapper method,replace with toList()
3. rewrite toList method,support constructCollectionType.
4. parseObject(resp, Map.class) replace to toMap() method
5. rename JacksonSerialize to JsonSerializer

rewrite createArrayNode,createObjectNode method will Affect too many lines ,i will fix it in the next pr.
2020-06-09 14:15:13 +08:00
simon824
6e907f150c 1. specified jackson Date format.
2. remove getMapper method,replace with toList()
3. rewrite toList method,support constructCollectionType.
4. parseObject(resp, Map.class) replace to toMap() method
5. rename JacksonSerialize to JsonSerializer

rewrite createArrayNode,createObjectNode method will Affect too many lines ,i will fix it in the next pr.
2020-06-09 13:02:50 +08:00
bao liang
422fe5136d
[license] add license and notice (#2916)
* fix bug: send task result to master until success.

* fix bug: send task result to master until success.

* add sleep

* set null if send result failed

* set null if send result failed

* change max length of username to 39

* test for gc

* change version to 1.3.0

* check and add some license files

* check and add some license files

* revert

Co-authored-by: baoliang <baoliang@analysys.com.cn>
2020-06-06 22:21:37 +08:00
Rubik-W
7c9f5960e5
Fixbug datax task (#2909)
* fix:  local param bug

* fix: UT bug

Co-authored-by: Rubik-W <whm_777@163.com>
2020-06-06 21:12:41 +08:00
simon824
572ee0fc64 Using Jackson instead of Fastjson 2020-06-04 16:23:59 +08:00
simon
8bab8abcd2 ut fix 2020-06-03 19:21:19 +08:00
simon
bb089288c4 Using Jackson instead of Fastjson 2020-06-03 12:40:28 +08:00
simon
dea2d4634d Using Jackson instead of Fastjson 2020-06-02 20:13:23 +08:00
simon
f26e246706 Using Jackson instead of Fastjson 2020-06-02 20:00:49 +08:00
张世鸣
e386af700d Using Jackson instead of Fastjson 2020-06-02 17:22:39 +08:00
张世鸣
a9f72c2081 Using Jackson instead of Fastjson 2020-06-02 15:54:34 +08:00
bao liang
41e4986eb2
change max length of user name to 39 (#2856)
* fix bug: send task result to master until success.

* fix bug: send task result to master until success.

* add sleep

* set null if send result failed

* set null if send result failed

* change max length of username to 39

Co-authored-by: baoliang <baoliang@analysys.com.cn>
2020-06-01 17:25:25 +08:00
Han Gao
8f8ddae030
Using Jackson instead of Fastjson (#2850)
* Using Jackson instead of Fastjson

* Fix some json bugs

Co-authored-by: dailidong <dailidong66@gmail.com>
2020-05-31 18:11:10 +08:00
CalvinKirs
c8ab27acb0
Optimization of enumerated values method (#2677)
* The value method of enum is a deep copy. If there are many parameters, it is not recommended to use the value method. Specifically see jmh results

* beautiful misunderstanding

Co-authored-by: dailidong <dailidong66@gmail.com>
2020-05-31 10:29:46 +08:00
itbasketplayer
1513aae3ce
add job history to judge application status/2625 (#2848)
* job history status url when application number threshold is reached(default 10000,maybe it was set to 1000)

* job history status url when application number threshold is reached(default 10000,maybe it was set to 1000)

* job history status url when application number threshold is reached(default 10000,maybe it was set to 1000)

Co-authored-by: yuhaibin@lizhi.fm <35716fc5847f6d154cf556296453ca91>
Co-authored-by: dailidong <dailidong66@gmail.com>
2020-05-30 23:07:17 +08:00
lgcareer
ef4c1e7930
add source license full text into source license (#2840)
* add source license full text into source license

* exclude ScriptRunner.java on rat check
2020-05-29 17:23:34 +08:00
lgcareer
fadb69b181
When update the data source,the host and port show error (#2821)
* When update the data source,the host and port show error

* update DataSourceServiceTest

* update DataSourceServiceTest

* update DataSourceServiceTest
2020-05-27 15:51:58 +08:00