Commit Graph

6348 Commits

Author SHA1 Message Date
mazhong
8821b91829
[FIX-9471][Script] fix run install.sh error (#9472) 2022-04-13 17:53:05 +08:00
Eric Gao
eb21b6cc52
[Feature][Doc] Switch shell task instruction screenshots to latest (#9434) 2022-04-13 14:13:42 +08:00
Amy0104
099bde0f78
[Fix][Next-UI] Add LIST type to the custom parameter types of task node. (#9468) 2022-04-13 13:45:39 +08:00
Tq
04d4e4e0c4
fix the OUT direct param could pass to the current script (#9463) 2022-04-13 10:43:12 +08:00
labbomb
8440baa5e8
[Bug][UI Next]Fix global variable validation condition for save button in workflow definition. (#9454) 2022-04-12 17:41:46 +08:00
Jiajie Zhong
51c1d8f2aa
[doc] Migrate develop docs from website (#9457)
* [doc] Migrate develop docs from website

developer should also into main repo, because
development setup should follow with the latest
code

* Add missing change from website
2022-04-12 17:23:56 +08:00
Amy0104
ac2e1df2bf
[Fix][UI][2.0.5] Add LIST type to the custom parameter types of task node. (#9455) 2022-04-12 17:04:58 +08:00
xiangzihao
d18cf184c6
[CI] Improve CI (#9436)
* test

* test

* test

* test

* test

* test

* test

* test

* test

* improve ci

* improve ci

* improve ci
2022-04-12 14:43:15 +08:00
Tq
fb0139e959
[Feature] [ALERT-9406]enrich alert info when insert new row into DB (#9445)
* enrich alert info when insert new row into DB

* fix comments
2022-04-12 14:31:38 +08:00
worry
69923546a1
[DS-9263][Improvement][master]optimize failover (#9281)
- add FailoverService.java
- move failover method  from MasterRegistryClient to FailoverService
- move failover code from FailoverExecuteThread to FailoverService

This closes #9263
2022-04-12 11:53:18 +08:00
Lyle Shaw
9d7223b038
[doc] fix url wrong in zh doc (#9421)
Co-authored-by: xiaoliangyu <xiaoliangyu@bytedance.com>
2022-04-12 09:45:25 +08:00
Jiajie Zhong
3d78859fe5
[python] Recover python release properties (#9444)
This patch recovers the properties `python.sign.skip=false`
when the combined profile `release,python` is used.

also close: #9433
2022-04-12 09:42:59 +08:00
Jiajie Zhong
aaf2042ec4
[python] Add missing doc about config and connect remote server (#9443)
which includes `configuration`, `run example`
`how to connect remote server`

close: #9286, #9284, #8917
2022-04-12 09:42:28 +08:00
Jiajie Zhong
62284ae689
[doc] Add some dev missing doc (#9427)
* [doc] Add some dev missing doc

Including general-setting, task-definition, audit-log
and they related img

* Change base on suggestion

Co-authored-by: Tq <tianqitobethefirst@gmail.com>
2022-04-11 21:46:40 +08:00
caishunfeng
b285ccf930
[Future-9396]Support output parameters transfer from parent workflow to child work flow (#9410)
* [Future-9396]Support output parameters transfer from parent workflow to child work flow

* fix note
2022-04-11 20:03:16 +08:00
kezhenxu94
14d71d1462
[UI] Migrate NPM to PNPM in CI builds (#9431) 2022-04-11 14:16:23 +08:00
Kerwin
923f3f38e3
[Fix-9316] [Task] Configure DB2 data source SQL script execution report ResultSet has been closed exception in SQL task (#9317)
* fix db2 error in the sql task

* update limit in sql task
2022-04-11 13:19:48 +08:00
nobolity
037692517a
[Fix-9251] [WORKER] reslove the sql task about of add the udf resource failed (#9319)
* feat(resource  manager): extend s3 to the storage of ds

1.fix some spell question
2.extend the type of storage
3.add the s3utils
to manager resource
4.automatic inject the storage in addition to your
config

* fix(resource  manager): update the dependency

* fix(resource  manager): extend s3 to the storage of ds

fix the constant of hadooputils

* fix(resource  manager): extend s3 to the storage of ds

1.fix some spell question
2.delete the import *

* fix(resource  manager):

merge  the unitTest:
1.TenantServiceImpl
2.ResourceServiceImpl
3.UserServiceImpl

* fix(resource  manager): extend s3 to the storage of ds

merge the resourceServiceTest

* fix(resource  manager): test  cancel the test method

createTenant verifyTenant

* fix(resource  manager): merge the code  follow the check-result of sonar

* fix(resource  manager): extend s3 to the storage of ds

fit the spell question

* fix(resource  manager): extend s3 to the storage of ds

revert the common.properties

* fix(resource  manager): extend s3 to the storage of ds

update the storageConfig with None

* fix(resource  manager): extend s3 to the storage of ds

fix the judge of resourceType

* fix(resource  manager): extend s3 to the storage of ds

undo the compile-mysql

* fix(resource  manager): extend s3 to the storage of ds

delete hadoop aws

* fix(resource  manager): extend s3 to the storage of ds

update the know-dependencies to delete aws 1.7.4
update the e2e
file-manager common.properties

* fix(resource  manager): extend s3 to the storage of ds

update the aws-region

* fix(resource  manager): extend s3 to the storage of ds

fix the storageconfig init

* fix(resource  manager): update e2e docker-compose

update e2e docker-compose

* fix(resource  manager): extend s3 to the storage of ds

revent the e2e common.proprites

print the resource type in propertyUtil

* fix(resource  manager): extend s3 to the storage of ds
1.println the properties

* fix(resource  manager): println the s3 info

* fix(resource  manager): extend s3 to the storage of ds

delete the info  and upgrade the s3 info to e2e

* fix(resource  manager): extend s3 to the storage of ds

add the bucket init

* fix(resource  manager): extend s3 to the storage of ds

1.fix some spell question
2.delete the import *

* fix(resource  manager): extend s3 to the storage of ds

upgrade the s3 endpoint

* fix(resource  manager): withPathStyleAccessEnabled(true)

* fix(resource  manager): extend s3 to the storage of ds

1.fix some spell question
2.delete the import *

* fix(resource  manager): upgrade the  s3client builder

* fix(resource  manager): correct  the s3 point to s3client

* fix(resource  manager): update the constant BUCKET_NAME

* fix(resource  manager): e2e  s3 endpoint -> s3:9000

* fix(resource  manager): extend s3 to the storage of ds

1.fix some spell question
2.delete the import *

* style(resource  manager): add info to createBucket

* style(resource  manager): debug the log

* ci(resource  manager): test

test s3

* ci(ci): add INSERT INTO dolphinscheduler.t_ds_tenant (id, tenant_code, description, queue_id, create_time, update_time) VALUES(1, 'root', NULL, 1, NULL, NULL); to h2.sql

* fix(resource  manager): update the h2 sql

* fix(resource  manager): solve to delete the tenant

* style(resource  manager): merge the style end delete the unuse s3 config

* fix(resource  manager): extend s3 to the storage of ds

UPDATE the rename resources when s3

* fix(resource  manager): extend s3 to the storage of ds

1.fix the code style of QuartzImpl

* fix(resource  manager): extend s3 to the storage of ds

1.impoort restore_type to CommonUtils

* fix(resource  manager): update the work thread

* fix(resource  manager): update  the baseTaskProcessor

* fix(resource  manager): upgrade dolphinscheduler-standalone-server.xml

* fix(resource  manager): add  user Info to dolphinscheduler_h2.sql

* fix(resource  manager): merge  the resourceType to NONE

* style(upgrade the log level to info):

* fix(resource  manager): sysnc the h2.sql

* fix(resource  manager): update the merge the user tenant

* fix(resource  manager): merge the resourcesServiceImpl

* fix(resource  manager):

when the storage is s3 ,that the directory can't be renamed

* fix(resource  manager): in s3 ,the directory cannot be renamed

* fix(resource  manager): delete the deleteRenameDirectory in E2E

* fix(resource  manager): check the style and  recoverd the test

* fix(resource  manager): delete the log.print(LoginUser)

* fix(server): fix the  udf serialize

* fix(master  task): update the udfTest to update the json string

* fix(test): update the udfFuncTest

* fix(common): syn the common.properties

* fix(udfTest): upgrade the udfTest

* fix(common): revent the common.properties
2022-04-11 10:49:46 +08:00
BaoLiang
fea9ce391b
Update inappropriate characters (#9413) 2022-04-10 21:09:48 +08:00
Tq
e2759a8f42
[Feature] [ALERT-9406]add new properties to alert class (#9408)
* add new properties to Alert.java and do minor changes to comments

* fix Integer to int

* fix Integer to int

* fix sql files

* fix not null properties to default null
2022-04-08 23:54:34 +08:00
worry
dce3c132ca
[DS-9387][refactor]Remove the lock in the start method of the MasterRegistryClient class (#9389) 2022-04-08 21:06:28 +08:00
Jiajie Zhong
2c49c248f3
[doc] Remove observability (#9402)
SkyWalking v9 is coming soon and there are without
DolphinScheduler menus anymore, So we should remove
the SW agent to avoid confusion.

close: #9242
2022-04-08 15:52:57 +08:00
labbomb
2db4cd4f14
[Bug][UI Next]Modify the display state logic of save buttons under workflow definition (#9403)
* Modifying site Configurations

* Modify the display state logic of save buttons under workflow definition
2022-04-08 14:55:49 +08:00
Eric Gao
d23616fcbc
[CI] Enable CI to remove unexpected files in /docs/img dir (#9393) 2022-04-08 13:46:35 +08:00
Jiajie Zhong
a8f6bf3831
Add new code owner of docs module (#9388) 2022-04-08 13:04:50 +08:00
Jiajie Zhong
c828809b46
[doc] Change get help from dev mail list to slack (#9377)
* Change all get help from dev mailing list to slack, because
  we find out mailing list have many users ask for subscribe
  and they maybe subscribe by accident.
* remove join dev mailing list in faq.md because we already
  have it in https://dolphinscheduler.apache.org/en-us/community/development/subscribe.html
2022-04-08 10:56:35 +08:00
songjianet
72bdf6f03c
[Fix][UI Next][V1.0.0-Alpha] Fix the task instance forced success button multi-language support error. (#9392) 2022-04-08 10:02:48 +08:00
guoshupei
ca95d2f928
[Fix-9221] [alert-server] optimization and gracefully close (#9246)
* [Fix-9221] [alert-server] optimization and gracefully close

This closes #9221

* [Fix-9221] [alert-server] remove unused mock data

This closes #9221

* [Fix-9221] [alert-server] remove unused mock data

This closes #9221

* [Fix-9221] [alert-server] remove unnecessary Mockito stubbings

* [Fix-9221] [alert-server] init AlertPluginManager in AlertServer

* [Fix-9221] [alert-server] AlertServerTest add AlertPluginManager installPlugin

* [Fix-9221] [alert-server] replace @Eventlistener with @PostConstruct

* [Fix-9221] [alert-server] sonar check solution

* [Improvement-9221] [alert] update constructor injection and replace IStoppable with Closeable

Co-authored-by: guoshupei <guoshupei@lixiang.com>
2022-04-08 10:02:10 +08:00
caishunfeng
ab0357027d
[Improvement] change method access (#9390)
* change method to protected

* change method access
2022-04-07 21:51:21 +08:00
Tq
2a4fa9cdb1
[BUG][WORKER-9349]fix param priority (#9379)
* fix param priority

* fix params priority code logic
2022-04-07 20:09:53 +08:00
Tq
f186b0d391
[Bug][API-9364]fix ProcessInstance wrong alert group id (#9383)
* fix ProcessInstance wrong alert group id

* change  createComplementCommandList method to protected
2022-04-07 19:19:38 +08:00
Eric Gao
fd6b43bc81
[Dev] Switch version in pom.xml to dev-SNAPSHOT (#9223) (#9299) 2022-04-07 18:15:19 +08:00
Amy0104
e2c36324b3
[Fix][UI Next][V1.0.0-Alpha] Add light color theme to echarts. (#9381) 2022-04-07 17:35:17 +08:00
Jiajie Zhong
3457cee960
[python] Migrate pythonGatewayServer into api server (#9372)
Currently the size of our distribute package is up to
800MB, this patch is migrate python gateway server into
api server

The distribute package size before and after this patch is:

```sh
# before
796M   apache-dolphinscheduler-2.0.4-SNAPSHOT-bin.tar.gz

# after
647M   apache-dolphinscheduler-2.0.4-SNAPSHOT-bin.tar.gz
```
2022-04-07 14:41:15 +08:00
mans2singh
950f32e1d6
[task-spark][docs] Corrected notice section (#9375) 2022-04-07 14:03:21 +08:00
gaojun2048
5ef3f9d668
[optimization] [Service] Optimization ProcessService and add ProcessService interface (#9370) 2022-04-07 12:21:34 +08:00
xiangzihao
80ea8049e0
[CI] try to fix ci (#9366)
* try to fix ci

* try to fix ci

* try to fix ci

* try to fix ci

* try to fix ci

* try to fix ci

* try to fix ci

* try to fix ci

* try to fix ci

* try to fix ci

* try to fix ci

* try to fix ci

* try to fix ci
2022-04-06 21:57:52 +08:00
xiangzihao
1679f15a50
[FIX-9355] Fix scheduleTime of start-process-instance api in api-doc (#9359)
* fix #9355

* fix #9355

* fix ut error

* fix ut error
2022-04-06 21:34:22 +08:00
Tq
c294979e2f
[Bug-9235][Alert]Fix wechat markdown message and change wechat form structure (#9367)
* fix wechat issues:
1. change table msg type to markdown.
2. change userId to not required and enrich hints
3. change 'app id' to 'app id and chat id'

* fix wechat issues:
1. revert table showtype and add markdown showtype.
2. enrich hints.
3. delete 'chatid', rename agentid to weChatAgentIdChatId.
4. modify code to send markdown message.

* fix wechat issues: Change the language pack of agentId to agentId/chatId.

* fix format

* fix param name

Co-authored-by: Amy <amywang0104@163.com>
2022-04-06 20:29:30 +08:00
worry
2bab12f2c8
[Feature-9204][alert] Implement alert send status (#9208)
* [DS-9204][feat][alert,dao] Implement alert send status
- implement alert send status
- add alert send status entity、mapper
- modify alert dao
- modify alert sender
- add test
- add sql

This closes #9204

* [DS-9204][feat][alert,dao] Implement alert send status
- add license header

This closes #9204
2022-04-06 18:08:00 +08:00
Devosend
ce7740b9fc
[Fix][UI Next][V1.0.0-Alpha]Add zh for dag execution policy (#9363) 2022-04-06 16:31:20 +08:00
Devosend
aba257084b
[Fix][UI-Next] Rename process to workflow (#9350) 2022-04-06 10:23:30 +08:00
Devosend
c21f8c650b
[Fix] [UI-Next][V1.0.0-Alpha] Rename node to task in the task creation modal (#9351) 2022-04-06 10:17:20 +08:00
Tq
4873ec8a45
fix task definition page info total count (#9354) 2022-04-06 09:57:20 +08:00
xiangzihao
40b73f7962
[Improve][CI] improve ci checking (#9325) 2022-04-06 09:33:41 +08:00
zchong
4a29c6a6c8
[Improvement-9338][API] show more create datasource exception message (#9336)
* Update DataSourceServiceImpl.java

fix  error message miss

* Update DataSourceServiceImpl.java

import optional jar
2022-04-06 09:17:29 +08:00
labbomb
ee0e3beb9d
[docs] Modifying site Configurations (#9341) 2022-04-03 21:48:22 +08:00
xiangzihao
3c5568c16c
[Doc] There are some version errors in the 2.0.1-release (#9335) 2022-04-03 12:55:25 +08:00
songjianet
a866465d01
[Fix][UI Next][V1.0.0-Alpha] Fix the time columns cannot be broken in the cron manage table. (#9334) 2022-04-02 18:12:31 +08:00
songjianet
d7a4dc1dbc
[Fix][UI Next][V1.0.0-Alpha] Fix the two hover prompts appear. (#9333)
* [Fix][UI Next][V1.0.0-Alpha] Fix the two hover prompts appear.

* [Fix][UI Next][V1.0.0-Alpha] Fix the two hover prompts appear.
2022-04-02 17:01:07 +08:00