Commit Graph

25 Commits

Author SHA1 Message Date
Wenjun Ruan
742944c60d
[Fix] [Worker] Fix worker will hang if fails to start (#10501)
* [Fix] [Worker] Fix worker will hang if fails to start (#10342)

* Fix worker will hang if fails to start

* Add .run to ignore

Signed-off-by: ruanwenjun <wenjun@apache.org>

* Add import Epoll

* Add cpu_quota in h2 to fix UT

* Remove cpu_quota,memory_max in TaskDefiniitionMapper
2022-06-22 10:13:55 +08:00
songjianet
7a7807e520
[Feature] Remove old ui. (#9909)
* [Feature] Remove old ui.

* [Docs] Add license file for screenfull.
2022-05-06 14:04:35 +08:00
Jiajie Zhong
90f9aed11d
[python] Add tox for out of box test (#8837)
* [python] Add tox for out of box test

* Add `tox` for easy and complete test
* Correct package data attribute
* Add more dirs in flake8 ignore

* Change GA from raw to tox

* Fix env setter

* Temp skip sanity

* Rm not exists job

* Add code test

* fix

* rm all need

* fix

* fix

* fix

* fix

* fix

* fix

* Migrate pip upgrede to tox setting

* Change install_commands to commands

* fix config of install command

* Add GA env to do that

* Fix env error

* Finial commit

* remove duplicate

* Change doc

* Change dependence of ci

* Change dependence of ci
2022-03-12 21:27:36 +08:00
kezhenxu94
d2a360480a
[CI][UI NEXT] Package new UI into release tar and remove old UI from it (#8670)
* [CI][UI NEXT] Package new UI into release tar and remove old UI from it

* Update e2e.yml
2022-03-07 11:20:24 +08:00
Wenjun Ruan
f88def8ef7
[IMPROVEMENT-8178] Add Netty processor in Spring container (#8179)
* Add Netty processor in Spring container
2022-01-25 21:33:09 +08:00
kezhenxu94
67cc260d52
Split the components into individual package, refactor configurations to use Spring properties (#7160)
* Split the components into individual package

A follow-up PR will be made to build dedicated Docker images for each
component, so that every component Docker image has minimal jars, which
is easy to maintain and good for security fixes.

* Split the components into individual package

A follow-up PR will be made to build dedicated Docker images for each
component, so that every component Docker image has minimal jars, which
is easy to maintain and good for security fixes.

* Split the components into individual package

A follow-up PR will be made to build dedicated Docker images for each
component, so that every component Docker image has minimal jars, which
is easy to maintain and good for security fixes.
2021-12-10 11:28:53 +08:00
Jiajie Zhong
54933b33e3
[ci][python] Add coverage check in CI (#6861)
* [ci] Add coverage check in CI

* Coverage add dependent

* Install pydolphinscheduler before run coverage

* Up test coverage to 87% and down threshold to 85%

* Fix code style

* Add doc about coverage
2021-11-17 09:46:40 +08:00
kezhenxu94
4114cb07f6
Add actuator and enable metrics, add Grafana configs (#6840) 2021-11-14 20:22:13 +08:00
Jiajie Zhong
dd6ed36f65
Add Python API implementation of workflows-as-code (#6269)
* Init DS python SDK pydolphinscheduler: python code definition

* Doc first

* Add quick start and developer doc

* Java documentation change

* Add LICENSE-py4j.txt

* Add py4j to release-docs/LICENSE

* Move dependency version to parent pom

* Remove outdated code

* Add tenant parameter to tutorial
2021-10-31 20:35:46 +08:00
Shiwen Cheng
cbada7c16d
[Improvement][Docker] Reduce docker image size, remove frontend/nginx, remove stdout log, improve compose/swarm deployment and fix random PSQLException (#4643)
* [Improvement][Docker] Reduce docker image size, remove frontend/nginx, remove stdout log and improve compose/swarm deployment

* [Improvement][Docker] Fix random PSQLException
2021-02-14 17:42:04 +08:00
Shiwen Cheng
0b8fcac37b
[Fix-4439][*] Fix cors problem (#4490)
* [Fix][UI] Fix warning in npm install

* [Fix-4439][*] Fix CORS problem

* [Improvement][API] Improve locale change interceptor

* [Improvement][API] Add test for locale change interceptor

* [Improvement][API] Fix code style

* [Improvement][Build] Update .gitignore
2021-01-24 15:41:58 +08:00
baoliang
381d23ea4f merge from 1.3.3-release 2020-11-06 00:03:20 +08:00
rockxsj
2b46db0bf7
[Feature]add presto datasource support (#3308)
* * add presto datasource support
update .gitigonre to igonre some files

* * use another presto driver

* * add LICENSE files about presto-jdbc

* * just for test sonar

Co-authored-by: rockxsj <junjun@mgtv.com>
2020-08-14 08:52:31 +08:00
kezhenxu94
e11fda8af3 Add a script to check the license (#2552)
* Add a script to check the license

### Motivation

Check licenses in the distribution package

### Modification

1. Add a script to check licenses, and list known licenses.
1. Remove unused dep license, lombok.
1. Remove unnecessary ignored patterns in .gitignore.
1. Add missing config items to apache-rat plugin, which just checked a small part of files before.
1. Add check to GitHub Actions process

### Result

1. Newly-added dependencies should be checked and confirmed.
1. Closes #1578

* Separate rat and dependency check

* Tee dependencies to stdout

* Add more debug messages

* Check light-weight task first
2020-06-29 17:39:06 +08:00
itbasketplayer
d03f9daa0e .gitignore remove taskInstance/index.vue(#2300) and no need to findProcessDefineById 2 times(#2373) (#2374)
* some optimization

* revert

* ProcessDefinitionService#updateProcessDefinition findProcessDefineById 1 time
2020-06-29 17:36:42 +08:00
kezhenxu94
6e08b29254
Add a script to check the license (#2552)
* Add a script to check the license

### Motivation

Check licenses in the distribution package

### Modification

1. Add a script to check licenses, and list known licenses.
1. Remove unused dep license, lombok.
1. Remove unnecessary ignored patterns in .gitignore.
1. Add missing config items to apache-rat plugin, which just checked a small part of files before.
1. Add check to GitHub Actions process

### Result

1. Newly-added dependencies should be checked and confirmed.
1. Closes #1578

* Separate rat and dependency check

* Tee dependencies to stdout

* Add more debug messages

* Check light-weight task first
2020-05-02 22:19:20 +08:00
qiaozhanwei
45ea2ccfdb Merge remote-tracking branch 'remotes/upstream/dev' into dev 2020-04-07 17:33:25 +08:00
itbasketplayer
d3990dfdb4
.gitignore remove taskInstance/index.vue(#2300) and no need to findProcessDefineById 2 times(#2373) (#2374)
* some optimization

* revert

* ProcessDefinitionService#updateProcessDefinition findProcessDefineById 1 time
2020-04-07 15:14:08 +08:00
dailidong
ce2853ad4c
add master and worker properties and remove data quality config (#2211)
* update logback

* update log

* refactor worker registry (#2107)

* Refactor worker (#2115)

* refactor worker registry

* refactor master server

* Modify workgroupid parameter name (#2105)

* Delete worker group management page

* Modify workgroupid parameter name

* Refactor worker (#2121)

* refactor worker registry

* refactor master server

* refactor MasterSchedulerService

* cancelTaskInstance set TaskExecutionContext host,logPath,executePath (#2126)

* 1, master persistent task
2. extract  master and worker communication model

* 1, master persistent task
2. extract  master and worker communication model

* 1, master persistent task
2. extract  master and worker communication model

* add license

* modify javadoc error

* TaskExecutionContext create modify

* buildAckCommand taskInstanceId not set modify

* java doc error modify

* add comment

* ExecutorManager interface add generic type

* add TaskInstanceCacheManager receive Worker report result

* TaskInstance setExecutePath

* add TaskInstanceCacheManager to receive Worker Task result report

* TaskInstanceCacheManager add remove method

* add license

* add dispatcht task method

* AbstractCommandExecutor remove db access

* AbstractCommandExecutor remove db access

* AbstractCommandExecutor remove db access

* AbstractCommandExecutor remove db access

* AbstractCommandExecutor remove db access

* AbstractCommandExecutor remove db access

* AbstractCommandExecutor remove db access

* taskInstanceCache is null ,need load from db

* taskInstanceCache is null ,need load from db

* taskInstanceCache is null ,need load from db

* 1,worker TaskPros use TaskExecutionContext replase
2,Master kill Task , KillTaskProcessor modify

* worker remove db

* ShellTask modify

* master persistence processId and appIds

* master persistence processId and appIds

* master add kill task logic

* master add kill task logic

* master add kill task logic

* javadoc error modify

* remove chinese log

* executeDirectly method add Override

* remote module modify

* TaskKillResponseProcessor command type modify

* create buildKillCommand

* host add host:port format

* host add host:port format

* TaskAckProcessor modify

* TaskAckProcessor modify

* task prioriry refator

* remove ITaskQueue

* task prioriry refator

* remove ITaskQueue

* TaskPriority refactor

* remove logs

* WorkerServer refactor

* MasterSchedulerService modify

* WorkerConfig listen port modify

* modify master and worker listen port

* cancelTaskInstance set TaskExecutionContext host,logPath,executePath

* cancelTaskInstance set TaskExecutionContext host,logPath,executePath

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* not exist in openjdk,just delete

* add master and worker properties

* add master and worker properties

* add master and worker properties

Co-authored-by: Tboy <guo.jiwei@immomo.com>
Co-authored-by: break60 <790061044@qq.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
2020-03-19 00:07:36 +08:00
gaojun2048
fd93b2bbac
fix https://github.com/apache/incubator-dolphinscheduler/issues/1420 (#1443) 2019-12-11 15:04:24 +08:00
lgcareer
31f9fbf335 change escheduler to dolphinscheduler (#961)
* rename from DatasourceUserMapper to DataSourceUserMapper

* add unit test in UserMapper and WorkerGroupMapper

* change cn.escheduler to org.apache.dolphinscheduler

* add unit test in UdfFuncMapperTest

* add unit test in UdfFuncMapperTest

* remove DatabaseConfiguration

* add ConnectionFactoryTest

* cal duration in processInstancesList

* change desc to description

* change table name in mysql ddl

* change table name in mysql ddl

* change escheduler to dolphinscheduler

* change escheduler to dolphinscheduler
2019-10-08 18:30:40 +08:00
dailidong
d2fe0b10f1 add monitor by lidong 2019-04-24 17:59:27 +08:00
dailidong
1e43309661 Merge remote-tracking branch 'upstream/dev-20190415' into dev-20190415
# Conflicts:
#	escheduler-api/src/main/java/cn/escheduler/api/enums/Status.java
2019-04-24 17:04:21 +08:00
baoliang
df1ba0b85d Merge remote-tracking branch 'upstream/dev' into dev
# Conflicts:
#	docs/zh_CN/前端部署文档.md
2019-04-12 11:55:16 +08:00
gongzijian
5070f86bc2 update .gitignore 2019-04-12 11:32:35 +08:00