2021-07-26 15:17:20 +08:00
|
|
|
# Scale Tests
|
|
|
|
## Goal
|
|
|
|
Scale tests are designed to check the scalability of Milvus.
|
|
|
|
|
|
|
|
For instance, if the dataNode pod expands from one to two:
|
|
|
|
- verify the consistency of existing data
|
|
|
|
|
|
|
|
- verify that the DDL and DML operation is working
|
|
|
|
|
|
|
|
## Prerequisite
|
2021-12-21 15:55:46 +08:00
|
|
|
- Kubernetes Cluster
|
|
|
|
- Milvus Operator (refer to [Milvus Operator](https://github.com/milvus-io/milvus-operator))
|
2021-07-26 15:17:20 +08:00
|
|
|
|
|
|
|
## Test Scenarios
|
|
|
|
### Milvus in cluster mode
|
2021-12-21 15:55:46 +08:00
|
|
|
- scale dataNode replicas
|
2021-07-26 15:17:20 +08:00
|
|
|
|
2021-12-21 15:55:46 +08:00
|
|
|
- expand / shrink indexNode replicas
|
2021-07-26 15:17:20 +08:00
|
|
|
|
2021-12-21 15:55:46 +08:00
|
|
|
- scale queryNode replicas
|
2021-07-26 15:17:20 +08:00
|
|
|
|
2021-12-21 15:55:46 +08:00
|
|
|
- scale proxy replicas
|
2021-07-26 15:17:20 +08:00
|
|
|
|
|
|
|
## How it works
|
|
|
|
|
2021-12-21 15:55:46 +08:00
|
|
|
- Milvus scales the number of pods in a deployment based on the milvus operator
|
2021-07-26 15:17:20 +08:00
|
|
|
|
|
|
|
- Scale test decouple the milvus deployment from the test code
|
|
|
|
|
|
|
|
- Each test scenario is carried out along the process:
|
|
|
|
<br> deploy milvus -> operate milvus -> scale milvus -> verify milvus
|
|
|
|
|
2021-12-21 15:55:46 +08:00
|
|
|
- Milvus deployment and milvus scaling are designed in `./customize/milvus_operator.py`
|
2021-07-26 15:17:20 +08:00
|
|
|
|
|
|
|
## Run
|
|
|
|
### Manually
|
|
|
|
Run a single test scenario manually(take scale dataNode as instance):
|
|
|
|
|
2021-12-21 15:55:46 +08:00
|
|
|
- update
|
|
|
|
update milvus image tag `IMAGE_TAG` in `scale/constants.py`
|
2021-07-26 15:17:20 +08:00
|
|
|
|
|
|
|
- run the commands below:
|
|
|
|
```bash
|
2021-09-14 13:11:54 +08:00
|
|
|
cd /milvus/tests/python_client/scale
|
2021-07-26 15:17:20 +08:00
|
|
|
|
2021-09-14 13:11:54 +08:00
|
|
|
pytest test_data_node_scale.py::TestDataNodeScale::test_expand_data_node -v -s
|
2021-07-26 15:17:20 +08:00
|
|
|
```
|
|
|
|
|
|
|
|
### Nightly
|
|
|
|
still in planning
|