Commit Graph

6803 Commits

Author SHA1 Message Date
FluorineDog
88f5642603
Add plan proto and support basic boolean expr parser (#5088)
**What type of PR is this?**
- [x] Feature

**What this PR does / why we need it:**
This PR supports boolean expression as DSL.
1. The goal of this PR is to support predicates
    like `A > 3 && not B < 5 or C in [1, 2, 3]`. 
2. Defines `plan.proto`, as Intermediate Representation (IR) 
    used between go and cpp. 
3. Support expr parser, convert predicate expr to IR
    in proxynode, while doing static check there
4. Support IR to AST in cpp, enable the execution
2021-04-29 08:48:06 +00:00
quicksilver
136cd7a704
Run KinD Inside A Kubernetes Cluster For Continuous Integration (#5085)
**What type of PR is this?**

- [x] Improvement

**Which issue(s) this PR fixes:**

Fixes #5038

**What this PR does / why we need it:**

Run KinD Inside A Kubernetes Cluster For Continuous Integration
2021-04-29 08:45:17 +00:00
bigsheeper
88e1d651cb
Remove partition and segment ids properly in query node (#5090)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

**What type of PR is this?**

- [ ] API-change
- [x] BUG
- [ ] Improvement
- [ ] Documentation
- [ ] Feature
- [ ] Test and CI
- [ ] Code Refactoring

**Which issue(s) this PR fixes:**

Fixes #

**What this PR does / why we need it:**

Not Available
2021-04-29 07:59:08 +00:00
quicksilver
7a188a8ae5
Update openblas images (#5082)
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-04-29 11:28:27 +08:00
quicksilver
a042090b86
Fix publish pytest image bug (#5084)
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-04-29 11:21:35 +08:00
Xiangyu Wang
d82f6d2764
[skip ci]Update mergify rules (#5083)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-04-29 10:39:28 +08:00
Xiangyu Wang
409564909e
[skip ci]Update mergify (#5074)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-04-29 10:19:41 +08:00
XuanYang-cn
cebdfb34f7
Enable multiple datanode (#5068)
Now it's able to start multiple datanode, however, they will process the same insert data.
Further changes are needed to truly enable multiple datanode:

While registering, dataservice need to return different insert channels for different datanode.

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-04-28 18:43:48 +08:00
godchen
004598f796
Remove field in desc index interface. (#5080)
* remove field in desc index interface

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* trigger GitHub actions

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-04-28 09:48:38 +00:00
quicksilver
eb338d4620
Provide install dependencies for Ubuntu and CentOS (#5076)
* Provide install dependencies for Ubuntu and CentOS

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

* Update scripts/install_deps.sh

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-04-28 14:46:18 +08:00
godchen
584051b2bf
Fix trace unavailable on distributed version (#5072)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-04-28 11:48:44 +08:00
godchen
94f0966225
Add get index build progress interface implementation (#5067)
Add get index build progress interface implementation.

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-04-28 03:15:28 +00:00
quicksilver
27af87e4d8
Used for local development or CI in Kubernetes in Docker (#5061)
* Used for local development or CI in Kubernetes in Docker
2021-04-27 19:27:50 +08:00
godchen
c64f3a2a52
Fix the version of protobuf in go.mod not consistence with MakeFile (#5047)
Fix the version of protobuf in go.mod not consistence with MakeFile

Signed-off-by: godchen qingxiang.chen@zilliz.com
2021-04-27 18:18:04 +08:00
godchen
060f7ca0d2
Add get index build progress proto (#5049)
Add get index build progress proto.

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-04-27 07:46:45 +00:00
shengjun.li
a3e4339027
improve rhnsw (#5059)
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2021-04-27 06:17:52 +00:00
zhenshan.cao
5a89b2668e
Update CODEOWNERS (#5062)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-04-27 12:01:57 +08:00
neza2017
262c484cec
the return struct of "DescribeIndex" should contain "filed_name" (#5055)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-04-27 02:30:55 +00:00
quicksilver
521f423326
Update milvus builder image version (#5058)
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-04-27 02:11:50 +00:00
Cai Yudong
28f636151c
remove tests/python (#5057)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-04-26 20:54:54 +08:00
XuanYang-cn
6ad2252f2d
Fix panic bug when server stop with alias (#5054)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-04-26 12:45:16 +00:00
Cai Yudong
4cec3924cc
rename all singlenode to standalone (#5051)
* rename all singlenode to standalone

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update docker image name

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix merge conflict

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* Rename all singlenode to standalone (#1)

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

Co-authored-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-04-26 10:35:57 +00:00
neza2017
d7c0a72611
index name is unique in collection scope (#5046)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-04-26 15:53:47 +08:00
quicksilver
3b529f3b5d
Update milvus project name (#5032)
* Update milvus project name
2021-04-26 13:59:28 +08:00
Cai Yudong
d4e7b4d1a4
remove .env and update docker-compose yml (#5048)
* remove .env and update docker-compose yml
2021-04-26 11:31:23 +08:00
Cai Yudong
74f457bd14
add more testcase to improve code coverage for data service (#5041)
* fix TestChannel run error

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* comment validateAllocRequest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add test APIs

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix static-check

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-04-26 01:45:54 +00:00
neza2017
0d68c3f2fd
fix flock, lock the pid file during the processing is running (#5037)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-04-26 01:28:23 +00:00
Xiangyu Wang
53b5092b3a
[skip ci]Update mergify rules (#5035)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-04-25 17:38:48 +08:00
bigsheeper
4197b094bf
Use internal dsl and placeholder group (#5033)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-04-25 06:08:28 +00:00
Cai Yudong
7c14682c89
update docker-compose configuration (#5029)
remove pulsar from docker-compose.standalone.yml
2021-04-25 11:16:07 +08:00
sunby
4d03da9c4f
Remove SetSegmentState and seal segment in segment allocator (#4994)
Segment should be sealed once it exceeds the limited size. This operation
should be done in segment allocator. Watcher only check the status to
decide whether the segment will be flushed.

Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-04-25 01:51:57 +00:00
neza2017
ada58b5bde
add unit test for master's grpc server, the coverage of package "internal/distributed/masterservice" increase from 46.6% to 82.9% (#5019)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-04-24 09:23:35 +00:00
Xiangyu Wang
efdc2f1a3f
[skip ci]Update codeowners (#5026)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-04-24 17:23:27 +08:00
bigsheeper
c4b379be8b
Set search graceful time from config (#5014)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-04-24 07:53:37 +00:00
Xiangyu Wang
746fc556a0
[skip ci]Add go code coverage (#5024)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-04-24 15:10:24 +08:00
sunby
b855c283cb
Remove meaningless log test (#5025)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-04-24 05:07:58 +00:00
Cai Yudong
5a75e23795
improve data service code coverage (#5020)
* optimize TestAssignSegmentID

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update mockMasterService to improve code coverage

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename client to kvClient in Server

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update server.go

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add TestChannel

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update TestChannel

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update TestChannel

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update TestChannel

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-04-24 03:29:15 +00:00
bigsheeper
2b5bedf736
Use multiple search channels (#5018)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-04-24 02:44:28 +00:00
Xiangyu Wang
4495cf84f0
[skip ci]Add a mergify rule (#5021)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-04-24 10:24:18 +08:00
Xiangyu Wang
3c58e7e9b2
[skip ci]Remove unused content in pull request template (#5015)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-04-23 18:14:56 +08:00
bigsheeper
00bb1014cd
Add search channels to query node (#5012)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-04-23 10:11:26 +00:00
shengjun.li
a2875f9d95
Update knowhere (#5006)
Import performance of ivf::train and hnsw, and fix bugs
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2021-04-23 18:03:49 +08:00
Xiangyu Wang
7aaae3f98c
[skip ci]Fix broken links in README.md and README_CN.md (#5002)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-04-23 02:55:29 +00:00
Xiangyu Wang
e8d9d2b030
[skip ci]Add an automerge rule for document changes (#5005)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-04-23 10:54:53 +08:00
Xiangyu Wang
0629b9c773
[skip ci]Update automerge conditions (#5001)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-04-23 10:27:54 +08:00
Cai Yudong
111376ba40
improve code coverage for data service (#4998)
* rename mock.go to mock_test.go

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add allocator_test.go to improve code coverage

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update meta_test.go

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update watcher_test.go

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix race

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-04-23 02:15:30 +00:00
godchen
4f7e393bfd
Remove useless config (#4997)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-04-23 10:10:46 +08:00
bigsheeper
dadb02db75
Remove unused params in query node (#4987)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-04-23 10:07:45 +08:00
Xiangyu Wang
82ccd4cec0
Rename module (#4988)
* Rename module

Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-04-22 14:45:57 +08:00
XuanYang-cn
5645718311
edit datanode config files (#4992)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-04-22 11:19:48 +08:00