Commit Graph

224 Commits

Author SHA1 Message Date
Cai Yudong
880e375b00
[skip ci] Remove comments (#10541)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-10-25 15:25:12 +08:00
Cai Yudong
f6802589eb
Optimize proxy reduce code readability (#10537)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-10-25 14:29:12 +08:00
zhenshan.cao
3af852eafb
Fix golint error in proxy (#10488)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-10-23 23:17:11 +08:00
zhenshan.cao
51d82fc71d
Fix golint error in proxy (#10487)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-10-23 18:25:37 +08:00
zhenshan.cao
169359448f
Fix golint error in proxy (#10486)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-10-23 18:23:44 +08:00
zhenshan.cao
43ef5d5eb9
Fix golint error in proxy (#10485)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-10-23 10:53:12 +08:00
Cai Yudong
824005115a
Remove duplicated search results in proxy (#10355)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-10-21 19:52:28 +08:00
Cai Yudong
b099179ac0
Optimize proxy reduce search result data (#10327)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-10-21 10:46:36 +08:00
Cai Yudong
b40513b211
Use timerecord to calculate time span (#10287)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-10-20 17:56:38 +08:00
godchen
24a6fc096b
Update query node bloom filter (#10109)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-18 20:08:42 +08:00
Cai Yudong
dc0269cf4c
Optimize proxy getChannels (#9964)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-10-15 20:37:04 +08:00
yukun
92815f4542
Change timestamp to timestamps in internalproto.DeleteRequest (#9802)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-10-14 14:48:34 +08:00
dragondriver
8f5bf46705
Fix golint warning of plan_parser.go (#9827)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-10-13 20:30:32 +08:00
Cai Yudong
84e568ee32
Support Delete in datanode (#8505)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-10-11 16:31:44 +08:00
Cai Yudong
bd3a8ed3cf
Support delete in proxy (#9588)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-10-11 07:48:55 +08:00
Xiangyu Wang
75716e0deb
Update partition name validation in DeleteTask (#9560)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-10-09 22:50:39 +08:00
dragondriver
302ca04d6c
Describe collection by collection ID (#9471)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-10-09 16:10:56 +08:00
elfisworking
d4232f88a2
For search funtion adding a round_decimal paramter to precision control (#8574)
Signed-off-by: elfisworking <zymustb@126.com>
2021-10-08 17:39:55 +08:00
congqixia
827a18a7c1
Fix Assigner typo (#9075)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-02 18:43:56 +08:00
godchen
4bfa85eda3
Fix searchTask result nil (#8844)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-09-29 15:48:00 +08:00
zhenshan.cao
a662c62880
Fix golint error of segment.go in proxy package (#8629)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-09-28 23:06:15 +08:00
dragondriver
6065bb50c4
Fix golint warning of funcutil (#8652)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-27 23:18:02 +08:00
zhenshan.cao
b71bffe26f
Replace ChannelID with ShardName in DML Request (#8539)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-09-27 10:01:59 +08:00
Cai Yudong
3938d68034
Optimize proxy query execute (#8479)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-09-25 00:21:57 +08:00
Cai Yudong
ee3eaa2966
Rename CreateQueryExprPlan to CreateExprPlan (#8438)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-09-24 10:27:54 +08:00
XuanYang-cn
4ee8275b88
Fix master compile error (#8333)
Resolves: #8336

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-22 20:31:54 +08:00
cai.zhang
729d409a32
Add trace log for search (#8302)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-09-22 19:36:00 +08:00
cai.zhang
79b7dcc75a
Add trace log for insert (#8301)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-09-22 19:33:54 +08:00
dragondriver
b2e8ba7b33
Fix reduce algorithm in proxy search task (#8206)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-22 10:15:54 +08:00
congqixia
bdfb8768d4
Enable gosec (#8202)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-18 14:45:50 +08:00
Yusup
ccb57ba4b2
add collection alias (#7255)
Signed-off-by: Yusup <yusup@lsgrep.com>
2021-09-18 11:13:51 +08:00
yukun
46871fcaec
Remove unused code in queryTask (#8077)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-09-17 21:57:58 +08:00
congqixia
4fe22bcaaa
Fix query task panicking (#8060)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-16 20:03:50 +08:00
dragondriver
5f3b961502
Add test case for delete task (#7939)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-15 14:04:54 +08:00
Xiaofan
d0667b58d7
Change Error Msg When dimension of vector field not exist (#7931)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-09-15 10:01:30 +08:00
dragondriver
2382be82a5
Add test case for the workflow of insert task (#7912)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-14 19:07:48 +08:00
dragondriver
ec776a30dc
Add test case for the workflow of query task (#7886)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-14 16:19:08 +08:00
dragondriver
f122e93383
Proxy ut cover case that search task process the fields data (#7874)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-14 14:09:05 +08:00
zhenshan.cao
e5e4b24954
Fix bug: describe collection return wrong shard number (#7841)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-09-14 11:59:47 +08:00
dragondriver
bcf9b4e240
Coverage error case of searchTask.PreExecute (#7826)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-14 10:24:00 +08:00
dragondriver
8d239b6473
Add test cases for search task (#7796)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-13 17:12:19 +08:00
Xiaofan
619c4393f3
Refine logs in proxy search path(#7357) (#7753)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-09-12 17:54:01 +08:00
dragondriver
5a3fc37584
Add test cases for create collection (#7759)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-11 18:06:02 +08:00
dragondriver
a872a4357c
Add more test cases for proxy implementation (#7685)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-11 11:36:22 +08:00
dragondriver
596357dc85
Add more data definition test cases (#7623)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-09 19:02:08 +08:00
zhenshan.cao
b0524adbd8
Support specified shard number when create collection (#7482)
Add shards_num to CollectionInfo

Modify dmlChannelNum config

Compatible with old meta

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-09-08 15:00:00 +08:00
dragondriver
42b687bf48
Add unittest for task scheduler (#7508)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-06 20:49:04 +08:00
yukun
94272bba87
Support query by expression (#7386)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-09-03 17:12:55 +08:00
groot
8701c477e2
Add delete api into proto (#7304)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-08-26 12:15:52 +08:00
Cai Yudong
0ab70271a6
correct empty search result handle (#7244)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-08-24 11:13:52 +08:00
Cai Yudong
8405d90f5e
proxy add illegal check for search result (#7227)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-08-23 17:03:51 +08:00
yukun
671b2737d2
Remove RetrieveRequest and RetrieveTask (#6848)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-08-16 17:06:10 +08:00
Cai Yudong
e771bda92f
optimize retrieve output vector code structure (#7102)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-08-14 11:18:10 +08:00
xige-16
87c5a49d9a
Refactor showCollection/showPartition to get load state in memory (#6935)
* refactor showCollection/showPartition to get load state in memory

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* update proto

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* update proto

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* update milvus.proto

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* add show result log to debug

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* fix wrong err message

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* update pymilvus-orm version

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* fix repeated load error

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* update proto

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* fix status nil error

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-08-02 22:39:25 +08:00
Cai Yudong
fe50f97726
Use id to tell search result validation in proxy reduce stage (#6905)
* use id to tell search result validation

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* enable test_search_binary_hamming_flat_index

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code optimize

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix merge retrieve result issue

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-08-02 10:25:49 +08:00
cai.zhang
d51fec6678
Enhance log when create index failed (#6871)
* Enhance log when create index failed

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Fix bug for repeatedly drop index

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>
2021-07-29 14:47:22 +08:00
zhenshan.cao
8a1a841011
FixBug: Incorrect handling of time synchronization (#6814)
* FixBug: proxy send wrong sync timetick and DataCoord did not filter channel

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-07-27 14:34:48 +08:00
dragondriver
3cc8ee298e
Filter NaN when reducing search results (#6806)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-07-27 10:19:21 +08:00
zhenshan.cao
2962d31a5f
Fixbug: segment allocate but return value incorrect (#6788)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-07-24 20:49:20 +08:00
dragondriver
cf8e9d52b1
Reduce the log of proxy (#6766)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-07-23 20:33:34 +08:00
dragondriver
67c1fdd2c0
Show/Describe collections/partitions with created timestamp (#6700)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-07-21 18:00:14 +08:00
Cai Yudong
cbb4052fdb
Support search and query output fields using wildcard (#6671)
* update wildcard polocy for search and query

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* mark xfail cases

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* mark xfail cases

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize debug log

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix static-check

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-07-21 14:52:12 +08:00
dragondriver
2d9361e0ba
Fix bug: remove duplicated primary field id when retrieve entities (#6577)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-07-17 10:53:30 +08:00
Cai Yudong
a992dcf6a8
Support query return vector output field (#6570)
* improve code readibility

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add offset in RetrieveResults

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add VectorFieldInfo into Segment struct

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add new interface for query vector

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update load vector field logic

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update load vector field logic

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fill in field name in query result

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add FieldId into FieldData

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add fillVectorOutputFieldsIfNeeded

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update data_codec_test.go

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add DeserializeFieldData

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* realize query return vector output field

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix static-check

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* disable query vector case

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-07-16 17:19:55 +08:00
dragondriver
5016e509be
Support wildcard match on search/query output fields (#6510)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-07-14 18:51:54 +08:00
Cai Yudong
1c29b42434
change output_fields to out_fields_id for RetrieveRequest (#6506)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-07-14 13:11:54 +08:00
dragondriver
d896e3119e
Fix 6419, check if num_rows is greater than zero in proxy (#6439)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-07-13 16:33:55 +08:00
dragondriver
bd317a5461
Check if field data is empty (#6404)
* Check if field data is empty

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* FixBug: fast fail when output_fields contain fields that are not in schema

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Co-authored-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-07-11 19:39:52 +08:00
neza2017
8b89744c6b
search with exp (#6409)
* process float vector and binary vector field
on the search result

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* add more debug info on etcd

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* add debug info when grpc failed

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-07-10 10:21:52 +08:00
dragondriver
166548447b
Fix 6249, use more clear error message (#6316)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-07-06 14:40:04 +08:00
zhenshan.cao
ff4c6c7ff4
Fix travel timestamp and guarantee timestamp (#6234)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-06-30 21:02:13 +08:00
yukun
95aa3c85b6
Add index_name check for drop_index (#6217)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-06-30 17:56:12 +08:00
yukun
b87baa108a
Add topks in searchResult (#6168)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-06-28 15:28:11 +08:00
dragondriver
d3c503f3aa
Calculate the real topk in proxy (#6132)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-06-28 14:30:14 +08:00
yukun
ae66446f16
Fix issue-5992 (#6158)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-06-28 12:16:13 +08:00
yukun
effcbc7cb3
Fix issue-6143 (#6145)
* Fix issue-6143

Signed-off-by: kun yu <kun.yu@zilliz.com>

* Return error when output_fields contains vector field

Signed-off-by: kun yu <kun.yu@zilliz.com>

* Fix when output_fields contain primarykey

Signed-off-by: kun yu <kun.yu@zilliz.com>

* Add primarykey

Signed-off-by: kun yu <kun.yu@zilliz.com>
2021-06-28 10:12:13 +08:00
xige-16
ca4cba152c
return loaded collection when ShowCollectionsType_InMemory (#6149)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-06-27 12:10:08 +08:00
yukun
eb818d05ab
Remove vector fields from output fields (#6112)
* Remove vector fields from output fields

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add empty_collection error_code

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-06-25 16:52:11 +08:00
yukun
9b549e0fa4
Add primaryKey to output fields (#6094)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-06-24 21:08:14 +08:00
sunby
46f86a0354
Return sealed segments when calling flush (#6011)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-23 16:56:11 +08:00
dragondriver
a493151f70
Fix #5951 (#6010)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-06-23 12:06:10 +08:00
yukun
792dba4ae3
Fix id field insert failed when auto_id is true (#5984)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-06-22 18:46:04 +08:00
zhenshan.cao
0f05622c09
Modify error msg when index is not created (#5978)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-06-22 18:04:07 +08:00
xige-16
2af1c27811
Rename queryservice to querycoordinator (#5931)
* Rename queryservice to querycoordinator

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* fix param table

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* code format

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* fix unittest

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* service to coordinator

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* code format

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* set querycoord in ci

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-06-22 16:44:09 +08:00
dragondriver
fd57554d32
Add index params check in proxy (#5958)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-06-22 15:28:04 +08:00
Cai Yudong
2926a78968
Rename proxynode to proxy (#5961)
* rename package name from proxynode to proxy

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move distributed/proxynode to distributed/proxy

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move internal/proxynode to internal/proxy

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename proxynode to proxy

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-22 14:40:07 +08:00
quicksilver
7fe061b846 Raneme init_devcontainer.sh to devcontainer.sh
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-01-15 14:38:36 +08:00
neza2017
56ab2bda2d Update proto:add partition id
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-15 11:53:22 +08:00
zhenshan.cao
63b21321d6 Refactor indexservice and update doc
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-01-15 13:50:27 +08:00
cai.zhang
3d885742ee Reopen ruleguard and format code
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-01-13 13:36:27 +08:00
bigsheeper
2d2af1505a Improve mutex usage in query node
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-12 18:03:24 +08:00
GuoRentong
f3d2afa4f4 Update doc:detailed interface definition
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
2021-01-11 18:35:54 +08:00
quicksilver
a6690dbc99 Fix syntax error: Bad fd number
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-01-11 15:25:19 +08:00
yukun
18cfd31d40 Fix Load and Remove with prefix
Signed-off-by: yukun <kun.yu@zilliz.com>
2021-01-11 15:17:06 +08:00
bigsheeper
4fad3b189c Fix search without insertion, improve nil hits behavior (#622)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-09 17:58:18 +08:00
xige-16
f4566731fc Fix search error in regression test
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-01-09 09:47:22 +08:00
sunby
5dfe9448ae Modify test_index
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-01-08 16:00:16 +08:00
GuoRentong
82ad488555 Update doc: rocksmq
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
2021-01-08 18:37:24 +08:00
dragondriver
018466a256 Avoid crash when index parameters are invalid
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-01-08 17:45:57 +08:00
XuanYang-cn
65089ea362 Fix flush sync test bug and remove unecessary lock in meta_table
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-08 17:20:59 +08:00