Commit Graph

58 Commits

Author SHA1 Message Date
zhenshan.cao
f95ef2260b
Add GetFlushState grpc method (#12220)
Update sdk version

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-11-23 22:57:17 +08:00
Xiaofan
f654621650
Remove the etcd deletion of segment data path on querynode (#11863)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-11-17 12:13:10 +08:00
zhenshan.cao
8c6c031e80
FixBug:update serviceableTime incorrectly (#11769)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-11-15 10:31:09 +08:00
godchen
5357e301f4
Fix retrieve search error (#11501)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-11 00:54:45 +08:00
godchen
639800241d
Make flow graph for delta channel (#11379)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-09 09:27:04 +08:00
Cai Yudong
021f4c0f8a
Update query reduce log (#11401)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-11-08 16:05:51 +08:00
yukun
4e5c8811da
Add flow_graph for delta channel in querynode (#11293)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-11-06 11:02:58 +08:00
Cai Yudong
0fac38000c
Correct empty retrieve result handling (#11219)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-11-04 18:08:32 +08:00
godchen
a0a3a889e7
Add common endian for global use (#11092)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-02 18:16:32 +08:00
Cai Yudong
7763615780
Remove primary key duplicated query result on query node (#10834)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-10-30 15:12:41 +08:00
bigsheeper
6053d693df
Add query mutex to fix crashed with panic (#10832)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-10-29 17:12:43 +08:00
zhenshan.cao
c0ea7ee780
Remove unused code in querynode (#10859)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-10-28 23:58:52 +08:00
congqixia
5317c8abab
Fix waitNewTSafe not waiting (#10724)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-27 14:22:22 +08:00
xige-16
0cedd9d47e
Compaction multiSegmentChangeInfo to a single info (#10587)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-10-26 15:18:22 +08:00
xige-16
a414d16c06
Fix remove segment when querynode receive segmentChangeInfo (#10619)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-10-26 13:04:18 +08:00
xige-16
8a046a6495
BroadCast sealedSegmentChangeInfo to queryChannel (#10084)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-10-22 19:07:15 +08:00
bigsheeper
132f57f5b7
Set gracefulTime to 0 (#10114)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-10-21 10:55:12 +08:00
godchen
24a6fc096b
Update query node bloom filter (#10109)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-18 20:08:42 +08:00
bigsheeper
4c420dac21
Update excluded segments info when receive changeInfo (#10011)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-10-16 17:00:33 +08:00
bigsheeper
d567b21fd4
Add globalSegmentManager and consume changeInfoMsg (#9575)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-10-11 15:40:38 +08:00
congqixia
6f7f71fdc0
Add lock before reading tSafeWatcher map (#8562)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-26 12:37:56 +08:00
bigsheeper
52126f2d5a
Dynamic add tSafe watcher, use sync.Cond instead of selectCase, add ref count (#8050)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-09-24 13:57:54 +08:00
congqixia
bdfb8768d4
Enable gosec (#8202)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-18 14:45:50 +08:00
sunby
db01040b53
Fix query request lost bug (#7715)
Issue: #7633
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-09-10 16:34:44 +08:00
bigsheeper
bcccd767ac
Add unittests for querynode (#7510)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-09-07 15:45:59 +08:00
yukun
94272bba87
Support query by expression (#7386)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-09-03 17:12:55 +08:00
godchen
c333af0dcf
Add query node segment filter (#7303)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-08-28 10:12:00 +08:00
godchen
2dc2cb1a28
Fix collection nil when add query collection (#7311)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-08-27 17:51:56 +08:00
bigsheeper
1e60cccaf9
Set inMemoryPercentage to 0 when releasing partition (#7310)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-08-27 16:51:11 +08:00
Cai Yudong
0ab70271a6
correct empty search result handle (#7244)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-08-24 11:13:52 +08:00
godchen
8cc24e09c9
Make users get vectors without using local storage (#7134)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-08-18 16:30:11 +08:00
xige-16
b035ec5c7b
Enable search when querynode only load segment (#7104)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-08-16 12:24:09 +08:00
Cai Yudong
e771bda92f
optimize retrieve output vector code structure (#7102)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-08-14 11:18:10 +08:00
Cai Yudong
6c75301c70
optimize search reduce logic (#7066)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-08-12 18:00:11 +08:00
godchen
7fc7623b92
Add vector data memroy existence check (#7044)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-08-11 19:50:09 +08:00
Cai Yudong
fe50f97726
Use id to tell search result validation in proxy reduce stage (#6905)
* use id to tell search result validation

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* enable test_search_binary_hamming_flat_index

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code optimize

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix merge retrieve result issue

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-08-02 10:25:49 +08:00
bigsheeper
e8ea8b51d5
improve query node log (#6897)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-07-31 10:47:22 +08:00
godchen
db94d7771f
Read vector from disk (#6707)
* Read vector from disk

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* go fmt

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix git action error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix test error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix action error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix caculate error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* change var name

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* remove unused method

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* remove unused method

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix len error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* remove unused code

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* change bytes to float method

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* change float to bytes method

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix action error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-07-24 09:25:22 +08:00
bigsheeper
21bc5810c0
remove redundant log of query node (#6768)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-07-23 20:09:33 +08:00
Cai Yudong
cbb4052fdb
Support search and query output fields using wildcard (#6671)
* update wildcard polocy for search and query

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* mark xfail cases

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* mark xfail cases

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize debug log

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix static-check

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-07-21 14:52:12 +08:00
Cai Yudong
a7b27db63e
update fillRetrieveResults parameter (#6598)
* update fillRetrieveResults parameter

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize fillVectorFieldData process

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-07-17 15:17:30 +08:00
Cai Yudong
a992dcf6a8
Support query return vector output field (#6570)
* improve code readibility

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add offset in RetrieveResults

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add VectorFieldInfo into Segment struct

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add new interface for query vector

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update load vector field logic

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update load vector field logic

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fill in field name in query result

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add FieldId into FieldData

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add fillVectorOutputFieldsIfNeeded

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update data_codec_test.go

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add DeserializeFieldData

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* realize query return vector output field

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix static-check

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* disable query vector case

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-07-16 17:19:55 +08:00
Cai Yudong
1c29b42434
change output_fields to out_fields_id for RetrieveRequest (#6506)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-07-14 13:11:54 +08:00
bigsheeper
e50d39dfe1
Query result get global sealed segments from ETCD (#6377)
* get global sealed segments from ETCD

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* fix race error

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-07-14 11:15:54 +08:00
Cai Yudong
724f10b9a0
Unify the usage of query and search (#6467)
Unify the usage of query and search

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-07-13 22:20:33 +08:00
dragondriver
bd317a5461
Check if field data is empty (#6404)
* Check if field data is empty

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* FixBug: fast fail when output_fields contain fields that are not in schema

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Co-authored-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-07-11 19:39:52 +08:00
neza2017
8b89744c6b
search with exp (#6409)
* process float vector and binary vector field
on the search result

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* add more debug info on etcd

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* add debug info when grpc failed

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-07-10 10:21:52 +08:00
zhenshan.cao
ff4c6c7ff4
Fix travel timestamp and guarantee timestamp (#6234)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-06-30 21:02:13 +08:00
Cai Yudong
a7a9eeeffe
Add time record for build index and search process (#6231)
* add timerecord for build index process

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix build error

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add timerecord for query process

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-30 19:46:14 +08:00
xige-16
ff93d1611f
merge retrive and search code in query node (#6227)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-06-30 17:50:15 +08:00