Cai Yudong
b0054f1b48
Update C API NewSegment ( #11809 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-11-15 16:57:09 +08:00
Cai Yudong
34ae61d8e2
Move HandleCProto into query node ( #11763 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-11-15 11:05:09 +08:00
Cai Yudong
5fdc6626cb
Change API retrieve return type from CProtoResult to CProto ( #11555 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-11-12 10:04:49 +08:00
Cai Yudong
ca356d2811
Use API HandleCStatus to handle CStatus returned by CGo in query node ( #11559 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-11-11 10:18:42 +08:00
xige-16
b6c7bc19d1
[skip ci] Add comments for VectorFieldInfo ( #11595 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-11-10 23:48:06 +08:00
Cai Yudong
b5bc9b26d7
Rename getEntityByIds to retrieve ( #11548 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-11-10 17:15:37 +08:00
zhenshan.cao
34c4ef06fc
[skip ci]Fix golint error in segment.go ( #11203 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-11-03 23:50:14 +08:00
godchen
0d97549d84
Remove useless log ( #11105 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-02 19:20:29 +08:00
godchen
a0a3a889e7
Add common endian for global use ( #11092 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-02 18:16:32 +08:00
yukun
f7c0f5b3da
Add segmentLoadDeletedRecord in querynode ( #10449 )
...
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-10-22 18:51:14 +08:00
yukun
fe1927c05d
Support delete in SegmentSealed ( #10181 )
...
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-10-22 13:11:11 +08:00
godchen
8daeb0d519
Change bloom filter use pk ( #10193 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-19 20:18:47 +08:00
godchen
24a6fc096b
Update query node bloom filter ( #10109 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-18 20:08:42 +08:00
yukun
997e88d8c6
Add deleteMessage in flow_graph_insert_node ( #9505 )
...
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-10-15 11:02:32 +08:00
elfisworking
d4232f88a2
For search funtion adding a round_decimal paramter to precision control ( #8574 )
...
Signed-off-by: elfisworking <zymustb@126.com>
2021-10-08 17:39:55 +08:00
bigsheeper
691e4c5bc9
Use definitional type instead of raw type ( #7797 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-09-14 10:25:26 +08:00
Cai Yudong
27dcf698d3
Support set segcore chunk_size via config file ( #7635 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-09-11 14:40:01 +08:00
bigsheeper
1a5e5989c8
Add segment unittests for query node ( #7632 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-09-09 17:36:02 +08:00
bigsheeper
bcccd767ac
Add unittests for querynode ( #7510 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-09-07 15:45:59 +08:00
yukun
94272bba87
Support query by expression ( #7386 )
...
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-09-03 17:12:55 +08:00
godchen
c333af0dcf
Add query node segment filter ( #7303 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-08-28 10:12:00 +08:00
godchen
8cc24e09c9
Make users get vectors without using local storage ( #7134 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-08-18 16:30:11 +08:00
Cai Yudong
e771bda92f
optimize retrieve output vector code structure ( #7102 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-08-14 11:18:10 +08:00
Cai Yudong
6c75301c70
optimize search reduce logic ( #7066 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-08-12 18:00:11 +08:00
bigsheeper
b9d3897deb
Return error when any failed detected in loadSegment ( #7005 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-08-12 14:14:08 +08:00
bigsheeper
e8ea8b51d5
improve query node log ( #6897 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-07-31 10:47:22 +08:00
bigsheeper
eee0f1e077
Add mutex for segment of segCore in querynode ( #6870 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-07-29 16:03:22 +08:00
godchen
db94d7771f
Read vector from disk ( #6707 )
...
* Read vector from disk
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* go fmt
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* fix git action error
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* fix error
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* fix test error
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* fix action error
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* fix caculate error
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* change var name
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* remove unused method
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* remove unused method
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* fix error
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* fix len error
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* remove unused code
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* change bytes to float method
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* change float to bytes method
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* fix action error
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-07-24 09:25:22 +08:00
Cai Yudong
a7b27db63e
update fillRetrieveResults parameter ( #6598 )
...
* update fillRetrieveResults parameter
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* optimize fillVectorFieldData process
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-07-17 15:17:30 +08:00
Cai Yudong
a992dcf6a8
Support query return vector output field ( #6570 )
...
* improve code readibility
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add offset in RetrieveResults
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add VectorFieldInfo into Segment struct
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add new interface for query vector
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update load vector field logic
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update load vector field logic
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fill in field name in query result
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add FieldId into FieldData
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add fillVectorOutputFieldsIfNeeded
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update data_codec_test.go
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add DeserializeFieldData
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* realize query return vector output field
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix static-check
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* disable query vector case
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-07-16 17:19:55 +08:00
Cai Yudong
724f10b9a0
Unify the usage of query and search ( #6467 )
...
Unify the usage of query and search
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-07-13 22:20:33 +08:00
FluorineDog
bf8b2be4a7
Deprecate num_groups to simplify search API ( #6230 )
...
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-07-01 10:32:15 +08:00
Cai Yudong
a7a9eeeffe
Add time record for build index and search process ( #6231 )
...
* add timerecord for build index process
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix build error
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add timerecord for query process
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-30 19:46:14 +08:00
bigsheeper
58bf92a407
Add vChannel,pChannel in query node and get vChannel from master in query service ( #5784 )
...
* add vChannel,pChannel in query node and get vChannel from master in query service
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-06-15 20:06:10 +08:00
FluorineDog
bec9f2c182
Split segcore and plan proto for future feature ( #5767 )
...
* Split segcore and plan proto for future feature
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
* lint
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-06-15 14:43:57 +08:00
bigsheeper
cdbc6d2c94
Refactor query node and query service ( #5751 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
Co-authored-by: xige-16 <xi.ge@zilliz.com>
Co-authored-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-15 12:41:40 +08:00
bigsheeper
8aae0f7cc9
Refactor flow graph and load/watchDML in query node ( #5682 )
...
* Refactor flow graph and load/watchDML in query node
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-06-09 11:37:55 +08:00
zhenshan.cao
7dac20c35c
Add log for poxynode transform data and querynode segment insert ( #5627 )
...
* Add log for poxynode transform data and querynode segment insert
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
* Decrease print times
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-06-05 16:21:36 +08:00
FluorineDog
9a90313390
Support GetEntityByIDs in CGo, fix segcore bugs ( #5563 )
...
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-06-04 10:38:34 +08:00
Xiangyu Wang
82ccd4cec0
Rename module ( #4988 )
...
* Rename module
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-04-22 14:45:57 +08:00
bigsheeper
71b8f309b2
Add licence to query node and query service
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-04-19 13:47:10 +08:00
xige-16
332b14c915
Delete querynode’s redundant load IO
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-04-07 18:29:19 +08:00
xige-16
c10b36f188
fix getting wrong offset when segment preinsert
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-03-31 16:16:58 +08:00
xige-16
a6f1de036b
Optimize search performance in query node
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-03-30 22:16:58 +08:00
bigsheeper
dfb68a7e46
Remove unused segment
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-03-25 16:08:08 -05:00
zhenshan.cao
c2734fa55f
Fix bug and enchance system
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-03-22 16:36:10 +08:00
xige-16
8a5c039137
Fix search error when running single node
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-03-12 19:23:06 +08:00
xige-16
09ae985daa
Fix wrong error code in master_service_test.go
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-03-10 22:06:22 +08:00
XuanYang-cn
8567679888
Check enableIndex before load index
...
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-03-10 14:51:00 +08:00
quicksilver
455cc59dbb
Update publish-test-images.yaml
...
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-03-09 13:55:35 +08:00
sunby
ddddd65d10
Delete internal/errors package
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-03-05 10:15:27 +08:00
bigsheeper
447a15207e
Add zap log to query node
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-03-05 09:21:35 +08:00
xige-16
f62078c027
Fix crash error when search
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-02-20 10:14:03 +08:00
xige-16
a6442cb600
Fix querynode memory leak
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-02-20 09:20:51 +08:00
sunby
5769c000a1
Find failed python tests and add skip mark "r0.3-test"
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-02-19 15:37:04 +08:00
bigsheeper
d8f41db823
Fix search crash error
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-02-19 13:56:01 +08:00
bigsheeper
12b2eaf196
Rename apis
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-02-07 21:26:03 +08:00
zhenshan.cao
1ba8e2448f
Add tsLoop for indexservice
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-02-07 18:38:58 +08:00
bigsheeper
a64c831ed9
Fix segment replacement bug
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-02-07 18:37:18 +08:00
bigsheeper
c6c99ef32b
Implement getSegmengInfo function
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-02-04 11:40:14 +08:00
bigsheeper
f12a0490bb
Do not drop collections or partitions in flow graph
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-02-03 18:12:48 +08:00
Xiangyu Wang
fd9e42d3ed
Update msgstream/unmarshal interface
...
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-02-03 17:30:10 +08:00
bigsheeper
abffdbff62
Remove partition tag
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-02-03 16:44:38 +08:00
godchen
66d5189601
Add get collection statistics implemetation
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-02-02 19:54:31 +08:00
dragondriver
e4e3ec88d1
Add usage count of insert channels
...
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-02-02 15:49:13 +08:00
bigsheeper
09bdbee944
Refactor load service, remove seg manager
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-02-02 15:13:08 +08:00
bigsheeper
3573e01093
Check index periodically and load index, add indexing segment type
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-30 16:02:10 +08:00
bigsheeper
73d2b6a101
Get index param from minio and filter by vector fields
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-29 15:22:24 +08:00
cai.zhang
83c60fd84f
Update proto about index
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-01-21 15:20:23 +08:00
zhenshan.cao
2ec573d2f9
Recover delete docs
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-01-20 09:36:50 +08:00
cai.zhang
fb493cc235
Update interface name of GetIndexStates
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-01-19 18:32:57 +08:00
neza2017
918e279869
Set msg stream
...
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-19 16:14:13 +08:00
neza2017
d5e53aa28a
Add Meta table implementation
...
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-19 14:13:49 +08:00
bigsheeper
2064b014a1
Add loadFieldData, use partition id instead of tag
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-19 14:44:03 +08:00
XuanYang-cn
6a13386393
Add DataNode package
...
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-19 11:37:16 +08:00
neza2017
1ce32d8775
Add master service
...
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-19 09:45:42 +08:00
bigsheeper
aaf8b0ad40
Add sealed segment, segment type and manager, send loadIndexReq to service's buffer
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-19 10:29:16 +08:00
bigsheeper
f02bd8c8f5
Rename query node package, implement watchDmChannel
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-16 10:12:14 +08:00
bigsheeper
b98b226d8c
Refactor query node and implement channelRemove, channelAdd
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-15 15:28:54 +08:00
quicksilver
8e9d8e36e1
Update run_go_unittest.sh
...
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-01-13 10:40:46 +08:00
FluorineDog
4cd42c553f
Rename field_name, make field_id strongly typed, skip multithread test
...
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2021-01-12 18:31:52 +08:00
bigsheeper
2d2af1505a
Improve mutex usage in query node
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-12 18:03:24 +08:00
GuoRentong
f3d2afa4f4
Update doc:detailed interface definition
...
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
2021-01-11 18:35:54 +08:00
quicksilver
a6690dbc99
Fix syntax error: Bad fd number
...
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-01-11 15:25:19 +08:00
yukun
18cfd31d40
Fix Load and Remove with prefix
...
Signed-off-by: yukun <kun.yu@zilliz.com>
2021-01-11 15:17:06 +08:00
xige-16
004d0027b3
Fix error that different loadIndex messages have same indexParam
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-01-09 15:57:55 +08:00
zhenshan.cao
02df310fa5
Fix data race
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-01-09 12:30:22 +08:00
XuanYang-cn
c0797333ce
Add ddl binlog metaTable in writenode
...
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-24 20:55:40 +08:00
neza2017
4dbf4ade41
Add test for master
...
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-24 16:19:42 +08:00
xige-16
f5198bcaee
Add LoadIndexService in query node
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-12-24 14:51:55 +08:00
godchen
7ab5b5d80d
Add MinIO kv implements
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2020-12-03 19:00:11 +08:00
neza2017
cec903da19
Add parquet payload
...
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-02 18:31:56 +08:00
xige-16
36cf8a8ea7
Support return primary key when doing search
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-12-03 17:18:06 +08:00
neza2017
32445dd02b
Add cpu profile
...
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-26 16:01:31 +08:00
cai.zhang
57da66ba72
Update read config logic in docker
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-11-26 15:18:36 +08:00
zhenshan.cao
045fc3d808
Fix segment fault
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-11-26 14:29:01 +08:00
XuanYang-cn
0a16a9a651
Add logic of getting milvus distributed config from master by grpc
...
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-11-26 13:21:06 +08:00
dragondriver
8ce0deca4e
Add default partition tag to InsertTask in Proxy
...
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-11-26 11:38:04 +08:00
xige-16
ae78aa1ac1
Warp cpp reduce and repack searchResultMsg to specified pulsar channel
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-11-26 11:12:33 +08:00
sunby
f4c643f1bd
Add number limitation of partition and field. And Change default partition tag to "_default"
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-11-25 18:39:05 +08:00