Commit Graph

187 Commits

Author SHA1 Message Date
zhenshan.cao
bbfcbbdd68 Fix ci: paramstable and startup logic refactor
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-02-23 11:40:30 +08:00
xige-16
450ea631bc Fix queryService assign queryChannel failure
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-02-18 16:26:02 +08:00
xige-16
70fe5233cf Watch queryChannel on query node
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-02-09 17:09:26 +08:00
groot
833f1d49ab Let singlenode use rocksmq
Signed-off-by: groot <yihua.mo@zilliz.com>
2021-02-09 15:57:10 +08:00
groot
dc3736281a Integrate message stream
Signed-off-by: groot <yihua.mo@zilliz.com>
2021-02-08 14:30:54 +08:00
bigsheeper
b07b2484dc Fix error when loading
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-02-07 10:29:58 +08:00
xige-16
cdcf006690 Fix loadCollection error when query service assign insert channels and query nodes
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-02-07 09:30:48 +08:00
bigsheeper
e9ee9a273e Refactor load service and check insertion binLog periodically
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-02-06 11:35:35 +08:00
sunby
0111cba8ae Fix startup bugs in some modules
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-02-06 10:28:53 +08:00
bigsheeper
1578c13224 Refactor collection replica
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-02-05 10:53:11 +08:00
godchen
c2ca2c276f Fix response check error
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-02-04 19:34:35 +08:00
dragondriver
e0e8e1605e Implement the release-related task and load-related task in Proxy
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-02-04 17:47:19 +08:00
sunby
7270d18c98 Fix timesync startup bugs
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-02-04 17:31:02 +08:00
xige-16
aaac839075 Joint debug query service success with other components
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-02-04 17:09:18 +08:00
zhenshan.cao
b89e5a3240 Add GetQuerySegmentInfo
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-02-04 14:37:12 +08:00
sunby
71b6c88c6e Defer lock's unlock in meta::DropSegment
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-02-04 11:52:10 +08:00
Xiangyu Wang
8d35e76ad1 Change Msgstream interface
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-02-04 13:55:47 +08:00
bigsheeper
c6c99ef32b Implement getSegmengInfo function
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-02-04 11:40:14 +08:00
XuanYang-cn
1aafe86ffe Change MsgPosition Logic
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-02-04 11:19:48 +08:00
zhenshan.cao
1b616fb874 Add persistent_segmentinfo
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-02-03 18:55:00 +08:00
Xiangyu Wang
fd9e42d3ed Update msgstream/unmarshal interface
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-02-03 17:30:10 +08:00
godchen
66d5189601 Add get collection statistics implemetation
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-02-02 19:54:31 +08:00
dragondriver
e4e3ec88d1 Add usage count of insert channels
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-02-02 15:49:13 +08:00
bigsheeper
09bdbee944 Refactor load service, remove seg manager
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-02-02 15:13:08 +08:00
xige-16
4cd315408b Refactor query service’s meta interface
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-02-02 11:52:41 +08:00
bigsheeper
3573e01093 Check index periodically and load index, add indexing segment type
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-30 16:02:10 +08:00
bigsheeper
68cd15af63 Get index info from master
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-27 14:41:56 +08:00
bigsheeper
4f914a2c30 Add docker file and main function of query node
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-27 09:50:52 +08:00
dragondriver
c6a6b1436c Add soft time tick in proxy service
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-01-26 13:41:41 +08:00
cai.zhang
f940cc455a Add dockerfile for index
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-01-26 09:38:40 +08:00
FluorineDog
a8fd6e7686 Enable drop index/field_data and extract info visitor
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2021-01-25 18:59:44 +08:00
bigsheeper
7554246ace Add segment seeking and use real client
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-25 18:33:10 +08:00
bigsheeper
445675a588 Refactor loadSegment request and load strategy
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-24 18:02:08 +08:00
XuanYang-cn
b794921b7e Add enum Role in internal.proto
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-23 18:56:08 +08:00
xige-16
511aa419fa Refactor query service
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-01-22 14:28:06 +08:00
bigsheeper
46a0a6133a Implement distributed api, delete partition states api
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-22 11:17:18 +08:00
neza2017
9828a52999 Add create index
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-21 10:01:29 +08:00
FluorineDog
7f044fff82 Enable search for sealed
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2021-01-21 15:29:52 +08:00
sunby
be6775671b Add error returned by Init,Start,Stop functions
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-01-19 17:38:01 +08:00
cai.zhang
83c60fd84f Update proto about index
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-01-21 15:20:23 +08:00
Xiangyu Wang
7ee8623e1a Refactor msgstream
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-01-20 10:02:59 +08:00
zhenshan.cao
2ec573d2f9 Recover delete docs
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-01-20 09:36:50 +08:00
cai.zhang
fb493cc235 Update interface name of GetIndexStates
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-01-19 18:32:57 +08:00
neza2017
918e279869 Set msg stream
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-19 16:14:13 +08:00
neza2017
d5e53aa28a Add Meta table implementation
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-19 14:13:49 +08:00
bigsheeper
2064b014a1 Add loadFieldData, use partition id instead of tag
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-19 14:44:03 +08:00
XuanYang-cn
6a13386393 Add DataNode package
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-19 11:37:16 +08:00
neza2017
1ce32d8775 Add master service
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-19 09:45:42 +08:00
bigsheeper
aaf8b0ad40 Add sealed segment, segment type and manager, send loadIndexReq to service's buffer
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-19 10:29:16 +08:00
GuoRentong
dd6c4bfb75 Update doc:interface definitions related to LoadCollection
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
2021-01-18 10:09:17 +08:00
XuanYang-cn
37688559f1 Fix multi rowdata writenode crash bug
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-16 15:06:19 +08:00
bigsheeper
46a14da3eb Implement segment manager and loadSegment
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-16 15:04:32 +08:00
bigsheeper
f02bd8c8f5 Rename query node package, implement watchDmChannel
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-16 10:12:14 +08:00
sunby
67e6ea8051 Add data service
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-01-15 17:09:41 +08:00
bigsheeper
b98b226d8c Refactor query node and implement channelRemove, channelAdd
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-15 15:28:54 +08:00
godchen
0971cf65b2 Remove opentracing stdlog
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-01-15 15:28:27 +08:00
bigsheeper
2d2af1505a Improve mutex usage in query node
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-12 18:03:24 +08:00
GuoRentong
f3d2afa4f4 Update doc:detailed interface definition
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
2021-01-11 18:35:54 +08:00
quicksilver
a6690dbc99 Fix syntax error: Bad fd number
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-01-11 15:25:19 +08:00
yukun
18cfd31d40 Fix Load and Remove with prefix
Signed-off-by: yukun <kun.yu@zilliz.com>
2021-01-11 15:17:06 +08:00
sunby
5dfe9448ae Modify test_index
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-01-08 16:00:16 +08:00
GuoRentong
82ad488555 Update doc: rocksmq
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
2021-01-08 18:37:24 +08:00
XuanYang-cn
65089ea362 Fix flush sync test bug and remove unecessary lock in meta_table
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-08 17:20:59 +08:00
GuoRentong
c4599e26de Update doc: node api
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
2021-01-08 16:59:19 +08:00
bigsheeper
e13822043d Fix search test when partition not exists, add binary vector search validation
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-08 16:41:27 +08:00
godchen
fa73f17f94 Add opentracing doc
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-01-08 15:32:38 +08:00
zhenshan.cao
6a3ec57fef Fix docker lib dependency
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-01-07 21:08:23 +08:00
dragondriver
a4860beae2 Parse type params and index params when buildindex is invoked
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-01-07 17:38:46 +08:00
xige-16
d9d2f33a23 Add binary test for loadIndex and fix loadIndexService can't close correctly
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-01-07 17:22:10 +08:00
dragondriver
ced56c84f8 Add support to build binary vector index
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-01-07 14:56:17 +08:00
godchen
8f1f118b26 Add query opentracing
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-01-07 14:24:35 +08:00
XuanYang-cn
c0797333ce Add ddl binlog metaTable in writenode
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-24 20:55:40 +08:00
neza2017
4dbf4ade41 Add test for master
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-24 16:19:42 +08:00
xige-16
f5198bcaee Add LoadIndexService in query node
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-12-24 14:51:55 +08:00
bigsheeper
762b1e1f1f Add ddService, fieldID, requestID and refactor params table, fix nil hits search result bug
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-12-10 16:31:09 +08:00
XuanYang-cn
32977e270c Add impl cgo of parquet
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-08 14:41:04 +08:00
zhenshan.cao
7bbbc14637 Fix bug: address already used
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-07 15:22:20 +08:00
xige-16
37d7526d31 Fix search test failure
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-12-07 14:37:42 +08:00
neza2017
aeca8f85a5 Run cwrapper unittest in ci
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-05 17:39:58 +08:00
cai.zhang
806a97505c Add query node for CI progress
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-12-05 16:12:45 +08:00
neza2017
70710dee47 Add parquet payload
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-05 16:11:03 +08:00
zhenshan.cao
0b6fe0a8cf Refactor paramtable
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-04 14:18:17 +08:00
neza2017
32445dd02b Add cpu profile
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-26 16:01:31 +08:00
cai.zhang
57da66ba72 Update read config logic in docker
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-11-26 15:18:36 +08:00
zhenshan.cao
045fc3d808 Fix segment fault
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-11-26 14:29:01 +08:00
XuanYang-cn
0a16a9a651 Add logic of getting milvus distributed config from master by grpc
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-11-26 13:21:06 +08:00
sunby
f4c643f1bd Add number limitation of partition and field. And Change default partition tag to "_default"
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-11-25 18:39:05 +08:00