Commit Graph

195 Commits

Author SHA1 Message Date
neza2017
3a866dab3d Update proto
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-28 17:25:43 +08:00
bigsheeper
68cd15af63 Get index info from master
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-27 14:41:56 +08:00
sunby
dbfe68512d Add dbID to data service proto
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-01-27 10:33:17 +08:00
neza2017
d1b23cb8c5 Add debug info
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-26 19:24:09 +08:00
sunby
f632ed0272 Fix data service init
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-01-26 18:45:12 +08:00
zhenshan.cao
e83ac41bca Add component for index
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-01-26 18:01:32 +08:00
xige-16
7c210310dc Add load function for query service
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-01-26 15:13:20 +08:00
dragondriver
d972b75aaa Add GetTimeTickChannel, GetComponentStates to proxy service
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-01-26 14:55:57 +08:00
cai.zhang
f940cc455a Add dockerfile for index
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-01-26 09:38:40 +08:00
neza2017
1fe099775d Refactor data service
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-25 15:17:17 +08:00
neza2017
e89e89b894 Add main for masterservice
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-25 12:03:44 +08:00
sunby
54fcd486bb Refactor data service.
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-01-25 13:45:56 +08:00
XuanYang-cn
5d9d35b733 Add Supplementary statistics
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-25 13:36:40 +08:00
XuanYang-cn
9fe672ab86 Enchance datanode interface
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-24 21:20:11 +08:00
neza2017
5aec8bc5e6 Refactor masterservice
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-24 20:26:35 +08:00
bigsheeper
445675a588 Refactor loadSegment request and load strategy
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-24 18:02:08 +08:00
XuanYang-cn
b794921b7e Add enum Role in internal.proto
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-23 18:56:08 +08:00
XuanYang-cn
50e2369000 Change name and tag to ID
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-22 19:36:09 +08:00
bigsheeper
067c30c422 Add sealedSegment cgo unittest, fix growingSegment field id check
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-22 15:41:54 +08:00
xige-16
511aa419fa Refactor query service
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-01-22 14:28:06 +08:00
neza2017
2d2dbd7d19 Add logic of segment flush complete
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-22 11:06:12 +08:00
dragondriver
cd52adc18b Add proxy service to ci workflow
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-01-22 12:57:23 +08:00
neza2017
90a0b8b2f5 Update proto
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-22 11:44:02 +08:00
bigsheeper
46a0a6133a Implement distributed api, delete partition states api
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-22 11:17:18 +08:00
sunby
ef79b8d8c7 Add client and grpc service
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-01-22 11:07:07 +08:00
XuanYang-cn
377ee8687d Modify flush logic
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-22 09:36:40 +08:00
dragondriver
eba6bb288c Refactor the sructure of proto file
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-01-22 09:36:18 +08:00
neza2017
9828a52999 Add create index
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-21 10:01:29 +08:00
sunby
be6775671b Add error returned by Init,Start,Stop functions
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-01-19 17:38:01 +08:00
cai.zhang
83c60fd84f Update proto about index
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-01-21 15:20:23 +08:00
XuanYang-cn
ec27788628 Add segment statistics updates logic for datanode
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-21 09:55:25 +08:00
yukun
527c0c49df Add rocksmq unittest with goroutines
Signed-off-by: yukun <kun.yu@zilliz.com>
2021-01-20 16:46:58 +08:00
BossZou
4588342fa1 Add IdAlocator service
Signed-off-by: BossZou <yinghao.zou@zilliz.com>
2021-01-20 15:02:23 +08:00
sunby
2af61fc196 Change ComponentStates in proto
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-01-20 11:08:04 +08:00
sunby
3602376a10 Rename GetServiceStates to GetComponentStates and reconstruct NodeStates and ServiceStates
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-01-20 11:02:29 +08:00
zhenshan.cao
2ec573d2f9 Recover delete docs
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-01-20 09:36:50 +08:00
cai.zhang
fb493cc235 Update interface name of GetIndexStates
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-01-19 18:32:57 +08:00
neza2017
918e279869 Set msg stream
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-19 16:14:13 +08:00
sunby
7e8162a79e Add dd handler
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-01-19 15:35:40 +08:00
neza2017
d5e53aa28a Add Meta table implementation
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-19 14:13:49 +08:00
bigsheeper
2064b014a1 Add loadFieldData, use partition id instead of tag
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-19 14:44:03 +08:00
sunby
76a7684d96 Add data service
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-01-19 12:10:49 +08:00
XuanYang-cn
6a13386393 Add DataNode package
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-19 11:37:16 +08:00
neza2017
1ce32d8775 Add master service
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-19 09:45:42 +08:00
zhenshan.cao
76d92e73d1 Replace old proto with new proto
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-01-18 21:39:56 +08:00
xige-16
d22c2cd203 Add proto for query service loading collection
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-01-18 19:32:08 +08:00
xige-16
0164e78bb5 Add query service module
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-01-16 15:31:10 +08:00
XuanYang-cn
37688559f1 Fix multi rowdata writenode crash bug
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-16 15:06:19 +08:00
yukun
44c75bdd2a Add consume function in rocksmq
Signed-off-by: yukun <kun.yu@zilliz.com>
2021-01-15 20:12:26 +08:00
sunby
67e6ea8051 Add data service
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-01-15 17:09:41 +08:00
dragondriver
e6225a7030 Add service interface to milvus.proto
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-01-15 16:51:21 +08:00
bigsheeper
b98b226d8c Refactor query node and implement channelRemove, channelAdd
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-15 15:28:54 +08:00
godchen
0971cf65b2 Remove opentracing stdlog
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-01-15 15:28:27 +08:00
quicksilver
7fe061b846 Raneme init_devcontainer.sh to devcontainer.sh
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-01-15 14:38:36 +08:00
neza2017
56ab2bda2d Update proto:add partition id
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-15 11:53:22 +08:00
zhenshan.cao
63b21321d6 Refactor indexservice and update doc
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-01-15 13:50:27 +08:00
zhenshan.cao
04e2062750 Update proto
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-01-14 20:30:27 +08:00
yukun
6e36f10a26 Fix rocksdb illegal instruction in some machine
Signed-off-by: yukun <kun.yu@zilliz.com>
2021-01-14 14:24:14 +08:00
cai.zhang
f5ea46b1a3 Simplify proto and fix logic bug of checking vector type
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-01-07 16:13:28 +08:00
neza2017
4015d7245d Merge operation
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-06 14:45:50 +08:00
cai.zhang
1cfc2ff0a5 Add make clean in Makefile
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-01-06 13:50:48 +08:00
zhenshan.cao
b06e01f523 Initmate querynode collection manager
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-01-06 11:31:36 +08:00
XuanYang-cn
03b5d32569 Add default fields when create collection
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-04 12:03:29 +08:00
cai.zhang
7a28cfb9e7 Fix bug of sdk : unnecessary is_primary_key
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-01-04 10:58:51 +08:00
dragondriver
e61903ea36 Add query interface for cgo index wrapper
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-01-04 10:42:51 +08:00
bigsheeper
5b9e8c4f60 Add write node client segment flushing
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-12-28 15:25:16 +08:00
GuoRentong
e8ec0424d8 Update doc: service api
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
2020-12-27 09:05:24 +08:00
XuanYang-cn
e316533a3a Add Flush Sync service
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-26 16:28:55 +08:00
bigsheeper
5acc9681d6 Fix search error about metric type
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-12-26 15:29:48 +08:00
XuanYang-cn
c0797333ce Add ddl binlog metaTable in writenode
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-24 20:55:40 +08:00
neza2017
4dbf4ade41 Add test for master
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-24 16:19:42 +08:00
xige-16
f5198bcaee Add LoadIndexService in query node
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-12-24 14:51:55 +08:00
sunby
5d58606349 Move Status to SegIDAssignment
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-24 10:11:34 +08:00
cai.zhang
a8b78f11c3 Update proto about index
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-12-23 19:06:11 +08:00
sunby
0b9cffaf87 Remove index_name in IndexRequest and remove indexType in FieldIndexMeta
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-23 11:34:35 +08:00
sunby
2aa483c922 Add index meta to meta table
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-22 20:16:32 +08:00
cai.zhang
05918f55d2 Impl CreateIndex interface for proxy server
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-12-22 15:39:10 +08:00
cai.zhang
b582cf7b8e Add the interface of index in proto
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-12-21 19:28:54 +08:00
dragondriver
d023c1afba Refactor index wrapper, use protobuf instead of json to pass data
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-12-21 16:36:07 +08:00
sunby
06620935ab Move expire timestamp to SegIDAssignment
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-21 10:51:37 +08:00
zhenshan.cao
9694203f8b Add writer client and metatable for writer
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-21 10:06:46 +08:00
zhenshan.cao
3f6e926aef Refactor indexbuilder
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-13 06:48:05 +08:00
XuanYang-cn
d64aa37db3 Add KFlush msg type
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-11 12:01:20 +08:00
xige-16
0067e2d989 Add load index client in query node
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-12-11 11:29:54 +08:00
zhenshan.cao
fc054874d3 Add index_builder
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-10 17:55:55 +08:00
bigsheeper
762b1e1f1f Add ddService, fieldID, requestID and refactor params table, fix nil hits search result bug
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-12-10 16:31:09 +08:00
zhenshan.cao
7bbbc14637 Fix bug: address already used
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-07 15:22:20 +08:00
xige-16
37d7526d31 Fix search test failure
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-12-07 14:37:42 +08:00
neza2017
aeca8f85a5 Run cwrapper unittest in ci
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-05 17:39:58 +08:00
cai.zhang
806a97505c Add query node for CI progress
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-12-05 16:12:45 +08:00
FluorineDog
6412ebc0d4 Add support of metric type in schema, enable binary vector, fix segfault
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-12-05 06:46:01 +08:00
zhenshan.cao
0b6fe0a8cf Refactor paramtable
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-04 14:18:17 +08:00
neza2017
cec903da19 Add parquet payload
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-02 18:31:56 +08:00
zhenshan.cao
23aebe7e2a Fix proxy unittest
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-01 19:53:53 +08:00
neza2017
17cc7bc337 Use MetaRootPath instead
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-01 17:03:39 +08:00
neza2017
d3858500c6 Add static check
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-30 19:38:23 +08:00
quicksilver
86573d0053 Create Jenkins pipeline
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2020-11-30 10:07:04 +08:00
FluorineDog
77fa75b1ec Add binary insert and warper of binary search, rename vector
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-11-30 05:18:44 +08:00
cai.zhang
85a544b79b Add dockerfile for each component
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-11-28 19:06:48 +08:00
cai.zhang
115d0732af Update docker-compose file for unittest
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-11-28 16:43:39 +08:00