Commit Graph

202 Commits

Author SHA1 Message Date
neza2017
1ce32d8775 Add master service
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-19 09:45:42 +08:00
zhenshan.cao
76d92e73d1 Replace old proto with new proto
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-01-18 21:39:56 +08:00
xige-16
d22c2cd203 Add proto for query service loading collection
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-01-18 19:32:08 +08:00
xige-16
0164e78bb5 Add query service module
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-01-16 15:31:10 +08:00
XuanYang-cn
37688559f1 Fix multi rowdata writenode crash bug
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-16 15:06:19 +08:00
yukun
44c75bdd2a Add consume function in rocksmq
Signed-off-by: yukun <kun.yu@zilliz.com>
2021-01-15 20:12:26 +08:00
sunby
67e6ea8051 Add data service
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-01-15 17:09:41 +08:00
dragondriver
e6225a7030 Add service interface to milvus.proto
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-01-15 16:51:21 +08:00
bigsheeper
b98b226d8c Refactor query node and implement channelRemove, channelAdd
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-15 15:28:54 +08:00
godchen
0971cf65b2 Remove opentracing stdlog
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-01-15 15:28:27 +08:00
quicksilver
7fe061b846 Raneme init_devcontainer.sh to devcontainer.sh
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-01-15 14:38:36 +08:00
neza2017
56ab2bda2d Update proto:add partition id
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-15 11:53:22 +08:00
zhenshan.cao
63b21321d6 Refactor indexservice and update doc
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-01-15 13:50:27 +08:00
zhenshan.cao
04e2062750 Update proto
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-01-14 20:30:27 +08:00
yukun
6e36f10a26 Fix rocksdb illegal instruction in some machine
Signed-off-by: yukun <kun.yu@zilliz.com>
2021-01-14 14:24:14 +08:00
cai.zhang
f5ea46b1a3 Simplify proto and fix logic bug of checking vector type
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-01-07 16:13:28 +08:00
neza2017
4015d7245d Merge operation
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-06 14:45:50 +08:00
cai.zhang
1cfc2ff0a5 Add make clean in Makefile
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-01-06 13:50:48 +08:00
zhenshan.cao
b06e01f523 Initmate querynode collection manager
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-01-06 11:31:36 +08:00
XuanYang-cn
03b5d32569 Add default fields when create collection
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-04 12:03:29 +08:00
cai.zhang
7a28cfb9e7 Fix bug of sdk : unnecessary is_primary_key
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-01-04 10:58:51 +08:00
dragondriver
e61903ea36 Add query interface for cgo index wrapper
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-01-04 10:42:51 +08:00
bigsheeper
5b9e8c4f60 Add write node client segment flushing
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-12-28 15:25:16 +08:00
GuoRentong
e8ec0424d8 Update doc: service api
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
2020-12-27 09:05:24 +08:00
XuanYang-cn
e316533a3a Add Flush Sync service
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-26 16:28:55 +08:00
bigsheeper
5acc9681d6 Fix search error about metric type
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-12-26 15:29:48 +08:00
XuanYang-cn
c0797333ce Add ddl binlog metaTable in writenode
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-24 20:55:40 +08:00
neza2017
4dbf4ade41 Add test for master
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-24 16:19:42 +08:00
xige-16
f5198bcaee Add LoadIndexService in query node
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-12-24 14:51:55 +08:00
sunby
5d58606349 Move Status to SegIDAssignment
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-24 10:11:34 +08:00
cai.zhang
a8b78f11c3 Update proto about index
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-12-23 19:06:11 +08:00
sunby
0b9cffaf87 Remove index_name in IndexRequest and remove indexType in FieldIndexMeta
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-23 11:34:35 +08:00
sunby
2aa483c922 Add index meta to meta table
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-22 20:16:32 +08:00
cai.zhang
05918f55d2 Impl CreateIndex interface for proxy server
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-12-22 15:39:10 +08:00
cai.zhang
b582cf7b8e Add the interface of index in proto
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-12-21 19:28:54 +08:00
dragondriver
d023c1afba Refactor index wrapper, use protobuf instead of json to pass data
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-12-21 16:36:07 +08:00
sunby
06620935ab Move expire timestamp to SegIDAssignment
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-21 10:51:37 +08:00
zhenshan.cao
9694203f8b Add writer client and metatable for writer
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-21 10:06:46 +08:00
zhenshan.cao
3f6e926aef Refactor indexbuilder
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-13 06:48:05 +08:00
XuanYang-cn
d64aa37db3 Add KFlush msg type
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-11 12:01:20 +08:00
xige-16
0067e2d989 Add load index client in query node
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-12-11 11:29:54 +08:00
zhenshan.cao
fc054874d3 Add index_builder
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-10 17:55:55 +08:00
bigsheeper
762b1e1f1f Add ddService, fieldID, requestID and refactor params table, fix nil hits search result bug
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-12-10 16:31:09 +08:00
zhenshan.cao
7bbbc14637 Fix bug: address already used
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-07 15:22:20 +08:00
xige-16
37d7526d31 Fix search test failure
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-12-07 14:37:42 +08:00
neza2017
aeca8f85a5 Run cwrapper unittest in ci
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-05 17:39:58 +08:00
cai.zhang
806a97505c Add query node for CI progress
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-12-05 16:12:45 +08:00
FluorineDog
6412ebc0d4 Add support of metric type in schema, enable binary vector, fix segfault
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-12-05 06:46:01 +08:00
zhenshan.cao
0b6fe0a8cf Refactor paramtable
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-04 14:18:17 +08:00
neza2017
cec903da19 Add parquet payload
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-02 18:31:56 +08:00
zhenshan.cao
23aebe7e2a Fix proxy unittest
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-01 19:53:53 +08:00
neza2017
17cc7bc337 Use MetaRootPath instead
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-01 17:03:39 +08:00
neza2017
d3858500c6 Add static check
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-30 19:38:23 +08:00
quicksilver
86573d0053 Create Jenkins pipeline
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2020-11-30 10:07:04 +08:00
FluorineDog
77fa75b1ec Add binary insert and warper of binary search, rename vector
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-11-30 05:18:44 +08:00
cai.zhang
85a544b79b Add dockerfile for each component
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-11-28 19:06:48 +08:00
cai.zhang
115d0732af Update docker-compose file for unittest
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-11-28 16:43:39 +08:00
dragondriver
eb6c0e0a28 Fix the bug when reduce the multiple search result from query nodes
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-11-28 15:57:49 +08:00
godchen
92bf09c7e6 Add ddstream and scheduler implements
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2020-11-28 15:26:39 +08:00
dragondriver
568cef0730 Fix InsertTask and SearchTask
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-11-28 10:48:29 +08:00
FluorineDog
c9fb34142c Enable primary_key switch
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-11-28 09:16:00 +08:00
dragondriver
235e723eb6 Read proxy id from param table instead of hardcode
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-11-27 11:17:12 +08:00
XuanYang-cn
3e605ae25f Rename reader to querynode
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-11-27 10:56:53 +08:00
neza2017
32445dd02b Add cpu profile
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-26 16:01:31 +08:00
cai.zhang
57da66ba72 Update read config logic in docker
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-11-26 15:18:36 +08:00
zhenshan.cao
045fc3d808 Fix segment fault
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-11-26 14:29:01 +08:00
XuanYang-cn
0a16a9a651 Add logic of getting milvus distributed config from master by grpc
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-11-26 13:21:06 +08:00
dragondriver
8ce0deca4e Add default partition tag to InsertTask in Proxy
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-11-26 11:38:04 +08:00
xige-16
ae78aa1ac1 Warp cpp reduce and repack searchResultMsg to specified pulsar channel
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-11-26 11:12:33 +08:00
zhenshan.cao
50d59d91ee Add segmentIDAssigner for proxy
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-11-19 21:02:31 +08:00
neza2017
c3de667bf5 Implement segment management in master
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-18 19:46:18 +08:00
bigsheeper
99ea7e9f05 Update param table
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-11-18 19:33:56 +08:00
sunby
9e4ce8454e Implement segment management in master
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-11-18 15:04:17 +08:00
cai.zhang
0b478b614a Change Id to ID in go files according to golint check
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-11-13 15:17:18 +08:00
sunby
e541041b28 Add QueryNodeSegStats proto
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-11-13 14:20:51 +08:00
XuanYang-cn
88cb274644 Refactor master scheduler: drop_collection
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-11-13 12:46:24 +08:00
FluorineDog
f0ec7a8c78 Support Placeholder Parser and refine CreatePlan on predicate-less
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-11-12 12:18:07 +08:00
cai.zhang
1cd2fd9954 Add the standard of golangci-lint
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-11-12 12:04:12 +08:00
zhenshan.cao
87a4495c59 Fix proxy bug
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-11-09 17:52:47 +08:00
bigsheeper
b441685635 Refactor segment, partition and collection, refactor search and services
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-11-09 16:27:11 +08:00
bigsheeper
88a3fa31fa Refactor query node and services
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-11-05 10:52:50 +08:00
zhenshan.cao
e9122921b9 Use type alias
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-11-04 17:58:43 +08:00
neza2017
d18c17e786 Add Meta table implementation
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-04 16:28:14 +08:00
godchen
bade037fb2 Add meta_table implementation
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2020-11-04 16:01:28 +08:00
zhenshan.cao
2e2e059f8a Add msgheader and change ReqType to MsgType
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-11-04 15:13:42 +08:00
zhenshan.cao
da18bc4952 Update proto
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-11-03 16:38:16 +08:00
zhenshan.cao
e962b57fbb Add timestamp allocator
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-11-03 14:53:36 +08:00
GuoRentong
fbdc5df96f Update docs
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
2020-11-03 12:40:13 +08:00
zhenshan.cao
3e596fa474 Update proto and doc
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-11-02 16:01:04 +08:00
GuoRentong
15b6963be0 Replace pdf figs with png figs
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
2020-10-31 15:11:47 +08:00
zhenshan.cao
27b9a51938 Add logic of allocate id
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-10-30 16:27:58 +08:00
GuoRentong
f7bd6ebfad Add docs
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
2020-10-29 20:42:47 +08:00
zhenshan.cao
6eee7d1342 Fix bug and update proto
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-10-29 19:55:57 +08:00
zhenshan.cao
f0cc9c8ad3 Add tso for master and add timestamp
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-10-29 09:31:08 +08:00
GuoRentong
1cfc6c7f86 Add docs
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
2020-10-28 10:09:35 +08:00
FluorineDog
a48ca80286 Format Code and duplicate class Segment
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-10-24 18:04:57 +08:00
FluorineDog
e84b0180c9 Refactor cmake and build script and add timed benchmark
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-10-23 18:01:24 +08:00
zhenshan.cao
251bc2a19e Fix conf.LoadConfig failed to load yaml file
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-10-19 18:31:00 +08:00
GuoRentong
974496a2b2 Update issue templates
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
2020-10-19 17:45:56 +08:00
zhenshan.cao
6a136c1a76 Change import path
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-10-19 17:39:51 +08:00
zhenshan.cao
4965a315ad Refact proto generating procedure
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-10-19 10:36:40 +08:00
zhenshan.cao
64295db471 Refact master and proxy and add etcdutil
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-10-15 21:31:50 +08:00