Commit Graph

124 Commits

Author SHA1 Message Date
Cai Yudong
c795cdbe2a Change minSegmentSizeToEnableIndex default to 1024
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
2021-03-07 13:37:25 -06:00
neza2017
014c4fe8ce Add config of minSegmentSizeToEnableIndex
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-03-08 15:46:51 +08:00
bigsheeper
d3db2b0d82 Add zap log in query service
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-02-27 10:11:52 +08:00
neza2017
85c1404393 Refactor master:index ids
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-02-11 08:41:59 +08:00
godchen
c2ca2c276f Fix response check error
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-02-04 19:34:35 +08:00
Xiangyu Wang
fd9e42d3ed Update msgstream/unmarshal interface
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-02-03 17:30:10 +08:00
bigsheeper
abffdbff62 Remove partition tag
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-02-03 16:44:38 +08:00
sunby
208700b0e7 Set fixed insert channel number, remove channel manager
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-02-03 15:07:48 +08:00
neza2017
854accf95b Remove master and writenode
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-30 16:41:47 +08:00
bigsheeper
68cd15af63 Get index info from master
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-27 14:41:56 +08:00
dragondriver
a0226c09f0 Change the logic node id allocator in Proxy
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-01-27 13:52:01 +08:00
sunby
b4bcaaf847 Fix data service startup bug
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-01-27 11:40:10 +08:00
dragondriver
c6a6b1436c Add soft time tick in proxy service
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-01-26 13:41:41 +08:00
neza2017
5af23cf018 Init params and add segment start/end position
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-26 09:43:41 +08:00
bigsheeper
7554246ace Add segment seeking and use real client
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-25 18:33:10 +08:00
sunby
7c0c835b49 Init params and add segment start/end position
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-01-25 15:16:28 +08:00
XuanYang-cn
5015a1035d Add logic of get meta from MasterService for datanode
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-25 17:10:32 +08:00
neza2017
1fe099775d Refactor data service
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-25 15:17:17 +08:00
neza2017
e89e89b894 Add main for masterservice
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-25 12:03:44 +08:00
neza2017
e2b45f9866 Init param table and open unittest at ci
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-23 17:56:57 +08:00
neza2017
9828a52999 Add create index
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-21 10:01:29 +08:00
cai.zhang
83c60fd84f Update proto about index
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-01-21 15:20:23 +08:00
XuanYang-cn
ec27788628 Add segment statistics updates logic for datanode
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-21 09:55:25 +08:00
XuanYang-cn
6a13386393 Add DataNode package
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-19 11:37:16 +08:00
quicksilver
7fe061b846 Raneme init_devcontainer.sh to devcontainer.sh
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-01-15 14:38:36 +08:00
neza2017
56ab2bda2d Update proto:add partition id
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-15 11:53:22 +08:00
zhenshan.cao
63b21321d6 Refactor indexservice and update doc
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-01-15 13:50:27 +08:00
XuanYang-cn
3a2e2a04b6 Fix writenode bug
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-29 21:15:00 +08:00
XuanYang-cn
c0797333ce Add ddl binlog metaTable in writenode
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-24 20:55:40 +08:00
neza2017
4dbf4ade41 Add test for master
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-24 16:19:42 +08:00
xige-16
f5198bcaee Add LoadIndexService in query node
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-12-24 14:51:55 +08:00
XuanYang-cn
7ce0f27ebc Add buffer to minIO for binlogs
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-23 18:06:04 +08:00
bigsheeper
585d3f9831 Refactor param table, and add ddNode
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-12-11 17:20:14 +08:00
bigsheeper
762b1e1f1f Add ddService, fieldID, requestID and refactor params table, fix nil hits search result bug
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-12-10 16:31:09 +08:00
XuanYang-cn
881be7f3e3 Impl minimum runable writenode and writenode flowgraph
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-09 20:07:27 +08:00
XuanYang-cn
32977e270c Add impl cgo of parquet
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-08 14:41:04 +08:00
zhenshan.cao
7bbbc14637 Fix bug: address already used
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-07 15:22:20 +08:00
xige-16
37d7526d31 Fix search test failure
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-12-07 14:37:42 +08:00
neza2017
aeca8f85a5 Run cwrapper unittest in ci
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-05 17:39:58 +08:00
cai.zhang
806a97505c Add query node for CI progress
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-12-05 16:12:45 +08:00
neza2017
70710dee47 Add parquet payload
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-05 16:11:03 +08:00
FluorineDog
6412ebc0d4 Add support of metric type in schema, enable binary vector, fix segfault
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-12-05 06:46:01 +08:00
zhenshan.cao
0b6fe0a8cf Refactor paramtable
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-04 14:18:17 +08:00
neza2017
cec903da19 Add parquet payload
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-02 18:31:56 +08:00
zhenshan.cao
23aebe7e2a Fix proxy unittest
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-01 19:53:53 +08:00
neza2017
17cc7bc337 Use MetaRootPath instead
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-01 17:03:39 +08:00
cai.zhang
85a544b79b Add dockerfile for each component
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-11-28 19:06:48 +08:00
cai.zhang
115d0732af Update docker-compose file for unittest
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-11-28 16:43:39 +08:00
dragondriver
eb6c0e0a28 Fix the bug when reduce the multiple search result from query nodes
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-11-28 15:57:49 +08:00
godchen
92bf09c7e6 Add ddstream and scheduler implements
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2020-11-28 15:26:39 +08:00
neza2017
32445dd02b Add cpu profile
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-26 16:01:31 +08:00
cai.zhang
57da66ba72 Update read config logic in docker
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-11-26 15:18:36 +08:00
neza2017
d9e5a8774d Add dimension validation
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-26 15:07:47 +08:00
FluorineDog
8ae573b580 Fix bool bug
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-11-26 14:56:58 +08:00
sunby
e8e21a0308 Add dimension validation.
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-11-26 14:54:03 +08:00
zhenshan.cao
045fc3d808 Fix segment fault
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-11-26 14:29:01 +08:00
XuanYang-cn
0a16a9a651 Add logic of getting milvus distributed config from master by grpc
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-11-26 13:21:06 +08:00
dragondriver
8ce0deca4e Add default partition tag to InsertTask in Proxy
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-11-26 11:38:04 +08:00
xige-16
ae78aa1ac1 Warp cpp reduce and repack searchResultMsg to specified pulsar channel
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-11-26 11:12:33 +08:00
neza2017
ae26a81550 Change to use Marshal as text
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-26 10:13:23 +08:00
sunby
f4c643f1bd Add number limitation of partition and field. And Change default partition tag to "_default"
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-11-25 18:39:05 +08:00
bigsheeper
4ed11d9775 Reopen segment test assertion
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-11-25 16:24:57 +08:00
bigsheeper
3b24c52a8c Catch cpp runtime error
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-11-25 16:20:07 +08:00
dragondriver
a7b3efecd7 Read query node num from param table instead of hardcode
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-11-25 16:17:33 +08:00
cai.zhang
4bcb460c98 Add clang-format
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-11-25 15:25:00 +08:00
FluorineDog
a6f950e5b6 Fix boost::dynamic_bitset misuse
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-11-25 15:17:17 +08:00
sunby
24b29bec30 Add collectionName/partitionTag/fieldName validation.
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-11-25 14:44:30 +08:00
bigsheeper
84f3d974fa Delete pulsar address test and refactor master param table
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-11-23 18:04:09 +08:00
zhenshan.cao
7c39fc24a9 Simplify proxy config
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-11-21 19:12:59 +08:00
bigsheeper
dd3654f953 Fix query node param table
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-11-20 17:10:24 +08:00
neza2017
ce89f64bb3 Set master config
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-19 10:46:17 +08:00
neza2017
c3de667bf5 Implement segment management in master
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-18 19:46:18 +08:00
bigsheeper
99ea7e9f05 Update param table
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-11-18 19:33:56 +08:00
FluorineDog
e7dd30a884 Add framework of ExecExprVisitor
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-11-18 17:32:52 +08:00