Commit Graph

168 Commits

Author SHA1 Message Date
bigsheeper
a1e2ff3c26 Add gcNode to write node
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-06 16:44:12 +08:00
neza2017
4015d7245d Merge operation
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-06 14:45:50 +08:00
cai.zhang
1cfc2ff0a5 Add make clean in Makefile
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-01-06 13:50:48 +08:00
sunby
da6eeddbf8 Add flush timestamp
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-01-06 13:14:38 +08:00
zhenshan.cao
b06e01f523 Initmate querynode collection manager
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-01-06 11:31:36 +08:00
godchen
1c87faf8b3 Add insert request opentracing
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-01-06 11:17:35 +08:00
sunby
d0c787628b Pass type params when building index
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-01-05 16:09:14 +08:00
XuanYang-cn
05f1350e19 Add writenode to ci
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-05 15:14:59 +08:00
quicksilver
ca89ece576 Fix go format bug
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-01-05 14:20:45 +08:00
sunby
a1151d9bf5 Change kvRootPath to MetaRootPath in master
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-01-05 11:31:42 +08:00
sunby
95b162ccfd Refactor flush scheduler
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-01-05 10:57:59 +08:00
bigsheeper
33e1e17045 Fix writer client DescribeSegment api
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-04 20:16:15 +08:00
sunby
0a383608f3 Remove mock client in master
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-31 11:20:03 +08:00
FluorineDog
89f38e459a Fix bool type
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2021-01-04 17:52:42 +08:00
zhenshan.cao
6bd5702441 Add indexbuilder address to paramstable
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-01-04 14:53:20 +08:00
bigsheeper
cb3ca1ab7c Refactor kvPath param and unify the segment flush kvPath in write node
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-04 14:17:59 +08:00
XuanYang-cn
03b5d32569 Add default fields when create collection
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-04 12:03:29 +08:00
zhenshan.cao
7ed63c79ce Remove system field of describle_collection in proxy
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-01-04 11:41:19 +08:00
dragondriver
6cd8c2528d Add parameter check for almost all index type
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-12-31 15:16:03 +08:00
xige-16
29b21b4bf2 Refactor reduce strategy in query node
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-12-31 14:49:27 +08:00
sunby
4808b46246 Filter indexable fields in flush scheduler
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-30 19:56:54 +08:00
sunby
e7998da797 Fix bug when starting indexbuilder
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-30 16:41:24 +08:00
XuanYang-cn
3a2e2a04b6 Fix writenode bug
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-29 21:15:00 +08:00
XuanYang-cn
884f66d56d Fix timesync start twice in master
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-29 14:10:22 +08:00
godchen
d489183798 Random generate querynode channel in unittest
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2020-12-28 15:24:20 +08:00
sunby
a56f8f7b9e Add runtime stats in master
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-25 18:32:03 +08:00
sunby
e21397d68a Add runtime stats unittest
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-25 16:08:35 +08:00
sunby
a03ab05c6c Add DescribeIndex and DescribeIndexProgress RPC
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-25 15:39:56 +08:00
XuanYang-cn
c0797333ce Add ddl binlog metaTable in writenode
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-24 20:55:40 +08:00
neza2017
4dbf4ade41 Add test for master
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-24 16:19:42 +08:00
zhenshan.cao
a0244b7683 Add segment expire logic for proxy
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-24 16:53:31 +08:00
bigsheeper
4ecdea698f Refactor query node and query serviceMain changes:1. Add ddBuffer and save binLog.
2. Trans to insertData.
3. Change dataFields data to Data, dim to Dim4. Add float vector and binary vector.5. Deserialize data and convert to InsertData.6. Move all data into InsertData.7. Add insert buffer and hash string.
8. Add minIOkV in insertBuffer node.
9. Init write node insertBuffer maxSize from writeNode.yaml.
10. Add ddBuffer.
11. Add ddBuffer binLog and minio.
12. Add ddNode unittest.
13. Remove redundant call.
14. Increase test time.
15. Delete ddl const, use request's timestamp instead.

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-12-24 15:38:29 +08:00
xige-16
f5198bcaee Add LoadIndexService in query node
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-12-24 14:51:55 +08:00
sunby
add31188ba Add CreateIndex RPC
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-24 14:50:16 +08:00
sunby
5d58606349 Move Status to SegIDAssignment
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-24 10:11:34 +08:00
sunby
e7ebfcb05a Save index meta to meta table
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-23 15:13:45 +08:00
sunby
0b9cffaf87 Remove index_name in IndexRequest and remove indexType in FieldIndexMeta
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-23 11:34:35 +08:00
sunby
2aa483c922 Add index meta to meta table
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-22 20:16:32 +08:00
sunby
3a1c210098 Add runtime stats to record field stats
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-21 20:24:40 +08:00
godchen
8f87ee632e Add field id when create collection
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2020-12-21 19:31:24 +08:00
godchen
e90ef211c2 Add flush, build index, load index scheduler
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2020-12-21 19:30:24 +08:00
cai.zhang
b582cf7b8e Add the interface of index in proto
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-12-21 19:28:54 +08:00
sunby
06620935ab Move expire timestamp to SegIDAssignment
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-21 10:51:37 +08:00
sunby
f360ed7004 Add unittests
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-21 09:56:35 +08:00
sunby
c71cd40f68 Refactor segment manager
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-19 12:55:24 +08:00
godchen
475d36a2c9 Add data codec
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2020-12-18 15:21:25 +08:00
XuanYang-cn
d64aa37db3 Add KFlush msg type
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-11 12:01:20 +08:00
xige-16
0067e2d989 Add load index client in query node
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-12-11 11:29:54 +08:00
bigsheeper
762b1e1f1f Add ddService, fieldID, requestID and refactor params table, fix nil hits search result bug
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-12-10 16:31:09 +08:00
sunby
16071d92b5 Modify metatable close segment interface.
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-10 13:35:09 +08:00