Commit Graph

153 Commits

Author SHA1 Message Date
bigsheeper
cb3ca1ab7c Refactor kvPath param and unify the segment flush kvPath in write node
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-04 14:17:59 +08:00
XuanYang-cn
03b5d32569 Add default fields when create collection
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-04 12:03:29 +08:00
zhenshan.cao
7ed63c79ce Remove system field of describle_collection in proxy
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-01-04 11:41:19 +08:00
dragondriver
6cd8c2528d Add parameter check for almost all index type
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-12-31 15:16:03 +08:00
xige-16
29b21b4bf2 Refactor reduce strategy in query node
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-12-31 14:49:27 +08:00
sunby
4808b46246 Filter indexable fields in flush scheduler
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-30 19:56:54 +08:00
sunby
e7998da797 Fix bug when starting indexbuilder
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-30 16:41:24 +08:00
XuanYang-cn
3a2e2a04b6 Fix writenode bug
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-29 21:15:00 +08:00
XuanYang-cn
884f66d56d Fix timesync start twice in master
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-29 14:10:22 +08:00
godchen
d489183798 Random generate querynode channel in unittest
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2020-12-28 15:24:20 +08:00
sunby
a56f8f7b9e Add runtime stats in master
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-25 18:32:03 +08:00
sunby
e21397d68a Add runtime stats unittest
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-25 16:08:35 +08:00
sunby
a03ab05c6c Add DescribeIndex and DescribeIndexProgress RPC
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-25 15:39:56 +08:00
XuanYang-cn
c0797333ce Add ddl binlog metaTable in writenode
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-24 20:55:40 +08:00
neza2017
4dbf4ade41 Add test for master
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-24 16:19:42 +08:00
zhenshan.cao
a0244b7683 Add segment expire logic for proxy
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-24 16:53:31 +08:00
bigsheeper
4ecdea698f Refactor query node and query serviceMain changes:1. Add ddBuffer and save binLog.
2. Trans to insertData.
3. Change dataFields data to Data, dim to Dim4. Add float vector and binary vector.5. Deserialize data and convert to InsertData.6. Move all data into InsertData.7. Add insert buffer and hash string.
8. Add minIOkV in insertBuffer node.
9. Init write node insertBuffer maxSize from writeNode.yaml.
10. Add ddBuffer.
11. Add ddBuffer binLog and minio.
12. Add ddNode unittest.
13. Remove redundant call.
14. Increase test time.
15. Delete ddl const, use request's timestamp instead.

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-12-24 15:38:29 +08:00
xige-16
f5198bcaee Add LoadIndexService in query node
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-12-24 14:51:55 +08:00
sunby
add31188ba Add CreateIndex RPC
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-24 14:50:16 +08:00
sunby
5d58606349 Move Status to SegIDAssignment
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-24 10:11:34 +08:00
sunby
e7ebfcb05a Save index meta to meta table
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-23 15:13:45 +08:00
sunby
0b9cffaf87 Remove index_name in IndexRequest and remove indexType in FieldIndexMeta
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-23 11:34:35 +08:00
sunby
2aa483c922 Add index meta to meta table
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-22 20:16:32 +08:00
sunby
3a1c210098 Add runtime stats to record field stats
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-21 20:24:40 +08:00
godchen
8f87ee632e Add field id when create collection
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2020-12-21 19:31:24 +08:00
godchen
e90ef211c2 Add flush, build index, load index scheduler
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2020-12-21 19:30:24 +08:00
cai.zhang
b582cf7b8e Add the interface of index in proto
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-12-21 19:28:54 +08:00
sunby
06620935ab Move expire timestamp to SegIDAssignment
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-21 10:51:37 +08:00
sunby
f360ed7004 Add unittests
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-21 09:56:35 +08:00
sunby
c71cd40f68 Refactor segment manager
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-19 12:55:24 +08:00
godchen
475d36a2c9 Add data codec
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2020-12-18 15:21:25 +08:00
XuanYang-cn
d64aa37db3 Add KFlush msg type
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-11 12:01:20 +08:00
xige-16
0067e2d989 Add load index client in query node
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-12-11 11:29:54 +08:00
bigsheeper
762b1e1f1f Add ddService, fieldID, requestID and refactor params table, fix nil hits search result bug
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-12-10 16:31:09 +08:00
sunby
16071d92b5 Modify metatable close segment interface.
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-10 13:35:09 +08:00
XuanYang-cn
32977e270c Add impl cgo of parquet
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-08 14:41:04 +08:00
zhenshan.cao
65ce1f97e7 Fix bug: pulsar dirty msg
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-07 19:41:39 +08:00
zhenshan.cao
7bbbc14637 Fix bug: address already used
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-07 15:22:20 +08:00
xige-16
37d7526d31 Fix search test failure
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-12-07 14:37:42 +08:00
neza2017
aeca8f85a5 Run cwrapper unittest in ci
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-05 17:39:58 +08:00
cai.zhang
806a97505c Add query node for CI progress
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-12-05 16:12:45 +08:00
neza2017
70710dee47 Add parquet payload
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-05 16:11:03 +08:00
FluorineDog
6412ebc0d4 Add support of metric type in schema, enable binary vector, fix segfault
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-12-05 06:46:01 +08:00
zhenshan.cao
0b6fe0a8cf Refactor paramtable
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-04 14:18:17 +08:00
godchen
7ab5b5d80d Add MinIO kv implements
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2020-12-03 19:00:11 +08:00
neza2017
cec903da19 Add parquet payload
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-02 18:31:56 +08:00
zhenshan.cao
23aebe7e2a Fix proxy unittest
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-01 19:53:53 +08:00
neza2017
17cc7bc337 Use MetaRootPath instead
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-01 17:03:39 +08:00
neza2017
d3858500c6 Add static check
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-30 19:38:23 +08:00
quicksilver
86573d0053 Create Jenkins pipeline
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2020-11-30 10:07:04 +08:00