Commit Graph

6200 Commits

Author SHA1 Message Date
cai.zhang
f80536ed77 Update the logic of running go unittest
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-12-29 11:56:37 +08:00
XuanYang-cn
9577da9fce Add writenode main
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-28 18:52:34 +08:00
FluorineDog
eb8717188e Enable load index
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-12-28 17:55:48 +08:00
dragondriver
f67aae9596 Fix the codec bug of cgo wrapper of index builder
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-12-28 16:55:50 +08:00
bigsheeper
5b9e8c4f60 Add write node client segment flushing
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-12-28 15:25:16 +08:00
godchen
d489183798 Random generate querynode channel in unittest
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2020-12-28 15:24:20 +08:00
XuanYang-cn
aab6dede4a Add Flush for DDBuffer and insertBuffer
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-28 11:29:17 +08:00
dragondriver
11cef6e978 Add support to build binary index
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-12-28 10:00:02 +08:00
GuoRentong
e8ec0424d8 Update doc: service api
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
2020-12-27 09:05:24 +08:00
neza2017
7e8b203fe2 Add time tick in wite node
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-26 17:05:24 +08:00
XuanYang-cn
e316533a3a Add Flush Sync service
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-26 16:28:55 +08:00
bigsheeper
5acc9681d6 Fix search error about metric type
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-12-26 15:29:48 +08:00
xige-16
d599407e2b Add unittest for loadIndex service
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-12-26 14:16:51 +08:00
cai.zhang
2031c54746 Add unittest about index in proxy
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-12-26 11:51:07 +08:00
sunby
a56f8f7b9e Add runtime stats in master
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-25 18:32:03 +08:00
sunby
e21397d68a Add runtime stats unittest
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-25 16:08:35 +08:00
sunby
a03ab05c6c Add DescribeIndex and DescribeIndexProgress RPC
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-25 15:39:56 +08:00
dragondriver
84a0cf7097 Add unittest to c index wrapper
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-12-25 11:10:31 +08:00
XuanYang-cn
c0797333ce Add ddl binlog metaTable in writenode
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-24 20:55:40 +08:00
neza2017
4dbf4ade41 Add test for master
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-24 16:19:42 +08:00
zhenshan.cao
a0244b7683 Add segment expire logic for proxy
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-24 16:53:31 +08:00
bigsheeper
4ecdea698f Refactor query node and query serviceMain changes:1. Add ddBuffer and save binLog.
2. Trans to insertData.
3. Change dataFields data to Data, dim to Dim4. Add float vector and binary vector.5. Deserialize data and convert to InsertData.6. Move all data into InsertData.7. Add insert buffer and hash string.
8. Add minIOkV in insertBuffer node.
9. Init write node insertBuffer maxSize from writeNode.yaml.
10. Add ddBuffer.
11. Add ddBuffer binLog and minio.
12. Add ddNode unittest.
13. Remove redundant call.
14. Increase test time.
15. Delete ddl const, use request's timestamp instead.

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-12-24 15:38:29 +08:00
xige-16
f5198bcaee Add LoadIndexService in query node
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-12-24 14:51:55 +08:00
sunby
add31188ba Add CreateIndex RPC
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-24 14:50:16 +08:00
xige-16
14332d49fb Add minio for regression
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-12-24 14:34:14 +08:00
neza2017
0afeaa78c8 Fix unittest of flow graph
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-24 14:10:00 +08:00
sunby
5d58606349 Move Status to SegIDAssignment
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-24 10:11:34 +08:00
cai.zhang
a8b78f11c3 Update proto about index
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-12-23 19:06:11 +08:00
FluorineDog
342f4cb741 Support dynamic chunk size
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-12-23 19:02:37 +08:00
XuanYang-cn
7ce0f27ebc Add buffer to minIO for binlogs
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-23 18:06:04 +08:00
sunby
e7ebfcb05a Save index meta to meta table
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-23 15:13:45 +08:00
godchen
e52b130ba6 Refactor data codec code
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2020-12-23 15:46:16 +08:00
sunby
0b9cffaf87 Remove index_name in IndexRequest and remove indexType in FieldIndexMeta
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-23 11:34:35 +08:00
zhenshan.cao
11316525e5 Fix bug of running unittest in parallel
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-22 22:08:03 +08:00
sunby
2aa483c922 Add index meta to meta table
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-22 20:16:32 +08:00
dragondriver
adfb5187cf Dock build index task with storage & indexbuilder
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-12-22 00:14:36 +00:00
cai.zhang
05918f55d2 Impl CreateIndex interface for proxy server
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-12-22 15:39:10 +08:00
GuoRentong
9d5c93744c Fix figure path
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
2020-12-22 14:47:55 +08:00
sunby
3a1c210098 Add runtime stats to record field stats
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-21 20:24:40 +08:00
godchen
8f87ee632e Add field id when create collection
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2020-12-21 19:31:24 +08:00
godchen
e90ef211c2 Add flush, build index, load index scheduler
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2020-12-21 19:30:24 +08:00
cai.zhang
b582cf7b8e Add the interface of index in proto
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-12-21 19:28:54 +08:00
dragondriver
d023c1afba Refactor index wrapper, use protobuf instead of json to pass data
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-12-21 16:36:07 +08:00
XuanYang-cn
87521adfbd Add logic of deserilize rawData into InsertData
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-21 16:27:03 +08:00
FluorineDog
d5daa18392 Enable most DSL-related cases
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-12-21 14:45:00 +08:00
sunby
06620935ab Move expire timestamp to SegIDAssignment
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-21 10:51:37 +08:00
zhenshan.cao
9694203f8b Add writer client and metatable for writer
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-21 10:06:46 +08:00
sunby
f360ed7004 Add unittests
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-21 09:56:35 +08:00
sunby
c71cd40f68 Refactor segment manager
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-19 12:55:24 +08:00
FluorineDog
e65cfe1e3d Enable complex dsl parser
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-12-19 10:36:49 +08:00