test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-02 11:59:00 +08:00
Code Issues Actions 12 Packages Projects Releases Wiki Activity
6,072 Commits 94 Branches 105 Tags 184 MiB
6ffe8739bf
Commit Graph

360 Commits

Author SHA1 Message Date
sunby
4808b46246 Filter indexable fields in flush scheduler 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-12-30 19:56:54 +08:00
dragondriver
2e198e2783 Add support for idmap, nsg, sq8 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2020-12-30 19:37:45 +08:00
FluorineDog
09620efadf Merge from master 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2020-12-30 18:50:56 +08:00
zhenshan.cao
adf77c423c Update knowhere 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-12-30 18:30:51 +08:00
bigsheeper
4835d82060 Add index builder test cases 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2020-12-30 18:16:34 +08:00
sunby
e7998da797 Fix bug when starting indexbuilder 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-12-30 16:41:24 +08:00
XuanYang-cn
3a2e2a04b6 Fix writenode bug 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2020-12-29 21:15:00 +08:00
bigsheeper
bb8da15691 Return status and add try catch exception in index builder 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2020-12-29 17:53:27 +08:00
quicksilver
b1b8747477 Add logic of incremental check for go code 
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
 2020-12-29 17:19:44 +08:00
dragondriver
243346247b Add unittest case which used in regression to index builder 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2020-12-29 16:31:03 +08:00
bigsheeper
1e3b7cd55e Fix binary vector metric type validation 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2020-12-29 16:14:52 +08:00
xige-16
be32a33eff Refactor loadIndexTest to test diffrent index params 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2020-12-29 14:43:40 +08:00
XuanYang-cn
884f66d56d Fix timesync start twice in master 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2020-12-29 14:10:22 +08:00
cai.zhang
f80536ed77 Update the logic of running go unittest 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-12-29 11:56:37 +08:00
XuanYang-cn
9577da9fce Add writenode main 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2020-12-28 18:52:34 +08:00
FluorineDog
eb8717188e Enable load index 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2020-12-28 17:55:48 +08:00
dragondriver
f67aae9596 Fix the codec bug of cgo wrapper of index builder 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2020-12-28 16:55:50 +08:00
bigsheeper
5b9e8c4f60 Add write node client segment flushing 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2020-12-28 15:25:16 +08:00
godchen
d489183798 Random generate querynode channel in unittest 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2020-12-28 15:24:20 +08:00
XuanYang-cn
aab6dede4a Add Flush for DDBuffer and insertBuffer 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2020-12-28 11:29:17 +08:00
dragondriver
11cef6e978 Add support to build binary index 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2020-12-28 10:00:02 +08:00
GuoRentong
e8ec0424d8 Update doc: service api 
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
 2020-12-27 09:05:24 +08:00
neza2017
7e8b203fe2 Add time tick in wite node 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-12-26 17:05:24 +08:00
XuanYang-cn
e316533a3a Add Flush Sync service 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2020-12-26 16:28:55 +08:00
bigsheeper
5acc9681d6 Fix search error about metric type 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2020-12-26 15:29:48 +08:00
xige-16
d599407e2b Add unittest for loadIndex service 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2020-12-26 14:16:51 +08:00
cai.zhang
2031c54746 Add unittest about index in proxy 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-12-26 11:51:07 +08:00
sunby
a56f8f7b9e Add runtime stats in master 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-12-25 18:32:03 +08:00
sunby
e21397d68a Add runtime stats unittest 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-12-25 16:08:35 +08:00
sunby
a03ab05c6c Add DescribeIndex and DescribeIndexProgress RPC 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-12-25 15:39:56 +08:00
dragondriver
84a0cf7097 Add unittest to c index wrapper 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2020-12-25 11:10:31 +08:00
XuanYang-cn
c0797333ce Add ddl binlog metaTable in writenode 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2020-12-24 20:55:40 +08:00
neza2017
4dbf4ade41 Add test for master 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-12-24 16:19:42 +08:00
zhenshan.cao
a0244b7683 Add segment expire logic for proxy 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-12-24 16:53:31 +08:00
bigsheeper
4ecdea698f Refactor query node and query serviceMain changes:1. Add ddBuffer and save binLog. 
2. Trans to insertData.
3. Change dataFields data to Data, dim to Dim4. Add float vector and binary vector.5. Deserialize data and convert to InsertData.6. Move all data into InsertData.7. Add insert buffer and hash string.
8. Add minIOkV in insertBuffer node.
9. Init write node insertBuffer maxSize from writeNode.yaml.
10. Add ddBuffer.
11. Add ddBuffer binLog and minio.
12. Add ddNode unittest.
13. Remove redundant call.
14. Increase test time.
15. Delete ddl const, use request's timestamp instead.

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2020-12-24 15:38:29 +08:00
xige-16
f5198bcaee Add LoadIndexService in query node 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2020-12-24 14:51:55 +08:00
sunby
add31188ba Add CreateIndex RPC 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-12-24 14:50:16 +08:00
neza2017
0afeaa78c8 Fix unittest of flow graph 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-12-24 14:10:00 +08:00
sunby
5d58606349 Move Status to SegIDAssignment 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-12-24 10:11:34 +08:00
cai.zhang
a8b78f11c3 Update proto about index 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-12-23 19:06:11 +08:00
FluorineDog
342f4cb741 Support dynamic chunk size 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2020-12-23 19:02:37 +08:00
XuanYang-cn
7ce0f27ebc Add buffer to minIO for binlogs 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2020-12-23 18:06:04 +08:00
sunby
e7ebfcb05a Save index meta to meta table 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-12-23 15:13:45 +08:00
godchen
e52b130ba6 Refactor data codec code 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2020-12-23 15:46:16 +08:00
sunby
0b9cffaf87 Remove index_name in IndexRequest and remove indexType in FieldIndexMeta 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-12-23 11:34:35 +08:00
zhenshan.cao
11316525e5 Fix bug of running unittest in parallel 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-12-22 22:08:03 +08:00
sunby
2aa483c922 Add index meta to meta table 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-12-22 20:16:32 +08:00
dragondriver
adfb5187cf Dock build index task with storage & indexbuilder 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2020-12-22 00:14:36 +00:00
cai.zhang
05918f55d2 Impl CreateIndex interface for proxy server 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-12-22 15:39:10 +08:00
sunby
3a1c210098 Add runtime stats to record field stats 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-12-21 20:24:40 +08:00
godchen
8f87ee632e Add field id when create collection 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2020-12-21 19:31:24 +08:00
godchen
e90ef211c2 Add flush, build index, load index scheduler 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2020-12-21 19:30:24 +08:00
cai.zhang
b582cf7b8e Add the interface of index in proto 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-12-21 19:28:54 +08:00
dragondriver
d023c1afba Refactor index wrapper, use protobuf instead of json to pass data 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2020-12-21 16:36:07 +08:00
XuanYang-cn
87521adfbd Add logic of deserilize rawData into InsertData 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2020-12-21 16:27:03 +08:00
FluorineDog
d5daa18392 Enable most DSL-related cases 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2020-12-21 14:45:00 +08:00
sunby
06620935ab Move expire timestamp to SegIDAssignment 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-12-21 10:51:37 +08:00
zhenshan.cao
9694203f8b Add writer client and metatable for writer 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-12-21 10:06:46 +08:00
sunby
f360ed7004 Add unittests 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-12-21 09:56:35 +08:00
sunby
c71cd40f68 Refactor segment manager 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-12-19 12:55:24 +08:00
FluorineDog
e65cfe1e3d Enable complex dsl parser 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2020-12-19 10:36:49 +08:00
zhenshan.cao
0d75840ed6 Fix missing file 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-12-18 18:35:03 +08:00
neza2017
0110ba6bd2 List events of binlog 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-12-12 16:52:09 +08:00
dragondriver
da9ea7d2b9 Add cgo wrapper to index builder 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2020-12-18 15:44:27 +08:00
godchen
475d36a2c9 Add data codec 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2020-12-18 15:21:25 +08:00
zhenshan.cao
3f6e926aef Refactor indexbuilder 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-12-13 06:48:05 +08:00
FluorineDog
1540cffbb1 Add executor for expression 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2020-12-12 18:17:53 +08:00
cai.zhang
54f2b79f1b Add indexbuilder client 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-12-12 17:10:42 +08:00
neza2017
07d68de2a3 Fix data codec 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-12-11 18:14:19 +08:00
bigsheeper
585d3f9831 Refactor param table, and add ddNode 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2020-12-11 17:20:14 +08:00
neza2017
8df05714a8 Add binlog unittest 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-12-11 12:01:38 +08:00
FluorineDog
e54d5ccf19 Fix push failed because git hook detects "wrong error" 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2020-12-11 14:59:13 +08:00
XuanYang-cn
d64aa37db3 Add KFlush msg type 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2020-12-11 12:01:20 +08:00
zhenshan.cao
ca34353e9b Add writernode client 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-12-11 11:38:32 +08:00
xige-16
0067e2d989 Add load index client in query node 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2020-12-11 11:29:54 +08:00
cai.zhang
62e9002348 Add git-hooks for check code 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-12-11 11:29:07 +08:00
FluorineDog
02f855359a Fix term bug and add todo note 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2020-12-10 20:13:37 +08:00
zhenshan.cao
fc054874d3 Add index_builder 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-12-10 17:55:55 +08:00
bigsheeper
762b1e1f1f Add ddService, fieldID, requestID and refactor params table, fix nil hits search result bug 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2020-12-10 16:31:09 +08:00
neza2017
9585819154 Add binlog unittest 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-12-10 15:50:09 +08:00
neza2017
9cbebc0221 Add binlog unittest 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-12-10 14:52:42 +08:00
sunby
16071d92b5 Modify metatable close segment interface. 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-12-10 13:35:09 +08:00
quicksilver
21721d2e03 Cache go mod workflow for jenkins pipeline 
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
 2020-12-10 09:57:14 +08:00
neza2017
5512ce8d8a Add binlog unittest 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-12-10 09:31:29 +08:00
XuanYang-cn
881be7f3e3 Impl minimum runable writenode and writenode flowgraph 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2020-12-09 20:07:27 +08:00
neza2017
de12fa5a10 Add binlog event 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-12-09 11:18:13 +08:00
FluorineDog
968ddc6016 Fix thrift.a not found 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2020-12-09 14:57:07 +08:00
sunby
ed54d3e26c Fix binlog reader bug 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-12-09 09:55:56 +08:00
quicksilver
d09ad77fb2 Add archiving file workflow for cache thirdparty 
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
 2020-12-08 19:53:22 +08:00
sunby
8ace1c3837 Add binlog serialization and deserialization 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-12-08 19:13:39 +08:00
XuanYang-cn
e6f726e73a Add cache for thirdparty files cache 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2020-12-08 18:51:07 +08:00
XuanYang-cn
32977e270c Add impl cgo of parquet 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2020-12-08 14:41:04 +08:00
zhenshan.cao
65ce1f97e7 Fix bug: pulsar dirty msg 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-12-07 19:41:39 +08:00
zhenshan.cao
7bbbc14637 Fix bug: address already used 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-12-07 15:22:20 +08:00
xige-16
37d7526d31 Fix search test failure 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2020-12-07 14:37:42 +08:00
neza2017
0d8273c7cc Add string type payload cgo wrapper 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-12-06 15:15:11 +08:00
neza2017
aeca8f85a5 Run cwrapper unittest in ci 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-12-05 17:39:58 +08:00
cai.zhang
806a97505c Add query node for CI progress 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-12-05 16:12:45 +08:00
neza2017
70710dee47 Add parquet payload 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-12-05 16:11:03 +08:00
FluorineDog
63c8f60c6e Enable term parser and executor 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2020-12-05 09:51:27 +08:00