test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-03 20:39:36 +08:00
Code Issues Actions 14 Packages Projects Releases Wiki Activity
6,126 Commits 96 Branches 105 Tags 185 MiB
f02bd8c8f5
Commit Graph

196 Commits

Author SHA1 Message Date
XuanYang-cn
3a2e2a04b6 Fix writenode bug 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2020-12-29 21:15:00 +08:00
XuanYang-cn
884f66d56d Fix timesync start twice in master 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2020-12-29 14:10:22 +08:00
godchen
d489183798 Random generate querynode channel in unittest 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2020-12-28 15:24:20 +08:00
sunby
a56f8f7b9e Add runtime stats in master 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-12-25 18:32:03 +08:00
sunby
e21397d68a Add runtime stats unittest 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-12-25 16:08:35 +08:00
sunby
a03ab05c6c Add DescribeIndex and DescribeIndexProgress RPC 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-12-25 15:39:56 +08:00
XuanYang-cn
c0797333ce Add ddl binlog metaTable in writenode 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2020-12-24 20:55:40 +08:00
neza2017
4dbf4ade41 Add test for master 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-12-24 16:19:42 +08:00
zhenshan.cao
a0244b7683 Add segment expire logic for proxy 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-12-24 16:53:31 +08:00
bigsheeper
4ecdea698f Refactor query node and query serviceMain changes:1. Add ddBuffer and save binLog. 
2. Trans to insertData.
3. Change dataFields data to Data, dim to Dim4. Add float vector and binary vector.5. Deserialize data and convert to InsertData.6. Move all data into InsertData.7. Add insert buffer and hash string.
8. Add minIOkV in insertBuffer node.
9. Init write node insertBuffer maxSize from writeNode.yaml.
10. Add ddBuffer.
11. Add ddBuffer binLog and minio.
12. Add ddNode unittest.
13. Remove redundant call.
14. Increase test time.
15. Delete ddl const, use request's timestamp instead.

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2020-12-24 15:38:29 +08:00
xige-16
f5198bcaee Add LoadIndexService in query node 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2020-12-24 14:51:55 +08:00
sunby
add31188ba Add CreateIndex RPC 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-12-24 14:50:16 +08:00
sunby
5d58606349 Move Status to SegIDAssignment 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-12-24 10:11:34 +08:00
sunby
e7ebfcb05a Save index meta to meta table 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-12-23 15:13:45 +08:00
sunby
0b9cffaf87 Remove index_name in IndexRequest and remove indexType in FieldIndexMeta 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-12-23 11:34:35 +08:00
sunby
2aa483c922 Add index meta to meta table 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-12-22 20:16:32 +08:00
sunby
3a1c210098 Add runtime stats to record field stats 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-12-21 20:24:40 +08:00
godchen
8f87ee632e Add field id when create collection 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2020-12-21 19:31:24 +08:00
godchen
e90ef211c2 Add flush, build index, load index scheduler 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2020-12-21 19:30:24 +08:00
cai.zhang
b582cf7b8e Add the interface of index in proto 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-12-21 19:28:54 +08:00
sunby
06620935ab Move expire timestamp to SegIDAssignment 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-12-21 10:51:37 +08:00
sunby
f360ed7004 Add unittests 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-12-21 09:56:35 +08:00
sunby
c71cd40f68 Refactor segment manager 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-12-19 12:55:24 +08:00
godchen
475d36a2c9 Add data codec 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2020-12-18 15:21:25 +08:00
XuanYang-cn
d64aa37db3 Add KFlush msg type 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2020-12-11 12:01:20 +08:00
xige-16
0067e2d989 Add load index client in query node 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2020-12-11 11:29:54 +08:00
bigsheeper
762b1e1f1f Add ddService, fieldID, requestID and refactor params table, fix nil hits search result bug 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2020-12-10 16:31:09 +08:00
sunby
16071d92b5 Modify metatable close segment interface. 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-12-10 13:35:09 +08:00
XuanYang-cn
32977e270c Add impl cgo of parquet 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2020-12-08 14:41:04 +08:00
zhenshan.cao
65ce1f97e7 Fix bug: pulsar dirty msg 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-12-07 19:41:39 +08:00
zhenshan.cao
7bbbc14637 Fix bug: address already used 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-12-07 15:22:20 +08:00
xige-16
37d7526d31 Fix search test failure 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2020-12-07 14:37:42 +08:00
neza2017
aeca8f85a5 Run cwrapper unittest in ci 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-12-05 17:39:58 +08:00
cai.zhang
806a97505c Add query node for CI progress 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-12-05 16:12:45 +08:00
neza2017
70710dee47 Add parquet payload 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-12-05 16:11:03 +08:00
FluorineDog
6412ebc0d4 Add support of metric type in schema, enable binary vector, fix segfault 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2020-12-05 06:46:01 +08:00
zhenshan.cao
0b6fe0a8cf Refactor paramtable 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-12-04 14:18:17 +08:00
godchen
7ab5b5d80d Add MinIO kv implements 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2020-12-03 19:00:11 +08:00
neza2017
cec903da19 Add parquet payload 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-12-02 18:31:56 +08:00
zhenshan.cao
23aebe7e2a Fix proxy unittest 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-12-01 19:53:53 +08:00
neza2017
17cc7bc337 Use MetaRootPath instead 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-12-01 17:03:39 +08:00
neza2017
d3858500c6 Add static check 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-11-30 19:38:23 +08:00
quicksilver
86573d0053 Create Jenkins pipeline 
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
 2020-11-30 10:07:04 +08:00
FluorineDog
77fa75b1ec Add binary insert and warper of binary search, rename vector 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2020-11-30 05:18:44 +08:00
cai.zhang
85a544b79b Add dockerfile for each component 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-11-28 19:06:48 +08:00
cai.zhang
115d0732af Update docker-compose file for unittest 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-11-28 16:43:39 +08:00
dragondriver
eb6c0e0a28 Fix the bug when reduce the multiple search result from query nodes 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2020-11-28 15:57:49 +08:00
godchen
92bf09c7e6 Add ddstream and scheduler implements 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2020-11-28 15:26:39 +08:00
FluorineDog
c9fb34142c Enable primary_key switch 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2020-11-28 09:16:00 +08:00
dragondriver
235e723eb6 Read proxy id from param table instead of hardcode 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2020-11-27 11:17:12 +08:00
XuanYang-cn
3e605ae25f Rename reader to querynode 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2020-11-27 10:56:53 +08:00
bigsheeper
1decc1a4ca Add metaRootPath and kvRootPath 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2020-11-27 09:59:40 +08:00
neza2017
c507abdeaa Fix start time sync 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-11-26 16:51:44 +08:00
dragondriver
72a7af3f4a Add time tick logic to Proxy 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2020-11-26 17:47:46 +08:00
neza2017
32445dd02b Add cpu profile 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-11-26 16:01:31 +08:00
cai.zhang
57da66ba72 Update read config logic in docker 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-11-26 15:18:36 +08:00
zhenshan.cao
045fc3d808 Fix segment fault 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-11-26 14:29:01 +08:00
XuanYang-cn
0a16a9a651 Add logic of getting milvus distributed config from master by grpc 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2020-11-26 13:21:06 +08:00
dragondriver
8ce0deca4e Add default partition tag to InsertTask in Proxy 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2020-11-26 11:38:04 +08:00
xige-16
ae78aa1ac1 Warp cpp reduce and repack searchResultMsg to specified pulsar channel 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2020-11-26 11:12:33 +08:00
neza2017
ae26a81550 Change to use Marshal as text 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-11-26 10:13:23 +08:00
sunby
f4c643f1bd Add number limitation of partition and field. And Change default partition tag to "_default" 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-11-25 18:39:05 +08:00
dragondriver
a7b3efecd7 Read query node num from param table instead of hardcode 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2020-11-25 16:17:33 +08:00
cai.zhang
4bcb460c98 Add clang-format 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-11-25 15:25:00 +08:00
sunby
24b29bec30 Add collectionName/partitionTag/fieldName validation. 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-11-25 14:44:30 +08:00
bigsheeper
84f3d974fa Delete pulsar address test and refactor master param table 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2020-11-23 18:04:09 +08:00
FluorineDog
76001b29b6 Support Range Query 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2020-11-23 11:56:45 +08:00
sunby
578e952c60 Add segment threshold factor 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-11-23 10:20:30 +08:00
sunby
2ebf3b55e4 Modify pulsar address in unittest 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-11-21 18:36:14 +08:00
sunby
3afab6ecb9 Add unittest and fix a bug in segment manager 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-11-21 17:11:05 +08:00
dragondriver
e36b565ec0 Normalize the naming of variable and function in Proxy 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2020-11-21 16:31:38 +08:00
FluorineDog
9f35dee2df Support range executor 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2020-11-21 15:56:13 +08:00
neza2017
73e3ff09f5 Refactor variable name in Master 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-11-21 15:53:01 +08:00
XuanYang-cn
03c996d71a Fix timesync bug 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2020-11-21 15:25:23 +08:00
cai.zhang
17c3a70d64 update ParameterTable 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-11-21 15:06:46 +08:00
neza2017
2514e8b8a9 Optimize code in Master module 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-11-20 17:46:49 +08:00
bigsheeper
dd3654f953 Fix query node param table 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2020-11-20 17:10:24 +08:00
quicksilver
1728a4f419 Update .devcontainer.json 
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
 2020-11-20 15:02:26 +08:00
neza2017
cda47a9252 Remove unused code 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-11-20 09:13:29 +08:00
zhenshan.cao
50d59d91ee Add segmentIDAssigner for proxy 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-11-19 21:02:31 +08:00
neza2017
ce89f64bb3 Set master config 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-11-19 10:46:17 +08:00
neza2017
c3de667bf5 Implement segment management in master 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-11-18 19:46:18 +08:00
bigsheeper
99ea7e9f05 Update param table 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2020-11-18 19:33:56 +08:00
sunby
9e4ce8454e Implement segment management in master 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-11-18 15:04:17 +08:00
dragondriver
a18ac17a98 Increase timeout interval of task in Proxy 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2020-11-18 15:44:17 +08:00
neza2017
544c701f98 Start timeSyncMsgProducer in master 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-11-18 10:14:11 +08:00
dragondriver
b32e55d5f0 Add timeout logic to task in Proxy 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2020-11-17 20:00:23 +08:00
quicksilver
72ed65d7f8 Check builder images 
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
 2020-11-17 18:36:03 +08:00
godchen
4f04be4a28 Add master dd task unittest 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2020-11-17 17:17:10 +08:00
xige-16
4e8efe85df Delete wrong useage of *TsMsg 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2020-11-17 14:10:07 +08:00
cai.zhang
5bcabffdaf Replace conf struct with ParamsTable 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-11-16 21:10:43 +08:00
dragondriver
aff1e79f82 Fix SearchTask in Proxy 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2020-11-16 17:01:10 +08:00
FluorineDog
8c48cf30c0 Format and and extra parameters for checking 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2020-11-16 15:41:56 +08:00
neza2017
9d3a21a9c9 Rewrite NewTSOKVBase 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-11-16 12:37:46 +08:00
zhenshan.cao
9f948d3095 Remove mockmaster 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-11-14 16:40:10 +08:00
quicksilver
00b4b67a1f Update the version of build environment 
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
 2020-11-14 16:01:12 +08:00
neza2017
e349508e78 Fix list-collections 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-11-14 15:57:39 +08:00
XuanYang-cn
fd6d0c78e5 Refactor master scheduler: show_collections 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2020-11-14 15:37:07 +08:00
zhenshan.cao
e94138816f Add tsLoop for master 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-11-14 15:26:14 +08:00
neza2017
ce969e9568 Update segment 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-11-14 14:39:52 +08:00