Commit Graph

857 Commits

Author SHA1 Message Date
xige-16
158787811e
Move assemble/disassemble func to core (#19420)
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-10-16 21:17:25 +08:00
SimFG
a55f739608
Separate public proto files (#19782)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-10-16 20:49:27 +08:00
xige-16
a1db9038fb
Move disk index params to config file (#19714)
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-10-14 17:51:24 +08:00
xige-16
8c9c1672ae
Assign different storage config for indexes (#19517)
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-10-14 14:45:23 +08:00
congqixia
089912baa7
Fix segcore unit test name (#19783)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-10-14 14:29:24 +08:00
congqixia
a20d83bccd
Assemble RAW_DATA files before storing into index.raw_data (#19771)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-10-14 12:03:24 +08:00
mumon
b48c37cb44
Fix typo in core (#19661)
Signed-off-by: Ziyu Wang <15871035978@163.com>

Signed-off-by: Ziyu Wang <15871035978@163.com>
2022-10-12 17:05:23 +08:00
jaime
31db01b6ae
Support TTL for collection level (#19383)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-10-10 20:31:22 +08:00
wayblink
85e04d845c
Move GetComponentStates to milvus.proto, make it an external interface (#19594)
Signed-off-by: wayblink <anyang.wang@zilliz.com>

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-10-10 15:55:22 +08:00
xige-16
49daee2e17
Fix lost disk index data (#19636)
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-10-09 14:36:58 +08:00
congqixia
8c1c8000ac
Add TargetID in MsgBase and querynode v2 API check (#19627)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-10-08 20:26:57 +08:00
liliu-z
faa9d2176a
Upgrade Knowhere to 1.3.2 to support large dim in DiskANN (#19531)
Signed-off-by: Li Liu <liliu@LideMacBook-Pro.local>

Signed-off-by: Li Liu <liliu@LideMacBook-Pro.local>
Co-authored-by: Li Liu <liliu@LideMacBook-Pro.local>
2022-09-29 19:40:55 +08:00
aoiasd
a8cabbfaa9
Fix offset in search should not equal to insert barrier (#19522)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2022-09-28 21:04:54 +08:00
Ten Thousand Leaves
b30c9d4f60
Support Bulk Load in Milvus 2.2 (#18982)
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-09-26 18:06:54 +08:00
Cai Yudong
87d78a4a85
Ignore cases when comparing metric type in segcore (#19437)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-09-26 17:58:52 +08:00
aoiasd
2b58bd5c0a
Optimize large memory usage of InsertRecord by using vector instead of unordered_map if InsertRecord used in sealed segment (#19245)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2022-09-23 17:08:51 +08:00
xige-16
d4bc00423c
Fix start milvus failed on macos (#19394)
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-09-23 16:54:50 +08:00
SimFG
68a257458b
Implement the hook interceptor (#19294)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-09-23 09:50:52 +08:00
xige-16
428840178c
Support diskann index for vector field (#19093)
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-09-21 20:16:51 +08:00
Jenny Li
d8b11121a8
Decrease build time by removing compiling openblas (#19207)
Signed-off-by: Jenny Li <jing.li@zilliz.com>

Signed-off-by: Jenny Li <jing.li@zilliz.com>
2022-09-16 17:46:47 +08:00
SimFG
d7f38a803d
Separate some proto files (#19218)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-09-16 16:56:49 +08:00
aoiasd
58d3a49b62
Optimize large memory usage of Variant used in InsertRecord (#19197)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2022-09-16 10:52:47 +08:00
bigsheeper
5cdd6ca251
Add quotaCenter, rateLimiter and rateCollector (#18636)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-09-16 09:56:47 +08:00
Bingyi Sun
626854cf0c
Refactor QueryCoord (#18836)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Wei Liu <wei.liu@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Wei Liu <wei.liu@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
2022-09-15 18:48:32 +08:00
xige-16
4de1bfe5bc
Add cpp data codec (#18538)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Co-authored-by: zhagnlu lu.zhang@zilliz.com

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-09-09 22:12:34 +08:00
Cai Yudong
7b5eb85205
Remove redundant parameter for segcore Search() (#19025)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-09-06 12:05:11 +08:00
Cai Yudong
765907ab77
Optimize segcore Reduce (#18902)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-09-06 10:55:12 +08:00
Jiquan Long
a5e2d6b6fb
Refactor RootCoord (#18930)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
Co-authored-by: xaxys <tpnnghd@163.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
Co-authored-by: xaxys <tpnnghd@163.com>
2022-09-05 13:29:11 +08:00
Cai Yudong
686b0ce796
Upgrade to knowhere-v1.3.0, remove following index support: (#18935)
- IVF_SQ8H
- RHNSW_FLAT/RHNSW_PQ/RHNSW_SQ
- NGT
- NSG
- SPTAG

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-09-05 10:41:11 +08:00
Cai Yudong
da96659569
[skip e2e] Add more testcases with different parameter combinations in test_reduce (#18967)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-09-01 16:59:02 +08:00
Cai Yudong
4ded45394a
[skip e2e] Optimize test_reduce (#18957)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-09-01 11:09:00 +08:00
Cai Yudong
9dc3bbecbd
Change the type of slice_nqs and slice_topks from int32_t[] to int64_t[] (#18867)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-08-29 11:36:56 +08:00
Letian Jiang
ef7742507d
Remove redundant code (#18832)
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>

Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-08-25 19:48:54 +08:00
Cai Yudong
305601ad25
Add API SearchOnSealedChunk() to search on sealed segment without index (#18830)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-08-25 18:16:55 +08:00
Cai Yudong
dcf45df029
Optimize API vector_search parameter in segcore (#18827)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-08-25 16:16:54 +08:00
cai.zhang
c924f73105
Refactor for IndexCoord to support cloud (#18643)
Co-authored-by: Zach41 <zongmei.zhang@zilliz.com>
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: Zach41 <zongmei.zhang@zilliz.com>
2022-08-25 15:48:54 +08:00
Ji Bin
f3c755602b
Adaptation for latest MinGW64 (#18699)
Signed-off-by: Ji Bin <matrixji@live.com>

Signed-off-by: Ji Bin <matrixji@live.com>
2022-08-25 10:56:53 +08:00
Cai Yudong
94ffa8a275
Upgrade to knowhere-v1.2.0 (#18746)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-08-23 10:20:58 +08:00
Letian Jiang
c6d9d0df5e
Fix segcore benchmark (#18675)
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>

Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-08-17 20:24:49 +08:00
congqixia
0db281c4ff
Fix DataType::bool cast to double (#18690)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-08-17 16:14:49 +08:00
Cai Yudong
d878cc2ec0
Rename real_topK_per_nq_ to topks_per_nq_prefix_sum_ in SearchResult struct (#18634)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-08-16 00:32:48 +08:00
xige-16
111c608513
Add script to auto clang format (#18559)
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-08-11 11:04:37 +08:00
xige-16
dbb96a8b70
Fix pk2offset occupy too much memory (#18325)
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-08-11 10:48:38 +08:00
SimFG
c267ad612d
Support Role-Based Access Control II (#18528)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-08-05 16:28:35 +08:00
SimFG
ff0200210a
Support Role-Based Access Control (#18425)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-08-04 11:04:34 +08:00
Cai Yudong
7cd37fc6dd
Merge FloatSearch() and BinarySearch() into SearchOnGrowing() (#18498)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-08-03 11:56:36 +08:00
Enwei Jiao
283f5731d2
config from etcd (#18421)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-08-01 10:04:33 +08:00
zhenshan.cao
a287a2b3fd
Return empty result in advance if all data filtered out (#18329) (#18438)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-07-28 12:36:30 +08:00
bigsheeper
cef8b1e7cc
Enable jemalloc (#18349)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-07-20 22:22:31 +08:00
Ten Thousand Leaves
0ebe407221
Make DataNode not panic if stale segment is not found during flush (#18322)
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-07-20 15:06:30 +08:00