Commit Graph

192 Commits

Author SHA1 Message Date
congqixia
89fc9aad82
Improve sync target version logic (#26630)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-08-29 23:12:27 +08:00
congqixia
1cf6e00fa6
Improve segment manager interface (#26637)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-08-29 15:46:27 +08:00
wei liu
7af0f7d90c
avoid concurrent sub/unsub on same channel (#26454)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-08-23 10:10:22 +08:00
Xiaofan
7b3a1dc93f
Fix load index panic (#26457)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-08-21 09:56:20 +08:00
wei liu
3b11ad2695
fix dead lock in concurrent search/release segment (#26435)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-08-18 15:28:18 +08:00
cqy123456
fd37860e57
init knowhere build/search thread pool; (#26449)
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2023-08-18 14:00:19 +08:00
yihao.dai
63b86b32a6
Add server id validation interceptor (#26395)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-08-17 20:20:20 +08:00
congqixia
123ad921e0
Fix querynodev2 search/query segments return unexpected grpc err (#26341)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-08-17 18:24:18 +08:00
Xiaofan
ba806429fa
Remove frequent log on search growing segment (#26315)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-08-17 15:44:53 +08:00
congqixia
855a04539a
Set delegator lastUpdateTs to startTs (#26416)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-08-17 12:24:18 +08:00
yah01
5dae6a654a
Protect segment from being released while query/search (#26322)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-08-16 18:38:17 +08:00
yah01
3d8871b30e
Fix data race while updating segment version (#26387)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-08-16 16:30:18 +08:00
MrPresent-Han
d30a920226
add log trace for segcore(#26277) (#26339)
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-08-16 11:41:33 +08:00
wei liu
23baecd70f
set sealed segment to unreadable before sync target version (#26338)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-08-15 17:27:35 +08:00
congqixia
ec65a4e048
Use float64 for resource estimation log (#26335)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-08-15 14:25:33 +08:00
Enwei Jiao
7d61355ab0
Refactor log for Query (#26310)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-08-14 18:57:32 +08:00
yah01
a173486d2e
Fix calculation of memory usage prediction for mmap mode (#26264)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-08-12 17:19:31 +08:00
Gao
9ca8f28005
Fix missing totalChannelNum bug (#26295)
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2023-08-11 18:35:29 +08:00
yah01
48422dd4c5
Fix spawn too many threads (#26293)
- Low the thread pool cap
- Limit CGO calls concurrency

Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-08-11 18:29:29 +08:00
Gao
b6fcbb0998
Support ScaNN index (#26099)
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2023-08-11 14:21:29 +08:00
wei liu
b47a72bfcf
fix set dirty segment distribution to leader view (#26180)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-08-11 11:21:32 +08:00
xige-16
1971d98897
Add disk metric info (#25675)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-08-11 10:35:42 +08:00
Enwei Jiao
ca1349708b
Remove time travel ralted testcase (#26119)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-08-10 18:53:17 +08:00
xige-16
1055c90456
Add default retrieve limit (#24782)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-08-10 14:11:15 +08:00
yah01
300fef446b
Enable mmap for vector index (#25877)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-08-10 13:59:15 +08:00
yah01
889424b3f9
Fix load index with empty file list (#26236)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-08-09 18:39:16 +08:00
congqixia
eea9197306
Perform alter distribution under mutex protection (#26229)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-08-09 13:05:15 +08:00
wei liu
f33a89387f
enable config shard level cost metrics whether contains worker's cost (#26132)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-08-09 11:37:15 +08:00
yah01
99c633954a
Downgrade the log level in search path (#26188)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-08-08 18:35:14 +08:00
Bingyi Sun
54c0e64059
Fix search on empty segments set bug (#26136)
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2023-08-08 11:17:08 +08:00
MrPresent-Han
68ecf49ed5
fix lost parameter for threadCoefficient(#25781) (#26109)
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-08-03 12:05:06 +08:00
yah01
d9074bcb06
Remove unused code (#26100)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-08-03 09:45:05 +08:00
MrPresent-Han
5634ba777d
add new threadpool with various priority to avoid deadlock(#25781) (#26028)
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-08-03 09:31:07 +08:00
smellthemoon
3fc18c53a3
Fix wrong log (#26022)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-08-02 11:01:07 +08:00
yah01
9c55a7f422
Add worker num as one of load resource (#26045)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-08-01 21:47:06 +08:00
Jiquan Long
5c1f79dc54
Push down the limit operator to segcore (#25959)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-08-01 20:29:05 +08:00
aoiasd
7fec0d61cc
Refine test chunkmanager of querynode v2 (#25888)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-08-01 10:33:04 +08:00
congqixia
8b5e276361
Use different cgo pool for SQ and other operations (#26021)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-08-01 09:19:05 +08:00
yah01
39b00b97a6
Add more logs for committed resources (#26026)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-07-31 19:27:03 +08:00
yah01
8245e078c0
Add max segment size log (#26015)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-07-31 14:37:04 +08:00
zhagnlu
833674c1cb
add glog configurable function and redirect aws log to segcore log (#25664)
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-07-27 19:49:02 +08:00
congqixia
5fbe6e99fd
Remove debug log from segment loader (#25937)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-07-26 18:51:01 +08:00
wei liu
302897f866
refine look aside balance logic (#25837)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-07-25 18:51:01 +08:00
yah01
3735a097e6
Refine the memeory usage prediction (#25889)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-07-25 17:41:01 +08:00
congqixia
0bc03ede0d
Add eventlog pkg and support grpc streaming event observation (#25812)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-07-25 17:23:01 +08:00
wei liu
a132122370
fix time unit in insert/delete cost metrics value (#25893)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-07-25 15:07:01 +08:00
aoiasd
b533c68632
Forbid update checkpoint without msgID (#25694)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-07-25 10:43:04 +08:00
wei liu
0e26486abc
fix segment num in log (#25876)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-07-25 09:13:00 +08:00
yah01
168f346620
Remove deprecated interface (#25864)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-07-24 17:47:00 +08:00
wei liu
fc19b85a40
fix count(*)retrieve redundant growing segment (#25825)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-07-24 14:09:00 +08:00