Commit Graph

205 Commits

Author SHA1 Message Date
congqixia
af5c01082b
Refine delegator lifetime control (#26881)
- Add SafeChan interface in lifetime package
- Embed SafeChan into  interface
- Replace private lifetime struct in delegator package with
- Refine delegator on-going task lifetime control and wait all accepted task done
- Fix potential goroutine leakage from  if delegator closed concurrently

/kind improvement

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-07 10:11:15 +08:00
SimFG
28681276e2
Improve the retry of the rpc client (#26795)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-06 17:43:14 +08:00
MrPresent-Han
528948559f
fix false load failure for long unserviable period(#26813) (#26818)
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-09-06 12:57:15 +08:00
Enwei Jiao
fb0705df1b
Decouple basetable and componentparam (#26725)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-09-05 10:31:48 +08:00
congqixia
4b58c71908
Add ctx parameter for organizeTask and GetWorker method (#26835)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-05 10:05:48 +08:00
congqixia
1a8cf5c415
Organize all mockery generation commands in Makefile (#26826)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-04 21:19:48 +08:00
yah01
3349db4aa7
Refine errors to remove changes breaking design (#26521)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-04 09:57:09 +08:00
congqixia
e8f1b1736e
Remove log.Error(err.error())-style log (#26783)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-01 13:09:01 +08:00
XuanYang-cn
ef75784715
Fix LoadSegmentLatency metric p99 (#26761)
See also: #26743

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-09-01 11:07:07 +08:00
MrPresent-Han
8330c18dc9
add log for loading segment(#26564) (#26640)
/kind improvement

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-08-31 12:03:00 +08:00
Gao
7e36f819b4
Make autoindex config clean (#26732)
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2023-08-31 10:25:01 +08:00
yah01
bfcc691129
Fix segment leaked if task canceled (#26685)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-08-30 14:17:03 +08:00
congqixia
95dcf7fa06
Fix memory leak when Put duplicated segments (#26693)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-08-30 11:37:00 +08:00
congqixia
89fc9aad82
Improve sync target version logic (#26630)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-08-29 23:12:27 +08:00
congqixia
1cf6e00fa6
Improve segment manager interface (#26637)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-08-29 15:46:27 +08:00
wei liu
7af0f7d90c
avoid concurrent sub/unsub on same channel (#26454)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-08-23 10:10:22 +08:00
Xiaofan
7b3a1dc93f
Fix load index panic (#26457)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-08-21 09:56:20 +08:00
wei liu
3b11ad2695
fix dead lock in concurrent search/release segment (#26435)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-08-18 15:28:18 +08:00
cqy123456
fd37860e57
init knowhere build/search thread pool; (#26449)
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2023-08-18 14:00:19 +08:00
yihao.dai
63b86b32a6
Add server id validation interceptor (#26395)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-08-17 20:20:20 +08:00
congqixia
123ad921e0
Fix querynodev2 search/query segments return unexpected grpc err (#26341)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-08-17 18:24:18 +08:00
Xiaofan
ba806429fa
Remove frequent log on search growing segment (#26315)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-08-17 15:44:53 +08:00
congqixia
855a04539a
Set delegator lastUpdateTs to startTs (#26416)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-08-17 12:24:18 +08:00
yah01
5dae6a654a
Protect segment from being released while query/search (#26322)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-08-16 18:38:17 +08:00
yah01
3d8871b30e
Fix data race while updating segment version (#26387)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-08-16 16:30:18 +08:00
MrPresent-Han
d30a920226
add log trace for segcore(#26277) (#26339)
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-08-16 11:41:33 +08:00
wei liu
23baecd70f
set sealed segment to unreadable before sync target version (#26338)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-08-15 17:27:35 +08:00
congqixia
ec65a4e048
Use float64 for resource estimation log (#26335)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-08-15 14:25:33 +08:00
Enwei Jiao
7d61355ab0
Refactor log for Query (#26310)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-08-14 18:57:32 +08:00
yah01
a173486d2e
Fix calculation of memory usage prediction for mmap mode (#26264)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-08-12 17:19:31 +08:00
Gao
9ca8f28005
Fix missing totalChannelNum bug (#26295)
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2023-08-11 18:35:29 +08:00
yah01
48422dd4c5
Fix spawn too many threads (#26293)
- Low the thread pool cap
- Limit CGO calls concurrency

Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-08-11 18:29:29 +08:00
Gao
b6fcbb0998
Support ScaNN index (#26099)
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2023-08-11 14:21:29 +08:00
wei liu
b47a72bfcf
fix set dirty segment distribution to leader view (#26180)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-08-11 11:21:32 +08:00
xige-16
1971d98897
Add disk metric info (#25675)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-08-11 10:35:42 +08:00
Enwei Jiao
ca1349708b
Remove time travel ralted testcase (#26119)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-08-10 18:53:17 +08:00
xige-16
1055c90456
Add default retrieve limit (#24782)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-08-10 14:11:15 +08:00
yah01
300fef446b
Enable mmap for vector index (#25877)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-08-10 13:59:15 +08:00
yah01
889424b3f9
Fix load index with empty file list (#26236)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-08-09 18:39:16 +08:00
congqixia
eea9197306
Perform alter distribution under mutex protection (#26229)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-08-09 13:05:15 +08:00
wei liu
f33a89387f
enable config shard level cost metrics whether contains worker's cost (#26132)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-08-09 11:37:15 +08:00
yah01
99c633954a
Downgrade the log level in search path (#26188)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-08-08 18:35:14 +08:00
Bingyi Sun
54c0e64059
Fix search on empty segments set bug (#26136)
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2023-08-08 11:17:08 +08:00
MrPresent-Han
68ecf49ed5
fix lost parameter for threadCoefficient(#25781) (#26109)
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-08-03 12:05:06 +08:00
yah01
d9074bcb06
Remove unused code (#26100)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-08-03 09:45:05 +08:00
MrPresent-Han
5634ba777d
add new threadpool with various priority to avoid deadlock(#25781) (#26028)
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-08-03 09:31:07 +08:00
smellthemoon
3fc18c53a3
Fix wrong log (#26022)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-08-02 11:01:07 +08:00
yah01
9c55a7f422
Add worker num as one of load resource (#26045)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-08-01 21:47:06 +08:00
Jiquan Long
5c1f79dc54
Push down the limit operator to segcore (#25959)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-08-01 20:29:05 +08:00
aoiasd
7fec0d61cc
Refine test chunkmanager of querynode v2 (#25888)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-08-01 10:33:04 +08:00