Commit Graph

53 Commits

Author SHA1 Message Date
SimFG
dd9c61831d
enhance: Support to get the param value in the runtime (#29297)
/kind improvement
issue: #29299

Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-12-22 18:36:44 +08:00
yah01
1997fd150c
enhance: use local path as mmap file path if no mmap path provided (#28992)
the mmap mode isn't controlled by the config anymore, so we don't
require user to set it when they enabled mmap mode.

Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-12-21 18:12:45 +08:00
wei liu
cb0676153f
enhance: add metrics for stopping querynode balance progress (#29201)
This PR add three metrics to track the stopping balance progress.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-12-21 10:14:52 +08:00
zhagnlu
a602171d06
enhance: Refactor runtime and expr framework (#28166)
#28165

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-12-18 12:04:42 +08:00
yihao.dai
b4353ca4ce
enhance: Remove vector chunk manager (#28569)
We have implemented the chunkcache (in cpp) to retrieve vectors, hence
rendering the vectorchunkcache (in golang) obsolete.

issue: https://github.com/milvus-io/milvus/issues/28568

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-11-30 18:00:33 +08:00
SimFG
e3b7fdac61
Delay the cancellation of ctx when stopping the node (#28247)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-11-08 03:20:17 +08:00
cqy123456
4fbe3c9142
replace loaded binlog with binlog index for search performance (#27673)
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2023-11-01 02:20:15 +08:00
Filip Haltmayer
6b1a106a31
Moving etcd client into session (#27069)
Signed-off-by: Filip Haltmayer <filip.haltmayer@zilliz.com>
2023-10-27 07:36:12 +08:00
congqixia
bcbe98aba1
Add querynode client wrapper and avoid grpc in standalone mode (#27781)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-10-19 11:10:07 +08:00
jaime
ec1fe3549e
Add a stop hook to clean session (#27564)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-10-16 10:24:10 +08:00
yihao.dai
7d0dd0047d
Use path.Join when init ChunkCache path (#27433)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-09-30 17:05:27 +08:00
yihao.dai
106c17f304
Make read ahead policy in ChunkCache configurable (#27291)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-09-28 15:47:27 +08:00
congqixia
258e1ccd66
Refine querynode scheduler lifetime (#26915)
This PR refines scheduler lifetime control:
- Move private tri-state into lifetime package
- Make scheduler block incoming "Add" task
- Make scheduler Stop wait until all previously accepted task done

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-28 10:21:26 +08:00
congqixia
8c59dba329
Refine queryHook mockery (#27394)
This PR move `QueryHook` interface to `optimizers` pkg
Update all mockery generated files to latest
Add makefile entry for `QueryHook`

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-28 10:01:26 +08:00
foxspy
5db4a0489e
dynamic index version control (#27335)
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
2023-09-25 21:39:27 +08:00
SimFG
26f06dd732
Format the code (#27275)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-21 09:45:27 +08:00
yihao.dai
bb6711f28c
Add ChunkCache: support get vector from storage (#26142)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-09-15 10:21:20 +08:00
Enwei Jiao
fb0705df1b
Decouple basetable and componentparam (#26725)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-09-05 10:31:48 +08:00
congqixia
4b58c71908
Add ctx parameter for organizeTask and GetWorker method (#26835)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-05 10:05:48 +08:00
Gao
7e36f819b4
Make autoindex config clean (#26732)
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2023-08-31 10:25:01 +08:00
wei liu
7af0f7d90c
avoid concurrent sub/unsub on same channel (#26454)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-08-23 10:10:22 +08:00
cqy123456
fd37860e57
init knowhere build/search thread pool; (#26449)
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2023-08-18 14:00:19 +08:00
yihao.dai
63b86b32a6
Add server id validation interceptor (#26395)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-08-17 20:20:20 +08:00
Enwei Jiao
7d61355ab0
Refactor log for Query (#26310)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-08-14 18:57:32 +08:00
xige-16
1971d98897
Add disk metric info (#25675)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-08-11 10:35:42 +08:00
yah01
300fef446b
Enable mmap for vector index (#25877)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-08-10 13:59:15 +08:00
MrPresent-Han
5634ba777d
add new threadpool with various priority to avoid deadlock(#25781) (#26028)
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-08-03 09:31:07 +08:00
zhagnlu
833674c1cb
add glog configurable function and redirect aws log to segcore log (#25664)
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-07-27 19:49:02 +08:00
yah01
361db51337
Create dir if mmap enabled for validating path (#25750)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-07-19 18:48:58 +08:00
congqixia
f07c934558
Fix querynodev2 grace stop condition reversed (#25409)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-07-07 17:28:25 +08:00
yah01
4c0e36d28c
Stop blocking until all channels & segments released (#25328)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-07-06 10:04:25 +08:00
chyezh
d7d61f529c
[Feature|Pick] enable scheduler policy and add user-task-polling policy (#24839)
Signed-off-by: chyezh <ye.zhen@zilliz.com>
2023-07-03 18:24:25 +08:00
Gao
de6e4817a2
Support dynamic tuning config (#25152)
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2023-07-03 15:18:24 +08:00
yah01
7b5013664d
Support graceful stop (#25226)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-06-30 17:28:23 +08:00
xige-16
33c2012675
Add more metrics (#25081)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-06-26 17:52:44 +08:00
xige-16
04082b3de2
Migrate the ability to upload and download binlog to cpp (#22984)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-06-25 14:38:44 +08:00
congqixia
41af0a98fa
Use go-api/v2 for milvus-proto (#24770)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-09 01:28:37 +08:00
congqixia
54306a10b3
Fix confusing log while plugin is not enabled (#24741)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-08 15:40:35 +08:00
yiwangdr
37c02c9927
clean up meta interface (#24729)
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
2023-06-08 14:18:35 +08:00
xige-16
fe18109aab
Clean tmp disk data when start milvus (#24404)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-05-26 15:07:26 +08:00
congqixia
eb27a8a8c0
Fix integration test not wait for index built (#24037)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-05-12 09:51:25 +08:00
yah01
400364483d
Improve predict the resource usage (#23968)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-05-11 15:33:24 +08:00
foxspy
6f4ed517de
add growing segment index (#23615)
Signed-off-by: xianliang <xianliang.li@zilliz.com>
2023-04-26 10:14:41 +08:00
congqixia
4fe363c4b2
Add KNN cgo pool (#23526)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-04-19 23:56:31 +08:00
congqixia
eb690ef033
Fix querynodev2 stop logic (#23487)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-04-18 17:56:30 +08:00
cqy123456
7ff6a3a246
Set query node thread pool size in milvus.yaml (#23286)
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2023-04-17 10:44:29 +08:00
yah01
dc0eb433ae
Add IoPoolSize config (#23360)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-04-13 14:50:28 +08:00
cai.zhang
43a9e175a3
Exit component process when session key is deleted (#21658) (#22164)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-04-12 20:12:28 +08:00
jaime
c9d0c157ec
Move some modules from internal to public package (#22572)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-04-06 19:14:32 +08:00
congqixia
8db24e0a83
Support queryHook in querynodev2 (#23140)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-04-06 14:32:29 +08:00