Commit Graph

314 Commits

Author SHA1 Message Date
MrPresent-Han
f4e72cb170
remove sync segmentLastExpire every time when assigning(#25271) (#25316) (#25557)
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-07-24 14:11:07 +08:00
Bingyi Sun
9f31fc9a31
Add broker timeout config item. (#25855)
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2023-07-24 14:07:00 +08:00
cai.zhang
b15e34db21
Add contraint for compaction based indexed segments (#25709)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-07-23 21:31:00 +08:00
aoiasd
3545b1a608
Refine rocksmq (#25031)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-07-18 20:18:57 +08:00
congqixia
8d343bf75a
Make compaction rpc timeout and parallel maxium configurable (#25672)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-07-18 14:25:20 +08:00
Xiaofan
dbf0130803
Faster garbage collect on compacted data (#25088)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-07-12 10:14:29 +08:00
yah01
8b06941da3
Reduce the load memory usage predict factor to 2 (#25469)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-07-11 13:06:28 +08:00
smellthemoon
d63323d117
Add rate limit and deny write in upsert (#25351)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-07-11 11:20:34 +08:00
wei liu
342cfcad46
decrease default value of group max nq (#25380)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-07-07 15:40:25 +08:00
congqixia
efdd71c640
Make cgo pool size larger than worker pool size (#25318)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-07-05 16:56:26 +08:00
foxspy
31173727b2
growing segment index memory opt & get vector bugfix (#25272)
Signed-off-by: xianliang <xianliang.li@zilliz.com>
2023-07-05 00:04:25 +08:00
chyezh
d7d61f529c
[Feature|Pick] enable scheduler policy and add user-task-polling policy (#24839)
Signed-off-by: chyezh <ye.zhen@zilliz.com>
2023-07-03 18:24:25 +08:00
smellthemoon
b30517d303
Enlarge timeout to prevent health check failure (#25173)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-06-29 17:10:23 +08:00
jaime
18df2ba6fd
[Cherry-Pick] Support Database (#24769)
Support Database(#23742)
Fix db nonexists error for FlushAll (#24222)
Fix check collection limits fails (#24235)
backward compatibility with empty DB name (#24317)
Fix GetFlushAllState with DB (#24347)
Remove db from global meta cache after drop database (#24474)
Fix db name is empty for describe collection response (#24603)
Add RBAC for Database API (#24653)
Fix miss load the same name collection during recover stage (#24941)

RBAC supports Database validation (#23609)
Fix to list grant with db return empty (#23922)
Optimize PrivilegeAll permission check (#23972)
Add the default db value for the rbac request (#24307)

Signed-off-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
2023-06-25 17:20:43 +08:00
Xiaofan
72c5e2a41a
Fix channel reassigned to other datanodes (#25015)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-06-21 21:26:42 +08:00
cai.zhang
c9e456c6eb
Remove metric_type check and fix some minor bugs (#24921)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
Co-authored-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2023-06-19 09:54:41 +08:00
yihao.dai
c73219a54d
Limit the number of concurrent sync tasks and allow only one sync task for the same segment (#24881)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-06-16 14:14:39 +08:00
MrPresent-Han
ce5cb3c0c5
[skip e2e] modify default used balancer for 2.3 (#24937)
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-06-16 14:00:45 +08:00
wayblink
bfae6b49af
Remove datanode timetick mq, use rpc to report instead (#23156)
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-06-14 14:16:38 +08:00
chyezh
f97127ae55
add nats mq wrappers (#24445)
bug fixup, configurable natsmq, add unittest, pass e2e.



move natsmq to pkg project

Signed-off-by: chyezh <ye.zhen@zilliz.com>
Co-authored-by: yiwangdr <yiwangdr@gmail.com>
2023-06-07 10:00:37 +08:00
Jiquan Long
29ae1229b6
Support AutoIndex (#24387)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-05-29 20:35:28 +08:00
wei liu
1deac47069
reduce grpc client dial timout (#24427)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-05-29 10:03:28 +08:00
Bingyi Sun
46a8ca5b5b
Change log level to info (#24339)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2023-05-24 10:15:25 +08:00
MrPresent-Han
7744573d3d
support parms for import maxfilesize(#24191) (#24192)
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-05-18 18:59:27 +08:00
yihao.dai
7384d83d2c
Support rate limit based on growing segments size (#24121)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-17 09:57:22 +08:00
wei liu
4c956fab73
enable config collection level rate limit (#24012)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-05-12 18:13:26 +08:00
MrPresent-Han
b517bc9e6a
refine balance mechanism including:(#23454) (#23763) (#23791)
1. balance granuity to replica to avoid influence unrelated replicas
2. avoid balance back and forth

Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
2023-05-04 12:22:40 +08:00
yihao.dai
8c060215e9
Add collection number quota per DB (#23656)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-04-28 11:02:35 +08:00
smellthemoon
45fbe1d1a7
Change proxy max shard num (#23777)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-04-27 22:20:35 +08:00
yah01
eab94489ba
Refine the merge algorithm (#23767)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-04-27 18:26:35 +08:00
Bingyi Sun
f289aed63a
Fix nightly tests timeout (#23751)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2023-04-27 14:56:35 +08:00
smellthemoon
912cf4ef0f
Change some configurations, include change the defaultChannelNum to 16 (#23617)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-04-27 14:26:35 +08:00
yihao.dai
ed8836cd15
Add disk quota at the collection level (#23704)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-04-26 21:52:36 +08:00
SimFG
5cd21893c8
Fix superusers' password verification problem (#23733)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-04-26 21:16:34 +08:00
foxspy
6f4ed517de
add growing segment index (#23615)
Signed-off-by: xianliang <xianliang.li@zilliz.com>
2023-04-26 10:14:41 +08:00
yah01
4a2726b3f9
Remove dup config items (#23653)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-04-24 19:02:33 +08:00
aoiasd
695cc769df
Fix watch event timeout when IO Pool busy (#23293)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-04-17 11:02:30 +08:00
cqy123456
7ff6a3a246
Set query node thread pool size in milvus.yaml (#23286)
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2023-04-17 10:44:29 +08:00
Xiaofan
680ad482b7
Check balance checker chore to 10s (#23304)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-04-09 16:14:32 +08:00
smellthemoon
ce91f61e5a
Pre-create topics (#21907)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-04-03 16:44:23 +08:00
yah01
081572d31c
Refactor QueryNode (#21625)
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: aoiasd <zhicheng.yue@zilliz.com>
2023-03-27 00:42:00 +08:00
yihao.dai
f2ffb5db26
Use buffer size at memory sync policy (#22825)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-03-21 21:37:56 +08:00
MrPresent-Han
77c9e33e70
support dml channel balancer on datacoord (#22324) (#22377) (#22692)
Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
2023-03-20 10:01:56 +08:00
shaoyue
32581e6452
Support aliyun oss as object storage with ak or IAM (#22376)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2023-03-09 16:33:52 +08:00
Xiaofan
256aaa1944
Change Etcd session timeout to 20s (#22401)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-02-28 11:13:47 +08:00
Xiaofan
fa101e9751
Change RockMQ page size to 256MB (#22432)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-02-28 10:45:47 +08:00
wayblink
416866e42f
Add memory usage too large sync policy (#22244)
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-02-27 17:47:51 +08:00
Enwei Jiao
0851e05014
Gernate milvus.yaml by code (#22003)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-02-23 11:37:46 +08:00
aoiasd
148a024e05
Add tickle for datacoord watch event (#21193)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-02-15 16:20:34 +08:00
wayblink
77a27c6dfc
Support print log to file and console at the same time (#21946)
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-02-14 16:56:34 +08:00