congqixia
b44ef8207e
fix: [2.4] Check whether new collection name is alias ( #36981 ) ( #37208 )
...
Cherry pick from master
pr: #36981
Related to #36963
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-10-28 22:46:24 +08:00
SimFG
ae4ce9bbba
enhance: [2.4] allow to delete data when disk quota exhausted ( #37139 )
...
- issue: #37133
- pr: #37134
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-10-25 16:07:32 +08:00
congqixia
b958c56f48
fix: [2.4] Rectify delete buffer row count quota value ( #37060 ) ( #37068 )
...
Cherry pick from master
pr: #37060
Related to #37057
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-10-23 14:13:28 +08:00
XuanYang-cn
7847b662d4
enhance: [cp24]Log when limit writing by deletion ( #36955 )
...
See also: #36953
pr: #36954
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-10-18 17:47:30 +08:00
yihao.dai
f8d9c24256
enhance: Ignore db not found error in quota center ( #36821 ) ( #36850 )
...
In quota center, ignore the "DB not found error" to prevent it from
affecting the rate limiting of other databases.
/kind improvement
pr: https://github.com/milvus-io/milvus/pull/36821
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-10-15 20:41:24 +08:00
jaime
43b48a0f9b
enhance: skip alter operation when no change are detected ( #36786 )
...
issue: #36784
pr: #36785
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-10-11 21:43:21 +08:00
congqixia
a57626afa0
fix: [2.4] Use metrics values instead of hardcode mock values ( #36783 )
...
Cherry-pick from master
pr: #36782
Resolves #36781
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-10-11 19:49:21 +08:00
wei liu
975a9797a2
enhance: Enable dynamic update loaded collection's replica ( #36417 )
...
issue: #35821
pr: #35822
After collection loaded, if we need to increase/decrease collection's
replica, we need to release and load it again.
milvus offers 4 solution to update loaded collection's replica, this PR
aims to dynamic change the replica number without release, and after
replica number changed, milvus will execute load replica or release
replica in async, and the replica loaded status can be checked by
getReplicas API.
Notice that if set too much replicas than querynode can afford，the new
replica won't be loaded successfully until enough querynode joins.
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-09-26 10:43:15 +08:00
wei liu
ceca666e2a
fix: Fix privilege group hasn't been register for validate ( #35938 )
...
issue: #35471
pr: #35937
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-09-05 18:09:11 +08:00
congqixia
da0bc22a5f
enhance: [2.4] Add delete buffer related quota logic ( #35918 ) ( #35997 )
...
Cherry pick from master
pr: #35128 #35918
See also #35303
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: aoiasd <45024769+aoiasd@users.noreply.github.com>
2024-09-05 16:43:06 +08:00
jaime
2c1fa50412
enhance: remove cooling off in rate limiter for read requests ( #35936 )
...
issue: #35934
pr: #35935
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-09-04 14:39:10 +08:00
SimFG
8b706122a8
enhance: [2.4] support to drop the role which is related the privilege list ( #35863 )
...
- issue: #35545
- pr: #35727
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-08-31 21:57:02 +08:00
jaime
2fc111eb10
enhance: set database properties to restrict read access ( #35754 )
...
issue: #35744
pr: #35745
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-08-29 11:41:01 +08:00
jaime
c75f556769
fix: inconsistent meta view causes rate limit invalid ( #35664 )
...
issue: #35663
pr: #35665
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-08-28 11:21:04 +08:00
yihao.dai
8e6ec58652
fix: Fix rate wasn't limited to the expected value ( #35699 ) ( #35700 )
...
Each time the rate is reset, the token bucket is fully refilled, causing
the rate wasn't limited to the expected value. This PR addresses the
issue by preventing the token reset.
issue: https://github.com/milvus-io/milvus/issues/35675 ,
https://github.com/milvus-io/milvus/issues/35702
pr: https://github.com/milvus-io/milvus/pull/35699
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-08-28 10:29:00 +08:00
jaime
5c1a380d9d
fix: the incorrect number of partitions in rootcoord meta ( #35601 )
...
pr: #35600
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-08-25 16:30:59 +08:00
wei liu
421a00bfe8
enhance: Refresh proxy cache after restore rbac meta ( #35636 )
...
issue: #35443
pr: #35635
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-22 19:09:01 +08:00
wei liu
e2542a1bf5
enhance: Update protobuf-go to protobuf-go v2 ( #34394 ) ( #35555 )
...
issue: #34252
pr: #34394 #35072 #35084
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-21 18:50:58 +08:00
wei liu
248a6ea401
enhance: Add BackupRBAC/RestoreRBAC API to enable rbac backup ( #35444 ) ( #35513 )
...
issue: #35443
pr: #35444
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-18 13:42:55 +08:00
Jiquan Long
d14d00b0cc
enhance: add metric to record disk quota ( #35305 ) ( #35306 )
...
https://github.com/milvus-io/milvus/issues/35177
pr: https://github.com/milvus-io/milvus/pull/35305
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-08-07 10:24:27 +08:00
Jiquan Long
ae3657b23e
enhance: add metric to record maxInsertRate and queryNodeMemoryHighWaterLevel ( #35185 ) ( #35188 )
...
pr: https://github.com/milvus-io/milvus/pull/35185
fix: https://github.com/milvus-io/milvus/issues/35177
---------
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-08-02 10:16:15 +08:00
Jiquan Long
2822d8729c
feat: record the number of times milvus turns into force-deny-writing states ( #34960 ) ( #34989 )
...
/kind improvement
fix : #34961
pr: #34960
---------
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-07-25 15:43:09 +08:00
congqixia
7540714c1b
enhance: [2.4] Add l0 segment entry num quota ( #34733 ) ( #34837 )
...
Cherry-pick from master
pr: #34733
See also #34670
This PR add quota configuration for l0 segment entry number per
collection. If l0 compaction cannot keep up the insertion/upsertion
rate, this feature could back press the related rate.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-22 12:05:43 +08:00
SimFG
0e226502e4
enhance: [2.4] pick default root password and log level pr ( #34777 )
...
default root password
- issue: #33058
- pr: #34752
set log level
- issue: #34756
- pr: #34757
---------
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-07-18 13:45:43 +08:00
Patrick Weizhi Xu
3cf2e1831a
feat: [cherry-pick][2.4] support partition key isolation ( #34336 ) ( #34613 )
...
issue: #34332
pr: #34336
---------
Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
(cherry picked from commit 104d0966b7
2024-07-17 14:03:43 +08:00
wei liu
f0d12aa68d
fix: RootCoord panic during check health ( #34542 )
...
issue: #34540
pr: #34541
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-07-10 09:50:38 +08:00
jaime
60aab15ef4
enhance: support setting properties in create database request ( #34511 )
...
issue: #34493
pr: #34510
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-07-09 18:16:18 +08:00
jaime
326370c1be
enhance: add disk quota and max collections into db properties ( #34386 )
...
issue: https://github.com/milvus-io/milvus/issues/34385
pr: #34368
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-07-05 18:22:17 +08:00
jaime
3e0034bea2
enhance: cherry pick some improved PRs from the master branch ( #34391 )
...
issue:
https://github.com/milvus-io/milvus/issues/33205,https://github.com/milvus-io/milvus/issues/33342
pr: https://github.com/milvus-io/milvus/pull/33530
pr: #33343
pr: #33206
---------
Signed-off-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: xiaofanluan <xiaofan.luan@zilliz.com>
Co-authored-by: Xiaofan <83447078+xiaofan-luan@users.noreply.github.com>
2024-07-03 19:40:11 +08:00
xige-16
06d9d258c4
fix: [cherry-pick]Fix datacoord's properties overwriten ( #34240 )
...
issue: https://github.com/milvus-io/milvus/issues/34144
pr: https://github.com/milvus-io/milvus/pull/34146
/kind bug
Signed-off-by: xige-16 <xige2016@gmail.com>
2024-07-01 13:08:07 +08:00
jaime
0992f10694
enhance: improve check health ( #34265 )
...
issue: https://github.com/milvus-io/milvus/issues/34264
pr: #33800
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-07-01 10:18:07 +08:00
jaime
6423b6c718
enhance: move rocksmq from internal to pkg ( #34165 )
...
pr: https://github.com/milvus-io/milvus/pull/33881
issue: https://github.com/milvus-io/milvus/issues/33956
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-06-26 13:36:05 +08:00
jaime
c219dca001
fix: metrics database_num is 0 after restarting rootcoord ( #34010 )
...
issue: https://github.com/milvus-io/milvus/issues/34041
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-06-21 11:46:02 +08:00
SimFG
c331aa4ad3
enhance: [2.4] add the includeCurrentMsg param for the Seek method ( #33743 )
...
/kind improvement
- issue: #33325
- pr: #33326
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-06-11 15:01:55 +08:00
SimFG
766f47c9ca
fix: [2.4] reset the RootCoordQuotaStates metric before recording this metric ( #33601 )
...
- issue: #33539
- pr: #33553
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-06-04 20:17:51 +08:00
SimFG
23b0731a6a
fix: [2.4] show empty collection when has granted the all privilege ( #33454 )
...
- issue: #33382
- pr: #33445
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-29 20:49:45 +08:00
SimFG
cbeb6c169d
fix: [2.4] quotaCenter collect metrics failed because the wrong param ( #33399 )
...
- issue: #33397
- pr: #33398
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-27 17:05:42 +08:00
wayblink
deebae70a7
fix:[cherry-pick]Panic if ProcessActiveStandBy returns error ( #33372 )
...
pr:#33369
issue:#33368
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-05-27 10:13:59 +08:00
SimFG
37b2f90c90
enhance: [2.4] the panic when db isn't existed in the rate limit interceptor ( #33308 )
...
issue: #33243
pr: #33244
1. fix: the panic when db isn't existed in the rate limit interceptor
#33244
2. enhance: check the auth in some rest v2 api #33256
---------
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-23 17:25:41 +08:00
SimFG
2a38f5b598
enhance: [2.4] add config to control whether to init public role permissions ( #33174 )
...
issue: #33164
pr: #33165
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-21 22:39:46 +08:00
smellthemoon
89a7c34c7a
fix: exceed etcd limit ( #33041 )
...
#32974
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-05-14 18:45:32 +08:00
wei liu
c35797c399
enhance: expose DescribeDatabase api in proxy ( #32732 )
...
issue: #32707
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-05-09 22:51:30 +08:00
wayblink
42d0412e93
enhance: Add channelCPs in FlushResponse ( #32044 )
...
#32609
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-04-30 09:45:27 +08:00
SimFG
0359139299
enhance: return the auth collection list when using the show collection api ( #32576 )
...
issue: #32550
/kind improvement
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-04-29 19:09:34 +08:00
wei liu
04f355a802
enhance: Enable alter database props in rootcoord ( #32458 )
...
issue: #30040
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-04-25 10:53:25 +08:00
SimFG
31a29a2451
enhance: avoid the repeated metric info in the proxy ( #32380 )
...
issue: #30577
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-04-19 10:21:20 +08:00
Jiquan Long
7add7248e2
fix: remove useless log of quota center ( #32307 )
...
/kind improvement
issue: #31272
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-04-17 16:17:19 +08:00
wei liu
0d849a6c0a
fix: fix collectionInfo leak in datacoord ( #32175 )
...
issue: #32029
lack of logic to clean collection info in datacoord's meta, This PR
clean collection info after drop channel, to avoid collection info leak
in datacoord
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-04-15 16:33:19 +08:00
SimFG
c012e6786f
feat: support rate limiter based on db and partition levels ( #31070 )
...
issue: https://github.com/milvus-io/milvus/issues/30577
co-author: @jaime0815
---------
Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
Signed-off-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
2024-04-12 16:01:19 +08:00
SimFG
789e014c74
enhance: add the db id for the describe collection response ( #32114 )
...
/kind improvement
issue: #32110
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-04-10 21:23:20 +08:00
