jaime
24fb10114b
enhance: remove cooling off in rate limiter for read requests ( #35935 )
...
issue: #35934
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-09-04 14:39:10 +08:00
jaime
b0ac04d104
enhance: set database properties to restrict read access ( #35745 )
...
issue: #35744
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-08-29 13:17:01 +08:00
jaime
d8aa01bc1a
fix: inconsistent meta view causes rate limit invalid ( #35665 )
...
issue: #35663
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-08-28 11:21:04 +08:00
jaime
60be454db0
enhance: add disk quota and max collections into db properties ( #34368 )
...
issue: #34385
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-07-05 18:22:17 +08:00
SimFG
31a29a2451
enhance: avoid the repeated metric info in the proxy ( #32380 )
...
issue: #30577
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-04-19 10:21:20 +08:00
SimFG
c012e6786f
feat: support rate limiter based on db and partition levels ( #31070 )
...
issue: https://github.com/milvus-io/milvus/issues/30577
co-author: @jaime0815
---------
Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
Signed-off-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
2024-04-12 16:01:19 +08:00
wei liu
9111ecab82
enhance: Rootcoord's stop may block in quota_center's stop ( #31447 )
...
this PR fixed that rootcoord's stop may block in quota_center' stop
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-04-01 21:09:18 +08:00
congqixia
af315539d4
fix: Remove time tick delay metrics when nodes go offline ( #30833 )
...
See also #30832
This PR removes time tick delay metrics when rootcoord GetMetrics
response does not have previously existed querynode/datanode
Also add unit tests for this case
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Signed-off-by: Congqi.Xia <congqi.xia@zilliz.com>
2024-02-28 10:10:56 +08:00
wei liu
e41fd6fbde
enhance: Move proxy client manager to util package ( #28955 )
...
issue: #28898
This PR move the `ProxyClientManager` to util package, in case of
reusing it's implementation in querycoord
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-12-20 19:22:42 +08:00
yah01
b8674811cf
fix: data race in ProxyClientManager ( #29206 )
...
this PR changed the ProxyClientManager to thread-safe
fix #29205
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-12-14 18:22:39 +08:00
wei liu
fe1eeae2aa
enhance: Use mockery to replace manual mock code ( #29074 )
...
issue: #29043
This PR remove mannul mock code for proxy and data coord
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-12-13 10:46:44 +08:00
yah01
be980fbc38
Refine state check ( #27541 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-11 21:01:35 +08:00
jaime
7f7c71ea7d
Decoupling client and server API in types interface ( #27186 )
...
Co-authored-by:: aoiasd <zhicheng.yue@zilliz.com>
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-09-26 09:57:25 +08:00
SimFG
26f06dd732
Format the code ( #27275 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-21 09:45:27 +08:00
Enwei Jiao
fb0705df1b
Decouple basetable and componentparam ( #26725 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-09-05 10:31:48 +08:00
wei liu
cd7f811d40
fix rootcoord unstable ut ( #26460 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-08-30 09:42:27 +08:00
congqixia
b9850ce5c0
Fix copylocks linter errors ( #26217 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-08-08 20:41:15 +08:00
yihao.dai
b6effd7345
Disable deny writing when the growing segment size exceeds the watermark ( #26163 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-08-08 12:43:07 +08:00
yiwangdr
b9189b9f41
Organize mocks from types.go ( #25466 )
...
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
2023-07-14 10:12:31 +08:00
smellthemoon
d63323d117
Add rate limit and deny write in upsert ( #25351 )
...
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-07-11 11:20:34 +08:00
jaime
18df2ba6fd
[Cherry-Pick] Support Database ( #24769 )
...
Support Database(#23742 )
Fix db nonexists error for FlushAll (#24222 )
Fix check collection limits fails (#24235 )
backward compatibility with empty DB name (#24317 )
Fix GetFlushAllState with DB (#24347 )
Remove db from global meta cache after drop database (#24474 )
Fix db name is empty for describe collection response (#24603 )
Add RBAC for Database API (#24653 )
Fix miss load the same name collection during recover stage (#24941 )
RBAC supports Database validation (#23609 )
Fix to list grant with db return empty (#23922 )
Optimize PrivilegeAll permission check (#23972 )
Add the default db value for the rbac request (#24307 )
Signed-off-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
2023-06-25 17:20:43 +08:00
wei liu
0d51f55c48
enable config each collection has different rate limit ( #24139 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-06-21 15:02:42 +08:00
congqixia
41af0a98fa
Use go-api/v2 for milvus-proto ( #24770 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-09 01:28:37 +08:00
yah01
ebd0279d3f
Check error by Error() and NoError() for better report message ( #24736 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-06-08 15:36:36 +08:00
yihao.dai
1471da846d
Fix panic due to collection not exist in quota effect ( #24320 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-24 20:53:30 +08:00
wei liu
6cfb4559d2
fix deny writing use wrong errcode ( #24152 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-05-17 14:23:23 +08:00
yihao.dai
7384d83d2c
Support rate limit based on growing segments size ( #24121 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-17 09:57:22 +08:00
wei liu
4c956fab73
enable config collection level rate limit ( #24012 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-05-12 18:13:26 +08:00
yihao.dai
5a9a86ba74
Fix quota center assignment to entry in nil map ( #23819 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-04-29 18:22:38 +08:00
wei liu
537e068fb9
fix qupta_center calculate negative memory factor for rate limit ( #23750 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-04-28 11:32:35 +08:00
yihao.dai
017b15af6a
Fix disk quota when there's no binlog file ( #23776 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-04-28 11:10:35 +08:00
yihao.dai
ed8836cd15
Add disk quota at the collection level ( #23704 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-04-26 21:52:36 +08:00
wei liu
4fb8919a97
support collection level rate limit ( #22767 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-04-25 15:54:35 +08:00
jaime
c9d0c157ec
Move some modules from internal to public package ( #22572 )
...
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-04-06 19:14:32 +08:00
wei liu
13b508103e
refine mock querycoord ( #22198 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-02-16 15:38:34 +08:00
yah01
6e66c10114
Fix check time tick lag with no channel ( #21737 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-01-16 18:49:43 +08:00
bigsheeper
79535931ab
Return error code when deny to read/write ( #21561 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-01-06 14:31:37 +08:00
bigsheeper
c187de7754
Show trigger reason when deny to read/write ( #21368 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-12-30 18:35:32 +08:00
Enwei Jiao
89b810a4db
Refactor all params into ParamItem ( #20987 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-12-07 18:01:19 +08:00
bigsheeper
f0c616050a
Reduce and improve quota center log ( #20058 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-10-25 19:39:30 +08:00
bigsheeper
db0486c870
Fix missing import limit and rename functions in quotaCenter ( #19907 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-10-19 21:43:30 +08:00
bigsheeper
99df2d9b54
Add binlogSize and ttDelay metrics monitor ( #19872 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-10-19 15:13:26 +08:00
SimFG
a55f739608
Separate public proto files ( #19782 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-10-16 20:49:27 +08:00
bigsheeper
3a5aaeb7ad
Fix read queue metrics and memory protection ( #19787 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-10-14 18:05:24 +08:00
bigsheeper
f1e330a997
Add read result rate protection ( #19728 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-10-13 14:57:24 +08:00
bigsheeper
e6c76aba0e
Force deny writing if disk quota exceeded ( #19680 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-10-12 10:03:22 +08:00
bigsheeper
804c18df68
Update quota params ( #19351 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-09-26 16:48:53 +08:00
SimFG
d7f38a803d
Separate some proto files ( #19218 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-09-16 16:56:49 +08:00
bigsheeper
5cdd6ca251
Add quotaCenter, rateLimiter and rateCollector ( #18636 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-09-16 09:56:47 +08:00