congqixia
62bd51ec6d
fix: [2.4] Compare config value then swap when caching param value ( #33785 ) ( #33797 )
...
Cherry-pick from master
pr: #33785
See also #33784
This PR change the behavior of `SetCacheValue` of config manager:
- Use mutex and map instead of concurrent map for `configCache`
- Compare config raw value before set cache value
With this implementation, concurrent caching & eviction shall always
have current output:
|time|caching |eviction|config |cached |
|----|--------|------- |---------|---------|
|t0 |get | |old value|null |
|t1 |CAS OK | |old value|old value|
|t2 | |update |new value|old value|
|t3 | |eviction|new value|null |
|time|caching |eviction|config |cached |
|----|--------|------- |---------|---------|
|t0 |get | |old value|null |
|t1 | |update |new value|null |
|t2 |CAS fail| |old value|null |
|t3 | |eviction|new value|null |
|time|caching |eviction|config |cached |
|----|--------|------- |---------|---------|
|t0 | |update |new value|null |
|t1 |get | |new value|null |
|t2 |CAS OK | |new value|new value|
|t3 | |eviction|new value|null |
|time|caching |eviction|config |cached |
|----|--------|------- |---------|---------|
|t0 | |update |new value|null |
|t1 |get | |new value|null |
|t2 | |eviction|new value|null |
|t3 |CAS OK | |new value|new value|
|time|caching |eviction|config |cached |
|----|--------|------- |---------|---------|
|t0 | |update |new value|null |
|t1 | |eviction|new value|null |
|t2 |get | |new value|null |
|t3 |CAS OK | |new value|new value|
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-06-13 17:51:57 +08:00
aoiasd
aaf6c85095
fix: paramtable cache cause dynamic config non-dynamic ( #33473 ) ( #33590 )
...
relate: https://github.com/milvus-io/milvus/issues/33461
pr: https://github.com/milvus-io/milvus/pull/33473
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-06-05 10:15:51 +08:00
Xiaofan
b6fefee0cf
fix: etcd not connectable when auth enabled ( #31633 )
...
Fix etcd config source didn't respect auth enabled
Also removed pulsar recoverable error when pulsar return ConsumerBusy.
It could happen that pulsar didn't find the original consumer is dead
and recover takes some time.
fix #31631
Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
2024-04-01 15:23:19 +08:00
congqixia
74b7de3814
enhance: Cache formatted key for param item ( #31388 )
...
See also #30806
`formatKey` may cost lots of CPU on string processing under high QPS
scenario, this PR adds a formattedKeys cache preventing string operation
in each param get value.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-03-19 14:05:05 +08:00
aoiasd
b724753137
enhance: Add runtime config to paramtable ( #31006 )
...
relate: https://github.com/milvus-io/milvus/issues/30806
Avoid use string convert or format function when get some runtime
parameter
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-03-15 11:07:06 +08:00
Chun Han
3298e64bd3
enhance: cache config values for saving cpu cycles to parse config item ( #30947 )
...
related: #30958
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2024-03-12 11:09:04 +08:00
aoiasd
bbff9193d9
enhance: support clean paramtable config event in test ( #30534 )
...
relate: https://github.com/milvus-io/milvus/issues/30441
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-02-20 14:16:51 +08:00
aoiasd
8385157717
enhance: adjust config source for support config event use paramtable ( #29995 )
...
Adjust config source for support config event which for dynamic config
could use paramtable and not deadlock.
relate: https://github.com/milvus-io/milvus/issues/29807
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-01-26 19:07:00 +08:00
congqixia
d73b534f1e
fix: use atomic.Pointer to store EventHandler in case of data race ( #30205 )
...
Resolves #30204
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-01-23 19:09:04 +08:00
wei liu
797847904c
enhance: Change some frequency log to rated level ( #29720 )
...
This PR change some frequency log to rated level
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-01-11 16:30:50 +08:00
wei liu
1f759837c4
fix: remove Unnecessary lock in config manager ( #29836 )
...
issue: #29709 #291712
to avoid concurrent recursive RLock and Lock cause deadlock, This PR
remove the unnecessary lock in config manager
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-01-11 13:48:49 +08:00
MrPresent-Han
ed644983e2
enhance: add param for bloomfilter( #29388 ) ( #29490 )
...
related: #29388
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-12-28 18:10:46 +08:00
aoiasd
89d8ce2f73
enhance: refine access log to support format access log by yaml and print name info. ( #28319 )
...
relate: https://github.com/milvus-io/milvus/issues/28086
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-11-28 15:32:31 +08:00
congqixia
f1fc19e8a9
enhance: Add unittest for config.EventDispatcher ( #28552 )
...
These is no enough unittest case for EventDispatcher, see also #28540 .
This PR add unit test case for all methods for EventDispatcher
Related to #28538
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-11-18 19:12:20 +08:00
congqixia
c948a437a9
fix: protect EventDispatcher map with mutex ( #28540 )
...
Add mutex protection for `EventDispatcher.registry` map
Fix #28538
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-11-17 20:32:20 +08:00
SimFG
7dda2e8814
Change some log level in the pkg package ( #28181 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-11-08 23:34:22 +08:00
SimFG
26f06dd732
Format the code ( #27275 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-21 09:45:27 +08:00
Enwei Jiao
fb0705df1b
Decouple basetable and componentparam ( #26725 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-09-05 10:31:48 +08:00
yah01
c3f5856fbc
Fix data race for config with FileSource ( #26518 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-08-24 09:18:24 +08:00
congqixia
0bc03ede0d
Add eventlog pkg and support grpc streaming event observation ( #25812 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-07-25 17:23:01 +08:00
Gao
de6e4817a2
Support dynamic tuning config ( #25152 )
...
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2023-07-03 15:18:24 +08:00
yah01
ebd0279d3f
Check error by Error() and NoError() for better report message ( #24736 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-06-08 15:36:36 +08:00
wei liu
8e3ba74648
fix qc service unstable ut ( #24340 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-05-24 18:49:25 +08:00
congqixia
7e9ef36de4
Fix TestConfigFromRemote/close_manager ut is not stable ( #24279 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-05-22 09:59:25 +08:00
congqixia
3a66e1de65
Use suite for integration test ( #24253 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-05-19 15:57:24 +08:00
congqixia
de4b4dafef
Refresh etcd source config with Searializable option ( #23954 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-05-09 11:52:40 +08:00
Enwei Jiao
086f3bd748
Add it for refresh config ( #23773 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-05-06 17:34:39 +08:00
jaime
c9d0c157ec
Move some modules from internal to public package ( #22572 )
...
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-04-06 19:14:32 +08:00
