milvus/internal
wei liu b18de95817
enhance: Avoid assign too much segment/channels to new querynode (#34096) (#34245)
issue: #34095
pr: #34096

When a new query node comes online, the segment_checker,
channel_checker, and balance_checker simultaneously attempt to allocate
segments to it. If this occurs during the execution of a load task and
the distribution of the new query node hasn't been updated, the query
coordinator may mistakenly view the new query node as empty. As a
result, it assigns segments or channels to it, potentially overloading
the new query node with more segments or channels than expected.

This PR measures the workload of the executing tasks on the target query
node to prevent assigning an excessive number of segments to it.

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-07-01 10:32:06 +08:00
..
allocator enhance: move rocksmq from internal to pkg (#34165) 2024-06-26 13:36:05 +08:00
core fix: expr add skip using index while index exists (#34202) 2024-06-30 20:03:13 +08:00
datacoord enhance: decrease cpu overhead during filter segments on datacoord (#34231) 2024-07-01 10:20:08 +08:00
datanode enhance: [cherry-pick] Periodically synchronize segments to datanode watcher (#33420) (#34186) 2024-06-27 11:24:05 +08:00
distributed enhance: [cherry-pick] Decouple compaction from shard (#34157) 2024-06-25 20:22:03 +08:00
http enhance: add restful api to trigger component stop (#32076) (#33799) 2024-06-17 21:53:59 +08:00
indexnode enhance: [2.4] Avoid merging insert data when buffering insert msgs (#34205) 2024-06-27 10:14:05 +08:00
kv enhance: move rocksmq from internal to pkg (#34165) 2024-06-26 13:36:05 +08:00
metastore enhance: move rocksmq from internal to pkg (#34165) 2024-06-26 13:36:05 +08:00
mocks enhance: [cherry-pick] Decouple compaction from shard (#34157) 2024-06-25 20:22:03 +08:00
parser/planparserv2 enhance: optimize plan parser pool to avoid unnessary recycle (#32869) 2024-05-11 10:51:31 +08:00
proto enhance: [cherry-pick] Periodically synchronize segments to datanode watcher (#33420) (#34186) 2024-06-27 11:24:05 +08:00
proxy enhance: [cherry-pick]remove CheckVecIndexWithDataTypeExist function in pkg (#34104) 2024-06-26 21:14:26 +08:00
querycoordv2 enhance: Avoid assign too much segment/channels to new querynode (#34096) (#34245) 2024-07-01 10:32:06 +08:00
querynodev2 enhance: decrease cpu overhead during filter segments on datacoord (#34231) 2024-07-01 10:20:08 +08:00
registry Add querynode client wrapper and avoid grpc in standalone mode (#27781) 2023-10-19 11:10:07 +08:00
rootcoord enhance: improve check health (#34265) 2024-07-01 10:18:07 +08:00
storage enhance: [2.4] Avoid merging insert data when buffering insert msgs (#34205) 2024-06-27 10:14:05 +08:00
tso enhance: move rocksmq from internal to pkg (#34165) 2024-06-26 13:36:05 +08:00
types feat: support rate limiter based on db and partition levels (#31070) 2024-04-12 16:01:19 +08:00
util enhance: improve check health (#34265) 2024-07-01 10:18:07 +08:00