issue: #34595
When consuming insert data on the delegator node, QueryCoord will move
out some sealed segments to manage its memory usage. After the growing
segment gets flushed, some sealed segments from other workers will be
moved back to the delegator node. To avoid the frequent movement of
segments, we estimate the maximum growing row count and preserve a
fixed-size memory in the delegator node.
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
issue: #33285
- add two grpc resolver (by session and by streaming coord assignment
service)
- add one grpc balancer (by serverID and roundrobin)
- add lazy conn to avoid block by first service discovery
- add some utility function for streaming service
Signed-off-by: chyezh <chyezh@outlook.com>
The nodeID for compaction task initialization is 0. This PR adjusts the
task reassignment conditions to allow new compaction tasks to be
reassigned and executed.
issue: https://github.com/milvus-io/milvus/issues/34460
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
See also #34574
Add jitter for segment seal proportion to avoid seal operation burst in
short period of time.
This PR also fix license header in paramtable pkg.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
github.com/gogo/protobuf is deprecated and could be error prune after
upgrade protobuf message to v2.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
related: #33544
mainly changes in three aspects:
1. enable setting group_size for group by function
2. separate normal reduce and group by reduce
3. eleminate uncessary padding in search result for reducing
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
issue: # #34545
Print warn log instead of check health fail if orphan channel cp meta is
found in health check request.
Signed-off-by: jaime <yun.zhang@zilliz.com>
Related to #34508
The padding bytes shall be written only at the end of the mmap file not
the chunk of each field data file.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
issue: #34500
cause the sort in `GetLevel0Deletions` will broken the corresponed order
between pks and tss, then the pks and tss will be sorted in
segment.Delete() interface.
This PR remove this uncessary and incorrect sort progress to avoid query
may return deleted records.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
See also #34483
Some lint issues are introduced due to lack of static check run. This PR
fixes these problems.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
issue: #33285
- implement producing and consuming server of message
- implement management operation for streaming node server
---------
Signed-off-by: chyezh <chyezh@outlook.com>
related: #30376
fix: paritionIDs lost when no setting paritions
enhance: refine metrics for segment prune
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
issue: #34304
cosine is more widely used in float vectors, and cosine and hamming
distance are 'metrics' which have good geometric properties
Signed-off-by: chasingegg <chao.gao@zilliz.com>
