issue: #33285
- implement streaming service client.
- implement producing and consuming service client by streaming coord
client and streaming node client.
Signed-off-by: chyezh <chyezh@outlook.com>
issue: #35240
delete by expr shard the same err object between channels, so if one
channel's query failed, it will fail all channel, which will break
channel level retry policy, and make delete operation failed.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
before querycoord stop gracefully, we will save the current target to
meta store and recover it after querycoord start up, to speed the
querycoord's recovery time. but the target version hasn't been recovered
as expected, and it use latest timestamp as current target's version,
which has no effect to querycoord but an unnecessary syncTargetVersion
func call.
This PR recover the correct target version to avoid unnecessary
syncTargetVersion func call
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
The watch progress is always zero in CheckChannelOperationProgress
response, which is meaningless and confusing. This PR set progress value
in rpc response to fix this problem.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
issue: #35087
after qc restarts, and target is not ready yet, if dist_handler try to
update segment dist, it will set legacy level to l0 segment, which may
cause l0 segment be moved to other node, cause search/query failed.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
https://github.com/milvus-io/milvus/issues/35112
This pr would not affect milvus functionality by now.
It implments a Chunk memory layout that looks like
```
VariableColumn
|offset|offset|offset|
|data|data|data|
```
We maybe move offsets to the beginning and add null bitmaps later but
not in this PR.
And mmap test will also be added in another PR.
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
At most cases, data in each channel is almost evenly distributed, we
could utilize the channel num info to optimize searh param in queryHook
Signed-off-by: chasingegg <chao.gao@zilliz.com>
issue: #34715
if collection's segment list doesn't changes anymore, then the next
target will be empty at most time, and balance segment will check
whether segment exist in both current and next target, so the balance
cloud be blocked due to next target is empty.
This PR permit segment to be moved if next target is empty, to avoid
balance stuck.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Grow slice & map.growWork may cause a lot when segment number is large
for big K query. This PR pre-allocate space for reduce methods to avoid
this cost.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
add prometheus dependency for monitor module. Or else Some compilers may
report a compilation failure.
issue: #35077
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
1.fix compaction task not be cleaned correctly
2.add a new parameter to control compaction gc loop interval
3.remove some useless configs of clustering compaction
bug: #34764
Signed-off-by: wayblink <anyang.wang@zilliz.com>
after manual stop component by management restful api, `healthz` may
return unhealthy state. k8s may restart the pod to save the unhealthy
sate, and the manual stop operation will got unexpected result.
to solve this, we make `healthz` API skip the manual stopped component.
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
#34778#34849
fix two problems:
1. count(*) incorrect, if growing insert duplicated (pk, timestamp)
pairs that pk and timestamp all same, need to keep just one pair.
2. count(*) may core dump, if get_real_count interface get snapshot and
do mvcc at not consistency status, mainly happens under concurrency.
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
