See also: #30121#27675
This PR changes the delete buffering logic:
- Write buffer shall buffer insert first
- Then the delete messages shall be evaluated
- Whether PK matches previous Bloom filter, which ts is always smaller
- Whether PK matches insert data which has smaller timestamp
- Then the segment bloom filter is updates by the newly buffered pk rows
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
See also: #28873
When datanode returns error or go offline during GetCompactionResult
call, the compress binlog logic will panic since it was using a nil
result
This PR move it after the CheckRPCCall error to prevent this case.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
See also: #25323, #29969
many users reported log file name is incorrect when starting in mixture
type.
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
This PR also fixes bugs in l0 compactor where
l0 results would never be removed from datanode
See also: #30099
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
don't store logPath in meta to reduce memory, when service get
segmentinfo, generate logpath from logid.
#28885
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
issue: #30000
related to: [milvus-proto
#202](https://github.com/milvus-io/milvus-proto/pull/202)
1. replace collSchema.AutoID with primaryField.AutoID
2. show `enableDynamic` & `enableDynamicField` at the same time
3. avoid data race about the access to metacache
Signed-off-by: PowderLi <min.li@zilliz.com>
currently integration test may timeout if any case run time is above 3
minutes. This duration was hard coded.
This PR change this duration into a customized parameter and could be
passed via test running commands.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
issue: #29841
This PR fix leader checker use wrong check interval, which causes leader
checker trigger too frequency
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
issue: #29841
if segment loaded, submit load segment task for it isn't permitted, to
avoid load segment twice. but this logic blocks the leader checker to
correct leader view by `LoadSegment`
This PR remove the segment loaded check, to fix that leader checker
cann't submit load task
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
this avoids a corner case: after load index failed, this index can be
never loaded as it has been added into the segment's index map
Signed-off-by: yah01 <yang.cen@zilliz.com>
issue: #29677#29838
during get shard leaders, if qeurynode doesn't ack the heartbeat than
10s, querycoord will treat it as unavailable, and won't return shard
leader on it. but when querynode has a full cpu usage, it's easily to
stuck for more than 10s without ack the heartbeat, which cause no shard
leader to search/query.
This PR remove heartbeat lag logic during get shard leaders
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
the recent changes move the level 0 segments list to a new proto field,
which leads to the QueryCoord can't see the level 0 segments, handle the
new changes
fix#29907
Signed-off-by: yah01 <yang.cen@zilliz.com>
Ignoring .git may result "unusable report" in codecov. Use checkout
action to get .git information in upload codecov step
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
when apply dynamic config changes, we should format the value to proper
unit
This PR fix update rate limit config with wrong value.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
This fix will not augment the execution time of unit tests, but solely
enhances tolerance for waiting for failure.
issue: https://github.com/milvus-io/milvus/issues/29921
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
