milvus/internal/datanode/writebuffer
congqixia eaabe0293b
fix: Update segment compactTo when compactTo segment is compacted (#28755)
Related to #28736 #28748
See also #27675
Previous PR: #28646

This PR fixes `SegmentNotFound` issue when compaction happens multiple
times and the buffer of first generation segment is sync due to stale
policy

Now the `CompactSegments` API of metacache shall update the compactTo
field of segmentInfo if the compactTo segment is also compacted to keep
the bloodline clean

Also, add the `CompactedSegment` SyncPolicy to sync the compacted
segment asap to keep metacache clean

Now the `SyncPolicy` is an interface instead of a function type so that
when it selects some segments to sync, we colud log the reason and
target segment

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-11-27 19:48:26 +08:00
..
bf_write_buffer_test.go fix: Update segment compactTo when compactTo segment is compacted (#28755) 2023-11-27 19:48:26 +08:00
bf_write_buffer.go enhance: Add back clean compacted segment info logic (#28646) 2023-11-24 15:38:25 +08:00
delta_buffer_test.go Use writebuffer, sync manager refactory in datanode (#28320) 2023-11-15 15:24:18 +08:00
delta_buffer.go Use writebuffer, sync manager refactory in datanode (#28320) 2023-11-15 15:24:18 +08:00
insert_buffer_test.go enhance: Use bufSize instead of row number in sync policy (#28498) 2023-11-21 15:02:25 +08:00
insert_buffer.go enhance: Use bufSize instead of row number in sync policy (#28498) 2023-11-21 15:02:25 +08:00
l0_write_buffer_test.go fix: Skip statslog generation flushing empty L0 segment (#28733) 2023-11-25 15:10:25 +08:00
l0_write_buffer.go fix: Skip statslog generation flushing empty L0 segment (#28733) 2023-11-25 15:10:25 +08:00
manager_test.go feat: integrate storage v2 into the write path (#28440) 2023-11-23 17:26:24 +08:00
manager.go feat: integrate storage v2 into the write path (#28440) 2023-11-23 17:26:24 +08:00
mock_mananger.go feat: integrate storage v2 into the write path (#28440) 2023-11-23 17:26:24 +08:00
mock_write_buffer.go enhance: Remove commented code and fix naming issue (#28450) 2023-11-16 00:22:20 +08:00
options.go fix: Update segment compactTo when compactTo segment is compacted (#28755) 2023-11-27 19:48:26 +08:00
segment_buffer.go enhance: Remove commented code and fix naming issue (#28450) 2023-11-16 00:22:20 +08:00
sync_policy_test.go fix: Update segment compactTo when compactTo segment is compacted (#28755) 2023-11-27 19:48:26 +08:00
sync_policy.go fix: Update segment compactTo when compactTo segment is compacted (#28755) 2023-11-27 19:48:26 +08:00
write_buffer_test.go feat: integrate storage v2 into the write path (#28440) 2023-11-23 17:26:24 +08:00
write_buffer.go fix: Update segment compactTo when compactTo segment is compacted (#28755) 2023-11-27 19:48:26 +08:00