milvus/internal/datanode/writebuffer
congqixia a040692129
enhance: Use estimated batch size to initalize BF (#29842)
See also: #27675

The bloom filter set initialized new BF with fixed configured `n`. This
value is always larger than the actual batch size and causes generated
BF using more memory.

This PR make write buffer to initialize BF with estimated batch size
from schema & configuration value.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-01-10 20:36:50 +08:00
..
bf_write_buffer_test.go fix: Correct flush buffer size metrics (#29571) 2024-01-04 17:22:46 +08:00
bf_write_buffer.go enhance: separate serializer logic from sync task (#29413) 2023-12-26 10:40:47 +08:00
delta_buffer_test.go Use writebuffer, sync manager refactory in datanode (#28320) 2023-11-15 15:24:18 +08:00
delta_buffer.go Use writebuffer, sync manager refactory in datanode (#28320) 2023-11-15 15:24:18 +08:00
insert_buffer_test.go fix: Correct flush buffer size metrics (#29571) 2024-01-04 17:22:46 +08:00
insert_buffer.go fix: Correct flush buffer size metrics (#29571) 2024-01-04 17:22:46 +08:00
l0_write_buffer_test.go fix: Correct flush buffer size metrics (#29571) 2024-01-04 17:22:46 +08:00
l0_write_buffer.go fix: Correct flush buffer size metrics (#29571) 2024-01-04 17:22:46 +08:00
manager_test.go enhance: separate serializer logic from sync task (#29413) 2023-12-26 10:40:47 +08:00
manager.go feat: integrate storage v2 into the write path (#28440) 2023-11-23 17:26:24 +08:00
mock_mananger.go feat: integrate storage v2 into the write path (#28440) 2023-11-23 17:26:24 +08:00
mock_write_buffer.go enhance: Remove commented code and fix naming issue (#28450) 2023-11-16 00:22:20 +08:00
options.go enhance: Unify levelzero segment config in DN (#28720) 2023-11-28 18:04:26 +08:00
segment_buffer.go enhance: Write buffer time range when syncing logs (#28970) 2023-12-05 17:36:36 +08:00
sync_policy_test.go fix: Add jitter in GetSyncStaleBufferPolicy (#28626) 2023-12-20 15:00:52 +08:00
sync_policy.go fix: Add jitter in GetSyncStaleBufferPolicy (#28626) 2023-12-20 15:00:52 +08:00
write_buffer_test.go enhance: make ColumnBasedInsertMsgToInsertData check field missing (#29758) 2024-01-09 11:50:48 +08:00
write_buffer.go enhance: Use estimated batch size to initalize BF (#29842) 2024-01-10 20:36:50 +08:00