test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-03 12:29:36 +08:00
Code Issues Actions 13 Packages Projects Releases Wiki Activity
5e90f348fc
milvus/internal/datanode/compaction
History
yihao.dai 81879425e1
enhance: Optimize the performance of stats task (#37374) 
1. Increase the writer's `batchSize` to avoid multiple serialization
operations.
2. Perform asynchronous upload of binlog files to prevent blocking the
data processing flow.
3. Reduce multiple calls to `writer.Flush()`.

issue: https://github.com/milvus-io/milvus/issues/37373

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-11-08 10:08:27 +08:00
..
clustering_compactor_test.go enhance: Optimize the performance of stats task (#37374) 2024-11-08 10:08:27 +08:00
clustering_compactor.go enhance: Optimize the performance of stats task (#37374) 2024-11-08 10:08:27 +08:00
compactor_common.go enhance: Optimize the performance of stats task (#37374) 2024-11-08 10:08:27 +08:00
compactor.go enhance: Add compaction task slot usage logic (#34581) 2024-07-18 10:27:41 +08:00
executor_test.go enhance: Add compaction task slot usage logic (#34581) 2024-07-18 10:27:41 +08:00
executor.go enhance: Add compaction task slot usage logic (#34581) 2024-07-18 10:27:41 +08:00
l0_compactor_test.go feat: Support stats task to sort segment by PK (#35054) 2024-09-02 14:19:03 +08:00
l0_compactor.go feat: Support stats task to sort segment by PK (#35054) 2024-09-02 14:19:03 +08:00
load_stats.go feat: support embedding bm25 sparse vector and flush bm25 stats log (#36036) 2024-09-19 10:57:12 +08:00
merge_sort.go fix: merge sort segment loss data (#37400) 2024-11-07 11:18:26 +08:00
mix_compactor_test.go enhance: Optimize the performance of stats task (#37374) 2024-11-08 10:08:27 +08:00
mix_compactor.go enhance: avoid memory copy and serde in mix compaction (#37479) 2024-11-07 16:30:57 -08:00
mock_compactor.go enhance: Add compaction task slot usage logic (#34581) 2024-07-18 10:27:41 +08:00
priority_queue_test.go feat: supporing hybrid search group_by (#35982) 2024-09-08 17:09:04 +08:00
priority_queue.go feat: Support stats task to sort segment by PK (#35054) 2024-09-02 14:19:03 +08:00
segment_reader_from_binlogs.go feat: support bm25 logs mixcompaction (#36072) 2024-10-14 16:57:22 +08:00
segment_writer_bench_test.go enhance: Optimize the performance of stats task (#37374) 2024-11-08 10:08:27 +08:00
segment_writer_test.go enhance: Optimize the performance of stats task (#37374) 2024-11-08 10:08:27 +08:00
segment_writer.go enhance: Optimize the performance of stats task (#37374) 2024-11-08 10:08:27 +08:00