milvus/internal/querynodev2/segments
Bingyi Sun a75bb85f3a
feat: support chunked column for sealed segment (#35764)
This PR splits sealed segment to chunked data to avoid unnecessary
memory copy and save memory usage when loading segments so that loading
can be accelerated.

To support rollback to previous version, we add an option
`multipleChunkedEnable` which is false by default.

Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-10-12 15:04:52 +08:00
..
metricsutil enhance: add more metrics for cache and search (#31777) 2024-04-10 10:55:17 +08:00
state fix: async warmup will be blocked by state lock (#33686) 2024-06-10 21:59:53 +08:00
cgo_util.go enhance: optimize milvus core building (#35610) 2024-08-23 12:35:02 +08:00
collection.go feat: support load and query with bm25 metric (#36071) 2024-10-11 10:23:20 +08:00
count_reducer_test.go enhance: retrieve output fields after local reduce (#32346) 2024-04-25 09:49:26 +08:00
count_reducer.go enhance: retrieve output fields after local reduce (#32346) 2024-04-25 09:49:26 +08:00
default_limit_reducer.go fix: iterator cursor progress too fast(#36179) (#36180) 2024-09-24 11:45:13 +08:00
index_attr_cache_test.go fix: predict inverted index resource usage more reasonably (#31615) 2024-03-27 19:33:09 +08:00
index_attr_cache.go enhance: optimize milvus core building (#35610) 2024-08-23 12:35:02 +08:00
load_field_data_info.go enhance: optimize milvus core building (#35610) 2024-08-23 12:35:02 +08:00
load_index_info.go enhance: the estimate method when loading the collection (#36307) 2024-10-09 17:35:19 +08:00
manager_test.go enhance: growing segment support mmap (#32633) 2024-06-18 14:42:00 +08:00
manager.go enhance: optimize milvus core building (#35610) 2024-08-23 12:35:02 +08:00
mock_collection_manager.go fix: Remove QueryNodeEntitiesSize after segment/collection released (#31290) 2024-03-15 15:43:04 +08:00
mock_data.go feat: support load and query with bm25 metric (#36071) 2024-10-11 10:23:20 +08:00
mock_loader.go feat: support load and query with bm25 metric (#36071) 2024-10-11 10:23:20 +08:00
mock_segment_manager.go fix: load operation when segment is on releasing (#31340) 2024-06-14 15:35:56 +08:00
mock_segment.go feat: support load and query with bm25 metric (#36071) 2024-10-11 10:23:20 +08:00
OWNERS Fix offline set always nil (#23340) 2023-04-11 14:42:30 +08:00
plan_test.go enhance: Update protobuf-go to protobuf-go v2 (#34394) 2024-07-29 11:31:51 +08:00
plan.go enhance: optimize milvus core building (#35610) 2024-08-23 12:35:02 +08:00
pool_test.go enhance: Execute bloom filter apply in parallel to speed up segment predict (#33792) 2024-06-14 11:37:56 +08:00
pool.go enhance: Execute bloom filter apply in parallel to speed up segment predict (#33792) 2024-06-14 11:37:56 +08:00
reduce_test.go enhance: provide more general configuration to control mmap behavior (#35359) 2024-08-21 00:22:54 +08:00
reduce.go enhance: optimize milvus core building (#35610) 2024-08-23 12:35:02 +08:00
reducer_test.go fix: two-phase retrieval on lru-segment (#32945) 2024-05-15 17:53:34 +08:00
reducer.go fix: Return record with largest timestamp for entires with same PK (#33936) 2024-06-18 15:55:59 +08:00
result_sorter.go Fix bug of missing JSON type when sorting retrieve results (#25412) (#25455) 2023-07-10 20:02:28 +08:00
result_test.go enhance: Fix PR conflict in reduce unit test (#36470) 2024-09-24 18:01:13 +08:00
result.go fix: iterator cursor progress too fast(#36179) (#36180) 2024-09-24 11:45:13 +08:00
retrieve_test.go enhance: provide more general configuration to control mmap behavior (#35359) 2024-08-21 00:22:54 +08:00
retrieve.go enhance: Return segment id hint in QueryStream response (#36487) 2024-09-26 10:13:14 +08:00
search_reduce_test.go feat: supporing hybrid search group_by (#35982) 2024-09-08 17:09:04 +08:00
search_reduce.go feat: supporing hybrid search group_by (#35982) 2024-09-08 17:09:04 +08:00
search_test.go enhance: provide more general configuration to control mmap behavior (#35359) 2024-08-21 00:22:54 +08:00
search.go fix: add request resource timeout for lazy load, refactor context usage in cache (#32709) 2024-05-07 16:33:30 +08:00
segment_do.go fix: two-phase retrieval on lru-segment (#32945) 2024-05-15 17:53:34 +08:00
segment_interface.go feat: support load and query with bm25 metric (#36071) 2024-10-11 10:23:20 +08:00
segment_l0.go enhance: remove unused code for StorageV2 (#35132) 2024-08-01 12:08:13 +08:00
segment_loader_test.go feat: support load and query with bm25 metric (#36071) 2024-10-11 10:23:20 +08:00
segment_loader.go feat: support load and query with bm25 metric (#36071) 2024-10-11 10:23:20 +08:00
segment_test.go enhance: provide more general configuration to control mmap behavior (#35359) 2024-08-21 00:22:54 +08:00
segment.go feat: support chunked column for sealed segment (#35764) 2024-10-12 15:04:52 +08:00
statistics.go Protect segment from being released while query/search (#26322) 2023-08-16 18:38:17 +08:00
trace.go enhance: optimize milvus core building (#35610) 2024-08-23 12:35:02 +08:00
utils_test.go enhance: provide more general configuration to control mmap behavior (#35359) 2024-08-21 00:22:54 +08:00
utils.go enhance: the estimate method when loading the collection (#36307) 2024-10-09 17:35:19 +08:00
validate.go fix: Add defer Unpin when error happens (#36620) 2024-09-30 19:49:17 +08:00