14 Commits

Author	SHA1	Message	Date
aoiasd	139787371e	feat: support embedding bm25 sparse vector and flush bm25 stats log (#36036) ... relate: https://github.com/milvus-io/milvus/issues/35853 --------- Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>	2024-09-19 10:57:12 +08:00
cai.zhang	8395c8a8db	enhance: Update stats task to optional (#35947) ... issue: #33744 --------- Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>	2024-09-12 20:37:08 +08:00
XuanYang-cn	2687747278	fix: Set an empty segment if compaction deleted all inserts (#36044) ... See also: #36038 --------- Signed-off-by: yangxuan <xuan.yang@zilliz.com>	2024-09-09 14:23:05 +08:00
cai.zhang	2c9bb4dfa3	feat: Support stats task to sort segment by PK (#35054) ... issue: #33744 This PR includes the following changes: 1. Added a new task type to the task scheduler in datacoord: stats task, which sorts segments by primary key. 2. Implemented segment sorting in indexnode. 3. Added a new field `FieldStatsLog` to SegmentInfo to store token index information. --------- Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>	2024-09-02 14:19:03 +08:00
yihao.dai	1413ffe9b1	enhance: Rename preAllocatedSegments (#35871) ... Rename `preAllocatedSegments` to `preAllocatedSegmentIDs` to avoid confusion. Signed-off-by: bigsheeper <yihao.dai@zilliz.com>	2024-09-01 17:09:01 +08:00
XuanYang-cn	323400c190	enhance: Enable to write multiple segments in mix compactor (#35705) ... Prevent segments to be written larger than maxSize * expansionRate See also: #35584 Signed-off-by: yangxuan <xuan.yang@zilliz.com> --------- Signed-off-by: yangxuan <xuan.yang@zilliz.com>	2024-08-30 11:29:01 +08:00
congqixia	ab532ae199	enhance: Add back BF lazy load logic for datanode watch channel (#35646) ... Add back lazy loading statslog when watch dml channel on datanode. Related to #22994 #27675 Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>	2024-08-22 19:42:57 +08:00
yihao.dai	a4439cc911	enhance: Implement flusher in streamingNode (#34942) ... - Implement flusher to: - Manage the pipelines (creation, deletion, etc.) - Manage the segment write buffer - Manage sync operation (including receive flushMsg and execute flush) - Add a new `GetChannelRecoveryInfo` RPC in DataCoord. - Reorganize packages: `flushcommon` and `datanode`. issue: https://github.com/milvus-io/milvus/issues/33285 --------- Signed-off-by: bigsheeper <yihao.dai@zilliz.com>	2024-08-02 18:30:23 +08:00
yihao.dai	8aab6cbfac	enhance: Organize the common modules of streamingNode and dataNode (#34773) ... 1. Move the common modules of streamingNode and dataNode to flushcommon 2. Add new GetVChannels interface for rootcoord issue: https://github.com/milvus-io/milvus/issues/33285 --------- Signed-off-by: bigsheeper <yihao.dai@zilliz.com>	2024-07-22 11:33:51 +08:00
SimFG	16c002b51b	enhance: unified channel name generation and conversion (#34805) ... /kind improvement - issue: #34804 Signed-off-by: SimFG <bang.fu@zilliz.com>	2024-07-20 23:11:39 +08:00
yihao.dai	ca758c36cc	enhance: Pre-allocate ids for compaction (#34187) ... This PR removes the dependency of compaction on the ID allocator by pre-allocating the logID and segmentID. issue: https://github.com/milvus-io/milvus/issues/33957 --------- Signed-off-by: bigsheeper <yihao.dai@zilliz.com>	2024-07-17 13:23:42 +08:00
wayblink	a1232fafda	feat: Major compaction (#33620) ... #30633 Signed-off-by: wayblink <anyang.wang@zilliz.com> Co-authored-by: MrPresent-Han <chun.han@zilliz.com>	2024-06-10 21:34:08 +08:00
yihao.dai	7730b910b9	enhance: Decouple compaction from shard (#33138) ... Decouple compaction from shard, remove dependencies on shards (e.g. SyncSegments, injection). issue: https://github.com/milvus-io/milvus/issues/32809 --------- Signed-off-by: bigsheeper <yihao.dai@zilliz.com>	2024-05-24 09:07:41 +08:00
XuanYang-cn	22bddde5ff	enhance: Tidy compactor and remove dup codes (#32198) ... See also: #32451 Signed-off-by: yangxuan <xuan.yang@zilliz.com>	2024-05-23 09:53:40 +08:00