issue: #33744
This PR includes the following changes:
1. Added a new task type to the task scheduler in datacoord: stats task,
which sorts segments by primary key.
2. Implemented segment sorting in indexnode.
3. Added a new field `FieldStatsLog` to SegmentInfo to store token index
information.
---------
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
Prevent segments to be written larger than maxSize * expansionRate
See also: #35584
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
---------
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
Related to #28861
Move session manager, worker manager to session package. Also renaming
each manager to corresponding node name(datanode, indexnode).
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Related to #28861
Move allocator interface and implementation into separate package. Also
update some unittest logic.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
This PR removes the dependency of compaction on the ID allocator by
pre-allocating the logID and segmentID.
issue: https://github.com/milvus-io/milvus/issues/33957
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
The nodeID for compaction task initialization is 0. This PR adjusts the
task reassignment conditions to allow new compaction tasks to be
reassigned and executed.
issue: https://github.com/milvus-io/milvus/issues/34460
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
