Commit Graph

17 Commits

Author SHA1 Message Date
cai.zhang
8165961223
enhance: Optimeize clustering compaction (#34313)
issue: #30633

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-07-03 15:14:08 +08:00
jaime
d6afb31b94
enhance: make subfunctions of datanode component modular (#33992)
issue: #33994

also remove deprecated channel manager based on the etcd implementation

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-07-01 14:46:07 +08:00
wayblink
e5d691d854
Use new stream segment reader in clustering compaction (#34232)
#32939

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-06-30 20:26:07 +08:00
wayblink
fbe3231b1f
fix: fix error ignore in compactor (#34169)
#34170

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-06-26 10:24:03 +08:00
wayblink
380d3f4469
fix: Fix memory buffer error & some renaming (#33850)
#30633

---------

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-06-21 17:30:01 +08:00
XuanYang-cn
04edb07d82
enhance: Add deltaRowCount in l0 compaction (#33997)
See also: #33998

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-06-20 17:46:01 +08:00
wei liu
31ef0a1fe8
enhance: Add trace for bf cost in l0 compactor (#33860)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-06-20 10:10:05 +08:00
wayblink
5cb0760187
fix: Small fixs of major compaction (#33929)
#30633

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-06-18 16:53:58 +08:00
yihao.dai
1a9ab52f66
enhance: Ensure the idempotency of compaction task (#33872)
/kind enhancement

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-06-16 22:09:57 +08:00
yihao.dai
86a36b105a
enhance: Tidy compaction executor (#33778)
Move compaction executor to compaction pacakge.

issue: https://github.com/milvus-io/milvus/issues/32451

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-06-14 14:34:01 +08:00
wei liu
4987067375
enhance: Execute bloom filter apply in parallel to speed up segment predict (#33792)
issue: #33610

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-06-14 11:37:56 +08:00
wei liu
ab93d9c23d
enhance: Use BatchPkExist to reduce bloom filter func call cost (#33611)
issue:#33610

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-06-13 17:57:56 +08:00
wayblink
a1232fafda
feat: Major compaction (#33620)
#30633

Signed-off-by: wayblink <anyang.wang@zilliz.com>
Co-authored-by: MrPresent-Han <chun.han@zilliz.com>
2024-06-10 21:34:08 +08:00
XuanYang-cn
4dd0c54ca0
fix: Fix l0 compactor may cause DN from OOM (#33554)
See also: #33547

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-06-06 14:33:52 +08:00
cai.zhang
77637180fa
enhance: Periodically synchronize segments to datanode watcher (#33420)
issue: #32809

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-05-30 13:37:44 +08:00
yihao.dai
7730b910b9
enhance: Decouple compaction from shard (#33138)
Decouple compaction from shard, remove dependencies on shards (e.g.
SyncSegments, injection).

issue: https://github.com/milvus-io/milvus/issues/32809

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-05-24 09:07:41 +08:00
XuanYang-cn
22bddde5ff
enhance: Tidy compactor and remove dup codes (#32198)
See also: #32451

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-05-23 09:53:40 +08:00