Commit Graph

7 Commits

Author SHA1 Message Date
XuanYang-cn
04edb07d82
enhance: Add deltaRowCount in l0 compaction (#33997)
See also: #33998

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-06-20 17:46:01 +08:00
XuanYang-cn
def72947c7
fix: Trigger l0 compaction when l0 views don't change (#30729)
Trigger l0 compaction when l0 views don't change

So that leftover l0 segments would be compacted in the end.

1. Refresh LevelZero plans in comactionPlanHandler, remove the meta
dependency
of compaction trigger v2
2. Add ForceTrigger method for CompactionView interface
3. rename mu to taskGuard
4. Add a new TriggerTypeLevelZeroViewIDLE
5. Add an idleTicker for compaction view manager

See also: #30098, #30556

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-03-05 16:37:00 +08:00
XuanYang-cn
44d436d0b6
enhance: Add force trigger (#30641)
1. Increase maxCount of L0 compaction tasks to 30

This could reduce the l0 compaction task number by 30% for
high-frequently-generated-small l0 segments, with the maximum size 64MB
stay not changed. So that l0 segments would accumulate slower and
decrease the mem presure caused by L0 segment for QueryNode

2. Add force Trigger for later manual timely l0 compaction triggers.

See also: #30191, #30556

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-02-19 18:40:50 +08:00
XuanYang-cn
e0ed5647b3
fix: Limit L0 Compaction segment size and count (#30374)
See also: #30191

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-02-01 20:39:03 +08:00
XuanYang-cn
2c62bcbbf6
fix: Fix some left comments (#29027)
See also: #28814

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-12-13 18:48:39 +08:00
XuanYang-cn
5bac7f7897
fix: Fix L0 compaction in datacoord (#28814)
See also: #27606

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-12-05 18:44:37 +08:00
XuanYang-cn
9b371067d2
feat: Add Compaction views and triggers (#27906)
- Add Compaction l0 views
- Add Compaction scheduler
- Add Compaction triggerv2
- Add Compaction view manager

See also: #27606

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-11-23 17:30:25 +08:00