SimFG
|
d23f87a393
|
enhance: Add concurrency for datacoord segment GC (#29561)
issue: https://github.com/milvus-io/milvus/issues/29553
/kind improvement
Signed-off-by: SimFG <bang.fu@zilliz.com>
|
2024-01-03 13:16:57 +08:00 |
|
congqixia
|
8a63e53421
|
enhance: Add http method to control datacoord garbage collection (#29052)
See also #29051
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
|
2023-12-14 19:26:39 +08:00 |
|
congqixia
|
1d76565894
|
Add metrics for garbage collection (#27303)
Also fix second metrics usage in compaction
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
|
2023-09-22 18:47:25 +08:00 |
|
SimFG
|
26f06dd732
|
Format the code (#27275)
Signed-off-by: SimFG <bang.fu@zilliz.com>
|
2023-09-21 09:45:27 +08:00 |
|
congqixia
|
19e4deb792
|
Make garbage collector fetch meta after list from storage (#27203)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
|
2023-09-19 17:05:34 +08:00 |
|
PowderLi
|
4feb3fa7c6
|
support azure (#26398)
Signed-off-by: PowderLi <min.li@zilliz.com>
|
2023-09-19 10:01:23 +08:00 |
|
yah01
|
168e82ee10
|
Fix panic while handling with the nil status (#27040)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
|
2023-09-15 10:09:21 +08:00 |
|
congqixia
|
f8bcf60e55
|
Adjust some confusing Warning log to INFO (#26356)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
|
2023-08-15 17:49:33 +08:00 |
|
zhenshan.cao
|
7619db1edc
|
Fast GC compacted dropped segment (#26207)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
|
2023-08-09 19:39:15 +08:00 |
|
Zhihong Yu
|
150947a5f4
|
Conditionally log warning when DropSegment fails (#25774)
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
|
2023-07-25 10:39:01 +08:00 |
|
cai.zhang
|
b15e34db21
|
Add contraint for compaction based indexed segments (#25709)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
|
2023-07-23 21:31:00 +08:00 |
|
Enwei Jiao
|
66fdc71479
|
Refactor logs in DataCoord & DataNode (#25574)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
|
2023-07-14 15:56:31 +08:00 |
|
Xiaofan
|
dbf0130803
|
Faster garbage collect on compacted data (#25088)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
|
2023-07-12 10:14:29 +08:00 |
|
congqixia
|
41af0a98fa
|
Use go-api/v2 for milvus-proto (#24770)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
|
2023-06-09 01:28:37 +08:00 |
|
zhenshan.cao
|
dc5abe086c
|
Revert "Restrain gc files by collection (#22504)" (#23380)
This reverts commit fa86de530d.
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
|
2023-04-12 19:38:28 +08:00 |
|
cai.zhang
|
9288020da3
|
Fix bug for batch delete files on gcp of minio (#23052) (#23083) (#23090)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
|
2023-04-09 16:22:30 +08:00 |
|
jaime
|
c9d0c157ec
|
Move some modules from internal to public package (#22572)
Signed-off-by: jaime <yun.zhang@zilliz.com>
|
2023-04-06 19:14:32 +08:00 |
|
wayblink
|
aa6212d5b0
|
[Cherry-pick] Fix channel checkpoint issues and fix GC (#22559)
Signed-off-by: wayblink <anyang.wang@zilliz.com>
Co-authored-by: Ten Thousand Leaves <69466447+soothing-rain@users.noreply.github.com>
|
2023-03-09 14:13:52 +08:00 |
|
yah01
|
319f1773af
|
Fix segment not found if forward delete to empty segment (#22528)
Signed-off-by: yah01 <yang.cen@zilliz.com>
|
2023-03-03 14:13:49 +08:00 |
|
congqixia
|
fa86de530d
|
Restrain gc files by collection (#22504)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
|
2023-03-02 11:49:47 +08:00 |
|
zhenshan.cao
|
e768437681
|
Correct usage of Timer and Ticker (#22228)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
|
2023-02-23 18:59:45 +08:00 |
|
congqixia
|
787ce7c121
|
Add Rated group for checkpoint gc skip case (#21949)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
|
2023-02-03 14:23:56 +08:00 |
|
congqixia
|
42ab1b0a48
|
Add segment gc restraint of dml position before channel cp (#21564)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
|
2023-01-06 21:33:36 +08:00 |
|
cai.zhang
|
aa203acfb3
|
Low IndexCoord weight (#21548)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
|
2023-01-06 14:21:37 +08:00 |
|
cai.zhang
|
e5f408dceb
|
Merge IndexCoord and DataCoord (#21267)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
|
2023-01-04 19:37:36 +08:00 |
|
SimFG
|
a55f739608
|
Separate public proto files (#19782)
Signed-off-by: SimFG <bang.fu@zilliz.com>
Signed-off-by: SimFG <bang.fu@zilliz.com>
|
2022-10-16 20:49:27 +08:00 |
|
yah01
|
89c9cb3680
|
Fix scan may break GC limitation (#19670)
Signed-off-by: yah01 <yang.cen@zilliz.com>
Signed-off-by: yah01 <yang.cen@zilliz.com>
|
2022-10-15 15:13:24 +08:00 |
|
jaime
|
ab88dd77e7
|
Remove unnecessary log for datacoord gc (#19698)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
|
2022-10-13 14:43:23 +08:00 |
|
congqixia
|
2a2d32c4fc
|
Fix datacoord panics when collection info not found (#19706)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
|
2022-10-11 21:39:24 +08:00 |
|
jaime
|
1919353f02
|
Fix parse segment id error from delta log (#19566)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
|
2022-09-30 14:18:55 +08:00 |
|
congqixia
|
838a633584
|
Add ctx parameter for ChunkManager methods (#19546)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
|
2022-09-29 16:18:56 +08:00 |
|
Ten Thousand Leaves
|
b30c9d4f60
|
Support Bulk Load in Milvus 2.2 (#18982)
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
|
2022-09-26 18:06:54 +08:00 |
|
Xiaofan
|
928a213e31
|
Fix Chunk Manager Path mis use (#19141)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
|
2022-09-23 14:40:51 +08:00 |
|
SimFG
|
d7f38a803d
|
Separate some proto files (#19218)
Signed-off-by: SimFG <bang.fu@zilliz.com>
Signed-off-by: SimFG <bang.fu@zilliz.com>
|
2022-09-16 16:56:49 +08:00 |
|
yah01
|
11b352c3ae
|
Guarantee flushed segments are all indexed (#19062)
Signed-off-by: yah01 <yang.cen@zilliz.com>
Signed-off-by: yah01 <yang.cen@zilliz.com>
|
2022-09-16 11:32:48 +08:00 |
|
jaime
|
efa5dfaa7b
|
Fix Parse binlog path failure (#18583)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
|
2022-08-10 17:04:37 +08:00 |
|
xing.zhao
|
d90313a1a8
|
Use chunkManager rather than minio.Client in datanode garbage collection (#18092)
Signed-off-by: xingzhao <xing.zhao@zilliz.com>
Co-authored-by: xingzhao <xing.zhao@zilliz.com>
|
2022-07-22 22:10:28 +08:00 |
|
Xiaofan
|
6d82ef8c20
|
fix garbage collector err handling (#18277)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
|
2022-07-15 13:54:26 +08:00 |
|
cai.zhang
|
bcf3b7426a
|
Add distributed lock for segment refgerence (#16782)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
|
2022-05-31 16:36:03 +08:00 |
|
Xiaofan
|
bd7d294c6d
|
Refine log level in data (#15821)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
|
2022-03-02 15:35:55 +08:00 |
|
Bingyi Sun
|
daf25bcb81
|
Add log when remove keys in garbage collector. (#15453)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
|
2022-02-18 11:59:49 +08:00 |
|
godchen
|
a0534d3139
|
[skip e2e]Sort imports (#14814)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
|
2022-01-05 14:14:21 +08:00 |
|
godchen
|
d29243cbc5
|
[skip e2e]Sort imports (#14317)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
|
2021-12-30 16:49:37 +08:00 |
|
guoxiangzhou
|
1dc4418059
|
[skip e2e] Modify log text according to naming conventions (#13710)
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
|
2021-12-20 17:42:07 +08:00 |
|
congqixia
|
1dcd06cc78
|
Add log file info for all kind of segment binlogs (#13606)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
|
2021-12-19 20:00:42 +08:00 |
|
edward.zeng
|
dbae087d21
|
[skip ci] Refine datacoord garbage collector comment (#13430)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
|
2021-12-15 18:25:43 +08:00 |
|
congqixia
|
7b870e3392
|
Limit datacoord gc scan prefixes (#12231)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
|
2021-11-24 09:55:15 +08:00 |
|
Bingyi Sun
|
3be9442c0f
|
Clear dropped segment meta on etcd (#12191)
issue: #9613
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
|
2021-11-23 11:23:15 +08:00 |
|
congqixia
|
8c107ab15f
|
Set segment DroppedAt when compacted (#12182)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
|
2021-11-22 20:11:14 +08:00 |
|
congqixia
|
b99164f43d
|
Add datacoord GC configuration (#12061)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
|
2021-11-18 22:29:40 +08:00 |
|