congqixia
5f5fdb0789
Compaction bypass stale segments ( #20202 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-10-31 19:13:34 +08:00
SimFG
a55f739608
Separate public proto files ( #19782 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-10-16 20:49:27 +08:00
congqixia
2a2d32c4fc
Fix datacoord panics when collection info not found ( #19706 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-10-11 21:39:24 +08:00
jaime
31db01b6ae
Support TTL for collection level ( #19383 )
...
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-10-10 20:31:22 +08:00
Ten Thousand Leaves
b30c9d4f60
Support Bulk Load in Milvus 2.2 ( #18982 )
...
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-09-26 18:06:54 +08:00
zhagnlu
59f8e96522
Change compactor policy from loading all binlogs to loading serially ( #13716 ) ( #19280 )
...
Signed-off-by: zhagnlu <lu.zhang@zilliz.com>
Signed-off-by: zhagnlu <lu.zhang@zilliz.com>
Co-authored-by: zhagnlu <lu.zhang@zilliz.com>
2022-09-25 20:12:52 +08:00
yah01
ee9d634a43
Compaction requires the source segments indexed ( #19368 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-09-22 18:34:50 +08:00
yah01
71d4a32ce6
Fix deadlock ( #19340 )
...
The IsParentDropped() acquires read lock of meta,
which leads to recursively acquire read lock
Signed-off-by: yah01 <yang.cen@zilliz.com>
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-09-22 15:48:50 +08:00
SimFG
d7f38a803d
Separate some proto files ( #19218 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-09-16 16:56:49 +08:00
yah01
11b352c3ae
Guarantee flushed segments are all indexed ( #19062 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-09-16 11:32:48 +08:00
Enwei Jiao
096149e33f
fix GetCompactionState hang ( #18829 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-08-26 14:22:55 +08:00
cai.zhang
c924f73105
Refactor for IndexCoord to support cloud ( #18643 )
...
Co-authored-by: Zach41 <zongmei.zhang@zilliz.com>
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: Zach41 <zongmei.zhang@zilliz.com>
2022-08-25 15:48:54 +08:00
jaime
c9174d55ba
Refine merge operation during compacting phase ( #18399 )
...
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-07-28 14:52:31 +08:00
yah01
ef6859a4d6
Refactor handoff handler ( #17806 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: xiaofan-luan <xiaofan.luan@zilliz.com>
Co-authored-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-06-29 23:26:19 +08:00
Xiaofan
6fdf88f452
Fix compaction selection policy ( #17690 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-06-22 19:00:14 +08:00
cai.zhang
ed94ecf847
Segments can't be compacted when they have reference lock ( #17649 )
...
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-20 21:56:12 +08:00
Xiaofan
54aeb077e9
Refine compaction selection poliy ( #17486 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-06-15 23:14:10 +08:00
XuanYang-cn
c70af73005
Fix datacoord set wrong state for node registering ( #17376 )
...
Fix datacoord datarace
See also: #17335
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-06-06 15:10:05 +08:00
Xiaofan
bd7d294c6d
Refine log level in data ( #15821 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-03-02 15:35:55 +08:00
congqixia
5751759c85
Implement mix compaction logic ( #15542 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-02-18 14:51:49 +08:00
Xiaofan
be42627ab2
Move Logutil to Util dir ( #15447 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-02-09 18:55:46 +08:00
godchen
f2baa987fb
Handle trigger compaction error ( #13636 )
...
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-01-07 17:41:40 +08:00
godchen
3874edf163
[skip e2e]Sort imports ( #14815 )
...
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-01-05 14:16:17 +08:00
Xiangyu Wang
80ffa9c4d0
[skip e2e]Fix license missing issue in datacoord ( #14489 )
...
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-12-29 11:38:57 +08:00
Xiaofan
1408926cd9
Remove logs without enough info ( #14440 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-12-29 10:06:47 +08:00
godchen
b8dcbf6604
[skip e2e]Sort imports ( #14318 )
...
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2021-12-27 18:46:42 +08:00
Cai Yudong
891c202b73
Use GlobalParamTable for all components ( #13885 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-12-23 18:39:11 +08:00
congqixia
74ae8d3923
Fix nil segment caused panic in handleSignal ( #13990 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-22 21:23:10 +08:00
congqixia
1dcd06cc78
Add log file info for all kind of segment binlogs ( #13606 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-19 20:00:42 +08:00
edward.zeng
098e1375fb
[skip ci] Refine datacoord compaction trigger comment ( #13491 )
...
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-16 09:59:12 +08:00
edward.zeng
d6b0a47248
[skip ci] Refine datacoord compaction trigger comment ( #13483 )
...
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-16 09:57:25 +08:00
congqixia
8476791492
Process only flushed/flushing segment in global compaction ( #13339 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-14 15:15:10 +08:00
Bingyi Sun
3a2aace6f0
Fix ci issue caused by only compact flushed segments ( #13317 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-14 13:55:07 +08:00
congqixia
aea5a61e9a
Add switch for auto compaction ( #12974 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-08 19:47:05 +08:00
edward.zeng
f8c826c597
Fix typo in datacoord compaction trigger ( #12636 )
...
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-02 19:09:37 +08:00
Bingyi Sun
b8c5239784
Add compaction retention config ( #11793 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-11-16 11:47:10 +08:00
Bingyi Sun
54b40da4c5
Add segment drop logic ( #11621 )
...
issue: #11558
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-11-12 00:22:42 +08:00
Bingyi Sun
5574a104c9
Add triggerInfo in compactionTask ( #11517 )
...
issue: #9904
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-11-11 15:54:42 +08:00
sunby
05681364c9
Fix some review comments in compaction. related pr #9906 ( #11399 )
...
issue: #9904
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2021-11-08 21:45:00 +08:00
sunby
c26cd79532
Add compaction ( #9906 )
...
issue: #9904
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-11-05 22:25:00 +08:00