Commit Graph

64 Commits

Author SHA1 Message Date
zhenshan.cao
72910d65e0
Fix inconsistency in DataCoord (#18170) (#18171)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-07-08 13:48:21 +08:00
zhenshan.cao
9bd35dd84e
Refine logs to reduce print (#17759) (#18028)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-07-04 11:10:20 +08:00
zhenshan.cao
0baeb609dc
Fix bugs in monitoring metrics (#17894)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-06-30 17:26:19 +08:00
yah01
ef6859a4d6
Refactor handoff handler (#17806)
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Co-authored-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-06-29 23:26:19 +08:00
XuanYang-cn
ff3c4ab2a8
Fix compaction panic (#17840)
See also: #17823

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-06-29 16:54:17 +08:00
cai.zhang
ed94ecf847
Segments can't be compacted when they have reference lock (#17649)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-20 21:56:12 +08:00
Xiaofan
516d91470f
Fix Delta log loss after compaction (#17634)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-06-19 20:12:11 +08:00
Letian Jiang
b15c24a554
Acquire reference lock of compacted from segments during index building (#17619)
- Accquire reference lock during index building
- Keep dropped segment record

Co-authored-by: Letian Jiang <letian.jiang@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-18 01:24:11 +08:00
wayblink
074ec3060a
Support return dropped segments info in GetSegmentInfo rpc (#17617)
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-06-17 18:24:12 +08:00
Bingyi Sun
d4fdfab818
Fix VDC-510 (#17554)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-06-15 19:58:10 +08:00
Xiaofan
6426e3d037
Split Etcd save operation into small txns (#17386)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-06-08 15:08:07 +08:00
cai.zhang
bcf3b7426a
Add distributed lock for segment refgerence (#16782)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-05-31 16:36:03 +08:00
zhenshan.cao
97757405de
Fix bug: wrong metrics name and forget to register metrics (#16714)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-04-29 15:35:47 +08:00
Ten Thousand Leaves
289e468a7a
Make bulk load fully work (#16512)
issue: #15604

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-20 14:03:40 +08:00
zhenshan.cao
f3eeecf146
Reduce the number of metrics (#16030)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-03-15 21:51:21 +08:00
yah01
0cfb6a85be
Make compaction not generate empty segment (#15707)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2022-02-28 10:17:54 +08:00
Bingyi Sun
daf25bcb81
Add log when remove keys in garbage collector. (#15453)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-02-18 11:59:49 +08:00
godchen
43e8c4f6e5
[skip e2e]Sort imports (#14812)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-01-05 14:10:35 +08:00
congqixia
c3423a09c8
Add comment for chanPartSegments (#14748)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-01-04 17:33:22 +08:00
godchen
3d593a5c30
[skip e2e]Sort imports (#14315)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2021-12-29 20:36:03 +08:00
guoxiangzhou
032c1c5f72
[skip e2e] Modify code comments according to naming conventions (#13942)
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
2021-12-22 15:21:22 +08:00
JackLCL
16769f8bfa
[skip e2e]Fix error log output format in meta.go (#13675)
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
2021-12-22 09:57:06 +08:00
Xiangyu Wang
8e78aeccaf
Improve import statement (#13726)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-12-20 18:13:05 +08:00
congqixia
1dcd06cc78
Add log file info for all kind of segment binlogs (#13606)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-19 20:00:42 +08:00
edward.zeng
f44ede7b9d
[skip ci] Refine datacoord meta comment (#13432)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-15 16:57:23 +08:00
xige-16
97d5231a0a
Reorganize querycoord proto (#13181)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-12-15 16:53:12 +08:00
Xiangyu Wang
405b3cd932
Use the same lint rules with golint in revive (#13324)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-12-14 15:31:07 +08:00
edward.zeng
4459611126
[skip ci] Refine datacoord meta comment (#13112)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-09 22:07:10 +08:00
JackLCL
de00aa038a
[skip ci]Fix error log output format in meta.go (#12854)
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
2021-12-07 14:32:51 +08:00
congqixia
22fddc537c
Fix not all segment state is drop after DropVirtualChannel (#12702)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-03 18:55:34 +08:00
congqixia
8f9e62fa18
Apply DropVirtualChannel and FlushManager drop mode (#12563)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-02 16:39:33 +08:00
congqixia
06ced4d1d1
Add DropVirtualChannel for DataCoord (#12361)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-29 22:35:41 +08:00
congqixia
eb48013042
Set start position for compact result segment (#12280)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-25 17:19:17 +08:00
Bingyi Sun
3be9442c0f
Clear dropped segment meta on etcd (#12191)
issue: #9613
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-11-23 11:23:15 +08:00
congqixia
8c107ab15f
Set segment DroppedAt when compacted (#12182)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-22 20:11:14 +08:00
congqixia
7e43b568ee
Add dropped ts and apply when gc scan dropped segments (#11838)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-16 14:23:21 +08:00
Bingyi Sun
54b40da4c5
Add segment drop logic (#11621)
issue: #11558
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-11-12 00:22:42 +08:00
congqixia
3c9610cb1b
Add garbage collector for DataCoord (#11554)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-10 19:03:38 +08:00
sunby
05681364c9
Fix some review comments in compaction. related pr #9906 (#11399)
issue: #9904
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2021-11-08 21:45:00 +08:00
sunby
db6ecd58f2
Fix UpdateFlushSegmentsInfo bug (#11407)
issue: #7313
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2021-11-08 14:16:56 +08:00
sunby
c26cd79532
Add compaction (#9906)
issue: #9904
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-11-05 22:25:00 +08:00
XuanYang-cn
fe955b888f
[skip ci]Update license for dc meta (#10592)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-10-25 20:21:18 +08:00
xige-16
9ebabdac79
Add handoffTask (#10330)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-10-24 22:39:09 +08:00
sunby
4036830b2d
Fix modify segment without clone (#10498)
issue: #10430
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-10-23 13:31:09 +08:00
congqixia
7ce7cb7a5e
Add a timed flush trigger mechanism (#10197)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-20 15:02:36 +08:00
congqixia
6a886d63f2
Support stats and delta log SaveBinlog logic (#10156)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-19 14:32:41 +08:00
sunby
e811a66527
Fix save binlog path timeout bug (#9573)
issue: #9559
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-10-12 23:46:35 +08:00
godchen
d08370d06b
[skip ci]Fix meta go comment typo (#9378)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2021-10-07 10:36:03 +08:00
sunby
88b4439fdf
[skip ci]Add comment for package datacoord (#9014)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-10-05 22:33:01 +08:00
zhenshan.cao
45d6b8dcd0
[skip ci] Fix golint error in datacoord (#9184)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-10-04 17:16:54 +08:00