Commit Graph

51 Commits

Author SHA1 Message Date
Ten Thousand Leaves
289e468a7a
Make bulk load fully work (#16512)
issue: #15604

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-20 14:03:40 +08:00
zhenshan.cao
f3eeecf146
Reduce the number of metrics (#16030)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-03-15 21:51:21 +08:00
yah01
0cfb6a85be
Make compaction not generate empty segment (#15707)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2022-02-28 10:17:54 +08:00
Bingyi Sun
daf25bcb81
Add log when remove keys in garbage collector. (#15453)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-02-18 11:59:49 +08:00
godchen
43e8c4f6e5
[skip e2e]Sort imports (#14812)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-01-05 14:10:35 +08:00
congqixia
c3423a09c8
Add comment for chanPartSegments (#14748)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-01-04 17:33:22 +08:00
godchen
3d593a5c30
[skip e2e]Sort imports (#14315)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2021-12-29 20:36:03 +08:00
guoxiangzhou
032c1c5f72
[skip e2e] Modify code comments according to naming conventions (#13942)
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
2021-12-22 15:21:22 +08:00
JackLCL
16769f8bfa
[skip e2e]Fix error log output format in meta.go (#13675)
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
2021-12-22 09:57:06 +08:00
Xiangyu Wang
8e78aeccaf
Improve import statement (#13726)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-12-20 18:13:05 +08:00
congqixia
1dcd06cc78
Add log file info for all kind of segment binlogs (#13606)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-19 20:00:42 +08:00
edward.zeng
f44ede7b9d
[skip ci] Refine datacoord meta comment (#13432)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-15 16:57:23 +08:00
xige-16
97d5231a0a
Reorganize querycoord proto (#13181)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-12-15 16:53:12 +08:00
Xiangyu Wang
405b3cd932
Use the same lint rules with golint in revive (#13324)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-12-14 15:31:07 +08:00
edward.zeng
4459611126
[skip ci] Refine datacoord meta comment (#13112)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-09 22:07:10 +08:00
JackLCL
de00aa038a
[skip ci]Fix error log output format in meta.go (#12854)
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
2021-12-07 14:32:51 +08:00
congqixia
22fddc537c
Fix not all segment state is drop after DropVirtualChannel (#12702)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-03 18:55:34 +08:00
congqixia
8f9e62fa18
Apply DropVirtualChannel and FlushManager drop mode (#12563)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-02 16:39:33 +08:00
congqixia
06ced4d1d1
Add DropVirtualChannel for DataCoord (#12361)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-29 22:35:41 +08:00
congqixia
eb48013042
Set start position for compact result segment (#12280)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-25 17:19:17 +08:00
Bingyi Sun
3be9442c0f
Clear dropped segment meta on etcd (#12191)
issue: #9613
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-11-23 11:23:15 +08:00
congqixia
8c107ab15f
Set segment DroppedAt when compacted (#12182)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-22 20:11:14 +08:00
congqixia
7e43b568ee
Add dropped ts and apply when gc scan dropped segments (#11838)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-16 14:23:21 +08:00
Bingyi Sun
54b40da4c5
Add segment drop logic (#11621)
issue: #11558
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-11-12 00:22:42 +08:00
congqixia
3c9610cb1b
Add garbage collector for DataCoord (#11554)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-10 19:03:38 +08:00
sunby
05681364c9
Fix some review comments in compaction. related pr #9906 (#11399)
issue: #9904
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2021-11-08 21:45:00 +08:00
sunby
db6ecd58f2
Fix UpdateFlushSegmentsInfo bug (#11407)
issue: #7313
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2021-11-08 14:16:56 +08:00
sunby
c26cd79532
Add compaction (#9906)
issue: #9904
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-11-05 22:25:00 +08:00
XuanYang-cn
fe955b888f
[skip ci]Update license for dc meta (#10592)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-10-25 20:21:18 +08:00
xige-16
9ebabdac79
Add handoffTask (#10330)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-10-24 22:39:09 +08:00
sunby
4036830b2d
Fix modify segment without clone (#10498)
issue: #10430
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-10-23 13:31:09 +08:00
congqixia
7ce7cb7a5e
Add a timed flush trigger mechanism (#10197)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-20 15:02:36 +08:00
congqixia
6a886d63f2
Support stats and delta log SaveBinlog logic (#10156)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-19 14:32:41 +08:00
sunby
e811a66527
Fix save binlog path timeout bug (#9573)
issue: #9559
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-10-12 23:46:35 +08:00
godchen
d08370d06b
[skip ci]Fix meta go comment typo (#9378)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2021-10-07 10:36:03 +08:00
sunby
88b4439fdf
[skip ci]Add comment for package datacoord (#9014)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-10-05 22:33:01 +08:00
zhenshan.cao
45d6b8dcd0
[skip ci] Fix golint error in datacoord (#9184)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-10-04 17:16:54 +08:00
godchen
f18e0628ae
Make NewMeta private (#9057)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-02 23:09:56 +08:00
zhenshan.cao
d1d5f9e409
Replace proto.MarshalTextString with proto.Marshal (#8542)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-09-29 20:26:00 +08:00
sunby
e5bc8e7050
Check error in MoveSegmentBinlogs (#8704)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-09-27 22:44:18 +08:00
congqixia
664aeb09c6
Add ut and comments for datacoord meta (#7560)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-08 11:35:59 +08:00
sunby
4d56d53c0b
Add a tool to move binlog path to SegmentInfo (#7243)
issue: #7214
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-08-24 15:51:51 +08:00
sunby
3cc210da26
Add flush interval (#6996)
Time tick message is sent about per 0.2 seconds. DataCoord will call
`FlushSegment` too frequently. We add a min interval for segment flush. A
segment will called `FlushSegment` only once between the interval.

issue: #6995
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-08-19 14:08:10 +08:00
sunby
4f7c3e7d3a
Save binlog path in SegmentInfo (#6590)
issue: #6589
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-08-19 13:00:12 +08:00
groot
6b001fd76a
fix debug error in IDE (#6539)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-07-15 14:27:54 +08:00
sunby
83be910baa
Refactor segment manager (#6441)
issue: #6440
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-07-12 17:24:25 +08:00
sunby
017484e093
Refactor meta in data coord (#6300)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-07-07 14:02:01 +08:00
sunby
25d3fead9f
Change error type to error functions in data coord (#6076)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-24 19:05:42 +08:00
sunby
a529410b38
Fix data race in internal/datacoord/meta.go (#6005)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-23 12:20:06 +08:00
sunby
0978b93a21
Refactor data coordinator (#5982)
Rename variable name and make error msg more clear

Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-22 18:24:08 +08:00