Commit Graph

380 Commits

Author SHA1 Message Date
Xiangyu Wang
1341dcd3f8
Improve import statement (#13730)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-12-21 15:21:25 +08:00
guoxiangzhou
fc6c39cc88
[skip e2e] Modify log text according to naming conventions (#13842)
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
2021-12-21 12:03:22 +08:00
guoxiangzhou
094b5f48dc
[skip e2e] Modify log text according to naming conventions (#13841)
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
2021-12-21 12:01:31 +08:00
Bingyi Sun
7a8e49451f
Add log in datacoord (#13600)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-21 09:34:53 +08:00
godchen
c1cb69ed7e
[skip ci]Fix segment info typo (#13643)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2021-12-21 09:19:53 +08:00
godchen
d59647bb3f
[skip ci]Fix datanode compaction comment typo (#13631)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2021-12-21 09:19:10 +08:00
Bingyi Sun
9d44ea034c
[skip e2e]Improve log (#13704)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-21 09:17:00 +08:00
edward.zeng
38add97685
[skip e2e] Refine datacoord server comment (#13797)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-20 19:54:58 +08:00
edward.zeng
66b28259d9
[skip e2e] Refine datacoord session comment (#13796)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-20 19:48:53 +08:00
edward.zeng
1848dae589
[skip e2e] Refine datacoord policy comment (#13794)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-20 19:46:59 +08:00
Xiangyu Wang
8e78aeccaf
Improve import statement (#13726)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-12-20 18:13:05 +08:00
Xiangyu Wang
1fdd8fa823
Improve import statement (#13727)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-12-20 18:07:08 +08:00
guoxiangzhou
1dc4418059
[skip e2e] Modify log text according to naming conventions (#13710)
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
2021-12-20 17:42:07 +08:00
Xiangyu Wang
8960f54454
Improve import statement (#13731)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-12-20 15:47:38 +08:00
Xiangyu Wang
e704af8d1f
Improve import statement (#13725)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-12-20 15:46:03 +08:00
Xiangyu Wang
7806115752
Improve import statement (#13728)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-12-20 14:36:38 +08:00
Xiangyu Wang
562d53fb15
Improve import statement (#13724)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-12-20 14:30:55 +08:00
guoxiangzhou
71c595e696
[skip e2e] Modify log text according to naming conventions (#13711)
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
2021-12-20 11:50:43 +08:00
guoxiangzhou
8248bd7d01
[skip e2e] Modify code comments according to naming conventions (#13709)
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
2021-12-20 11:20:43 +08:00
Bingyi Sun
7ee0a2bf9f
[skip e2e]Improve log (#13707)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-20 11:18:49 +08:00
Bingyi Sun
229a727b3a
[skip e2e]Improve log (#13706)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-20 11:16:52 +08:00
Bingyi Sun
9e0219ec3f
[skip e2e]Improve log (#13702)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-20 11:14:51 +08:00
Bingyi Sun
06c4662e97
[skip e2e]Improve log (#13705)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-20 11:06:43 +08:00
Bingyi Sun
279316e09b
[skip e2e]Improve log (#13701)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-20 11:02:45 +08:00
Bingyi Sun
3e7f76e265
[skip e2e]Improve log (#13642)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-20 10:42:43 +08:00
congqixia
1dcd06cc78
Add log file info for all kind of segment binlogs (#13606)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-19 20:00:42 +08:00
godchen
f4caa94ea8
[skip e2e]Fix segment manager typo (#13644)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2021-12-17 22:34:42 +08:00
Bingyi Sun
63ebf76318
[skip e2e]Improve log print (#13640)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-17 19:48:42 +08:00
godchen
c983af85c5
[skip e2e]Fix policy typo (#13645)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2021-12-17 19:46:42 +08:00
Bingyi Sun
c7513bb1ad
[skip e2e]Improve log print (#13641)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-17 19:18:52 +08:00
godchen
ec33f3b494
[skip e2e]Fix datanode compaction policy typo (#13635)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2021-12-17 19:16:55 +08:00
godchen
e36f7287ae
[skip e2e]Fix datanode compaction variable typo error (#13630)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2021-12-17 19:14:59 +08:00
Bingyi Sun
371017ad39
Improve log print in datacoord (#13621)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-17 19:11:19 +08:00
edward.zeng
6b74aed32a
[skip e2e] Refine datacoord segment info comment (#13602)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-17 18:54:26 +08:00
edward.zeng
d5791289fc
[skip e2e] Refine datacoord segment manager comment (#13605)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-17 18:49:09 +08:00
edward.zeng
5431e475ae
[skip e2e] Add comment to datacoord handler (#13555)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-17 18:05:24 +08:00
congqixia
016514bee6
Normalize DataCoord in error string (#13557)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-17 11:05:15 +08:00
edward.zeng
098e1375fb
[skip ci] Refine datacoord compaction trigger comment (#13491)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-16 09:59:12 +08:00
edward.zeng
d6b0a47248
[skip ci] Refine datacoord compaction trigger comment (#13483)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-16 09:57:25 +08:00
edward.zeng
d9bd36b58a
[skip ci] Refine datacoord compaction policy comment (#13489)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-16 09:41:10 +08:00
Xiangyu Wang
25022fa800
Improve import statements (#13435)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-12-15 20:49:10 +08:00
Xiangyu Wang
c6908679b4
Improve import statements (#13436)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-12-15 20:47:10 +08:00
Xiangyu Wang
006a095370
Improve import statements (#13437)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-12-15 19:57:13 +08:00
edward.zeng
dbae087d21
[skip ci] Refine datacoord garbage collector comment (#13430)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-15 18:25:43 +08:00
edward.zeng
f44ede7b9d
[skip ci] Refine datacoord meta comment (#13432)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-15 16:57:23 +08:00
xige-16
97d5231a0a
Reorganize querycoord proto (#13181)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-12-15 16:53:12 +08:00
xiangzhou.guo
5e644a780c
correct spelling error (#13409)
Signed-off-by: xiangzhou.guo <xiangzhou.guo@zilliz.com>
2021-12-15 16:47:13 +08:00
Bingyi Sun
5c2302d711
Improve compaction log (#13394)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-15 16:07:10 +08:00
Bingyi Sun
bd03236b79
Improve log (#13396)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-15 16:03:12 +08:00
Bingyi Sun
663a3aa333
Improve compaction log (#13395)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-15 16:01:11 +08:00
xiangzhou.guo
a87e690d53
Correct spelling error in code comment (#13413)
Signed-off-by: xiangzhou.guo <xiangzhou.guo@zilliz.com>
2021-12-15 15:59:17 +08:00
xiangzhou.guo
78d583a3bb
Correct spelling error (#13412)
Signed-off-by: xiangzhou.guo <xiangzhou.guo@zilliz.com>
2021-12-15 15:57:09 +08:00
Bingyi Sun
fcd1ee9c8e
Improve log (#13397)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-15 13:15:10 +08:00
edward.zeng
f8b85c4646
[skip ci] Refine datacoord cluster comment (#13408)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-15 11:58:31 +08:00
cai.zhang
9f23fc7f2a
Register the service when the component state is healthy (#13248)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2021-12-15 11:47:10 +08:00
Bingyi Sun
0c39e0370c
Move segment statistics update to datanode tt loop (#13233)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-15 10:53:16 +08:00
Bingyi Sun
f480b314d9
Improve compaction log (#13393)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-15 10:43:15 +08:00
Bingyi Sun
4a5aa1cfea
Improve datacoord log (#13392)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-15 10:41:27 +08:00
congqixia
3b4d114b71
Add comment for DataCoord startFlushLoop (#13379)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-15 09:45:09 +08:00
Xiangyu Wang
405b3cd932
Use the same lint rules with golint in revive (#13324)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-12-14 15:31:07 +08:00
congqixia
8476791492
Process only flushed/flushing segment in global compaction (#13339)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-14 15:15:10 +08:00
Bingyi Sun
3a2aace6f0
Fix ci issue caused by only compact flushed segments (#13317)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-14 13:55:07 +08:00
edward.zeng
6593e277f7
[skip ci] Refine datacoord metrics info comment (#13114)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-09 22:14:14 +08:00
edward.zeng
d323b3bd84
[skip ci] Refine datacoord handler comment (#13113)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-09 22:10:23 +08:00
edward.zeng
4459611126
[skip ci] Refine datacoord meta comment (#13112)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-09 22:07:10 +08:00
XuanYang-cn
a9a332dbcf
Fix empty segment compact and load (#12710)
1. Fix compaction save empty segment bug
2. Fix load empty segment bug
3. Add UT for dmlchannels to 100%

Resolves: #12450

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-12-09 11:03:08 +08:00
Bingyi Sun
a78b2010e9
[skip ci]Improve log style (#13008)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-08 20:27:06 +08:00
Bingyi Sun
48c5632277
[skip ci]Improve log style (#13007)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-08 20:25:15 +08:00
Bingyi Sun
320cfbb606
[skip ci]Improve log style (#13006)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-08 20:19:26 +08:00
congqixia
aea5a61e9a
Add switch for auto compaction (#12974)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-08 19:47:05 +08:00
edward.zeng
260d3e124d
[skip ci] Refine datacoord channel store comment (#12991)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-08 17:57:05 +08:00
edward.zeng
2e84eef229
[skip ci] Refine datacoord channel manager comment (#12989)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-08 17:53:22 +08:00
edward.zeng
8ca248dce6
[skip ci] Refine datacoord allocator comment (#12985)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-08 17:35:06 +08:00
congqixia
de454956fe
Refactor session WatchSessions to allow rewatch when Rev Compacted (#12880)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-08 10:11:04 +08:00
JackLCL
f90410fe8e
[skip ci]Fix error log output format in segment_manager.go (#12856)
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
2021-12-07 14:36:27 +08:00
JackLCL
de00aa038a
[skip ci]Fix error log output format in meta.go (#12854)
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
2021-12-07 14:32:51 +08:00
congqixia
22fddc537c
Fix not all segment state is drop after DropVirtualChannel (#12702)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-03 18:55:34 +08:00
congqixia
58664bb6b9
Fix zero length slice declaration in datacoord/channel_manager.go (#12582)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-02 19:17:38 +08:00
edward.zeng
f8c826c597
Fix typo in datacoord compaction trigger (#12636)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-02 19:09:37 +08:00
congqixia
8f9e62fa18
Apply DropVirtualChannel and FlushManager drop mode (#12563)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-02 16:39:33 +08:00
Bingyi Sun
0778e6f715
Fix alloc segment with dropped segment (#12572)
issue: #12465
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-01 22:39:37 +08:00
godchen
5e3e3b3b41
Remove grpc returned error (#12537)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-12-01 22:17:46 +08:00
edward.zeng
5db458a44e
[skip ci] Add license for data coord compaction (#12561)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-01 19:33:32 +08:00
edward.zeng
446b5a3a48
[skip ci] Refine datacoord cluster comment (#12562)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-01 19:29:31 +08:00
Bingyi Sun
7ff15490c9
[skip ci]Add comment for GetVChanPosition (#12495)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-11-30 23:15:33 +08:00
Bingyi Sun
27898928b0
[skip ci]Modify WatchChannels comment in services.go (#12417)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@gmail.com>
2021-11-29 23:09:41 +08:00
congqixia
06ced4d1d1
Add DropVirtualChannel for DataCoord (#12361)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-29 22:35:41 +08:00
congqixia
eb48013042
Set start position for compact result segment (#12280)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-25 17:19:17 +08:00
congqixia
2ee0c7c074
Add compacted segment into check list when drop collection (#12272)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-25 15:13:15 +08:00
bigsheeper
e245ba87a9
Add excluded segments for dropped segemnts (#12247)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-11-25 14:25:15 +08:00
dragondriver
6c4c0ef6b5
Add more deploy metrics (#12248)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-11-24 15:43:15 +08:00
congqixia
7b870e3392
Limit datacoord gc scan prefixes (#12231)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-24 09:55:15 +08:00
congqixia
8e0c8febe9
Add dropped segment ids in vchaninfo (#12230)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-23 18:59:15 +08:00
Bingyi Sun
3be9442c0f
Clear dropped segment meta on etcd (#12191)
issue: #9613
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-11-23 11:23:15 +08:00
Bingyi Sun
01be91116e
Add GetFlushState interface (#12170)
issue: #12075
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-11-23 10:55:14 +08:00
congqixia
8c107ab15f
Set segment DroppedAt when compacted (#12182)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-22 20:11:14 +08:00
congqixia
4121e31df1
Send SIGINT to runner goroutine after etcd disconnects (#12163)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-22 16:23:17 +08:00
shaoyue
6777d24c65
[skip ci] Add comment for exported items (#12126)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2021-11-19 18:05:25 +08:00
dragondriver
ee0f753f7a
Fix datarace between GetComponentStates and Register (#11935)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-11-19 13:57:12 +08:00
congqixia
dfc70108c4
Add load collection for AssignSegmentID (#12077)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-18 22:31:34 +08:00