Commit Graph

46 Commits

Author SHA1 Message Date
XuanYang-cn
2b5340581c
Skip remove if reassigns to the original node (#17450)
Fix ut race

See also: #15966, #17432

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-06-10 10:54:07 +08:00
XuanYang-cn
1dbfebf827
Add debug log in removing assignment (#17427)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-06-08 10:44:06 +08:00
Xiaofan
af994b5e1c
Fix etcd watch error (#17364)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-06-06 16:26:06 +08:00
XuanYang-cn
c70af73005
Fix datacoord set wrong state for node registering (#17376)
Fix datacoord datarace

See also: #17335

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-06-06 15:10:05 +08:00
Ten Thousand Leaves
f5e63177d2
Fix an issue with bulk load with multiple DataNodes (#17324)
issue: #17300
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-06-02 18:54:04 +08:00
XuanYang-cn
42f643e727
Make DataNode release rather than delete when reassign (#17293)
1. Reassgin now will assign to the original Node if no other nodes
   avaliable
2. Make AddNode balance async: ToRealse + Reassign

See also: #16114, #17270

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-06-02 13:56:04 +08:00
XuanYang-cn
127dd34b37
Make proxy use roundrobin to choose replica (#17063)
Fixes: #17055

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-05-17 22:35:57 +08:00
Xiaofan
62658dcda6
Fix pulsar unscubsribe fail because of consumer not found (#16839)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-05-09 12:07:52 +08:00
XuanYang-cn
5494f3c318
Remove offLineNodes from watch states check (#16347)
This PR also
- adds more information in log
- makes Session able to logged by zap.Any/zap.String

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-04-01 22:33:29 +08:00
congqixia
376a7f6567
Fix Datacoord unsubAttempt using pchannel (#16337)
Fix the channel name used when Datacoord unsub for DataNode
This channel shall always be pChannel instead of vChannel

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-04-01 11:25:28 +08:00
XuanYang-cn
a77dd1034c
Adopt the extended state in DataCoord (#16200)
ChannelWatchInfo with ToWatch, ToRelease will trigger a timer.
ChannelManager now reacts to different ChannelWatch states.

- WatchSuccess > log this info

- WatchFailure/WatchTimeout > ToRelease

- ReleaseSuccess > Delete, reassign if not from DropCollection

- ReleaseFailure/ReleaseTimeout > Cleanup subscription and Delete,
  reassgin if not from DropCollection.

Some Notes:
1. Reassignment will add this channel to buffer if there's only one node.

See also: #15846

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-03-28 22:33:27 +08:00
congqixia
6513be5c44
Extend watch/release channel logic in DataNode (#15925)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-03-09 15:39:58 +08:00
Cai Yudong
3e862ecdfb
Merge msgChannelConfig and knowhereConfig into commonConfig (#15843)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-03-04 11:17:56 +08:00
jaime
7a3a721380
Reconstruct mqstream module (#15784)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-03-03 21:57:56 +08:00
Xiaofan
bd7d294c6d
Refine log level in data (#15821)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-03-02 15:35:55 +08:00
Nemo
3eb074a9b0
Update some comments for data coord with some minor code refactoring. (#15686)
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-02-22 13:15:51 +08:00
Cai Yudong
8255b713ac
Move ProxySubName/QueryNodeSubName/DataNodeSubName to msgChannelConfig (#15501)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-02-09 14:41:45 +08:00
Cai Yudong
4bccc85534
Use MsgChannelConfig in GlobalParams for all components (#15163)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-02-02 00:35:43 +08:00
Bingyi Sun
515fe962d5
Fix subscription path (#15373)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-01-25 13:27:44 +08:00
Bingyi Sun
3987574448
Unsubscribe topic in DataCoord (#15353)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-01-24 19:47:38 +08:00
godchen
1e0a0dee24
[skip e2e]Sort imports (#14817)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-01-05 14:20:04 +08:00
godchen
77ae88fc37
[skip e2e]Fix function typo (#14321)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-01-05 10:15:24 +08:00
zhenshan.cao
7abebe576c
Add more logs on write path (#14600)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-01-04 15:47:18 +08:00
godchen
d7da870cc4
[skip e2e]Fix variable typo (#14325)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2021-12-31 15:49:19 +08:00
godchen
f0e6e190df
[skip e2e]Fix variable typo (#14327)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2021-12-31 15:47:19 +08:00
godchen
ededf126c3
[skip e2e]Fix log typo (#14322)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2021-12-30 16:54:12 +08:00
godchen
ee6765e1db
[skip e2e]Fix function typo (#14320)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2021-12-27 18:49:11 +08:00
Xiangyu Wang
3f79e1bfda
Improve import statement (#13723)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-12-21 19:11:11 +08:00
Xiangyu Wang
c6908679b4
Improve import statements (#13436)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-12-15 20:47:10 +08:00
edward.zeng
2e84eef229
[skip ci] Refine datacoord channel manager comment (#12989)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-08 17:53:22 +08:00
congqixia
58664bb6b9
Fix zero length slice declaration in datacoord/channel_manager.go (#12582)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-02 19:17:38 +08:00
congqixia
8f9e62fa18
Apply DropVirtualChannel and FlushManager drop mode (#12563)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-02 16:39:33 +08:00
Bingyi Sun
175a656ff1
Unwatch dropped channel when init channel manager (#11986)
issue: #11558
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-11-17 23:25:12 +08:00
Bingyi Sun
e3e4dc7153
Improve GetVChanPosistion and fix unit tests (#11888)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-11-16 21:41:50 +08:00
Bingyi Sun
902e856a06
[skip ci]Remove redundancy log (#11732)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-11-15 10:25:09 +08:00
congqixia
d38043f235
Filter with partition id in GetVChanPosition (#11746)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-13 08:45:09 +08:00
Bingyi Sun
54b40da4c5
Add segment drop logic (#11621)
issue: #11558
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-11-12 00:22:42 +08:00
sunby
c26cd79532
Add compaction (#9906)
issue: #9904
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-11-05 22:25:00 +08:00
godchen
ad2cfdd99f
[skip ci]Fix datanode typo error (#10880)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-29 13:04:04 +08:00
godchen
7071ded008
[skip ci]Fix datanode typo error (#10879)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-29 12:03:38 +08:00
congqixia
c019e80dbb
Change SeekPosition to earliest of all segments (#10771)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-28 14:30:26 +08:00
sunby
7fbb469081
Change channel's field to upper case to log details (#10567)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-10-26 20:15:41 +08:00
sunby
2de3131bb3
Change default balance policy (#10563)
issue: #10198
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-10-26 19:40:44 +08:00
XuanYang-cn
acb4dd2069
[skip ci]Update license for dc channel manager (#10591)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-10-25 20:19:25 +08:00
groot
07599e6a35
[skip ci] Fix a typo (#10106)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-10-18 17:54:41 +08:00
sunby
044b06421a
Use etcd to watch channel on datanode (#7115)
issue: #6578
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-10-14 15:44:34 +08:00