Commit Graph

85 Commits

Author SHA1 Message Date
Xiaofan
bd7d294c6d
Refine log level in data (#15821)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-03-02 15:35:55 +08:00
bigsheeper
13177a90af
Add prometheus metrics for DataNode (#15650)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Co-authored-by: Cai Yudong <yudong.cai@zilliz.com>
2022-02-28 19:11:55 +08:00
zhenshan.cao
a5f8bdeb42
Fix bug: data race in MsgFactory (#14839)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-01-06 14:57:20 +08:00
godchen
e1eb721532
[skip e2e]Sort imports (#14819)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-01-05 14:23:52 +08:00
zhenshan.cao
7abebe576c
Add more logs on write path (#14600)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-01-04 15:47:18 +08:00
Cai Yudong
891c202b73
Use GlobalParamTable for all components (#13885)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-12-23 18:39:11 +08:00
congqixia
b20a238c38
Fix newDataSyncService panicking with normal segment has no dml pos (#12771)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-06 16:21:35 +08:00
congqixia
8f9e62fa18
Apply DropVirtualChannel and FlushManager drop mode (#12563)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-02 16:39:33 +08:00
congqixia
e813867652
Implement Drop mode for flush manager (#12469)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-01 10:11:39 +08:00
XuanYang-cn
8a8ebed23d
Fix DN GC bug (#12258)
Fixes: #12249

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-25 09:43:15 +08:00
congqixia
5e550a8793
Add close for flushManager (#11874)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-17 10:39:15 +08:00
congqixia
579416e3d3
Apply retry for flush manager execution (#11808)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-15 17:19:10 +08:00
XuanYang-cn
f3852c1db0
Add more specific logs (#11724)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-15 15:25:09 +08:00
Bingyi Sun
5c8978e5d6
Fix drop segment bug (#11709)
issue: #11708
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-11-12 17:31:11 +08:00
XuanYang-cn
c4bb73dc74
Complete drop collection procedure in DN (#11552)
drop, cancel compaction, flush all buffers and release fg

See also: #11426

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-11 20:56:49 +08:00
congqixia
3ac7d7a9de
Add detailed comments for dataSyncService in DataNode (#11285)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-05 22:59:04 +08:00
godchen
9186d5527e
Forward delete msg (#11210)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-05 11:59:02 +08:00
XuanYang-cn
8863c01ff7
Make DN fail faster when watching an invalid chan (#11102)
See also: #11098

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-04 15:56:28 +08:00
congqixia
169af67eb1
Fix segments not removed from flushingCache (#10738)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-27 16:06:24 +08:00
godchen
d2e0943f21
Fix delete node flush error (#10549)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-25 18:49:20 +08:00
godchen
d6bf0c928f
Get stats delta path from datacoord (#10362)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-22 14:31:13 +08:00
congqixia
ed4e516ce7
Save stats and delta logs from DataNode (#10159)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-19 14:34:35 +08:00
congqixia
5737e0075f
Apply flush manager logic in datanode (#10142)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-19 11:04:34 +08:00
congqixia
0438ed2bfe
Use single signal chan to notify flush (#10075)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-18 12:34:34 +08:00
XuanYang-cn
880056e664
[skip ci]Update license for datesync (#9959)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-10-15 18:01:38 +08:00
godchen
2e0f7dfe42
Check data sync service error (#9856)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-14 17:40:42 +08:00
XuanYang-cn
aac6323cff
Use config as parameter (#9707)
Make functions in flowgraph lesser parameters

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-10-13 11:16:32 +08:00
Cai Yudong
84e568ee32
Support Delete in datanode (#8505)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-10-11 16:31:44 +08:00
godchen
376be1b532
Add data sync service error check (#9498)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-09 15:38:57 +08:00
godchen
937dbf4279
Add flushed segments (#9476)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-08 19:09:12 +08:00
XuanYang-cn
3be6672753
Add flush channel for delete_node (#8762)
Resolves: #8761

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-28 18:22:16 +08:00
godchen
77a773dff0
Change segment replica struct (#8608)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-09-27 14:38:00 +08:00
congqixia
2aa9deda1c
Improve logic when watch dm channel fails (#8386)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-23 18:31:55 +08:00
XuanYang-cn
d816a81ee3
Make FlushSegments async (#8159)
Resolves: #8122

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-23 16:03:54 +08:00
Cai Yudong
d10c6abc9e
Apply for msgstream from pool when creating collection (#7738)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-09-18 09:13:50 +08:00
XuanYang-cn
3db76e8e59
[skip-ci]Add comment in data sync service (#8002)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-15 21:25:49 +08:00
Cai Yudong
4e23ed9a18
Revert apply physical channels from pool (d5f7e358 and 3b960969) (#7687)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-09-10 13:00:00 +08:00
XuanYang-cn
64aad49959
Add ut in datasync service to 88% (#7615)
See also: #6357

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-09 15:36:01 +08:00
XuanYang-cn
a5767e05c6
Add ut for flowgraph_delete_node to 100% (#7550)
See also: #6357
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-08 10:41:59 +08:00
Cai Yudong
3b9609692b
Apply for msgstream from pool when creating collection (#7377)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-08-31 18:35:58 +08:00
Cai Yudong
f4b53df727
Revert "Apply for msgstream from pool when creating collection (#7323)" (#7376)
This reverts commit e6de86a433.
2021-08-31 11:07:58 +08:00
Cai Yudong
e6de86a433
Apply for msgstream from pool when creating collection (#7323)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-08-31 09:47:58 +08:00
congqixia
65f168eac1
Fix datanode corner cases (#7336)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-08-30 10:03:58 +08:00
godchen
b4c55f6b3c
Add delete node (#7261)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-08-25 11:41:52 +08:00
sunby
4f7c3e7d3a
Save binlog path in SegmentInfo (#6590)
issue: #6589
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-08-19 13:00:12 +08:00
XuanYang-cn
f0a68704e2
Add unittests for getChannelName in DataNode (#6346)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-07-08 15:52:40 +08:00
sunby
7f2d7c1da3
Rename dataservice to datacoordinator (#5912)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-21 18:22:13 +08:00
XuanYang-cn
441300140d
Tidy replica (#5920)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-21 16:00:22 +08:00
sunby
3cc0ab1345
Remove panic in NewDataSyncService (#5888)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-19 15:18:06 +08:00
sunby
33f5225968
Use physical channel to init msg stream (#5815)
* Use physical channel to init msg stream

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* fix search collection error check (#1)

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Co-authored-by: bigsheeper <954206947@qq.com>
2021-06-18 16:02:05 +08:00