Commit Graph

224 Commits

Author SHA1 Message Date
godchen
83f256c31b
Remove datanode mock test error (#9211)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-04 17:34:16 +08:00
congqixia
becf6c8f7d
Add DataNode implementation Assertion (#9144)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-03 19:46:05 +08:00
congqixia
109186db9c
[skip ci] Add comment for datanode meta_service.go (#9036)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-01 22:25:24 +08:00
Xiaofan
2c2b800ff4
Change Log Name with NodeId (#8721)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-10-01 08:52:50 +08:00
congqixia
6bbf0a8f8f
Read datanode insertbuf from env (#8942)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-30 17:45:07 +08:00
congqixia
10a6f7f146
Defer buf removal from insertBuf after SaveBinlog succeeds (#8930)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-30 17:42:11 +08:00
zhenshan.cao
d1d5f9e409
Replace proto.MarshalTextString with proto.Marshal (#8542)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-09-29 20:26:00 +08:00
congqixia
db944cd09c
Refactor flowgraph and related invocation (#8770)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-29 20:19:59 +08:00
Cai Yudong
d94361fdcc
Add parameter partitionID for API filterSegments (#8813)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-09-29 10:27:58 +08:00
XuanYang-cn
3be6672753
Add flush channel for delete_node (#8762)
Resolves: #8761

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-28 18:22:16 +08:00
congqixia
d374ea71ba
Unify liveness check in datacoord and datanode (#8480)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-28 16:10:04 +08:00
XuanYang-cn
b150f57927
No buffer also means flushed (#8686)
Resolves: #8684

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-28 11:18:03 +08:00
Xiaofan
585989935a
Revert the logger change (#8677)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-09-27 17:37:57 +08:00
godchen
51151b7b95
Fix delete node ut error (#8674)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-09-27 17:07:58 +08:00
godchen
77a773dff0
Change segment replica struct (#8608)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-09-27 14:38:00 +08:00
congqixia
e7117f67ee
Add BroadcastMark for Msgstream returning MessageIDs (#8654)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-27 14:10:09 +08:00
zhenshan.cao
b71bffe26f
Replace ChannelID with ShardName in DML Request (#8539)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-09-27 10:01:59 +08:00
Xiaofan
d38adda8c2
Rename log file based on node id (#8517)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-09-26 21:18:11 +08:00
XuanYang-cn
1307782093
[skip-ci]Fix golint in param_table in datanode (#8602)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-26 20:57:59 +08:00
XuanYang-cn
ebd894d8b7
Refactor buffer size calculation (#8565)
This PR changes the datanode insertBufSize config
from 32000 number of rows to 16MB

Resolves: #7741

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-26 20:55:59 +08:00
XuanYang-cn
b376107ab3
[skip-ci]Fix golint in segmentRep (#8601)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-26 20:51:58 +08:00
XuanYang-cn
b13170f83e
[skip-ci]Fix golint in flowgraphnode.go (#8600)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-26 20:49:59 +08:00
groot
e4e762f2e8
Reduce go unittest time cost (#8503)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-09-26 12:50:24 +08:00
Cai Yudong
b2e5627061
Rename insertMsg to flowGraphMsg in datanode flow graph (#8546)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-09-26 10:43:57 +08:00
Cai Yudong
28d305148a
Update ddnode insert (#8536)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-09-26 09:43:57 +08:00
XuanYang-cn
3aabc1d3ee
[skip-ci]Fix golint in type_def.go (#8464)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-24 21:43:56 +08:00
sunby
816bac82ec
Add root path in minio and replace etcd.rootPath in datanode (#8272)
issue: #7905
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-09-24 20:50:23 +08:00
XuanYang-cn
e4f0d4bab6
[skip-ci]Fix golint in flow_graph_message.go (#8463)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-24 20:46:04 +08:00
XuanYang-cn
2539dd4593
[skip-ci]Fix golint in data_node.go (#8462)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-24 20:44:05 +08:00
XuanYang-cn
faeb1bf50f
Add comment on exposed struct (#8348)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-23 20:15:54 +08:00
XuanYang-cn
2b82482679
Fix golint and ruleguard (#8347)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-23 19:14:01 +08:00
XuanYang-cn
a830c28bd8
Fix golint in datanode (#8345)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-23 19:05:55 +08:00
congqixia
2aa9deda1c
Improve logic when watch dm channel fails (#8386)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-23 18:31:55 +08:00
SS
58a178fdf5
[skip ci] Fix grammar in comment (#8382)
Signed-off-by: sida shen <sida@sidas-MacBook-Pro.local>

Co-authored-by: sida shen <sida@sidas-MacBook-Pro.local>
2021-09-23 16:50:16 +08:00
XuanYang-cn
d816a81ee3
Make FlushSegments async (#8159)
Resolves: #8122

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-23 16:03:54 +08:00
congqixia
6469d83072
Add datanode etcd alive check (#8363)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-23 14:29:54 +08:00
Xiaofan
f5173b595f
Add a channel prefix for all channels (#8166)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-09-23 10:53:53 +08:00
godchen
050e27cbcf
[skip ci]Add delete node comment (#8323)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-09-22 20:02:09 +08:00
Xiangyu Wang
c314b2b4a8
Remove unused label rules (#7929)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-09-18 18:19:51 +08:00
groot
334297b49d
[skip ci] Update OWNERS files (#8211)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-09-18 15:30:37 +08:00
congqixia
bdfb8768d4
Enable gosec (#8202)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-18 14:45:50 +08:00
XuanYang-cn
46c72c57c2
Add BufferData in insertbufferNode (#8138)
Optimize bufferInsertMsg param to avoid a second transfer
of endposition

See also: #8058, #7741

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-18 14:25:50 +08:00
Cai Yudong
d10c6abc9e
Apply for msgstream from pool when creating collection (#7738)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-09-18 09:13:50 +08:00
dragondriver
daf001f683
Expose created time and updated time to datanode metrics (#8173)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-17 21:32:47 +08:00
XuanYang-cn
6c434b4801
Improve codes in insertbuffernode (#8054)
Add error handling in insertbuffernode

See also: #7624, #7684

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-17 16:27:56 +08:00
XuanYang-cn
82d4d19337
Remove not in use segmentCheckPoints (#8039)
Resolves: #8037

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-17 16:25:55 +08:00
XuanYang-cn
3db76e8e59
[skip-ci]Add comment in data sync service (#8002)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-15 21:25:49 +08:00
XuanYang-cn
376d39fa02
[skip-ci]Add comment for Cache and allocator (#7924)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-15 10:33:37 +08:00
dragondriver
a872a4357c
Add more test cases for proxy implementation (#7685)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-11 11:36:22 +08:00
XuanYang-cn
ce33855652
[skip-ci]Fix datanode comment typo (#7689)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-10 14:46:00 +08:00