Commit Graph

212 Commits

Author SHA1 Message Date
Xiaofan
585989935a
Revert the logger change (#8677)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-09-27 17:37:57 +08:00
godchen
51151b7b95
Fix delete node ut error (#8674)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-09-27 17:07:58 +08:00
godchen
77a773dff0
Change segment replica struct (#8608)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-09-27 14:38:00 +08:00
congqixia
e7117f67ee
Add BroadcastMark for Msgstream returning MessageIDs (#8654)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-27 14:10:09 +08:00
zhenshan.cao
b71bffe26f
Replace ChannelID with ShardName in DML Request (#8539)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-09-27 10:01:59 +08:00
Xiaofan
d38adda8c2
Rename log file based on node id (#8517)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-09-26 21:18:11 +08:00
XuanYang-cn
1307782093
[skip-ci]Fix golint in param_table in datanode (#8602)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-26 20:57:59 +08:00
XuanYang-cn
ebd894d8b7
Refactor buffer size calculation (#8565)
This PR changes the datanode insertBufSize config
from 32000 number of rows to 16MB

Resolves: #7741

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-26 20:55:59 +08:00
XuanYang-cn
b376107ab3
[skip-ci]Fix golint in segmentRep (#8601)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-26 20:51:58 +08:00
XuanYang-cn
b13170f83e
[skip-ci]Fix golint in flowgraphnode.go (#8600)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-26 20:49:59 +08:00
groot
e4e762f2e8
Reduce go unittest time cost (#8503)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-09-26 12:50:24 +08:00
Cai Yudong
b2e5627061
Rename insertMsg to flowGraphMsg in datanode flow graph (#8546)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-09-26 10:43:57 +08:00
Cai Yudong
28d305148a
Update ddnode insert (#8536)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-09-26 09:43:57 +08:00
XuanYang-cn
3aabc1d3ee
[skip-ci]Fix golint in type_def.go (#8464)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-24 21:43:56 +08:00
sunby
816bac82ec
Add root path in minio and replace etcd.rootPath in datanode (#8272)
issue: #7905
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-09-24 20:50:23 +08:00
XuanYang-cn
e4f0d4bab6
[skip-ci]Fix golint in flow_graph_message.go (#8463)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-24 20:46:04 +08:00
XuanYang-cn
2539dd4593
[skip-ci]Fix golint in data_node.go (#8462)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-24 20:44:05 +08:00
XuanYang-cn
faeb1bf50f
Add comment on exposed struct (#8348)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-23 20:15:54 +08:00
XuanYang-cn
2b82482679
Fix golint and ruleguard (#8347)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-23 19:14:01 +08:00
XuanYang-cn
a830c28bd8
Fix golint in datanode (#8345)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-23 19:05:55 +08:00
congqixia
2aa9deda1c
Improve logic when watch dm channel fails (#8386)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-23 18:31:55 +08:00
SS
58a178fdf5
[skip ci] Fix grammar in comment (#8382)
Signed-off-by: sida shen <sida@sidas-MacBook-Pro.local>

Co-authored-by: sida shen <sida@sidas-MacBook-Pro.local>
2021-09-23 16:50:16 +08:00
XuanYang-cn
d816a81ee3
Make FlushSegments async (#8159)
Resolves: #8122

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-23 16:03:54 +08:00
congqixia
6469d83072
Add datanode etcd alive check (#8363)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-23 14:29:54 +08:00
Xiaofan
f5173b595f
Add a channel prefix for all channels (#8166)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-09-23 10:53:53 +08:00
godchen
050e27cbcf
[skip ci]Add delete node comment (#8323)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-09-22 20:02:09 +08:00
Xiangyu Wang
c314b2b4a8
Remove unused label rules (#7929)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-09-18 18:19:51 +08:00
groot
334297b49d
[skip ci] Update OWNERS files (#8211)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-09-18 15:30:37 +08:00
congqixia
bdfb8768d4
Enable gosec (#8202)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-18 14:45:50 +08:00
XuanYang-cn
46c72c57c2
Add BufferData in insertbufferNode (#8138)
Optimize bufferInsertMsg param to avoid a second transfer
of endposition

See also: #8058, #7741

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-18 14:25:50 +08:00
Cai Yudong
d10c6abc9e
Apply for msgstream from pool when creating collection (#7738)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-09-18 09:13:50 +08:00
dragondriver
daf001f683
Expose created time and updated time to datanode metrics (#8173)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-17 21:32:47 +08:00
XuanYang-cn
6c434b4801
Improve codes in insertbuffernode (#8054)
Add error handling in insertbuffernode

See also: #7624, #7684

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-17 16:27:56 +08:00
XuanYang-cn
82d4d19337
Remove not in use segmentCheckPoints (#8039)
Resolves: #8037

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-17 16:25:55 +08:00
XuanYang-cn
3db76e8e59
[skip-ci]Add comment in data sync service (#8002)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-15 21:25:49 +08:00
XuanYang-cn
376d39fa02
[skip-ci]Add comment for Cache and allocator (#7924)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-15 10:33:37 +08:00
dragondriver
a872a4357c
Add more test cases for proxy implementation (#7685)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-11 11:36:22 +08:00
XuanYang-cn
ce33855652
[skip-ci]Fix datanode comment typo (#7689)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-10 14:46:00 +08:00
Cai Yudong
4e23ed9a18
Revert apply physical channels from pool (d5f7e358 and 3b960969) (#7687)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-09-10 13:00:00 +08:00
sunby
06cee85a39
Add data node unit tests (#7622)
Issue: #6357
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-09-09 16:40:00 +08:00
XuanYang-cn
64aad49959
Add ut in datasync service to 88% (#7615)
See also: #6357

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-09 15:36:01 +08:00
congqixia
69794fd32d
Refactor insertBufferNode and add unit tests (#7621)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-09 15:00:00 +08:00
XuanYang-cn
762060e670
Add ut for segmentReplica to 100% (#7589)
See also: #6357
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-09 10:14:00 +08:00
XuanYang-cn
074b610811
Add ut for dmstream_input_node to 100% (#7593)
See also: #6357

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-08 19:12:00 +08:00
XuanYang-cn
a5767e05c6
Add ut for flowgraph_delete_node to 100% (#7550)
See also: #6357
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-08 10:41:59 +08:00
XuanYang-cn
29756c6ce8
Add ut for flow_graph_dd_node to 100% (#7511)
This PR:
- Change mutex to sync.Map
- Add ut for flow_graph_dd_node.go to 100%

See also: #6357

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-07 15:41:59 +08:00
dragondriver
960741ee16
Support more detailed metrics information (#7472)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-03 19:35:47 +08:00
dragondriver
89748ac639
Expose metrics of data cluster (#7177)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-01 10:13:15 +08:00
Cai Yudong
3b9609692b
Apply for msgstream from pool when creating collection (#7377)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-08-31 18:35:58 +08:00
Cai Yudong
f4b53df727
Revert "Apply for msgstream from pool when creating collection (#7323)" (#7376)
This reverts commit e6de86a433.
2021-08-31 11:07:58 +08:00