Commit Graph

24 Commits

Author SHA1 Message Date
Xiangyu Wang
23c4de0eb8
Flush statistics for all int64 fields (#5318)
Resolves: #5262

Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-05-20 10:38:45 +00:00
XuanYang-cn
fc630bc1c8
Correct auto-flush behaviour (#5282)
Before this PR, DataNode considered auto-flush a valid flush
complete. It's wrong. So I open this PR to correct this behaviour
in DataNode.

Now binlog paths from auto-flush will be buffered in replica,
waiting until the manul flush to save into etcd all together.

See also: #5220, #5268
A follow up job of #5271 

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-05-18 11:45:00 +00:00
Xiangyu Wang
82ccd4cec0
Rename module (#4988)
* Rename module

Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-04-22 14:45:57 +08:00
XuanYang-cn
6deba1faa5 Add license in datanode package
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-04-19 15:16:33 +08:00
Xiangyu Wang
87a1e0b662 Reorganize msgstream
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-04-02 13:48:25 +08:00
XuanYang-cn
91f5c48037 Fix datanode goroutine leak
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-03-30 09:47:27 +08:00
xige-16
a48d46cfaf Alter msgID’s type to bytes
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-03-27 09:46:54 +08:00
quicksilver
7ad9b36207 Update reviewdog/action-hadolint github action version to v1.16.1
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-03-25 14:41:46 +08:00
XuanYang-cn
51f669f1cb Flush segments concurrently & change etcd meta
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-03-23 18:50:13 +08:00
XuanYang-cn
32660d547b Add endPositions in datanode
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-03-16 17:55:42 +08:00
Cai Yudong
84110d2684 Add tests/benchmark and tests/python_test using new python SDK
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
2021-02-25 17:35:36 +08:00
groot
dc3736281a Integrate message stream
Signed-off-by: groot <yihua.mo@zilliz.com>
2021-02-08 14:30:54 +08:00
Xiangyu Wang
fd9e42d3ed Update msgstream/unmarshal interface
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-02-03 17:30:10 +08:00
bigsheeper
abffdbff62 Remove partition tag
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-02-03 16:44:38 +08:00
XuanYang-cn
73a8e7de84 Fix datanode insertchannel refresh bug
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-02-03 15:18:05 +08:00
XuanYang-cn
b445587e11 Add datanode main
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-26 14:46:54 +08:00
bigsheeper
7554246ace Add segment seeking and use real client
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-25 18:33:10 +08:00
sunby
7c0c835b49 Init params and add segment start/end position
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-01-25 15:16:28 +08:00
XuanYang-cn
5015a1035d Add logic of get meta from MasterService for datanode
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-25 17:10:32 +08:00
XuanYang-cn
9fe672ab86 Enchance datanode interface
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-24 21:20:11 +08:00
XuanYang-cn
50e2369000 Change name and tag to ID
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-22 19:36:09 +08:00
XuanYang-cn
377ee8687d Modify flush logic
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-22 09:36:40 +08:00
XuanYang-cn
ec27788628 Add segment statistics updates logic for datanode
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-21 09:55:25 +08:00
XuanYang-cn
6a13386393 Add DataNode package
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-19 11:37:16 +08:00