Commit Graph

18 Commits

Author SHA1 Message Date
XuanYang-cn
d0e6b52ce9
[skip ci]Update license for datanode (#9963)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-10-15 18:09:00 +08:00
sunby
816bac82ec
Add root path in minio and replace etcd.rootPath in datanode (#8272)
issue: #7905
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-09-24 20:50:23 +08:00
Xiaofan
f5173b595f
Add a channel prefix for all channels (#8166)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-09-23 10:53:53 +08:00
dragondriver
daf001f683
Expose created time and updated time to datanode metrics (#8173)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-17 21:32:47 +08:00
XuanYang-cn
ab7f642740
Add alias in paramtable (#5878)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-19 12:38:06 +08:00
Cai Yudong
0f4bd50ce3 Support ETCD endpoints array (#5755) 2021-06-15 16:36:36 +08:00
sunby
189ac881f3 Fix bugs (#5676)
* Remove redundant session startup

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Register datanode after start success

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* fix meta snap shot

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* fix datanode message stream channel

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* Fix bugs when drop empty collection

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Fix bug of getting pchan statistics from task scheduler

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* Fix i/dist/dataservice test code

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* Fix epoch lifetime not applied

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* fix datanode flowgraph dd node

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* Fix handle datanode timetick bug

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Remove repack function of dml stream

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* fix proxynode

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Apply extended seal policy

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* add check for time tick

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* fix check

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Fix the repack function of dml stream

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* Fix the bug when send statistics of pchan

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* Fix the repack function when craete dml stream

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* fix bugs

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* fix describe collection

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Fix bug when send timestamp statistics

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* fix data node

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Add length check before flush request

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* add log for data node

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Fix SaveBinlog bugs

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Add more log in datanode

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* Put SegmentState.Flushing as the last one in enum to fit the client

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Fix params in GetInsertBinlogPaths

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Rename policy

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Remove unused ddl functions and fields

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* Remove pchan when drop collection

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* Add balanced assignment policy

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* fix master ut

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Add lock in session manager

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* add log for debug

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Fix some logic bug and typo

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* Fix recover bugs

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Get collection scheme of a specific timestamp

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* Change CheckPoint to SegmentInfo in VchannelInfo

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Recover Unflushed segment numOfRows

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* Fix dataservice unit tests

Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: yefu.chen <yefu.chen@zilliz.com>
Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: dragondriver <jiquan.long@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-15 16:06:11 +08:00
XuanYang-cn
7a8b34ee9f Remove not using code in DataNode (#5476)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-15 15:59:04 +08:00
XuanYang-cn
1c779f3efd
Refactor ddl binlog flush (#5303)
DDL won't auto-flush now

See also: #5289 , #5220 

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-05-19 11:42:07 +00:00
XuanYang-cn
cebdfb34f7
Enable multiple datanode (#5068)
Now it's able to start multiple datanode, however, they will process the same insert data.
Further changes are needed to truly enable multiple datanode:

While registering, dataservice need to return different insert channels for different datanode.

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-04-28 18:43:48 +08:00
XuanYang-cn
6deba1faa5 Add license in datanode package
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-04-19 15:16:33 +08:00
Cai Yudong
a2041abab4 improve code coverage for datanode
improve code coverage for collection_replica_test.go
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-04-19 10:34:56 +08:00
XuanYang-cn
51f669f1cb Flush segments concurrently & change etcd meta
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-03-23 18:50:13 +08:00
zhenshan.cao
bbfcbbdd68 Fix ci: paramstable and startup logic refactor
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-02-23 11:40:30 +08:00
XuanYang-cn
b445587e11 Add datanode main
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-26 14:46:54 +08:00
XuanYang-cn
9fe672ab86 Enchance datanode interface
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-24 21:20:11 +08:00
XuanYang-cn
ec27788628 Add segment statistics updates logic for datanode
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-21 09:55:25 +08:00
XuanYang-cn
6a13386393 Add DataNode package
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-19 11:37:16 +08:00