Commit Graph

198 Commits

Author SHA1 Message Date
Cai Yudong
7efb02a4c5
Use PulsarConfig in GlobalParams for all components (#15046)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-01-09 23:45:33 +08:00
Cai Yudong
f8f61fb116
Use BaseParams etcd configuration for all components (#15012)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-01-07 16:15:26 +08:00
zhenshan.cao
a5f8bdeb42
Fix bug: data race in MsgFactory (#14839)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-01-06 14:57:20 +08:00
zhenshan.cao
76a9687310
Adjust usage of lock in datanode (#14631)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Co-authored-by: yangxuan <xuan.yang@zilliz.com>
2022-01-04 19:35:32 +08:00
zhenshan.cao
7abebe576c
Add more logs on write path (#14600)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-01-04 15:47:18 +08:00
Xiaofan
69087ff8dd
Support embedded etcd (#14333)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-12-29 14:35:21 +08:00
congqixia
a8c7e0c827
[skip e2e] Add comment for Compaction of DataNode (#14350)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-28 09:38:07 +08:00
zhenshan.cao
9a2c91d326
Fix bug: refresh config after session init (#14194)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-12-24 22:22:17 +08:00
Cai Yudong
891c202b73
Use GlobalParamTable for all components (#13885)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-12-23 18:39:11 +08:00
cai.zhang
9f23fc7f2a
Register the service when the component state is healthy (#13248)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2021-12-15 11:47:10 +08:00
Bingyi Sun
0c39e0370c
Move segment statistics update to datanode tt loop (#13233)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-15 10:53:16 +08:00
congqixia
1195f441b2
Update DataNode state before stop (#12946)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-08 14:19:04 +08:00
congqixia
8f9e62fa18
Apply DropVirtualChannel and FlushManager drop mode (#12563)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-02 16:39:33 +08:00
godchen
5e3e3b3b41
Remove grpc returned error (#12537)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-12-01 22:17:46 +08:00
dragondriver
b6d0150dc5
Fix health check when disconnect to etcd (#12292)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-11-26 11:39:16 +08:00
XuanYang-cn
8a8ebed23d
Fix DN GC bug (#12258)
Fixes: #12249

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-25 09:43:15 +08:00
congqixia
4121e31df1
Send SIGINT to runner goroutine after etcd disconnects (#12163)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-22 16:23:17 +08:00
dragondriver
ee0f753f7a
Fix datarace between GetComponentStates and Register (#11935)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-11-19 13:57:12 +08:00
XuanYang-cn
9bd3664672
Fix compaction bug (#11816)
This PR fixes:
- timeout in compaction
- BinlogIterator using rawID as PKField bug

See also: #11757

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-17 10:07:13 +08:00
congqixia
5edbb82610
Add session revoke (#11908)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-16 22:31:14 +08:00
XuanYang-cn
c4bb73dc74
Complete drop collection procedure in DN (#11552)
drop, cancel compaction, flush all buffers and release fg

See also: #11426

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-11 20:56:49 +08:00
XuanYang-cn
3cd0584a04
Add compaction in datanode (#11353)
See also: #9530

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-08 19:49:07 +08:00
sunby
c26cd79532
Add compaction (#9906)
issue: #9904
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-11-05 22:25:00 +08:00
XuanYang-cn
225b2c66a0
Fix datanode ut using register bug (#10885)
Resolves: #10881
See also: #8058

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-01 11:01:49 +08:00
XuanYang-cn
1e3fc5076f
Change etcdkv clientv3 into MetaKv interface (#10903)
This pr:
- changed etcdkv clientv3 into MetaKv interface
- replaced fmt.Sprintf with path.Join for kv key

See also: #8058

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-01 10:19:55 +08:00
Xiaofan
2df2d488e1
Fix Unit test fatal fail (#10933)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-10-30 10:24:38 +08:00
Xiaofan
6fddcb3925
Fix Session checker for proxy (#10737)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-10-27 21:58:33 +08:00
congqixia
bd90a3831c
Allow flushed segments trigger flushe process (#10539)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-25 18:01:50 +08:00
godchen
31393d8801
[skip ci]Fix datanode comment typo error (#10502)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-23 16:57:35 +08:00
congqixia
7ce7cb7a5e
Add a timed flush trigger mechanism (#10197)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-20 15:02:36 +08:00
congqixia
0438ed2bfe
Use single signal chan to notify flush (#10075)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-18 12:34:34 +08:00
XuanYang-cn
9c706c6dc3
[skip ci]Update license on datanode (#9960)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-10-15 18:03:25 +08:00
godchen
bf71965f58
Check watchlist error (#9953)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-15 17:02:37 +08:00
congqixia
ec6e8e4127
Hide liveChn into sessionutil and fix liveness init order (#9860)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-14 16:40:35 +08:00
sunby
044b06421a
Use etcd to watch channel on datanode (#7115)
issue: #6578
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-10-14 15:44:34 +08:00
godchen
a2dc0d8808
Init segment replica bloom filter (#9770)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-14 10:24:33 +08:00
congqixia
c90a97f95a
Fix Watch Channel canceled due to revision compacted (#9788)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-13 17:02:33 +08:00
XuanYang-cn
f0bedc7046
Add ut for data_node.go (#9489)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-10-11 18:50:30 +08:00
godchen
55ec988544
[skip ci]Fix data node comment lint (#9534)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-09 11:28:57 +08:00
groot
735b02cf2b
Add unittest for distributed/datanode (#9503)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-10-09 10:10:59 +08:00
XuanYang-cn
30395cb1b1
Add more specific comment in datanode (#9488)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-10-09 00:09:47 +08:00
godchen
445f5426d1
Check data node error (#9303)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-07 22:16:56 +08:00
congqixia
becf6c8f7d
Add DataNode implementation Assertion (#9144)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-03 19:46:05 +08:00
Xiaofan
2c2b800ff4
Change Log Name with NodeId (#8721)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-10-01 08:52:50 +08:00
zhenshan.cao
d1d5f9e409
Replace proto.MarshalTextString with proto.Marshal (#8542)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-09-29 20:26:00 +08:00
congqixia
db944cd09c
Refactor flowgraph and related invocation (#8770)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-29 20:19:59 +08:00
XuanYang-cn
3be6672753
Add flush channel for delete_node (#8762)
Resolves: #8761

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-28 18:22:16 +08:00
congqixia
d374ea71ba
Unify liveness check in datacoord and datanode (#8480)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-28 16:10:04 +08:00
Xiaofan
585989935a
Revert the logger change (#8677)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-09-27 17:37:57 +08:00
Xiaofan
d38adda8c2
Rename log file based on node id (#8517)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-09-26 21:18:11 +08:00
XuanYang-cn
2539dd4593
[skip-ci]Fix golint in data_node.go (#8462)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-24 20:44:05 +08:00
congqixia
2aa9deda1c
Improve logic when watch dm channel fails (#8386)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-23 18:31:55 +08:00
XuanYang-cn
d816a81ee3
Make FlushSegments async (#8159)
Resolves: #8122

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-23 16:03:54 +08:00
congqixia
6469d83072
Add datanode etcd alive check (#8363)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-23 14:29:54 +08:00
dragondriver
daf001f683
Expose created time and updated time to datanode metrics (#8173)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-17 21:32:47 +08:00
XuanYang-cn
3db76e8e59
[skip-ci]Add comment in data sync service (#8002)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-15 21:25:49 +08:00
XuanYang-cn
ce33855652
[skip-ci]Fix datanode comment typo (#7689)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-10 14:46:00 +08:00
sunby
06cee85a39
Add data node unit tests (#7622)
Issue: #6357
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-09-09 16:40:00 +08:00
dragondriver
89748ac639
Expose metrics of data cluster (#7177)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-01 10:13:15 +08:00
Xiaofan
58b2089692
Import Embedded Etcd Kv Part 1(#7112) (#7174)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-08-24 09:45:51 +08:00
sunby
4f7c3e7d3a
Save binlog path in SegmentInfo (#6590)
issue: #6589
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-08-19 13:00:12 +08:00
XuanYang-cn
86ca96a02e
Fix master branch unittest error (#7083)
Resolves: #7080

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-08-13 14:00:14 +08:00
congqixia
e1b3273d1c
Add Datanode watch etcd channel (#6965)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-08-13 10:50:09 +08:00
XuanYang-cn
9d81a90402
Fix mem leak of flush cache in DataNode (#6862)
Resolves: #6858

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-08-11 14:24:09 +08:00
XuanYang-cn
47df4a2e57
Fix DataNode BackGroudGC problem (#6551)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-08-03 09:59:24 +08:00
XuanYang-cn
769cbb292a
Update doc in datanode (#6194)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-29 17:34:13 +08:00
Cai Yudong
7dbf33b788
Update component names in config files (#5989)
* rename master/indexService in .yaml

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename proxyNode to proxy in config files

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move master.yaml to root_coord.yaml, move proxy_node.yaml to proxy.yaml

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move proto/query_service.proto to proto/query_coord.proto

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move proxy_node_manager.go to proxy_manager.go

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add query_coord.pb.go

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update ci

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-22 19:08:03 +08:00
Cai Yudong
d5f4ee6f44
Rename proto master/index_service/proxy_service (#5973)
* rename proto proxy_service to proxy

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename proto index_service to index_coord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename proto master to root_coord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-22 16:14:09 +08:00
sunby
7f2d7c1da3
Rename dataservice to datacoordinator (#5912)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-21 18:22:13 +08:00
XuanYang-cn
688a587f4f
Add metrics in datanode (#5926)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-21 18:08:15 +08:00
Cai Yudong
92e429d812
Rename IndexService to IndexCoord (#5932)
* rename package indexservice to indexcoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename indexservice to indexcoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix queryservice static-check

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move distributed/indexservice to distributed/indexcoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move internal/indexservice to internal/indexcoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename indexservice to indexcoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename MasterComponent to RootCoordComponent

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename master to rootcoord for queryservice

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename master to rootcoord for dataservice

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename master to rootcoord for datanode

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename master to rootcoord for proxynode

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename master to rootcoord for querynode

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename master to rootcoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename IndexService to IndexCoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename IndexService to IndexCoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix rebase issue

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-21 17:28:03 +08:00
XuanYang-cn
441300140d
Tidy replica (#5920)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-21 16:00:22 +08:00
sunby
3cc0ab1345
Remove panic in NewDataSyncService (#5888)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-19 15:18:06 +08:00
sunby
33f5225968
Use physical channel to init msg stream (#5815)
* Use physical channel to init msg stream

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* fix search collection error check (#1)

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Co-authored-by: bigsheeper <954206947@qq.com>
2021-06-18 16:02:05 +08:00
XuanYang-cn
18eb27aa7a
Tidy replica in datanode (#5805)
- Remove not using functions
- Remove `initNode` logic in ParamTable
- Delete collection.go and collection_test.go

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-16 19:03:57 +08:00
Cai Yudong
0f4bd50ce3 Support ETCD endpoints array (#5755) 2021-06-15 16:36:36 +08:00
XuanYang-cn
dea0a4e522 Idempotent of FlushSegment (#5746)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-15 16:36:36 +08:00
XuanYang-cn
ab2fd34a2f fix double messages bug in datanode recovery (#5733)
* fix double messages bug in datanode recovery

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* remove debug log

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-15 16:14:29 +08:00
XuanYang-cn
46699c4c2b fix datanode dropcollection error (#5690)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-15 16:06:13 +08:00
sunby
189ac881f3 Fix bugs (#5676)
* Remove redundant session startup

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Register datanode after start success

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* fix meta snap shot

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* fix datanode message stream channel

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* Fix bugs when drop empty collection

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Fix bug of getting pchan statistics from task scheduler

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* Fix i/dist/dataservice test code

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* Fix epoch lifetime not applied

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* fix datanode flowgraph dd node

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* Fix handle datanode timetick bug

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Remove repack function of dml stream

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* fix proxynode

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Apply extended seal policy

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* add check for time tick

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* fix check

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Fix the repack function of dml stream

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* Fix the bug when send statistics of pchan

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* Fix the repack function when craete dml stream

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* fix bugs

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* fix describe collection

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Fix bug when send timestamp statistics

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* fix data node

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Add length check before flush request

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* add log for data node

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Fix SaveBinlog bugs

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Add more log in datanode

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* Put SegmentState.Flushing as the last one in enum to fit the client

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Fix params in GetInsertBinlogPaths

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Rename policy

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Remove unused ddl functions and fields

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* Remove pchan when drop collection

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* Add balanced assignment policy

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* fix master ut

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Add lock in session manager

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* add log for debug

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Fix some logic bug and typo

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* Fix recover bugs

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Get collection scheme of a specific timestamp

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* Change CheckPoint to SegmentInfo in VchannelInfo

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Recover Unflushed segment numOfRows

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* Fix dataservice unit tests

Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: yefu.chen <yefu.chen@zilliz.com>
Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: dragondriver <jiquan.long@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-15 16:06:11 +08:00
XuanYang-cn
e7fd3aa926 Flowgraph recovery (#5640)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-15 16:04:48 +08:00
XuanYang-cn
ac19711d74 Destroy DataNode when drop collections (#5638)
* Destroy DataNode when drop collections

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* golanci-lint

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

Co-authored-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-15 16:04:48 +08:00
neza2017
0a82c6381f add auto flush for data node (#5609)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-15 16:04:48 +08:00
congqixia
d0d845805b Change WatchDmChannelsRequest proto (#5577)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-15 16:04:48 +08:00
godchen
275881dbdb Remove address param (#5556)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-06-15 16:04:46 +08:00
XuanYang-cn
e96d39bf6a refactor datanode flowgraph (#5538)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-15 15:59:04 +08:00
XuanYang-cn
6c02ae4eb1 Remove Old msgposition logic (#5487)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-15 15:59:04 +08:00
congqixia
8657251f41 DataNode stores segment msgpack positions (#5472)
* DataNode stores segment msgpack positions

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* Fix typo caused deadlock

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* Use MsgPack EndPositions for flush pos

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-15 15:59:04 +08:00
XuanYang-cn
7a8b34ee9f Remove not using code in DataNode (#5476)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-15 15:59:04 +08:00
XuanYang-cn
55eac3ee7f Complete Flush procedure in DataNode (#5454)
* Complete Flush proceduer in DataNode

Resolves: #5220

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* fix unittest

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-15 15:59:04 +08:00
sunby
cfdd98b70f Fix recovery branch after rebasing master (#5444)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-15 15:59:04 +08:00
congqixia
e3956ad13f Change SaveBinLogPaths field2Path to repeated (#8)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-15 15:59:04 +08:00
yangxuan
ef563fda8f Updata flushe procedure
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-15 15:59:04 +08:00
sunby
c53afee616 DataNode scales flowgraph
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-15 15:59:04 +08:00
zhenshan.cao
03e3b4beb4
Add log to startup process (#5594)
* Add log to startup process

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

* Fixbug: avoid ctx exceed deadline

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

* Fmt code

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

* Fix bug: wrong time unit

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

* Fixbug: datanservice client reconnect use time ctx

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-06-04 16:29:35 +08:00
zhenshan.cao
b0b8f58192
Fix bug: search timeout (#5557)
* Fix bug: search timeout

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

* Add log and fix unittest bug

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-06-03 14:58:34 +08:00
neza2017
f80dbab6ec
watch proxynode (#5424)
watch proxy node and,
send InvalidataCollectionMetaCache to each proxynode

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

Resolves: #5423
2021-05-26 12:14:30 +00:00
neza2017
c28c34e852
let master use session.ServerID as nodeID (#5385)
let master use session.ServerID as nodeID

also see #5386 

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-05-25 07:06:05 +00:00
neza2017
e326eaad1e
master client,reconnect if grpc failed (#5360)
master client,reconnect if grpc failed

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-05-24 01:44:49 +00:00
godchen
0eee7e5252
Change session (#5329)
Change session.

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-05-21 11:28:52 +00:00