Commit Graph

329 Commits

Author SHA1 Message Date
congqixia
4c37e94a39
Add type assertion for compactTask (#11468)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-10 19:16:40 +08:00
Xiaofan
5f08b67156
Move node.yaml to component.yaml (#11363)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-11-10 14:27:37 +08:00
godchen
863f1bb34e
Fix multi delete data not effect (#11422)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-09 15:01:17 +08:00
Xiangyu Wang
0eb71f7ee5
Logging traceID in process of processing delete request (#11374)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-11-09 11:01:02 +08:00
godchen
639800241d
Make flow graph for delta channel (#11379)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-09 09:27:04 +08:00
XuanYang-cn
3cd0584a04
Add compaction in datanode (#11353)
See also: #9530

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-08 19:49:07 +08:00
congqixia
3ac7d7a9de
Add detailed comments for dataSyncService in DataNode (#11285)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-05 22:59:04 +08:00
sunby
c26cd79532
Add compaction (#9906)
issue: #9904
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-11-05 22:25:00 +08:00
congqixia
0e5931a3da
Add mergedTimetickSender for datanode (#11297)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-05 14:59:32 +08:00
godchen
9186d5527e
Forward delete msg (#11210)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-05 11:59:02 +08:00
XuanYang-cn
8863c01ff7
Make DN fail faster when watching an invalid chan (#11102)
See also: #11098

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-04 15:56:28 +08:00
congqixia
0ef95c5df1
Add timeticker logger for insert buffer node (#11216)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-04 15:40:14 +08:00
godchen
b63a9521c2
Make gen msg private (#10115)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-04 15:36:19 +08:00
godchen
a0a3a889e7
Add common endian for global use (#11092)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-02 18:16:32 +08:00
XuanYang-cn
225b2c66a0
Fix datanode ut using register bug (#10885)
Resolves: #10881
See also: #8058

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-01 11:01:49 +08:00
XuanYang-cn
1e3fc5076f
Change etcdkv clientv3 into MetaKv interface (#10903)
This pr:
- changed etcdkv clientv3 into MetaKv interface
- replaced fmt.Sprintf with path.Join for kv key

See also: #8058

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-01 10:19:55 +08:00
Xiaofan
2df2d488e1
Fix Unit test fatal fail (#10933)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-10-30 10:24:38 +08:00
congqixia
756be81bce
Add injectFlush method for flushManager interface (#10659)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-28 19:16:31 +08:00
Xiaofan
6fddcb3925
Fix Session checker for proxy (#10737)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-10-27 21:58:33 +08:00
XuanYang-cn
42ce9b18d1
Add methods in segment replica (#10582)
see also: #8058

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-10-27 17:14:28 +08:00
congqixia
169af67eb1
Fix segments not removed from flushingCache (#10738)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-27 16:06:24 +08:00
cai.zhang
aff5cd9598
Supprot to read the configuration of minio from environment variables (#10534)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-10-26 15:10:03 +08:00
godchen
095a2378ca
Add datanode ut (#10433)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-25 22:52:27 +08:00
congqixia
ae2a301662
Add injection logic for FlushManager (#10580)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-25 20:17:34 +08:00
XuanYang-cn
21d98c2825
Add binlog io for compaction (#10536)
See also: #9530

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-10-25 20:13:51 +08:00
godchen
d2e0943f21
Fix delete node flush error (#10549)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-25 18:49:20 +08:00
congqixia
2713988ff2
Merge auto manual flush with same segment id (#10550)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-25 18:03:42 +08:00
congqixia
bd90a3831c
Allow flushed segments trigger flushe process (#10539)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-25 18:01:50 +08:00
XuanYang-cn
cd06f50645
Remove schema in delete codec (#10517)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-10-24 09:59:10 +08:00
godchen
31393d8801
[skip ci]Fix datanode comment typo error (#10502)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-23 16:57:35 +08:00
godchen
ffc0c07610
Change delete data primary key to int64 (#10438)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-22 15:37:12 +08:00
godchen
d6bf0c928f
Get stats delta path from datacoord (#10362)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-22 14:31:13 +08:00
congqixia
7ce7cb7a5e
Add a timed flush trigger mechanism (#10197)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-20 15:02:36 +08:00
godchen
6279fb817a
Fix stats log load with prefix error (#10216)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-19 20:50:35 +08:00
godchen
8daeb0d519
Change bloom filter use pk (#10193)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-19 20:18:47 +08:00
sunby
4031b8746f
Fix panic when flushing segment (#10182)
issue: #10180
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-10-19 19:06:37 +08:00
congqixia
ed4e516ce7
Save stats and delta logs from DataNode (#10159)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-19 14:34:35 +08:00
congqixia
5737e0075f
Apply flush manager logic in datanode (#10142)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-19 11:04:34 +08:00
congqixia
3711fcd82f
Add more info for Delta log compaction (#10083)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-18 18:24:45 +08:00
congqixia
0438ed2bfe
Use single signal chan to notify flush (#10075)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-18 12:34:34 +08:00
congqixia
90f88c6559
Add flush manager structure (#9986)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-18 10:38:35 +08:00
Xiaofan
d845153de4
Add Buffer for consumer channel (#9578)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-10-15 20:31:16 +08:00
XuanYang-cn
d0e6b52ce9
[skip ci]Update license for datanode (#9963)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-10-15 18:09:00 +08:00
XuanYang-cn
dcd56a575d
[skip ci]Update license for flowgraph of dn (#9962)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-10-15 18:07:09 +08:00
XuanYang-cn
9c706c6dc3
[skip ci]Update license on datanode (#9960)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-10-15 18:03:25 +08:00
XuanYang-cn
880056e664
[skip ci]Update license for datesync (#9959)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-10-15 18:01:38 +08:00
godchen
bf71965f58
Check watchlist error (#9953)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-15 17:02:37 +08:00
godchen
9e9ad7c7e6
Check pk filter merge error (#9952)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-15 16:58:42 +08:00
Xiaofan
a0b98dacda
Skip reconsume tt msgstream when datacoord restart (#9684)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-10-15 11:46:33 +08:00
godchen
2e0f7dfe42
Check data sync service error (#9856)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-14 17:40:42 +08:00