Commit Graph

375 Commits

Author SHA1 Message Date
congqixia
8f87cc7ef1
Fix JoinIDPath logic problem (#13039)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-09 12:05:04 +08:00
XuanYang-cn
dbf757e08b
Change genKey into JoinIDPath (#12959)
1. Simplify genKey(), make genKey a rpc call
2. Unify the usage of JoinIDPath

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-12-09 11:09:06 +08:00
XuanYang-cn
a9a332dbcf
Fix empty segment compact and load (#12710)
1. Fix compaction save empty segment bug
2. Fix load empty segment bug
3. Add UT for dmlchannels to 100%

Resolves: #12450

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-12-09 11:03:08 +08:00
congqixia
1195f441b2
Update DataNode state before stop (#12946)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-08 14:19:04 +08:00
JackLCL
73b63b9a02
[skip ci]Fix error log output format in segment_replica.go (#12929)
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
2021-12-08 13:55:04 +08:00
JackLCL
a6ce7d7f53
[skip ci]Fix error log output format in meta_service.go (#12928)
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
2021-12-08 13:51:09 +08:00
JackLCL
e51ef53632
[skip ci]Fix error log output format in flush_manager.go (#12857)
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
2021-12-07 14:38:15 +08:00
congqixia
b20a238c38
Fix newDataSyncService panicking with normal segment has no dml pos (#12771)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-06 16:21:35 +08:00
congqixia
58c5b6093d
Fix close10000 ut has chances to fail (#12740)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-06 09:59:35 +08:00
Bingyi Sun
8bbaae04ff
Decrease compaction parallel (#12707)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-03 18:57:33 +08:00
congqixia
ffd05eb140
Fix deadlock in mergedTimeTickSender (#12697)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-03 16:39:33 +08:00
edward.zeng
45cbec1f7b
[skip ci] Refine datanode binlog io comment (#12638)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-02 19:11:32 +08:00
edward.zeng
c2b73e1b55
[skip ci] Refine data node flow graph comment (#12637)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-02 18:59:33 +08:00
congqixia
8f9e62fa18
Apply DropVirtualChannel and FlushManager drop mode (#12563)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-02 16:39:33 +08:00
godchen
5e3e3b3b41
Remove grpc returned error (#12537)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-12-01 22:17:46 +08:00
edward.zeng
271aaa7537
[skip ci] Refine data node param table comment (#12560)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-01 19:31:36 +08:00
congqixia
e813867652
Implement Drop mode for flush manager (#12469)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-01 10:11:39 +08:00
godchen
f31ed089b5
Add timeout for reader next (#12308)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-26 22:45:24 +08:00
XuanYang-cn
48b45d82e5
Add ut for binlog_io to 100 coverage (#12283)
Make DN ut coverage upto 90%
Resolves: #8058

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-26 17:43:17 +08:00
dragondriver
b6d0150dc5
Fix health check when disconnect to etcd (#12292)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-11-26 11:39:16 +08:00
bigsheeper
e245ba87a9
Add excluded segments for dropped segemnts (#12247)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-11-25 14:25:15 +08:00
XuanYang-cn
8a8ebed23d
Fix DN GC bug (#12258)
Fixes: #12249

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-25 09:43:15 +08:00
congqixia
59d19384dc
Fix flush manager injection logic with multiple segments (#12260)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-24 18:41:16 +08:00
godchen
e3f49858c5
Add subscription prefix (#12251)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-24 17:47:15 +08:00
dragondriver
6c4c0ef6b5
Add more deploy metrics (#12248)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-11-24 15:43:15 +08:00
XuanYang-cn
e72c5b9351
Filter dropped segments in DN (#12242)
See also: #12131, #12230

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-24 14:11:15 +08:00
XuanYang-cn
6bf0c4326d
Fix compaction generates wrong insert binlogs (#12204)
This PR
- changes Get() to GetRow() in BinlogIterator
- removes Get() and Length() of FieldData
- adds ut in BinlogIterator for floatvector and binaryvector
- fix a log bug

Fixes: #12146

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-23 17:09:15 +08:00
congqixia
4121e31df1
Send SIGINT to runner goroutine after etcd disconnects (#12163)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-22 16:23:17 +08:00
godchen
51b353b52c
Add msgstream reader (#12130)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-19 15:57:12 +08:00
congqixia
7e91bcf115
Normalize datanode dminput node log (#12128)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-19 14:21:13 +08:00
dragondriver
ee0f753f7a
Fix datarace between GetComponentStates and Register (#11935)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-11-19 13:57:12 +08:00
Bingyi Sun
038f42851e
Add more log for debug (#11893)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-11-18 10:37:41 +08:00
XuanYang-cn
7dbb4d10e5
Fix DeltaLogs empty yet appending a deltaInfo bug (#11984)
- Add more logs for startpositions

Fixes: #11970

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-17 17:57:11 +08:00
congqixia
5e550a8793
Add close for flushManager (#11874)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-17 10:39:15 +08:00
XuanYang-cn
9bd3664672
Fix compaction bug (#11816)
This PR fixes:
- timeout in compaction
- BinlogIterator using rawID as PKField bug

See also: #11757

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-17 10:07:13 +08:00
congqixia
5edbb82610
Add session revoke (#11908)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-16 22:31:14 +08:00
Xiangyu Wang
12f50cb22c
[skip ci]Update OWNERS files (#11898)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-11-16 15:41:11 +08:00
congqixia
fa6314393a
Fix compactor loop closure captured (#11814)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-15 19:44:10 +08:00
congqixia
579416e3d3
Apply retry for flush manager execution (#11808)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-15 17:19:10 +08:00
XuanYang-cn
f3852c1db0
Add more specific logs (#11724)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-15 15:25:09 +08:00
Bingyi Sun
5c8978e5d6
Fix drop segment bug (#11709)
issue: #11708
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-11-12 17:31:11 +08:00
XuanYang-cn
c4bb73dc74
Complete drop collection procedure in DN (#11552)
drop, cancel compaction, flush all buffers and release fg

See also: #11426

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-11 20:56:49 +08:00
godchen
5357e301f4
Fix retrieve search error (#11501)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-11 00:54:45 +08:00
godchen
14d2891a69
[skip ci]Fix download typo (#11577)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-10 20:11:35 +08:00
godchen
0fc1e12faf
[skip ci]Fix download typo (#11575)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-10 20:09:33 +08:00
godchen
038d0f1adf
[skip ci]Fix download typo (#11573)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-10 20:07:39 +08:00
congqixia
4c37e94a39
Add type assertion for compactTask (#11468)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-10 19:16:40 +08:00
Xiaofan
5f08b67156
Move node.yaml to component.yaml (#11363)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-11-10 14:27:37 +08:00
godchen
863f1bb34e
Fix multi delete data not effect (#11422)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-09 15:01:17 +08:00
Xiangyu Wang
0eb71f7ee5
Logging traceID in process of processing delete request (#11374)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-11-09 11:01:02 +08:00