Commit Graph

51 Commits

Author SHA1 Message Date
xige-16
99984b88e1
Support delete varChar value (#16229)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-04-02 17:43:29 +08:00
xige-16
205c92e54b
Support insert string data (#15993)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-03-25 14:27:25 +08:00
godchen
78557ca6d7
Replace minio kv with minio chunk manager (#15936)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-03-17 18:03:23 +08:00
zhenshan.cao
f3eeecf146
Reduce the number of metrics (#16030)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-03-15 21:51:21 +08:00
Letian Jiang
2be217bcf4
Specify appropriate timestamp in initPKBloomFilter (#15823) (#15826)
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-03-03 17:33:56 +08:00
Xiaofan
bd7d294c6d
Refine log level in data (#15821)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-03-02 15:35:55 +08:00
bigsheeper
13177a90af
Add prometheus metrics for DataNode (#15650)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Co-authored-by: Cai Yudong <yudong.cai@zilliz.com>
2022-02-28 19:11:55 +08:00
XuanYang-cn
8ab9d769de
Fix logs of compaction_executor (#15365)
- Remove not useful log in `stopExecutingtaskByVChannelName`
- Add planID info in `mergeFlushedSegments`

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-01-25 13:29:40 +08:00
Cai Yudong
b6a48817b0
Add MinioConfig in GlobalParams for all components (#15099)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-01-10 17:29:34 +08:00
zhenshan.cao
b10c361822
Fix bug: should use Lock instead of RLock (#14797)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-01-04 23:33:24 +08:00
zhenshan.cao
76a9687310
Adjust usage of lock in datanode (#14631)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Co-authored-by: yangxuan <xuan.yang@zilliz.com>
2022-01-04 19:35:32 +08:00
XuanYang-cn
3cfa88d6f2
Fix delete node not aware of merged segments error (#14190)
This PR made merge gracefully remove merged segment by moving
them into `compactedSegments`

And everytime when delete_node operates, if compactedSegments
have contents, delete_node will replace the latest segment in delBuf
merged from the merged segments, and truely remove the compactedSegments

Fixes: #14085

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-12-27 11:02:18 +08:00
Cai Yudong
891c202b73
Use GlobalParamTable for all components (#13885)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-12-23 18:39:11 +08:00
Jiquan Long
0871bd3d63
Add log when update pk range (#14006)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-23 11:57:08 +08:00
JackLCL
87071f3f41
[skip e2e]Fix error log output format in segment_replica.go (#13806)
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
2021-12-20 22:51:25 +08:00
congqixia
1dcd06cc78
Add log file info for all kind of segment binlogs (#13606)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-19 20:00:42 +08:00
JackLCL
20c956b675
[skip e2e]Fix error log output format in segment_replica.go (#13676)
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
2021-12-19 12:32:42 +08:00
Bingyi Sun
0c39e0370c
Move segment statistics update to datanode tt loop (#13233)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-15 10:53:16 +08:00
JackLCL
30afcf2311
[skip ci]Fix error log output format in segment_replica.go (#13350)
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
2021-12-14 16:51:09 +08:00
Xiangyu Wang
405b3cd932
Use the same lint rules with golint in revive (#13324)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-12-14 15:31:07 +08:00
JackLCL
73b63b9a02
[skip ci]Fix error log output format in segment_replica.go (#12929)
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
2021-12-08 13:55:04 +08:00
congqixia
b20a238c38
Fix newDataSyncService panicking with normal segment has no dml pos (#12771)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-06 16:21:35 +08:00
XuanYang-cn
9bd3664672
Fix compaction bug (#11816)
This PR fixes:
- timeout in compaction
- BinlogIterator using rawID as PKField bug

See also: #11757

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-17 10:07:13 +08:00
XuanYang-cn
c4bb73dc74
Complete drop collection procedure in DN (#11552)
drop, cancel compaction, flush all buffers and release fg

See also: #11426

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-11 20:56:49 +08:00
XuanYang-cn
3cd0584a04
Add compaction in datanode (#11353)
See also: #9530

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-08 19:49:07 +08:00
godchen
a0a3a889e7
Add common endian for global use (#11092)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-02 18:16:32 +08:00
XuanYang-cn
42ce9b18d1
Add methods in segment replica (#10582)
see also: #8058

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-10-27 17:14:28 +08:00
godchen
d2e0943f21
Fix delete node flush error (#10549)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-25 18:49:20 +08:00
congqixia
2713988ff2
Merge auto manual flush with same segment id (#10550)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-25 18:03:42 +08:00
godchen
d6bf0c928f
Get stats delta path from datacoord (#10362)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-22 14:31:13 +08:00
godchen
6279fb817a
Fix stats log load with prefix error (#10216)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-19 20:50:35 +08:00
godchen
8daeb0d519
Change bloom filter use pk (#10193)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-19 20:18:47 +08:00
congqixia
5737e0075f
Apply flush manager logic in datanode (#10142)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-19 11:04:34 +08:00
XuanYang-cn
d0e6b52ce9
[skip ci]Update license for datanode (#9963)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-10-15 18:09:00 +08:00
godchen
9e9ad7c7e6
Check pk filter merge error (#9952)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-15 16:58:42 +08:00
godchen
a2dc0d8808
Init segment replica bloom filter (#9770)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-14 10:24:33 +08:00
godchen
3e26c19bad
[skip ci]Fix segment replica comment (#9654)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-11 21:12:44 +08:00
Cai Yudong
84e568ee32
Support Delete in datanode (#8505)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-10-11 16:31:44 +08:00
godchen
937dbf4279
Add flushed segments (#9476)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-08 19:09:12 +08:00
XuanYang-cn
9526d22dc7
[skip ci]Add comment for normal2FlushedSegment (#9431)
Signed-off-by: Yang Xuan <xuan.yang@zilliz.com>
2021-10-07 23:46:37 +08:00
Cai Yudong
d94361fdcc
Add parameter partitionID for API filterSegments (#8813)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-09-29 10:27:58 +08:00
godchen
77a773dff0
Change segment replica struct (#8608)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-09-27 14:38:00 +08:00
XuanYang-cn
b376107ab3
[skip-ci]Fix golint in segmentRep (#8601)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-26 20:51:58 +08:00
XuanYang-cn
6c434b4801
Improve codes in insertbuffernode (#8054)
Add error handling in insertbuffernode

See also: #7624, #7684

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-17 16:27:56 +08:00
XuanYang-cn
762060e670
Add ut for segmentReplica to 100% (#7589)
See also: #6357
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-09 10:14:00 +08:00
congqixia
daae05641a
Construct bloom filter when inserting (#7268)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-08-25 10:29:52 +08:00
XuanYang-cn
9d81a90402
Fix mem leak of flush cache in DataNode (#6862)
Resolves: #6858

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-08-11 14:24:09 +08:00
XuanYang-cn
357af89d55
Add unittest in segment_replica (#6197)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-30 10:26:12 +08:00
Cai Yudong
a09a3a1905
rename service to coord (#6020)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-23 16:14:08 +08:00
Cai Yudong
92e429d812
Rename IndexService to IndexCoord (#5932)
* rename package indexservice to indexcoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename indexservice to indexcoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix queryservice static-check

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move distributed/indexservice to distributed/indexcoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move internal/indexservice to internal/indexcoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename indexservice to indexcoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename MasterComponent to RootCoordComponent

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename master to rootcoord for queryservice

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename master to rootcoord for dataservice

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename master to rootcoord for datanode

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename master to rootcoord for proxynode

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename master to rootcoord for querynode

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename master to rootcoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename IndexService to IndexCoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename IndexService to IndexCoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix rebase issue

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-21 17:28:03 +08:00