chyezh
2586c2f1b3
enhance: use WalkWithPrefix api for oss, enable piplined file gc ( #31740 )
...
issue: #19095,#29655,#31718
- Change `ListWithPrefix` to `WalkWithPrefix` of OOS into a pipeline
mode.
- File garbage collection is performed in other goroutine.
- Segment Index Recycle clean index file too.
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-04-25 20:41:27 +08:00
yah01
9658367a3c
Refine chunk manager errors ( #27590 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-31 12:18:15 +08:00
congqixia
2f201c25e2
Remove deprecated io/ioutil usage ( #27747 )
...
`io/ioutil` package is deprecated, use `io`,`os` package replacement
also added golangci-lint rule to block future reference
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: guoguangwu <guoguangwu@magic-shield.com>
2023-10-17 20:32:09 +08:00
jaime
c9d0c157ec
Move some modules from internal to public package ( #22572 )
...
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-04-06 19:14:32 +08:00
yah01
90a5aa6265
Refine errors, re-define error codes ( #22501 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-03-09 15:47:52 +08:00
congqixia
ea4ab9e4a4
Replace ErrorList with multiErrors ( #22532 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-03-03 14:59:49 +08:00
Enwei Jiao
697dedac7e
Use cockroachdb/errors to replace other error pkg ( #22390 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-02-26 11:31:49 +08:00
groot
bf2107ecf5
Fix a regression of local storage ( #20653 )
...
Signed-off-by: yhmo <yihua.mo@zilliz.com>
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-11-21 10:19:10 +08:00
Xiaofan
766957e75a
Recuce memory consumption in MinioChunkManager ( #20284 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-11-03 12:19:35 +08:00
congqixia
838a633584
Add ctx parameter for ChunkManager methods ( #19546 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-09-29 16:18:56 +08:00
congqixia
c91bb0b015
Unify rootPath in configs and ChunkManager ( #18808 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-08-25 19:32:53 +08:00
xing.zhao
d90313a1a8
Use chunkManager rather than minio.Client in datanode garbage collection ( #18092 )
...
Signed-off-by: xingzhao <xing.zhao@zilliz.com>
Co-authored-by: xingzhao <xing.zhao@zilliz.com>
2022-07-22 22:10:28 +08:00
cai.zhang
0ad15a22c8
IndexCoord handle events correctly ( #17878 )
...
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-07-07 14:44:21 +08:00
Xiaofan
5355153805
Minio Error is not Handled gracefully ( #17003 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-05-16 19:25:55 +08:00
congqixia
9769426e84
Use fmt.Errorf instead of string concat in local_chunk_manager.go ( #16442 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-04-19 18:39:39 +08:00
godchen
4781db8a2a
Add datanode import ( #16414 )
...
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-04-12 22:19:34 +08:00
godchen
bb7a0766fe
Add dependency factory ( #16204 )
...
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-04-07 22:05:32 +08:00
godchen
478f6ca11e
Add cache for vector chunk ( #15912 )
...
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-03-26 22:05:26 +08:00
godchen
fcf0887d92
Add and implement chunk manager interface. ( #15541 )
...
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-02-24 23:53:53 +08:00
congqixia
998ebcce4e
Fix file not close when ReadAll returns error in local chunk manager ( #15068 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-01-10 11:53:34 +08:00
XuanYang-cn
c29cf1bc21
[skip e2e]Update license for storage localchunck ( #14192 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-12-24 17:30:32 +08:00
jaime
dec2eba610
[skip ci] Fix grammar mistake in local_chunk_manager ( #13418 )
...
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
Co-authored-by: yun.zhang <yun.zhang@zilliz.com>
2021-12-15 11:56:07 +08:00
godchen
c23236c50e
Fix chunk manager ci lint ( #8779 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-09-28 20:46:04 +08:00
congqixia
bdfb8768d4
Enable gosec ( #8202 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-18 14:45:50 +08:00
godchen
31df0faaf5
[skip ci]Add chunk manager comment ( #7833 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-09-17 14:49:50 +08:00
bigsheeper
7307332fab
Add nil check ( #6832 )
...
* add nil check
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
* use grpc functions
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
* use grpc functions
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
* fix indexCoord crash
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-07-29 20:11:23 +08:00
godchen
db94d7771f
Read vector from disk ( #6707 )
...
* Read vector from disk
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* go fmt
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* fix git action error
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* fix error
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* fix test error
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* fix action error
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* fix caculate error
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* change var name
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* remove unused method
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* remove unused method
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* fix error
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* fix len error
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* remove unused code
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* change bytes to float method
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* change float to bytes method
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* fix action error
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-07-24 09:25:22 +08:00
godchen
b9f5949680
Add vector chunk manager ( #6613 )
...
* add file manager
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* add file manager
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* add vector file manager
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* rename
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* go fmt
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* fix ut error
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* fix action error
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* fix memory error
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-07-20 15:06:09 +08:00
