Commit Graph

37 Commits

Author SHA1 Message Date
xige-16
99984b88e1
Support delete varChar value (#16229)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-04-02 17:43:29 +08:00
XuanYang-cn
bccf65ec67
[skip e2e]Update license for storage datacodec (#14039)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-12-23 12:01:36 +08:00
godchen
7e56f08747
Add payload bytes interface. (#13467)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2021-12-16 16:35:42 +08:00
XuanYang-cn
48b45d82e5
Add ut for binlog_io to 100 coverage (#12283)
Make DN ut coverage upto 90%
Resolves: #8058

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-26 17:43:17 +08:00
godchen
9d5bcd3e3a
Close event and binlog reader (#12173)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-22 17:27:14 +08:00
godchen
863f1bb34e
Fix multi delete data not effect (#11422)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-09 15:01:17 +08:00
XuanYang-cn
cd06f50645
Remove schema in delete codec (#10517)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-10-24 09:59:10 +08:00
godchen
ffc0c07610
Change delete data primary key to int64 (#10438)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-22 15:37:12 +08:00
XuanYang-cn
2255fe0b45
Change deserialize deltelog from 1 blob to blobs (#10085)
See also: #9530

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-10-19 10:28:35 +08:00
godchen
59ab0e441c
Add bloom filter for stats (#9630)
* Add bloom filter for stats

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* trigger GitHub actions

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-13 10:22:33 +08:00
dragondriver
dedf745b76
Rename IndexParamsFile to IndexParamsKey (#9563)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-10-09 19:27:02 +08:00
dragondriver
818cf3ffa0
Split blob into several string rows when index file is large (#8919)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-30 17:57:01 +08:00
dragondriver
cf8600077f
Refactor the index file format (#8514)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-29 09:52:12 +08:00
godchen
af173dd2a0
Add delete codec (#8736)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-09-28 14:30:02 +08:00
godchen
db94d7771f
Read vector from disk (#6707)
* Read vector from disk

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* go fmt

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix git action error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix test error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix action error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix caculate error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* change var name

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* remove unused method

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* remove unused method

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix len error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* remove unused code

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* change bytes to float method

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* change float to bytes method

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix action error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-07-24 09:25:22 +08:00
Cai Yudong
a992dcf6a8
Support query return vector output field (#6570)
* improve code readibility

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add offset in RetrieveResults

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add VectorFieldInfo into Segment struct

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add new interface for query vector

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update load vector field logic

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update load vector field logic

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fill in field name in query result

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add FieldId into FieldData

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add fillVectorOutputFieldsIfNeeded

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update data_codec_test.go

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add DeserializeFieldData

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* realize query return vector output field

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix static-check

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* disable query vector case

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-07-16 17:19:55 +08:00
godchen
1c6786f85c
Add blob info (#5792)
* Add blob info

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-06-16 12:03:57 +08:00
Xiangyu Wang
23c4de0eb8
Flush statistics for all int64 fields (#5318)
Resolves: #5262

Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-05-20 10:38:45 +00:00
Xiangyu Wang
82ccd4cec0
Rename module (#4988)
* Rename module

Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-04-22 14:45:57 +08:00
godchen
0dfcb90881 Add storage copyright
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-04-19 11:32:24 +08:00
godchen
a5ad70a5ab Add unittest for storage
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-04-19 10:36:19 +08:00
godchen
f3649f0419 Refactor interface and proto
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-03-12 14:22:09 +08:00
bigsheeper
01e9dc8e3f Remove collection name
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-02-03 11:52:19 +08:00
bigsheeper
5e781b9370 Remove field name in query node and segCore
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-02-03 10:10:07 +08:00
sunby
45b99c0cf3 Save indexName and indexID in IndexCodec
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-02-02 19:56:11 +08:00
bigsheeper
73d2b6a101 Get index param from minio and filter by vector fields
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-29 15:22:24 +08:00
neza2017
3a866dab3d Update proto
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-28 17:25:43 +08:00
neza2017
2ebeade25e Fix get component states
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-28 17:13:00 +08:00
sunby
2be8cc1c4b Add index params serde to IndexCodec
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-01-28 16:41:24 +08:00
XuanYang-cn
7ce0f27ebc Add buffer to minIO for binlogs
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-23 18:06:04 +08:00
sunby
e7ebfcb05a Save index meta to meta table
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-23 15:13:45 +08:00
godchen
e52b130ba6 Refactor data codec code
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2020-12-23 15:46:16 +08:00
sunby
0b9cffaf87 Remove index_name in IndexRequest and remove indexType in FieldIndexMeta
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-23 11:34:35 +08:00
XuanYang-cn
87521adfbd Add logic of deserilize rawData into InsertData
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-21 16:27:03 +08:00
cai.zhang
62e9002348 Add git-hooks for check code
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-12-11 11:29:07 +08:00
quicksilver
21721d2e03 Cache go mod workflow for jenkins pipeline
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2020-12-10 09:57:14 +08:00
XuanYang-cn
881be7f3e3 Impl minimum runable writenode and writenode flowgraph
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-09 20:07:27 +08:00