test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-03 12:29:36 +08:00
Code Issues Actions 13 Packages Projects Releases Wiki Activity
21,166 Commits 96 Branches 105 Tags 184 MiB
a45a288a25
Commit Graph

46 Commits

Author SHA1 Message Date
aoiasd
139787371e
feat: support embedding bm25 sparse vector and flush bm25 stats log (#36036) 
relate: https://github.com/milvus-io/milvus/issues/35853

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-09-19 10:57:12 +08:00
shaoting-huang
88b373b024
enhance: binlog primary key turn off dict encoding (#34358) 
issue: #34357 

Go Parquet uses dictionary encoding by default, and it will fall back to
plain encoding if the dictionary size exceeds the dictionary size page
limit. Users can specify custom fallback encoding by using
`parquet.WithEncoding(ENCODING_METHOD)` in writer properties. However,
Go Parquet [fallbacks to plain
encoding](e65c1e295d/go/parquet/file/column_writer_types.gen.go.tmpl (L238))
rather than custom encoding method users provide. Therefore, this patch
only turns off dictionary encoding for the primary key.

With a 5 million auto ID primary key benchmark, the parquet file size
improves from 13.93 MB to 8.36 MB when dictionary encoding is turned
off, reducing primary key storage space by 40%.

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
 2024-07-17 17:47:44 +08:00
smellthemoon
ef3ced8138
fix: descriptor event in previous version not has nullable to parse error (#34235) 
#34176

---------

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-07-01 16:38:06 +08:00
smellthemoon
2a1356985d
enhance: support null in go payload (#32296) 
#31728

---------

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-06-19 17:08:00 +08:00
Cai Yudong
00438f408f
enhance: Unify data type check APIs for go (#31887) 
Issue: #22837

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2024-04-07 14:27:22 +08:00
Buqian Zheng
3c80083f51
feat: [Sparse Float Vector] add sparse vector support to milvus components (#30630) 
add sparse float vector support to different milvus components,
including proxy, data node to receive and write sparse float vectors to
binlog, query node to handle search requests, index node to build index
for sparse float column, etc.

https://github.com/milvus-io/milvus/issues/29419

---------

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
 2024-03-13 14:32:54 -07:00
SimFG
26f06dd732
Format the code (#27275) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-09-21 09:45:27 +08:00
congqixia
41af0a98fa
Use go-api/v2 for milvus-proto (#24770) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-06-09 01:28:37 +08:00
jaime
c9d0c157ec
Move some modules from internal to public package (#22572) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-04-06 19:14:32 +08:00
Xiaofan
633a749880
Recude IndexCodec Load Memory (#20621) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-11-18 10:47:08 +08:00
SimFG
a55f739608
Separate public proto files (#19782) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-10-16 20:49:27 +08:00
SimFG
d7f38a803d
Separate some proto files (#19218) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-09-16 16:56:49 +08:00
xige-16
4de1bfe5bc
Add cpp data codec (#18538) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Co-authored-by: zhagnlu lu.zhang@zilliz.com

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-09-09 22:12:34 +08:00
codeman
70254c3ec5
Unified catalog interface for segment (#18289) (#18290) 
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Signed-off-by: kejiang <ke.jiang@zilliz.com>
Co-authored-by: kejiang <ke.jiang@zilliz.com>
 2022-08-20 10:24:51 +08:00
congqixia
4369e08f2a
Fix storage memory leak caused by runtime.SetFinalizer (#15100) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-01-10 17:13:35 +08:00
godchen
384ceea223
Close payload writer when error occurs (#15013) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-01-07 18:27:23 +08:00
edward.zeng
94e65eaaa5
[skip e2e] Refine storage binlog writer comment (#14312) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-12-27 15:42:03 +08:00
XuanYang-cn
6eb0de1640
[skip e2e]Update license for storage writer (#14040) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-12-23 12:03:28 +08:00
godchen
9576ec4dcb
[skip e2e]Fix binlog writer comment (#13576) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2021-12-17 19:42:50 +08:00
godchen
7e56f08747
Add payload bytes interface. (#13467) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2021-12-16 16:35:42 +08:00
Xiangyu Wang
405b3cd932
Use the same lint rules with golint in revive (#13324) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-12-14 15:31:07 +08:00
godchen
febdda90f4
Change binlog writer close behavior (#13046) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-12-09 12:37:06 +08:00
shaoyue
e00a4242df
[skip ci] Add comments (#11408) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2021-11-08 11:31:00 +08:00
godchen
a0a3a889e7
Add common endian for global use (#11092) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-11-02 18:16:32 +08:00
congqixia
74233fcfd8
[skip ci] Fix golint for binlog_writer.go (#9388) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-10-07 22:13:09 +08:00
dragondriver
cf8600077f
Refactor the index file format (#8514) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-09-29 09:52:12 +08:00
godchen
af173dd2a0
Add delete codec (#8736) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-09-28 14:30:02 +08:00
godchen
10c07a3041
Add description event extras. (#8264) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-09-23 17:23:54 +08:00
Xiaofan
acfd7c7911
[skip ci] Fix typo in binlog writer (#8255) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-09-18 22:55:51 +08:00
yukun
269e041e34
[skip ci]Fix typo (#8251) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-18 19:53:51 +08:00
godchen
3f5e876188
[skip ci]Add binlog writer comment (#8231) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-09-18 18:39:56 +08:00
Cai Yudong
3387b07dfd
Optimize code under storage (#6335) 
* rename AddOneStringToPayload/GetOneStringFromPayload to AddStringToPayload/GetStringFromPayload

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code optimize

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename print_binglog_test to print_binlog_test

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update chap08_binlog.md

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* use SetEventTimestamp() to replace SetStartTimestamp() and SetEndTimestamp()

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code optimize

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename AddStringToPayload/GetStringFromPayload to AddOneStringToPayload/GetOneStringFromPayload

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-07-07 19:10:07 +08:00
Xiangyu Wang
82ccd4cec0
Rename module (#4988) 
* Rename module

Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-04-22 14:45:57 +08:00
godchen
0dfcb90881 Add storage copyright 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-04-19 11:32:24 +08:00
godchen
a5ad70a5ab Add unittest for storage 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-04-19 10:36:19 +08:00
sunby
ddddd65d10 Delete internal/errors package 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-03-05 10:15:27 +08:00
cai.zhang
54f2b79f1b Add indexbuilder client 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-12-12 17:10:42 +08:00
bigsheeper
585d3f9831 Refactor param table, and add ddNode 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2020-12-11 17:20:14 +08:00
neza2017
8df05714a8 Add binlog unittest 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-12-11 12:01:38 +08:00
cai.zhang
62e9002348 Add git-hooks for check code 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-12-11 11:29:07 +08:00
neza2017
9585819154 Add binlog unittest 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-12-10 15:50:09 +08:00
neza2017
9cbebc0221 Add binlog unittest 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-12-10 14:52:42 +08:00
neza2017
de12fa5a10 Add binlog event 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-12-09 11:18:13 +08:00
sunby
ed54d3e26c Fix binlog reader bug 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-12-09 09:55:56 +08:00
quicksilver
d09ad77fb2 Add archiving file workflow for cache thirdparty 
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
 2020-12-08 19:53:22 +08:00
sunby
8ace1c3837 Add binlog serialization and deserialization 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-12-08 19:13:39 +08:00