Commit Graph

66 Commits

Author SHA1 Message Date
smellthemoon
2a1356985d
enhance: support null in go payload (#32296)
#31728

---------

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-06-19 17:08:00 +08:00
congqixia
f993b2913b
enhance: Reserve space of payload writer when serialize data (#33817)
See also #33561 #33562

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-06-17 12:06:04 +08:00
Buqian Zheng
3c80083f51
feat: [Sparse Float Vector] add sparse vector support to milvus components (#30630)
add sparse float vector support to different milvus components,
including proxy, data node to receive and write sparse float vectors to
binlog, query node to handle search requests, index node to build index
for sparse float column, etc.

https://github.com/milvus-io/milvus/issues/29419

---------

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-03-13 14:32:54 -07:00
Ted Xu
12acaf3e4f
enhance: Adding a generic stream payload reader (#30682)
See: #30404

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-02-21 17:10:52 +08:00
aoiasd
a0537156c0
enhance: delete codc deserialize data by stream batch (#30407)
relate: https://github.com/milvus-io/milvus/issues/30404

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-02-06 17:04:25 +08:00
Xu Tong
e429965f32
Add float16 approve for multi-type part (#28427)
issue:https://github.com/milvus-io/milvus/issues/22837

Add bfloat16 vector, add the index part of float16 vector.

Signed-off-by: Writer-X <1256866856@qq.com>
2024-01-11 15:48:51 +08:00
Xu Tong
9166011c4a
Add float16 vector (#25852)
Signed-off-by: Writer-X <1256866856@qq.com>
2023-09-08 10:03:16 +08:00
Enwei Jiao
1ef8f0fceb
Remove cgo PayloadWriter (#24892)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-06-14 18:04:38 +08:00
yah01
a9dccec03a
Add go payload writer (#24656) (#24762)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-06-09 13:52:39 +08:00
congqixia
41af0a98fa
Use go-api/v2 for milvus-proto (#24770)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-09 01:28:37 +08:00
yah01
546080dcdd
Support to retrieve json (#23563)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-04-21 11:46:32 +08:00
Enwei Jiao
967a97b9bd
Support json & array types (#23408)
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
2023-04-20 11:32:31 +08:00
jaime
c9d0c157ec
Move some modules from internal to public package (#22572)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-04-06 19:14:32 +08:00
xige-16
9aa99aedbb
[Cherry-Pick] Remove arrow uasge in FieldData (#22726)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-03-20 10:41:56 +08:00
Enwei Jiao
697dedac7e
Use cockroachdb/errors to replace other error pkg (#22390)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-02-26 11:31:49 +08:00
SimFG
a55f739608
Separate public proto files (#19782)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-10-16 20:49:27 +08:00
SimFG
d7f38a803d
Separate some proto files (#19218)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-09-16 16:56:49 +08:00
groot
b161aec95e
Support input empty string (#19111) (#19144)
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-09-13 13:36:29 +08:00
xige-16
4de1bfe5bc
Add cpp data codec (#18538)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Co-authored-by: zhagnlu lu.zhang@zilliz.com

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-09-09 22:12:34 +08:00
Enwei Jiao
16c3aedc15
refine complie configuration (#17502)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-06-24 21:12:15 +08:00
Xiaofan
801eeffbcc
Replace cgo parquet reader to go parquet reader (#16199)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-03-30 15:21:28 +08:00
xige-16
205c92e54b
Support insert string data (#15993)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-03-25 14:27:25 +08:00
Ji Bin
3cd28420f1
Support compile under windows (#15786)
This patch makes compile milvus under windows(MSYS), including:
- some cpp adaptation for compile under msys/gcc-10.3
- install toolchain scripts for setup from MinGW/MSYS `scripts/install_deps_msys.sh`
- adaptation for POSIX API use in golang
  * using gofrs/flock instead of syscall.Flock
  * using x/exp/mmap instead of syscall.Mmap
- introducing github actions for build milvus.exe under windows/MSYS
- rocksdb's patch for MSYS
- adaptation for compile knowhere under windows
- a windows package script for pack zip file, `scripts/package_windows.sh`

issue #7706

Signed-off-by: Ji Bin <matrixji@live.com>
2022-03-17 17:17:22 +08:00
Cai Yudong
adca79facc
Fix centos build error (#15760)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-02-25 18:49:54 +08:00
jaime
307a8ce535
Support compile and run on Mac (#15491)
Co-authored-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: Cai Yudong <yudong.cai@zilliz.com>
Co-authored-by: Jenny Li <jing.li@zilliz.com>
Co-authored-by: Nemo <yuchen.gao@zilliz.com>
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: Cai Yudong <yudong.cai@zilliz.com>
Co-authored-by: Jenny Li <jing.li@zilliz.com>
Co-authored-by: Nemo <yuchen.gao@zilliz.com>
2022-02-09 14:27:46 +08:00
Ji Bin
d83f69bb21
Support windows compliation (#15448)
- remove dl dependency while compiling some lib under MSYS
- int types aligement in golang
- core/src/utils adaptation for compile under MSYS

issue: #7706

Signed-off-by: Ji Bin <matrixji@live.com>
2022-02-07 23:43:46 +08:00
shaoyue
c5e3efa874
[skip e2e] Add comment for payload (#14803)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2022-01-05 13:53:35 +08:00
shaoyue
756904dd3e
[skip e2e] Add coment for payloadwriter (#14724)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2022-01-04 10:49:27 +08:00
XuanYang-cn
1567aa3f53
[skip e2e]Update license for storage payload (#14347)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-12-28 09:43:26 +08:00
shaoyue
94265639fe
[skip e2e] Add comment for PayloadWriter (#14138)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2021-12-24 13:29:03 +08:00
jaime
d623ea5c37
[skip e2e] Fix goint error in payload (#14109)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: yun.zhang <yun.zhang@zilliz.com>
2021-12-23 22:07:38 +08:00
shaoyue
cd410c6069
[skip e2e] Add comment for internal/storage.PayloadWriter (#13993)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2021-12-22 19:33:39 +08:00
shaoyue
770adc17ae
Support library link in centos7 (#13974)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2021-12-22 19:22:19 +08:00
shaoyue
8998e1d29b
[skip e2e] Add comment for PayloadReader (#13899)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2021-12-21 21:06:28 +08:00
jaime
7723d7b571
[skip e2e] Fix golint in payload (#13803)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: yun.zhang <yun.zhang@zilliz.com>
2021-12-20 22:45:15 +08:00
shaoyue
83a59d2545
[skip e2e]Add comment for payload (#13799)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2021-12-20 20:00:53 +08:00
godchen
7e56f08747
Add payload bytes interface. (#13467)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2021-12-16 16:35:42 +08:00
shaoyue
1f2c098fe7
[skip ci]Add comment for PayloadReader (#13486)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2021-12-16 09:37:22 +08:00
shaoyue
18fbc167f8
[skip ci] Add comment for payload (#13367)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2021-12-14 20:03:20 +08:00
godchen
febdda90f4
Change binlog writer close behavior (#13046)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-12-09 12:37:06 +08:00
godchen
5c44ee8c0f
Close binlog event when error occur (#12995)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-12-08 21:11:39 +08:00
XuanYang-cn
383f14999d
Make Arrow compilation easier (#13005)
replace all the bundled libraries with arrow_bundled_dependencies

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-12-08 20:55:05 +08:00
shaoyue
ccf957e1a8
[skip ci] Add comment for PayloadReader (#12869)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2021-12-07 14:51:21 +08:00
shaoyue
ffda2e5945
[skip ci] Fix comment for PayloadReader (#12621)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2021-12-02 18:07:33 +08:00
Bingyi Sun
a3d4cbdd4c
Convert c array to a golang slice (#12066)
issue: #11974
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-11-22 10:05:14 +08:00
Cai Yudong
b3d85adb6e
Use API HandleCStatus to handle CStatus returned by CGo in storage (#11782)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-11-15 13:27:09 +08:00
Cai Yudong
3387b07dfd
Optimize code under storage (#6335)
* rename AddOneStringToPayload/GetOneStringFromPayload to AddStringToPayload/GetStringFromPayload

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code optimize

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename print_binglog_test to print_binlog_test

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update chap08_binlog.md

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* use SetEventTimestamp() to replace SetStartTimestamp() and SetEndTimestamp()

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code optimize

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename AddStringToPayload/GetStringFromPayload to AddOneStringToPayload/GetOneStringFromPayload

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-07-07 19:10:07 +08:00
Xiangyu Wang
82ccd4cec0
Rename module (#4988)
* Rename module

Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-04-22 14:45:57 +08:00
godchen
0dfcb90881 Add storage copyright
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-04-19 11:32:24 +08:00
godchen
a5ad70a5ab Add unittest for storage
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-04-19 10:36:19 +08:00