congqixia
4850336ca3
fix: Write padding at end of mmap file not chunk ( #34529 )
...
Related to #34508
The padding bytes shall be written only at the end of the mmap file not
the chunk of each field data file.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-10 11:12:14 +08:00
sre-ci-robot
0b3f317b14
[automated] Update Knowhere Commit ( #34549 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-07-10 01:26:14 +08:00
congqixia
6b4d977a10
fix: Write padding into mmap file in case of SIGBUS ( #34443 )
...
See also #34442
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-05 17:44:09 +08:00
sre-ci-robot
d65b6895d7
[automated] Update Knowhere Commit ( #34432 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-07-05 01:18:09 +08:00
Gao
ec5db7a57d
enhance: update knowhere version ( #34361 )
...
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2024-07-02 19:18:24 +08:00
yihao.dai
734415b8a2
fix: Reduce duplicate PKs in segcore ( #34267 )
...
issue: https://github.com/milvus-io/milvus/issues/34247
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-07-01 17:42:06 +08:00
smellthemoon
ef3ced8138
fix: descriptor event in previous version not has nullable to parse error ( #34235 )
...
#34176
---------
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-07-01 16:38:06 +08:00
congqixia
14e827dc6c
fix: Implement singleflight for segcore ChunkCache ( #34250 )
...
See also #34249
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-01 11:46:06 +08:00
zhagnlu
3030e4625e
enhance: refactor variable column to reduce memory cost ( #33875 )
...
#33874
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-06-30 20:16:06 +08:00
zhagnlu
cc1bc07bfd
enhance: add log to bitmap index ( #34197 )
...
#32900
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-06-30 20:02:06 +08:00
Cai Yudong
ad90360162
enhance: Update knowhere commit ( #34223 )
...
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
2024-06-27 18:20:06 +08:00
Gao
a933f6731b
fix: centroids file not removed when data skew in major compaction ( #34050 )
...
issue: https://github.com/milvus-io/milvus/issues/30633
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2024-06-26 10:48:04 +08:00
sre-ci-robot
cc54ab2e47
[automated] Update Knowhere Commit ( #34172 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-06-26 02:20:03 +08:00
sre-ci-robot
254b555d53
[automated] Update Knowhere Commit ( #34119 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-06-25 01:22:02 +08:00
Jiquan Long
aa36f9feed
fix: [ut] regex query under unsupported index ( #34087 )
...
/kind improvement
issue: https://github.com/milvus-io/milvus/issues/29988
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-06-24 14:08:03 +08:00
Patrick Weizhi Xu
b961767005
enhance: support integral type for MV and skip MV if there is only one category ( #33161 )
...
issue: #29892
---------
Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
2024-06-24 10:20:01 +08:00
zhagnlu
03a3f50892
enhance: add skip using array index when some situation ( #33947 )
...
#32900
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-06-23 21:26:02 +08:00
zhagnlu
0d7ea8ec42
enhance: Enhance and correct exception module ( #33705 )
...
#33704
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-06-23 21:22:01 +08:00
chyezh
259a682673
enhance: async search and retrieve in cgo ( #33228 )
...
issue: #30926 , #33132
related pr: #33133
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-06-22 09:38:02 +08:00
sre-ci-robot
d9f986439d
[automated] Update Knowhere Commit ( #34040 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-06-21 01:50:01 +08:00
presburger
dac20d41c0
fix: fix milvus gpu compile error ( #33995 )
...
NVIDIA RAFT uses customized versions of spdlog and fmt, which conflict
with the versions provided by Conan. Therefore, it is necessary to
isolate the build environment for knowhere to avoid these conflicts.
issue: https://github.com/milvus-io/milvus/issues/33991
Signed-off-by: yusheng.ma <yusheng.ma@zilliz.com>
2024-06-20 20:30:01 +08:00
cqy123456
dc4437ff82
enhance: use segment id and type to register in MmapChunkManager and opt malloc in variableChunk ( #33993 )
...
issue: https://github.com/milvus-io/milvus/issues/32984
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2024-06-20 17:42:02 +08:00
smellthemoon
2a1356985d
enhance: support null in go payload ( #32296 )
...
#31728
---------
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-06-19 17:08:00 +08:00
Jiquan Long
7b9462c0d3
enhance: fix copying hits of inverted index twice ( #33968 )
...
issue: https://github.com/milvus-io/milvus/issues/29793
The custom `VecCollector` have already transformed the results into
vector of offsets, no need to copy them twice.
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-06-19 12:40:01 +08:00
cqy123456
298e50b834
enhance: check index with data type ( #33880 )
...
issue: https://github.com/milvus-io/milvus/issues/22837
related: https://github.com/milvus-io/milvus/pull/33878
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2024-06-19 10:15:59 +08:00
Gao
0d20303e54
fix: fix binary vector data size ( #33750 )
...
issue: https://github.com/milvus-io/milvus/issues/22837
- fix byte size wrong for binary vectors
- fix the expect/actual error msg
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2024-06-18 21:39:59 +08:00
cqy123456
b460862537
fix: can't find Chunk struct after growing support mmap ( #33951 )
...
issue: https://github.com/milvus-io/milvus/issues/32984
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2024-06-18 18:37:58 +08:00
congqixia
3fdaae8792
fix: Return record with largest timestamp for entires with same PK ( #33936 )
...
See also #33883
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-06-18 15:55:59 +08:00
cqy123456
32f685ff12
enhance: growing segment support mmap ( #32633 )
...
issue: https://github.com/milvus-io/milvus/issues/32984
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2024-06-18 14:42:00 +08:00
presburger
e83ecd5074
enhance: fix not find arrow::parquet lib ( #33928 )
...
Signed-off-by: yusheng.ma <yusheng.ma@zilliz.com>
2024-06-18 00:22:01 +08:00
zhagnlu
d43ec4db0b
enhance: support array bitmap index ( #33527 )
...
#32900
---------
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-06-16 21:51:58 +08:00
zhagnlu
e422168f09
fix: readd timestamp index because segment timestamp not ordered ( #33856 )
...
#33533
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-06-16 21:48:01 +08:00
Yinzuo Jiang
7d74edd6dd
fix: update clang-tidy and clang-format from 10 to 12 ( #33141 )
...
Default llvm toolchain version in Ubuntu 20.04 is 10, while Ubuntu 22.04
does not have `clang-tidy-10` or `clang-format-10` by default.
issue: #33142
Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
Signed-off-by: Yinzuo Jiang <jiangyinzuo@foxmail.com>
2024-06-13 15:27:58 +08:00
Buqian Zheng
47b04ea167
enhance: support sparse cardinal hnsw index ( #33656 )
...
issue: #29419
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-06-12 16:57:55 +08:00
Buqian Zheng
8cb350598c
enhance: Improve GetVectorById of Sparse Float Vector ( #33209 )
...
issue: #29419
* sparse float vector to support raw data mmap
For get vector from chunk cache, I added a unit test but marking it as
skipped due to a known issue. I have tested it locally.
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-06-12 10:09:55 +08:00
Jiquan Long
ecf2bcee42
enhance: speed up array-equal operator via inverted index ( #33633 )
...
fix : #33632
---------
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-06-11 14:13:54 +08:00
chyezh
f53ab54c5d
enhance: async cgo utility ( #33133 )
...
issue: #30926 , #33132
- implement future-based cgo utility.
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-06-09 22:55:53 +08:00
zhagnlu
6ce9df913f
fix: clean vector memory ( #33692 )
...
#33533
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-06-07 10:37:54 +08:00
cai.zhang
27cc9f2630
enhance: Support analyze data ( #33651 )
...
issue: #30633
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
Co-authored-by: chasingegg <chao.gao@zilliz.com>
2024-06-06 17:37:51 +08:00
wei liu
b69740c8f3
enhance: Remove unnecessary log info during load segment ( #33663 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-06-06 14:13:50 +08:00
sre-ci-robot
fd191dd7db
[automated] Update Knowhere Commit ( #33655 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-06-06 01:53:50 +08:00
cqy123456
703fc73f71
enhance: disk index support binary vector ( #33631 )
...
issue:https://github.com/milvus-io/milvus/issues/22837
related https://github.com/milvus-io/milvus/pull/33575
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2024-06-05 19:37:57 +08:00
zhagnlu
8ad26093ba
fix: fix load failure ( #33599 )
...
issue: #33533
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-06-05 19:19:51 +08:00
Gao
545d4725fb
fix: correct get vector data size for bf16/fp16/binary vector ( #33377 )
...
related #22837
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2024-06-05 14:31:57 +08:00
congqixia
597f4c5e03
enhance: Make hasMoreResult accurate when hit number larger than limit ( #33609 )
...
See also milvus-io/milvus-sdk-go#756
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-06-05 11:51:51 +08:00
zhagnlu
c6f8a73bb2
enhance: optimize some cache to reduce memory usage ( #33534 )
...
#33533
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-06-04 14:09:47 +08:00
sre-ci-robot
d25c755480
[automated] Update Knowhere Commit ( #33573 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-06-04 01:55:46 +08:00
congqixia
2b285e5573
fix: Wrap init segcore tracing with golang timeout ( #33494 )
...
See also #33483
Wrap `C.InitTrace` & `C.SetTrace` with timeout preventing otlp
initializtion hangs forever when endpoint is not set correctly
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-06-03 19:25:51 +08:00
Buqian Zheng
4171414222
enhance: update knowhere version ( #33490 )
...
issue: https://github.com/milvus-io/milvus/issues/33489
update knowhere version to latest. remove usage of `seed_ef` as it be
replaced by existing `ef`.
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-05-31 10:17:50 +08:00
Jiquan Long
0c5d8660aa
feat: support inverted index for array ( #33452 )
...
issue: https://github.com/milvus-io/milvus/issues/27704
---------
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-05-31 09:47:47 +08:00
