sre-ci-robot
454984aa4e
[automated] Update Knowhere Commit ( #32181 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-04-12 01:25:19 +08:00
cai.zhang
1b767669a4
enhance: Throw error instead of crash when index cannot be built ( #31844 )
...
issue: #27589
---------
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-04-09 11:51:18 +08:00
Jiquan Long
03e0db109e
fix: udpate Cargo.lock ( #31859 )
...
issue: #31681
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-04-03 14:18:23 +08:00
Jiquan Long
9750e78f1d
enhance: lock tantivy dependencies ( #31688 )
...
issue: https://github.com/milvus-io/milvus/issues/31681
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-03-29 10:15:17 +08:00
Jiquan Long
e33dba8afe
fix: [skip-e2e] use zstd-sys 2.0.9 ( #31682 )
...
fix : #31681
/kind improvement
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-03-28 15:14:10 +08:00
sre-ci-robot
678cb187e8
[automated] Update Knowhere Commit ( #31630 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-03-27 01:15:10 +08:00
foxspy
b35ecebcc3
enhance: Update Knowhere version ( #31392 )
...
/kind branch-feature
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2024-03-19 15:11:07 +08:00
sammy.huang
d7727dd087
enhance: fetch simdjson directly in the format of targz ( #31369 )
...
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-03-18 18:55:11 +08:00
foxspy
1c930e560c
enhance: Update Knowhere version ( #31312 )
...
/kind branch-feature
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2024-03-18 11:29:04 +08:00
sre-ci-robot
53af6d8c59
[automated] Update Knowhere Commit ( #31151 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-03-09 01:55:02 +08:00
presburger
19c64067af
enhance: jemalloc aarch64 platform use 64k pagesize. ( #29522 )
...
enhance: jemalloc aarch64 platform use 64k pagesize.
issue: #28843
Signed-off-by: Yusheng.Ma <Yusheng.Ma@zilliz.com>
2024-03-07 21:01:01 +08:00
sre-ci-robot
2d9de233fc
[automated] Update Knowhere Commit ( #31089 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-03-07 12:05:02 +08:00
sre-ci-robot
c047f09110
[automated] Update Knowhere Commit ( #31015 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-03-05 16:31:00 +08:00
sre-ci-robot
3dc5e38240
[automated] Update Knowhere Commit ( #30989 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-03-04 16:34:59 +08:00
Buqian Zheng
f658dd5faa
enhance: update knowhere version to 60a5c9c ( #30788 )
...
/kind improvement
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-02-28 14:18:55 +08:00
sre-ci-robot
6e9f3ea531
[automated] Update Knowhere Commit ( #30744 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-02-28 10:50:57 +08:00
zhagnlu
976b6fc0e4
enhance: change opendal as compile configurable ( #30384 )
...
#30373
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-02-20 19:16:52 +08:00
foxspy
43e8cd531d
enhance: Update Knowhere version ( #30675 )
...
issue: #30669
Signed-off-by: xianliang <xianliang.li@zilliz.com>
2024-02-19 22:04:51 +08:00
sre-ci-robot
ebbe32df9a
[automated] Update Knowhere Commit ( #30515 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-02-05 01:32:44 +08:00
sre-ci-robot
20c9cfc587
[automated] Update Knowhere Commit ( #30487 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-02-04 01:23:04 +08:00
Jiquan Long
e549148a19
enhance: full-support for wildcard pattern matching ( #30288 )
...
issue: #29988
This pr adds full-support for wildcard pattern matching from end to end.
Before this pr, the users can only use prefix match in their expression,
for example, "like 'prefix%'". With this pr, more flexible syntax can be
combined.
To do so, this pr makes these changes:
- 1. support regex query both on index and raw data;
- 2. translate the pattern matching to regex query, so that it can be
handled by the regex query logic;
- 3. loose the limit of the expression parsing, which allows general
pattern matching syntax;
With the support of regex query in segcore backend, we can also add
mysql-like `REGEXP` syntax later easily.
---------
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-02-01 12:37:04 +08:00
sre-ci-robot
0542a0e7dc
[automated] Update Knowhere Commit ( #30332 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-01-29 01:05:01 +08:00
sre-ci-robot
e967949cc5
[automated] Update Knowhere Commit ( #30120 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-01-22 18:40:54 +08:00
Bingyi Sun
e1258b8cad
feat: integrate storagev2 into loading segment ( #29336 )
...
issue: #29335
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-01-12 18:10:51 +08:00
sre-ci-robot
4d11525f55
[automated] Update Knowhere Commit ( #29904 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-01-12 14:00:50 +08:00
Xu Tong
e429965f32
Add float16 approve for multi-type part ( #28427 )
...
issue：https://github.com/milvus-io/milvus/issues/22837
Add bfloat16 vector, add the index part of float16 vector.
Signed-off-by: Writer-X <1256866856@qq.com>
2024-01-11 15:48:51 +08:00
Jiquan Long
67ab5be15a
enhance: optimize search performance of inverted index ( #29794 )
...
issue: #29793
Use `DocSetCollector` instead of `TopDocsCollector`, which will avoid
scoring and sorting.
---------
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-01-11 11:12:49 +08:00
Jiquan Long
e9f3df3626
fix: inverted index file not found ( #29695 )
...
issue: https://github.com/milvus-io/milvus/issues/29654
---------
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-01-07 20:26:49 +08:00
Jiquan Long
3f46c6d459
feat: support inverted index ( #28783 )
...
issue: https://github.com/milvus-io/milvus/issues/27704
Add inverted index for some data types in Milvus. This index type can
save a lot of memory compared to loading all data into RAM and speed up
the term query and range query.
Supported: `INT8`, `INT16`, `INT32`, `INT64`, `FLOAT`, `DOUBLE`, `BOOL`
and `VARCHAR`.
Not supported: `ARRAY` and `JSON`.
Note:
- The inverted index for `VARCHAR` is not designed to serve full-text
search now. We will treat every row as a whole keyword instead of
tokenizing it into multiple terms.
- The inverted index don't support retrieval well, so if you create
inverted index for field, those operations which depend on the raw data
will fallback to use chunk storage, which will bring some performance
loss. For example, comparisons between two columns and retrieval of
output fields.
The inverted index is very easy to be used.
Taking below collection as an example:
```python
fields = [
FieldSchema(name="pk", dtype=DataType.VARCHAR, is_primary=True, auto_id=False, max_length=100),
FieldSchema(name="int8", dtype=DataType.INT8),
FieldSchema(name="int16", dtype=DataType.INT16),
FieldSchema(name="int32", dtype=DataType.INT32),
FieldSchema(name="int64", dtype=DataType.INT64),
FieldSchema(name="float", dtype=DataType.FLOAT),
FieldSchema(name="double", dtype=DataType.DOUBLE),
FieldSchema(name="bool", dtype=DataType.BOOL),
FieldSchema(name="varchar", dtype=DataType.VARCHAR, max_length=1000),
FieldSchema(name="random", dtype=DataType.DOUBLE),
FieldSchema(name="embeddings", dtype=DataType.FLOAT_VECTOR, dim=dim),
]
schema = CollectionSchema(fields)
collection = Collection("demo", schema)
```
Then we can simply create inverted index for field via:
```python
index_type = "INVERTED"
collection.create_index("int8", {"index_type": index_type})
collection.create_index("int16", {"index_type": index_type})
collection.create_index("int32", {"index_type": index_type})
collection.create_index("int64", {"index_type": index_type})
collection.create_index("float", {"index_type": index_type})
collection.create_index("double", {"index_type": index_type})
collection.create_index("bool", {"index_type": index_type})
collection.create_index("varchar", {"index_type": index_type})
```
Then, term query and range query on the field can be speed up
automatically by the inverted index:
```python
result = collection.query(expr='int64 in [1, 2, 3]', output_fields=["pk"])
result = collection.query(expr='int64 < 5', output_fields=["pk"])
result = collection.query(expr='int64 > 2997', output_fields=["pk"])
result = collection.query(expr='1 < int64 < 5', output_fields=["pk"])
```
---------
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-12-31 19:50:47 +08:00
sre-ci-robot
c2345daf3a
[automated] Update Knowhere Commit ( #29578 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2023-12-29 18:56:46 +08:00
sre-ci-robot
fce1a8dafb
[automated] Update Knowhere Commit ( #29412 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2023-12-25 17:58:46 +08:00
sre-ci-robot
3e66e78508
[automated] Update Knowhere Commit ( #29178 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2023-12-14 17:16:39 +08:00
Bingyi Sun
36f69ea031
feat: integrate storagev2 in building index of segcore ( #28768 )
...
issue: https://github.com/milvus-io/milvus/issues/28655
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2023-12-05 16:48:54 +08:00
sre-ci-robot
f01e507b15
[automated] Update Knowhere Commit ( #28965 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2023-12-05 15:56:35 +08:00
sre-ci-robot
9b6cbe956a
[automated] Update Knowhere Commit ( #28917 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2023-12-04 15:42:34 +08:00
sre-ci-robot
ecc3ca374c
[automated] Update Knowhere Commit ( #28882 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2023-12-01 02:28:31 +08:00
sre-ci-robot
86ccb8e146
[automated] Update Knowhere Commit ( #28704 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2023-11-24 16:56:24 +08:00
sre-ci-robot
b7b31ce0bc
Update knowhere commit ( #28285 )
...
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2023-11-09 14:24:19 +08:00
sre-ci-robot
b1df3ead0e
Update knowhere commit ( #28176 )
...
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2023-11-06 22:44:19 +08:00
sre-ci-robot
7f28e9d2f3
Update knowhere commit ( #28087 )
...
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2023-11-03 10:24:16 +08:00
Enwei Jiao
8ae9c947ae
Use OpenDAL to access object store ( #25642 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-11-01 09:00:14 +08:00
sre-ci-robot
1ae6e5d8c8
Update knowhere commit ( #27993 )
...
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2023-10-28 04:24:10 +08:00
sre-ci-robot
8c605ca858
Update knowhere commit ( #27865 )
...
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2023-10-24 02:34:08 +08:00
sre-ci-robot
b6e07d6fe3
Update knowhere commit ( #27812 )
...
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2023-10-20 03:54:09 +08:00
Gao
9dd369dd99
Update knowhere version to v2.2.2 ( #27810 )
...
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2023-10-19 21:34:07 +08:00
sre-ci-robot
6b79d2b7d6
Update knowhere commit ( #27752 )
...
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2023-10-18 07:14:09 +08:00
sre-ci-robot
75343b2cb4
Update knowhere commit ( #27706 )
...
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2023-10-14 03:03:37 +08:00
Enwei Jiao
0f2f4a0a75
Remove useless parameters for Makefile ( #27622 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-10-11 20:45:35 +08:00
Gao
7a65b6fb85
Limit faiss ivf index build thread num and fix ut ( #27567 )
...
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2023-10-11 10:33:33 +08:00
Sheldon
5ba158a3f5
fix knowhere version-changing ( #27508 )
...
Update KNOWHERE_VERSION for the first occurrence
Signed-off-by: Sheldon <chuanfeng.liu@zilliz.com>
2023-10-08 08:35:32 +08:00
