Commit Graph

3070 Commits

Author SHA1 Message Date
cai.zhang
ae227e3934
enhance: Add integration test for stats task (#37506)
issue: #33744

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-11-08 15:48:26 +08:00
congqixia
11f4fe0177
enhance: [GoSDK] move client pkg go files to sub one (#37492)
Related to #31293

Client source files under client pkg cannot be evaluate correctly by
codecov. This PR moves them to `milvusclient` sub-package to fix this
issue and follow go major version best practice.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-08 07:32:26 +08:00
smellthemoon
86fd3200be
enhance: refactor createIndex in RESTful API (#37235)
Make the parameter input method consistent with miluvs-client.

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-11-07 14:18:30 +08:00
Buqian Zheng
40b770cb7b
fix: add rescorer activation function for BM25 (#37481)
issue: #35853

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-11-07 12:08:25 +08:00
cai.zhang
aed3b94b5d
enhance: Refine error message for contains array (#37383)
issue: #36221

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-11-07 10:44:25 +08:00
wei liu
7cfd609ebc
fix: [skip e2e]unstable integration test TestNodeDownOnSingleReplica (#37480)
issue: #37289
cause pr #37116 introduce retry on get shard leader, which make search
won't fail during query node down.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-11-06 17:08:21 -08:00
nico
cf66fbee8a
test: update test cases (#37215)
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-11-06 17:50:25 +08:00
aoiasd
d67853fa89
feat: Tokenizer support build with params and clone for concurrency (#37048)
relate: https://github.com/milvus-io/milvus/issues/35853
https://github.com/milvus-io/milvus/issues/36751

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-11-06 17:48:24 +08:00
congqixia
c83b93946e
enhance: [GoSDK] Add alter collection API & expose options (#37365)
Related to #31293

This PR:

- Add `AlterCollection` API for collection property modification
- Expose hidden or missing option methods

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-06 14:36:25 +08:00
zhuwenxing
0fc6c634b0
test: fix tokenizer and monkey patch faker function (#37119)
/kind improvement

---------

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
Signed-off-by: zhuwenxing <wxzhuyeah@gmail.com>
2024-11-05 08:42:23 +08:00
cai.zhang
ba9f36ba33
enhance: Increase the concurrency of index tasks in CI (#37335)
#37096

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-11-04 12:04:23 +08:00
foxspy
eaf86f7649
fix: optimize invalid datatype error msg (#37376)
issue: #37151

Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2024-11-04 11:46:28 +08:00
cai.zhang
50de122dc7
enhance: Rename textmatch to text_match (#37290)
issue: #36672

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-11-03 18:40:27 +08:00
congqixia
be71b98146
enhance: Fix case error msg after knowhere upgrade (#37339)
Previous PR: #37315

Error message updated after knowhere behavior change, this PR update
corresponding test error message.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-01 14:00:22 +08:00
wei liu
f190e5d802
fix: [skip e2e] TestNodeDownOnSingleReplica has unstable result (#37288)
issue: #37289
those test case use search to verify replica's status, but if the search
gap is 1s, the node down's effect may be fixed up by balance.

This PR remove the 1 second gap between search operation.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-11-01 10:50:21 +08:00
zhuwenxing
247f75180f
test: add restful cases for full text search and some minor fix (#37148)
/kind improvement

---------

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-10-31 21:18:23 +08:00
zhuwenxing
d24970c090
test: fix potential unstable cases of full-text search (#37328)
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-10-31 21:12:21 +08:00
zhenshan.cao
63843dce33
fix: Fix conan gdal building problem (#37338)
issue:https://github.com/milvus-io/milvus/issues/27576

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2024-10-31 21:04:16 +08:00
Hao Tan
67c4340565
feat: Geospatial Data Type and GIS Function Support for milvus server (#35990)
issue:https://github.com/milvus-io/milvus/issues/27576

# Main Goals
1. Create and describe collections with geospatial fields, enabling both
client and server to recognize and process geo fields.
2. Insert geospatial data as payload values in the insert binlog, and
print the values for verification.
3. Load segments containing geospatial data into memory.
4. Ensure query outputs can display geospatial data.
5. Support filtering on GIS functions for geospatial columns.

# Solution
1. **Add Type**: Modify the Milvus core by adding a Geospatial type in
both the C++ and Go code layers, defining the Geospatial data structure
and the corresponding interfaces.
2. **Dependency Libraries**: Introduce necessary geospatial data
processing libraries. In the C++ source code, use Conan package
management to include the GDAL library. In the Go source code, add the
go-geom library to the go.mod file.
3. **Protocol Interface**: Revise the Milvus protocol to provide
mechanisms for Geospatial message serialization and deserialization.
4. **Data Pipeline**: Facilitate interaction between the client and
proxy using the WKT format for geospatial data. The proxy will convert
all data into WKB format for downstream processing, providing column
data interfaces, segment encapsulation, segment loading, payload
writing, and cache block management.
5. **Query Operators**: Implement simple display and support for filter
queries. Initially, focus on filtering based on spatial relationships
for a single column of geospatial literal values, providing parsing and
execution for query expressions.
6. **Client Modification**: Enable the client to handle user input for
geospatial data and facilitate end-to-end testing.Check the modification
in pymilvus.

---------

Signed-off-by: tasty-gumi <1021989072@qq.com>
2024-10-31 20:58:20 +08:00
cai.zhang
2ef6cbbf59
feat: The expression supports filling elements through templates (#37033)
issue: #36672

The expression supports filling elements through templates, which helps
to reduce the overhead of parsing the elements.

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-10-31 14:20:22 +08:00
zhuwenxing
6e37372619
test: update checker (#37275)
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-10-31 09:50:20 +08:00
yanliang567
3a3404658e
test: Add a test for iterator enhancement (#37296)
related issue: https://github.com/milvus-io/milvus/issues/37084

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-10-30 16:54:20 +08:00
Zhen Ye
889434691c
enhance: enable asan for e2e (#37149)
issue: #35854

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2024-10-29 14:14:24 +08:00
zhuwenxing
4c108b1564
test: update jieba tokenizer in test (#37199)
/kind improvement

---------

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-10-28 19:22:22 +08:00
zhuwenxing
cdee149191
test: fix testcases for verification after chaos (#37153)
/kind improvement

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-10-28 10:33:29 +08:00
zhuwenxing
c8dd665bf6
test: supplementing case for text match (#36693)
/kind improvement

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-10-28 10:31:40 +08:00
foxspy
d7b2ffe5aa
enhance: add an unify vector index config checker (#36844)
issue: #34298

Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2024-10-28 10:11:37 +08:00
yihao.dai
ed37c27bda
fix: Fix collection leak in querynode (#37061)
Unref the removed L0 segment count.

issue: https://github.com/milvus-io/milvus/issues/36918

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-10-25 19:59:29 +08:00
Yinzuo Jiang
3628593d20
feat: Implement custom function module in milvus expr (#36560)
OSPP 2024 project:
https://summer-ospp.ac.cn/org/prodetail/247410235?list=org&navpage=org

Solutions:

- parser (planparserv2)
    - add CallExpr in planparserv2/Plan.g4
    - update parser_visitor and show_visitor
- grpc protobuf
    - add CallExpr in plan.proto
- execution (`core/src/exec`)
- add `CallExpr` `ValueExpr` and `ColumnExpr` (both logical and
physical) for function call and function parameters
- function factory (`core/src/exec/expression/function`)
    - create a global hashmap when starting milvus (see server.go)
- the global hashmap stores function signatures and their function
pointers, the CallExpr in execution engine can get the function pointer
by function signature.
- custom functions
    - empty(string)
    - starts_with(string, string)
- add cpp/go unittests and E2E tests

closes: #36559

Signed-off-by: Yinzuo Jiang <jiangyinzuo@foxmail.com>
2024-10-25 15:25:30 +08:00
Buqian Zheng
088d5d7d76
fix: optimize BM25 err message (#37074)
issue: https://github.com/milvus-io/milvus/issues/37022

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-10-25 14:35:45 +08:00
zhuwenxing
ac2858d418
test: add full text search checker in test (#37122)
/kind improvement

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-10-25 14:09:29 +08:00
qixuan
80aa9ab4d6
test: Add insert and upsert related cases for null and default value support (#37088)
issue: #36129

---------

Signed-off-by: qixuan <673771573@qq.com>
2024-10-25 11:03:29 +08:00
binbin
c285853de8
test: Add test cases about expr for null and default support (#37121)
issue: #36129

Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2024-10-25 11:01:30 +08:00
smellthemoon
2b3f5bec07
fix: panic when create index on all none data (#37046)
#37045

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-10-24 17:09:28 +08:00
sre-ci-robot
53836f320a
[automated] Update Pytest image changes (#37067)
Update Pytest image changes
See changes:
3b024f9b36
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-10-24 16:53:28 +08:00
yanliang567
0d376f1a8f
test: Add query iterator test and seperate a new test file (#37085)
related issue: #37084

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-10-24 11:47:29 +08:00
jaime
4746f47282
feat: management WebUI homepage (#36822)
issue: #36784
1. Implement an embedded web server for WebUI access.  
2. Complete the homepage development.

Home page demo:
<img width="2177" alt="iShot_2024-10-10_17 57 34"
src="https://github.com/user-attachments/assets/38539917-ce09-4e54-a5b5-7f4f7eaac353">

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-10-23 11:29:28 +08:00
zhuwenxing
3b024f9b36
test: Add full-text search test cases (#36998)
/kind improvement

---------

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-10-23 09:51:27 +08:00
zhuwenxing
80d48f1e53
test: add text match checker in test (#37052)
/kind improvement

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-10-23 09:49:27 +08:00
qixuan
36147b10f5
test: Add insert and upsert related cases for null and default value support (#36932)
issue: #36129

Signed-off-by: qixuan <673771573@qq.com>
2024-10-22 10:13:27 +08:00
zhuwenxing
ef923958bb
test: add restful param check testcases (#36931)
/kind improvement

---------

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-10-19 19:13:25 +08:00
yanliang567
d17135d3bc
test: Add alias test with rename collection (#36978)
related issue: #36963 
1. add alias tests with rename collection for issue #36963
2. update some test levels
3. fix some xfail tests

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-10-18 16:27:24 +08:00
smellthemoon
eb3e4583ec
enhance: all op(Null) is false in expr (#35527)
#31728

---------

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-10-17 21:14:30 +08:00
wt
97ff012c67
test: check query response data (#36952)
Signed-off-by: wangting0128 <ting.wang@zilliz.com>
2024-10-17 19:13:25 +08:00
congqixia
e5948bd039
enhance: [GoSDK] Use variadic params for options (#36912)
Use variadic parameter function for options make client options easier
to use.

Related to #31293

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-10-16 19:33:23 +08:00
smellthemoon
44d80c1355
fix: not return err if consistencyLevel is not set to a valid value (#36714)
https://github.com/milvus-io/milvus/issues/36444

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-10-16 13:23:22 +08:00
nico
2291d0c9f5
test: update test cases (#36878)
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-10-15 19:13:24 +08:00
yanliang567
0ec92513a5
test: Update Diskann supporting group by (#36853)
related issue: #36813 

milvus diskann does not support group_search before, it supports now.

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-10-15 10:35:23 +08:00
ThreadDao
d566b0ceff
test: add stats task feature cases and DefaultVectorSearchParams (#36768)
issue: #36767

---------

Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2024-10-14 19:15:23 +08:00
nico
937ebec2ce
test: update test cases (#36841)
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-10-14 19:13:23 +08:00