Commit Graph

51 Commits

Author SHA1 Message Date
Xiaofan
36f1ea93a5
enhance: optimize plan parser pool to avoid unnessary recycle (#32869)
fix #32868
plan parser takes too much cpu on high qps,this pr try to avoid create
lexer and parser too freequent

Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
2024-05-11 10:51:31 +08:00
Cai Yudong
00438f408f
enhance: Unify data type check APIs for go (#31887)
Issue: #22837

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
2024-04-07 14:27:22 +08:00
Chun Han
c3264ca3e3
feat: support segment pruner (#31003)
related: #30376
2024-03-22 13:57:06 +08:00
Buqian Zheng
3c80083f51
feat: [Sparse Float Vector] add sparse vector support to milvus components (#30630)
add sparse float vector support to different milvus components,
including proxy, data node to receive and write sparse float vectors to
binlog, query node to handle search requests, index node to build index
for sparse float column, etc.

https://github.com/milvus-io/milvus/issues/29419

---------

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-03-13 14:32:54 -07:00
cai.zhang
de2c95d00c
enhance: Constraint dynamic field as key-value format (#31183)
issue: #31051

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-03-12 12:45:03 +08:00
cai.zhang
6a83f16871
feat: Support for multiple forms of JSON (#31052)
issue: #31051

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-03-11 19:55:02 +08:00
congqixia
9b3005f1be
enhance: Avoid create schema helper for each read task (#30981)
See also #30806

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-03-04 19:39:00 +08:00
cai.zhang
1aa97a5c21
enhance: Support more relational operators for binary expressions (#30902)
issue: #30677

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-03-01 16:57:00 +08:00
Jiquan Long
dcfc35316f
fix: underscore matching not work (#30826)
issue: https://github.com/milvus-io/milvus/issues/30721

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-02-29 10:41:00 +08:00
aoiasd
c863b82476
enhance: Return parse expression failed error with reason (#30548)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-02-20 16:02:52 +08:00
Jiquan Long
e549148a19
enhance: full-support for wildcard pattern matching (#30288)
issue: #29988 
This pr adds full-support for wildcard pattern matching from end to end.
Before this pr, the users can only use prefix match in their expression,
for example, "like 'prefix%'". With this pr, more flexible syntax can be
combined.

To do so, this pr makes these changes:
- 1. support regex query both on index and raw data;
- 2. translate the pattern matching to regex query, so that it can be
handled by the regex query logic;
- 3. loose the limit of the expression parsing, which allows general
pattern matching syntax;

With the support of regex query in segcore backend, we can also add
mysql-like `REGEXP` syntax later easily.

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-02-01 12:37:04 +08:00
Xu Tong
e429965f32
Add float16 approve for multi-type part (#28427)
issue:https://github.com/milvus-io/milvus/issues/22837

Add bfloat16 vector, add the index part of float16 vector.

Signed-off-by: Writer-X <1256866856@qq.com>
2024-01-11 15:48:51 +08:00
cai.zhang
31f442915b
fix: Fix bug for parsing expression that include quotes (#28416)
issue: #28365 
Fix bug for parsing error when a string enclosed in single quotes in an
expression contains multiple double quotes.
such as:
```
expr = "tag == '\"blue\"'"
```

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2023-11-23 17:18:32 +08:00
Bingyi Sun
d7145e2c06
enhance: Update golangci_lint version (#28535)
Update golangci lint and fix some warnings

Signed-off-by: sunby <sunbingyi1992@gmail.com>
2023-11-21 10:04:21 +08:00
yihao.dai
f9c630247d
Construct plan directly when search with vector output (#27928)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-10-26 19:30:10 +08:00
congqixia
5d558623fe
Add revive sub-lints and fix existing problems (#27495)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-10-07 20:53:38 +08:00
SimFG
26f06dd732
Format the code (#27275)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-21 09:45:27 +08:00
cai.zhang
a362bb1457
Support array datatype (#26369)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-09-19 14:23:23 +08:00
congqixia
cc9974979f
Add staticcheck linter and fix existing problems (#27174)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-19 10:05:22 +08:00
Xu Tong
9166011c4a
Add float16 vector (#25852)
Signed-off-by: Writer-X <1256866856@qq.com>
2023-09-08 10:03:16 +08:00
cai.zhang
82b0e39abe
Fix bug for concurrent parsing expr with strings (#26721)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-08-31 17:39:02 +08:00
Jiquan Long
36e29ef859
Auto generate parser code (#26716)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-08-31 11:13:01 +08:00
Jiquan Long
d9aa8ff545
Fix logical expression (#26513)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-08-22 15:44:22 +08:00
Jiquan Long
2a517d2da1
Fix expression incompatible between parser and executor (#26493)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-08-21 19:36:20 +08:00
cai.zhang
94846995bf
Refine error message for field not exist (#26330)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-08-21 11:52:20 +08:00
cai.zhang
90b12d04b4
Don't panic when receive unsupported expr (#26414)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-08-17 15:12:17 +08:00
cai.zhang
a0198ce8ae
Support json contains feature (#25384)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-08-11 17:09:30 +08:00
Jiquan Long
6578a27fe7
Support empty expression (#26153)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-08-10 08:37:15 +08:00
cai.zhang
3d78a452d7
Support escape string (#24848)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-07-25 10:29:01 +08:00
cai.zhang
f12574aaf3
Support JSON_CONTAINS or json_contains (#24814)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-06-13 10:50:37 +08:00
cai.zhang
853b7d0054
Support json contains for json field (#24720)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-06-09 16:10:36 +08:00
congqixia
41af0a98fa
Use go-api/v2 for milvus-proto (#24770)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-09 01:28:37 +08:00
xige-16
732fe54775
Support partition Key (#24047)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-06-06 10:24:34 +08:00
cai.zhang
dc02c5b064
Fix bug for supporting single quotes (#24464)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-05-29 09:55:27 +08:00
cai.zhang
6804bda8f5
Support single quotes with string (#24386)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-05-25 15:47:26 +08:00
cai.zhang
d16e18fd34
Add output fields for search/query results (#24302)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-05-23 10:19:26 +08:00
cai.zhang
008285f849
Support dynamic schema for create collection (#24176)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-05-18 09:33:24 +08:00
cai.zhang
ccd685013a
Json key must be enclosed in double quotes (#24113)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-05-16 16:19:22 +08:00
cai.zhang
9a0a41d2a8
Support access json key without quotation marks (#24062)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-05-14 22:51:21 +08:00
cai.zhang
9715a850fa
Support expr with json field (#23804)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-05-10 10:19:19 +08:00
jaime
c9d0c157ec
Move some modules from internal to public package (#22572)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-04-06 19:14:32 +08:00
Jiquan Long
8139106b51
Feat: count entities by expression (#22765)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-03-16 19:31:55 +08:00
congqixia
732986aa04
Remove fmt.Print from internal package (#22722)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-03-14 17:36:05 +08:00
SimFG
a55f739608
Separate public proto files (#19782)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-10-16 20:49:27 +08:00
SimFG
d7f38a803d
Separate some proto files (#19218)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-09-16 16:56:49 +08:00
Jiquan Long
b4f797506d
Fix err msg of invalid expression (#17296)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-06-01 10:00:07 +08:00
zhenshan.cao
08a4dff41d
Add logic for search merging and a simple task scheduler for read tasks (#17022)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Co-authored-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: bigsheeper <yihao.dai@zilliz.com>

Co-authored-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: bigsheeper <yihao.dai@zilliz.com>
2022-05-23 16:41:58 +08:00
Jiquan Long
e756fdccd4
Fix pool of lexer and parser (#17129)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-05-20 18:47:57 +08:00
Jiquan Long
276ba69819
Replace plan parser with antlr (#17090)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-05-19 17:15:57 +08:00
Jiquan Long
ac389f3e93
Limit and translate pattern match operations (#16818)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-05-07 16:31:52 +08:00