Commit Graph

57 Commits

Author SHA1 Message Date
cai.zhang
eb47150f66
enhance: [cherry-pick]Disallow the keywords as a field name or dynamic field name (#36108)
issue: #35873

master pr: #36101

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-09-15 15:19:14 +08:00
congqixia
9d0378ae84
enhance: Bump milvus & proto version to v2.4.11 (#36069)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-09-06 23:39:06 +08:00
congqixia
fedf3a7fe1
enhance: Bump milvus & proto version to v2.4.10 (#35798)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-29 14:55:01 +08:00
jaime
2fc111eb10
enhance: set database properties to restrict read access (#35754)
issue: #35744
pr: #35745

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-08-29 11:41:01 +08:00
congqixia
ab261d0f8b
feat: [2.4] Support field partial load collection (#35416) (#35696)
Cherry-pick from master
pr: #35416
Related to #35415

---------

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-27 14:07:00 +08:00
SimFG
5b5119a51f
feat: [2.4] provide more general configuration to control mmap behavior (#35609)
- issue: #35273
- pr: #35359

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-08-23 12:35:02 +08:00
congqixia
4e098f1b23
enhance: Bump milvus & proto to v2.4.9 (#35516)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-16 16:24:54 +08:00
congqixia
e44c63883b
enhance: Bump milvus & proto to v2.4.8 (#35460)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-14 12:52:52 +08:00
wei liu
2ac1bf7532
enhance: Enable setting the replica number and resource group during collection creation (#34403) (#34561)
issue: #30040
pr: #34403

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-06 15:06:17 +08:00
congqixia
eebd9804d6
enhance: Bump milvus & proto version to v2.4.7 (#35034)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-06 12:40:16 +08:00
Gao
be0123863f
enhance: add channel num for queryHook optimization (#35105)
pr: #35104

Signed-off-by: chasingegg <chao.gao@zilliz.com>
2024-07-31 18:23:51 +08:00
congqixia
d16320705e
enhance: [2.4] Add Segment Level in milvus segment info APIs (#34763) (#35023)
Cherry-pick from master
pr: #34763
See also #34746

This PR add segment level field in response of
`GetPersistentSegmentInfo` and `GetQuerySegmentInfo`

---------

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-29 10:11:52 +08:00
Patrick Weizhi Xu
3cf2e1831a
feat: [cherry-pick][2.4] support partition key isolation (#34336) (#34613)
issue: #34332
pr: #34336

---------

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
(cherry picked from commit 104d0966b7)
2024-07-17 14:03:43 +08:00
Gao
f6cd84161c
enhance: ensure autoindex default metric type compatibility (#34479)
issue: #34304 
pr: #34261

Signed-off-by: chasingegg <chao.gao@zilliz.com>
2024-07-08 19:48:14 +08:00
jaime
326370c1be
enhance: add disk quota and max collections into db properties (#34386)
issue: https://github.com/milvus-io/milvus/issues/34385
pr: #34368

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-07-05 18:22:17 +08:00
wayblink
c62bf8a0b0
fix: [Cherry-pick]Pick major compaction fixs and optimizations (#34360)
This PR cherry-picks the following commits:

- fix: sync partitiion stats blocking balance task #33742
- fix: Fix meta prefix overlap bug #33830
- fix: Small fixs of major compaction #33929 
- fix: Fix memory buffer error & some renaming #33850
- fix: sync part stats task cannot be finished #34027 
- Add an option to enable/disable vector field clustering key #34097
- fix: fix error ignore in compactor #34169
- fix:load major compaction partial result #34052
- Use new stream segment reader in clustering compaction #34232

issue: #30633
pr: #33742 #33830 #33929 #33850 #34027 #34097 #34169 #34052 #34232

---------

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
Signed-off-by: wayblink <anyang.wang@zilliz.com>
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: Chun Han <116052805+MrPresent-Han@users.noreply.github.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-07-03 09:53:37 +08:00
cai.zhang
c924b0b502
enhance: [cherry-pick] Refine index code and support analyze data (#34311)
This PR primary picks up the support analyzing functionality, including
the following commits:
- main functionality: https://github.com/milvus-io/milvus/pull/33651
- refine indexnode code: https://github.com/milvus-io/milvus/pull/33458
- related fixes:
  - https://github.com/milvus-io/milvus/pull/33832
  - https://github.com/milvus-io/milvus/pull/33161

issue: #30633 
master prs: #33651 , #33458 , #33832 , #33161

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
Co-authored-by: chasingegg <chao.gao@zilliz.com>
Co-authored-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
2024-07-02 09:50:39 +08:00
wei liu
061a00c58f
enhance: Enable database level replica num and resource groups for loading collection (#33052) (#33981)
pr: #33052

issue: #30040

This PR introduce two database level props:
1. database.replica.number
2. database.resource_groups

User can set those two database props by AlterDatabase API, then can
load collection without specified replica_num and resource groups. then
it will use database level load param when try to load collections.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-06-21 16:56:02 +08:00
Gao
5fc1370f6f
enhance: [2.4] autoindex for multi data type (#33867)
issue: #22837 
pr: https://github.com/milvus-io/milvus/pull/33868

- opensource autoindex support
- metric type check for different data types
- autoindex data type for search param

Signed-off-by: chasingegg <chao.gao@zilliz.com>
2024-06-14 23:26:00 +08:00
congqixia
59144d8246
enhance: Bump milvus & proto version to v2.4.5 (#33852)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-06-14 22:27:59 +08:00
Buqian Zheng
39e341e83a
fix: [2.4] update check for sparse hnsw index (#33714)
issue: #29419
pr: #33713

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-06-07 18:36:07 +08:00
congqixia
8e7f36d928
enhance: Bump milvus version v2.4.4 (#33504)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-31 14:55:44 +08:00
congqixia
ef9c1191ae
enhance: Bump milvus version to v2.4.3 (#33408)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-28 10:11:43 +08:00
congqixia
b2d83d3354
enhance: Bump milvus version to v2.4.2 (#33048)
Bumping version to v2.4.2. Also bump milvus-proto version to v2.4.3.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-15 13:57:35 +08:00
SimFG
4031abd2fa
enhance: change default partition num to 16 when using partition key (#32950)
/kind improvement

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-13 14:19:31 +08:00
congqixia
f07e78ec91
enhance: Bump Milvus & proto version to v2.4.1 (#32693)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-04-29 14:37:26 +08:00
wei liu
4822b109bd
fix: Skip to load l0 segment on old version query node (#32124)
issue: #32107

during rolling upgrade progress, skip to load l0 segment on old version
query node

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-04-15 11:23:23 +08:00
SimFG
c012e6786f
feat: support rate limiter based on db and partition levels (#31070)
issue: https://github.com/milvus-io/milvus/issues/30577
co-author: @jaime0815

---------

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
Signed-off-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
2024-04-12 16:01:19 +08:00
Buqian Zheng
2fdf1a6e76
feat: [Sparse Float Vector] added some integration tests (#31062)
add some integration tests for sparse float vector support

https://github.com/milvus-io/milvus/issues/29419

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-04-10 19:57:18 +08:00
Bingyi Sun
fbff46a005
enhance: add lazyload global config (#31610)
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-03-27 20:23:10 +08:00
Chun Han
c3264ca3e3
feat: support segment pruner (#31003)
related: #30376
2024-03-22 13:57:06 +08:00
congqixia
a647b84f3e
enhance: Add AllPartitionsID const to replace InvalidPartitionID (#31438)
"-1" as `InvalidPartitionID` previously used as All partition place
holder in delete cases. It's confusing and hard to maintain when a const
var has more than one meaning.

This PR add `AllPartitionsID` to replace these usages in delete
scenarios.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-03-20 19:01:05 +08:00
Bingyi Sun
bdc70dfc6a
feat: Add global mmap enable configuration (#31267)
https://github.com/milvus-io/milvus/issues/31279

Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-03-18 15:17:10 +08:00
Bingyi Sun
fd17a5f050
fix: check collection lazy load prop using schema (#30992)
issue: https://github.com/milvus-io/milvus/issues/30361

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-03-06 16:19:01 +08:00
Bingyi Sun
7783098ddd
feat: support lazy load on querycoord (#30372)
https://github.com/milvus-io/milvus/issues/30361

Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-03-01 18:15:29 +08:00
Gao
8a630f733a
enhance: add new optimize param for queryhook (#29495)
add a flag to indicate if we use search param optimizations, default is
on

---------

Signed-off-by: chasingegg <chao.gao@zilliz.com>
2023-12-28 10:04:46 +08:00
yah01
dc89730a50
Support collection-level mmap control (#26901)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-02 23:52:16 +08:00
congqixia
ee184a709e
Bump Milvus Version & milvus-proto/go-api to v2.3.2 (#27893)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-10-24 21:02:11 +08:00
yah01
15746b5b5f
Refine KV errors (#27588)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-13 17:15:34 +08:00
SimFG
26f06dd732
Format the code (#27275)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-21 09:45:27 +08:00
cai.zhang
a362bb1457
Support array datatype (#26369)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-09-19 14:23:23 +08:00
yah01
3349db4aa7
Refine errors to remove changes breaking design (#26521)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-04 09:57:09 +08:00
congqixia
145387fdcb
Bump proto go-api to v2.3.0 (#26561)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-08-23 20:18:23 +08:00
smellthemoon
d63323d117
Add rate limit and deny write in upsert (#25351)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-07-11 11:20:34 +08:00
Gao
de6e4817a2
Support dynamic tuning config (#25152)
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2023-07-03 15:18:24 +08:00
wei liu
0d51f55c48
enable config each collection has different rate limit (#24139)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-06-21 15:02:42 +08:00
PowderLi
3f4356df10
fix the spelling of field (#25008)
Signed-off-by: PowderLi <min.li@zilliz.com>
2023-06-21 14:00:42 +08:00
congqixia
41af0a98fa
Use go-api/v2 for milvus-proto (#24770)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-09 01:28:37 +08:00
Gao
ea62922829
Disable search params rewrite while searching with filter (#24625)
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2023-06-06 14:28:35 +08:00
xige-16
732fe54775
Support partition Key (#24047)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-06-06 10:24:34 +08:00