Commit Graph

20769 Commits

Author SHA1 Message Date
smellthemoon
e84c4f2a92
fix: to fill default value when nullable and default value are both enable (#36030)
#36003

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-09-19 17:35:11 +08:00
Zhen Ye
f65261215b
fix: streaming node health check panic (#36336)
issue: #36335

Signed-off-by: chyezh <chyezh@outlook.com>
2024-09-19 17:11:12 +08:00
aoiasd
139787371e
feat: support embedding bm25 sparse vector and flush bm25 stats log (#36036)
relate: https://github.com/milvus-io/milvus/issues/35853

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-09-19 10:57:12 +08:00
congqixia
c0317ce672
fix: Wait check node id goroutine in case of data race (#36302)
Resolves: #36301

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-09-19 10:41:10 +08:00
yanliang567
e013ef1908
test: Add tests for hybrid search group by (#36326)
related issue: #36295

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-09-19 09:41:12 +08:00
sre-ci-robot
167e4fb10d
[automated] Update Knowhere Commit (#36352)
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-09-19 01:01:10 +08:00
Zhen Ye
47da9023a6
fix: add future stateful lock (#36332)
issue: #36323

Signed-off-by: chyezh <chyezh@outlook.com>
2024-09-18 20:15:11 +08:00
sre-ci-robot
a03397ba70 Update all contributors
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
2024-09-18 12:00:41 +00:00
yellow-shine
9c695283c1
enhance: use new implementation of e2e pipeline (#36347)
https://github.com/milvus-io/milvus/issues/36348

---------

Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-09-18 19:57:11 +08:00
binbin
5ca4d5977a
test: Add bulk insert related test cases for default and null support (#36219)
issue: #36129

Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2024-09-18 19:33:17 +08:00
Bingyi Sun
23b95aeba3
fix: remove element type check (#35828)
https://github.com/milvus-io/milvus/issues/36275
Array's element type is not same with schema's. It is INT32 for INT16
and INT8

Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-09-18 11:37:10 +08:00
wt
526a672bae
test: add more bitmap test cases (#36290)
Signed-off-by: wangting0128 <ting.wang@zilliz.com>
2024-09-16 11:03:09 +08:00
zhenshan.cao
dcd904d2fa
fix: keep inner topK to avoid exceeding efSearch (#36284)
issue: #https://github.com/milvus-io/milvus/issues/36243

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2024-09-16 11:01:25 +08:00
yihao.dai
763fd0dfc5
enhance: Use a separate mmap config for chunk cache (#36276)
issue: https://github.com/milvus-io/milvus/issues/35273

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-09-15 16:23:09 +08:00
cai.zhang
517f8b3755
enhance: Refine the code for returning error (#36103)
issue: #36023

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-09-15 15:23:14 +08:00
cai.zhang
e486442ca6
enhance: Disallow the keywords as a field name or dynamic field name (#36101)
issue: #35873

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-09-15 15:19:14 +08:00
yihao.dai
a61668c77e
feat: Introduce stats task for import (#35868)
This PR introduce stats task for import:
1. Define new `Stats` and `IndexBuilding` states for importJob
2. Add new stats step to the import process: trigger the stats task and
wait for its completion
3. Abort stats task if import job failed

issue: https://github.com/milvus-io/milvus/issues/33744

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-09-15 15:17:08 +08:00
sre-ci-robot
5ce99e2535
[automated] Update Pytest image changes (#36250)
Update Pytest image changes
See changes:
2e434b2358
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-09-14 17:59:09 +08:00
nico
e3e88ffbe6
test: update pymilvus version and skip some cases (#36281)
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-09-14 17:57:14 +08:00
jaime
2ff3765058
enhance: catch std::stoi exception and improve error msg (#36267)
issue: #36255

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-09-14 16:17:08 +08:00
congqixia
3352030a84
enhance: Graceful stop flowgraph manager when stopping datanode (#36229)
Flowgraph manager is not stopped durong datanode stopping procedure
which may lead to unexpect flowgraph behavior during/after datanode stop
progress.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-09-14 15:55:08 +08:00
Chun Han
b8b4aea4f5
enhance: restrict max group size(#33544) (#36223)
related: #33544

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-09-14 15:45:08 +08:00
smellthemoon
fc1bdd4c84
fix: to forbid bulk insert with nullable field in numpy files (#36246)
#36241

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-09-14 15:35:07 +08:00
wei liu
329fb421cd
fix: fix search/query/count may access same growing and sealed segment (#36258)
issue: #36257
during syncTargetVersion, sealed segment should be excluded, to avoid
it's growing segment be conusmed from stream again.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-09-14 14:21:07 +08:00
congqixia
d07338b4b9
enhance: Ignore index check for non-loaded vector field (#36170)
Related to #35996

For `Field Partial Load` feature, Milvus shall ignore index check for
non-loaded vector field.

Also, this PR unifies the logic of index check for load collection and
load partitions tasks.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-09-14 11:01:08 +08:00
smellthemoon
2d05b7f219
fix: to add check in partition key field when set nullable==true (#36218)
#36213

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-09-14 10:53:08 +08:00
congqixia
3bc7d63be9
fix: overwrite correct selection when pk duplicated (#35826)
Related to #35505

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-09-14 10:27:08 +08:00
yellow-shine
ef8289ea9b
enhance: git clone with retry mechanism (#36256)
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-09-14 09:44:46 +08:00
zhenshan.cao
9d8d332c88
fix: Fix improper use of offset in HybridSearch (#36244)
issue :https://github.com/milvus-io/milvus/issues/36243

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2024-09-13 22:05:15 +08:00
zhagnlu
489087d18b
enhance: refactor executor framework V2 (#35251)
#32636

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-09-13 20:57:09 +08:00
jaime
22cce44afc
fix: metrics stored_index_files_size is never cleared (#36160)
issue: #36159

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-09-13 20:09:15 +08:00
congqixia
58d3200986
enhance: Filter out non-hit delete records during load delta (#36207)
Related to #35303

This PR utilizes pk index in segment to exclude non-hit delete record
during load delete records. This ability is crucial when l0/delete
forward policy only replies on segment itself(without BF filtering).

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-09-13 19:05:08 +08:00
wei liu
f7d950d465
fix: [skip e2e] Fix unstable ut TestCollectionObserver (#36231)
issue: #36237

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-09-13 19:01:09 +08:00
wei liu
fb2a41a94c
fix: Clean dirty segment/channel on querynode (#36202)
issue: #36201
after querynode has been remove from replica, all dirty segment/channel
on it should be released.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-09-13 18:15:08 +08:00
Zhen Ye
b4c1603a57
fix: skip recover flusher on streaming node when channel is dropped (#36149)
issue: #36016,#36230

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2024-09-13 17:19:08 +08:00
wei liu
bd658a6510
enhance: Enable dynamic update replica selection policy (#35860)
issue: #35859

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-09-13 17:05:15 +08:00
Zhen Ye
c03eb6f664
fix: streaming node consume blocks if recv message is too large (#36151)
issue: #36081

Signed-off-by: chyezh <chyezh@outlook.com>
2024-09-13 16:41:08 +08:00
yanliang567
2e434b2358
test: Add group size tests (#36240)
related issue: #36146

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-09-13 16:13:08 +08:00
aoiasd
c22a2cebb6
fix: split stream query result to avoid grpc response too large error (#36090)
relate: https://github.com/milvus-io/milvus/issues/36089

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-09-13 15:07:09 +08:00
Jiquan Long
f0f2fb4cf0
enhance: span tracing of c++ part (#36205)
fix: https://github.com/milvus-io/milvus/issues/36204

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-09-13 11:19:09 +08:00
wei liu
06a706e5f0
fix: fix ReadWrite privilege group deny all global API (#36144)
issue: #35471

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-09-13 10:33:08 +08:00
congqixia
c6eb6c7cb2
enhance: Add error handler for write buffer (#36216)
Related to #36215

This PR add error handler setting option providing the possibility to
change error handling behavior other than panicking.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-09-13 10:11:09 +08:00
Ted Xu
b9c037f558
feat: adding cache to expression parse (#36185)
See #36122

This PR improves the proxy node performance by adding cache to
expression parse.

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-09-13 10:03:09 +08:00
sre-ci-robot
1833f2d516 Update all contributors
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
2024-09-13 00:01:28 +00:00
cai.zhang
8395c8a8db
enhance: Update stats task to optional (#35947)
issue: #33744

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-09-12 20:37:08 +08:00
XuanYang-cn
e8840a1b41
enhance: Add metrics for Delete entries num of L0seg (#36175)
- Add metrics *DataCoordL0DeleteEntriesNum*
- Remove metrics *DataCoordRateStoredL0Segment*

See also: #36147

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-09-12 18:07:08 +08:00
binbin
b5ff348906
test: Add part of test cases for default and null support (#36186)
issue : #36129

Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2024-09-12 17:29:13 +08:00
zhagnlu
5e5e87cc2f
enhance: rename some params and reduce default bitmapCardinalityLimit… (#36138)
#32900

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-09-12 12:09:08 +08:00
congqixia
11dbe1e755
enhance: Add L0 forward policy to support remote load (#36189)
Related to #35303

This PR add a param item to support change l0 forward behavior from bf
filtering and forward to remote load.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-09-12 12:01:08 +08:00
aoiasd
da227ff9a1
feat: Support create collection with functions (#35973)
relate: https://github.com/milvus-io/milvus/issues/35853
Support create collection with functions. Prepare for support bm25
function.

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-09-12 10:43:06 +08:00