test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-04 04:49:08 +08:00
Code Issues Actions 14 Packages Projects Releases Wiki Activity
20,856 Commits 96 Branches 105 Tags 185 MiB
4e0ea39235
Commit Graph

20856 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
congqixia
4fd9b0a8e3
enhance: Return segment id hint in QueryStream response (#36487) 
Related to #36482

This PR reuses `SealedSegmentIDsRetrieved` field in `RetrieveResults`
struct to store segment id hint.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-09-26 10:13:14 +08:00
yihao.dai
9e8cafcbe2
enhance: Skip loading bf in datanode (#36367) 
Skip loading bf in datanode:
1. When watching vchannels, skip loading bloom filters for segments.
2. Bypass bloom filter checks for delete messages, directly writing to
L0 segments.
3. Remove flushed segments proactively after flush.

issue: https://github.com/milvus-io/milvus/issues/34585

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-09-26 10:11:15 +08:00
smellthemoon
a8c80abe36
enhance: upgrade pulsar-client-go to 0.11.1 (#36435) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-09-26 10:09:13 +08:00
sre-ci-robot
1970ab4c34
[automated] Update Pytest image changes (#36496) 
Update Pytest image changes
See changes:
58baeee8f1
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-09-26 10:05:13 +08:00
nico
cfd636ed5b
test: add different language tests and modify some cases (#36465) 
fix: #36396

Signed-off-by: nico <cheng.yuan@zilliz.com>
 2024-09-26 09:21:13 +08:00
sre-ci-robot
447e326629
[automated] Update Knowhere Commit (#36527) 
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-09-26 01:15:13 +08:00
wt
14be89df29
test: add case to enable offset cache and mmap at the same time (#36505) 
Signed-off-by: wangting0128 <ting.wang@zilliz.com>
 2024-09-25 19:59:13 +08:00
Buqian Zheng
8495bc6bbc
fix: fix broken Sparse Float Vector raw data mmap (#36183) 
issue: https://github.com/milvus-io/milvus/issues/36182

* improved `Column.h` to make the code much more readable and
maintainable, and added detailed comments.
* fixed an issue where `ArrayColumn::NumRows()` always returns 0 when
the mmap backing storage is a file.
* removed unused `ColumnBase` constructors and unnecessary members so we
don't get confused.
* Updated `test_chunk_cache.cpp` to make the tests parameterized: to
test both mmap enabled and disabled. Added sparse field in the test to
add coverage.
* re-enabled test `Sealed::GetSparseVectorFromChunkCache`. 
* But 2 other disabled tests `Sealed::WarmupChunkCache` and
`Sealed::GetVectorFromChunkCache` remain disabled, there seems to be
errors. @bigsheeper PTAL.

---------

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
 2024-09-25 18:59:13 +08:00
zhuwenxing
58baeee8f1
test: add query with text match filter (#36381) 
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
Co-authored-by: yanliang567 <82361606+yanliang567@users.noreply.github.com>
 2024-09-25 15:17:13 +08:00
zhuwenxing
aee046e52c
test: add some restful negative case to ci (#36479) 
/kind improvement

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2024-09-25 10:19:13 +08:00
wei liu
3cd0b26285
enhance: Enable dynamic update loaded collection's replica (#35822) 
issue: #35821
After collection loaded, if we need to increase/decrease collection's
replica, we need to release and load it again.

milvus offers 4 solution to update loaded collection's replica, this PR
aims to dynamic change the replica number without release, and after
replica number changed, milvus will execute load replica or release
replica in async, and the replica loaded status can be checked by
getReplicas API.

Notice that if set too much replicas than querynode can afford，the new
replica won't be loaded successfully until enough querynode joins.

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-09-25 10:13:18 +08:00
sre-ci-robot
da2d3ce763
[automated] Update Pytest image changes (#36463) 
Update Pytest image changes
See changes:
d55d9d6e1d
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-09-24 19:49:12 +08:00
congqixia
ed95568a05
enhance: Fix PR conflict in reduce unit test (#36470) 
Related to #36433 #36180

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-09-24 18:01:13 +08:00
wei liu
3bd7ec8751
fix: Fix cornor case that segment can't be move out from stopping node (#36431) 
issue: #36426
the old constriant requires only segment on current target can be
balanced, which is wrong, and caused that segment can't be move out from
stopping node, if it's only exist in next target.

by design, stopping balance need to move out all segment on it by
balance task, thus the unfair old constriant should be removed.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-09-24 17:01:14 +08:00
jaime
52cce4de58
fix: iaccurate size estimation for encoded array data (#36373) 
issue: #36029

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-09-24 14:51:14 +08:00
Zhen Ye
350dde666d
fix: streaming node dead lock (#36403) 
issue: #36388

- fix dead lock.
- fix barrier timetick failure.

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-09-24 14:33:13 +08:00
Chun Han
d55d9d6e1d
fix: change pymilvus version for hybridsearch-groupby(#36407) (#36451) 
related: #36407

---------

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2024-09-24 14:29:13 +08:00
congqixia
98a917c5d4
enhance: [skip e2e] Add unittest for reducing duplicated pk from multi segments (#36433) 
Related to #35505 #36362

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-09-24 14:11:13 +08:00
yihao.dai
8cda48a96a
enhance: Use mmap.scalarIndex config for text index (#36400) 
issue: https://github.com/milvus-io/milvus/issues/35273

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-09-24 12:21:13 +08:00
congqixia
b3f2d3db6f
enhance: [GoSDK] support embedded struct in row data (#36443) 
Related to milvus-io/milvus-sdk-go#818

This PR make Row-based insert data parsing embedded struct as flatten
fields instead.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-09-24 11:49:13 +08:00
Chun Han
df7ae08851
fix: iterator cursor progress too fast(#36179) (#36180) 
related: #36179

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2024-09-24 11:45:13 +08:00
zhuwenxing
4779c6cb8f
test: update restful v2 test cases (#36448) 
/kind improvement

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2024-09-24 10:21:12 +08:00
SimFG
ddadefcb62
enhance: get msg type from the msg header to reduce the Unmarshal usage (#36409) 
/kind improvement

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-09-24 10:19:20 +08:00
qixuan
51cdee1f84
test: Add query related test cases for default and null support (#36277) 
issue: #36129

Signed-off-by: qixuan <673771573@qq.com>
 2024-09-23 19:07:13 +08:00
congqixia
1833913f44
enhance: Add streaming forward policy switch for delegator (#36330) 
Related to #35303

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-09-23 18:01:12 +08:00
wt
701f3bf26e
test: add search group by test case for bitmap (#36410) 
Signed-off-by: wangting0128 <ting.wang@zilliz.com>
 2024-09-23 17:29:12 +08:00
smellthemoon
6e880d19a8
enhance: add search params in search request in restful (#36304) 
https://github.com/milvus-io/milvus/issues/36321

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-09-23 16:51:12 +08:00
SimFG
c50fe71163
fix: long buffering causes mq to be unable to receive messages. (#36420) 
- issue: #36397

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-09-23 16:33:18 +08:00
yanliang567
1fb8b46db0
test: Share one collection for group search tests (#36427) 
related issue: #36407
1. add partial load tests
2. use new test class to share one collection for all grouping search
tests

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
 2024-09-23 14:09:11 +08:00
congqixia
3b01b7dc9a
enhance: Produce messages of multiple topics in parallel (#36344) 
Related to #36343

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-09-23 11:23:11 +08:00
yellow-shine
bfd68cc092
enhance: use pull request number as image tag in ci (#36278) 
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
 2024-09-20 16:15:40 +08:00
Chun Han
eb23e23cd2
enhance: refine parameter relationship for hybridsearch_group_by(#35096) (#36289) 
related: #35096

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2024-09-20 14:55:11 +08:00
cai.zhang
4b077e1bd2
fix: Fix the compatibility bug between stats task and segment (#36359) 
issue: #33744

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-09-20 14:33:11 +08:00
congqixia
d2c774fb6d
fix: Return all compactTo segments after support split (#36361) 
Related to #36360

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-09-20 14:11:11 +08:00
wei liu
3b10085f61
enhance: Optimize workload based replica selection policy (#36181) 
issue: #35859

This PR introduce two new param: toleranceFactor and checkRequestNum,
after every checkRequestNum request has been assigned, try to compute
querynode's workload score.

if the diff is less than the toleranceFactor, replica selection policy
will fallback to round_robin, which reduce the average cost to about
500ns.

if the diff is larger than the toleranceFactor, replica selection policy
will compute querynode's score to select the target node with smallest
score in every assigment.

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-09-20 12:33:11 +08:00
Ted Xu
363004fd44
enhance: simplify reduction on single search result (#36334) 
See: #36122

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2024-09-20 11:59:10 +08:00
smellthemoon
89397d1e66
enhance: adjust parquet reader type check with null type (#36266) 
#36252 
remove no need type check. if users use null type writer to write
parquet, hope it successfully.

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-09-19 18:43:10 +08:00
smellthemoon
e84c4f2a92
fix: to fill default value when nullable and default value are both enable (#36030) 
#36003

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-09-19 17:35:11 +08:00
Zhen Ye
f65261215b
fix: streaming node health check panic (#36336) 
issue: #36335

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-09-19 17:11:12 +08:00
aoiasd
139787371e
feat: support embedding bm25 sparse vector and flush bm25 stats log (#36036) 
relate: https://github.com/milvus-io/milvus/issues/35853

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-09-19 10:57:12 +08:00
congqixia
c0317ce672
fix: Wait check node id goroutine in case of data race (#36302) 
Resolves: #36301

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-09-19 10:41:10 +08:00
yanliang567
e013ef1908
test: Add tests for hybrid search group by (#36326) 
related issue: #36295

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
 2024-09-19 09:41:12 +08:00
sre-ci-robot
167e4fb10d
[automated] Update Knowhere Commit (#36352) 
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-09-19 01:01:10 +08:00
Zhen Ye
47da9023a6
fix: add future stateful lock (#36332) 
issue: #36323

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-09-18 20:15:11 +08:00
sre-ci-robot
a03397ba70 Update all contributors 
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
 2024-09-18 12:00:41 +00:00
yellow-shine
9c695283c1
enhance: use new implementation of e2e pipeline (#36347) 
https://github.com/milvus-io/milvus/issues/36348

---------

Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
 2024-09-18 19:57:11 +08:00
binbin
5ca4d5977a
test: Add bulk insert related test cases for default and null support (#36219) 
issue: #36129

Signed-off-by: binbin lv <binbin.lv@zilliz.com>
 2024-09-18 19:33:17 +08:00
Bingyi Sun
23b95aeba3
fix: remove element type check (#35828) 
https://github.com/milvus-io/milvus/issues/36275
Array's element type is not same with schema's. It is INT32 for INT16
and INT8

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2024-09-18 11:37:10 +08:00
wt
526a672bae
test: add more bitmap test cases (#36290) 
Signed-off-by: wangting0128 <ting.wang@zilliz.com>
 2024-09-16 11:03:09 +08:00
zhenshan.cao
dcd904d2fa
fix: keep inner topK to avoid exceeding efSearch (#36284) 
issue: #https://github.com/milvus-io/milvus/issues/36243

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2024-09-16 11:01:25 +08:00