test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-11 09:46:26 +08:00
Code Issues Actions 16 Packages Projects Releases Wiki Activity
20,149 Commits 97 Branches 105 Tags 189 MiB
74adedf750
Commit Graph

20149 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
wei liu
a7ae45c91c
enhance: Add trace for bf cost in l0 compactor (#33898) 
pr: #33860

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-06-20 14:40:15 +08:00
elstic
e0e2ed073b
test: [cherry-pick] change the number of insertion vectors (#34015) 
pr: https://github.com/milvus-io/milvus/pull/34014

Signed-off-by: elstic <hao.wang@zilliz.com>
 2024-06-20 13:58:05 +08:00
congqixia
c28b69384a
fix: [2.4] Check the correct return error in MultiRemove (#33926) (#33966) 
Cherry-pick from master
pr: #33926
See also #33925

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-06-20 11:28:02 +08:00
Jiquan Long
032685239f
enhance: fix copying hits of inverted index twice (#33968) (#33975) 
issue: https://github.com/milvus-io/milvus/issues/29793
pr: https://github.com/milvus-io/milvus/pull/33968
The custom `VecCollector` have already transformed the results into
vector of offsets, no need to copy them twice.

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2024-06-20 10:06:06 +08:00
wei liu
87508c3390
enhance: Avoid to iterate whole segment list for each task's process(#33943) (#33976) 
pr: #33943

when querycoord process segment task, it will try to iterate whole
segment list to checke whether segment is loaded, which cost too much
cpu if there has thousands of segments.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-06-20 10:00:05 +08:00
zhuwenxing
c61740d407
test: [cherry-pick]add test case for bulkwriter (#33914) 
pr: https://github.com/milvus-io/milvus/pull/33879

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2024-06-20 09:22:18 +08:00
presburger
30b70b05d0
fix: fix milvus gpu compile error (#33990) 
NVIDIA RAFT uses customized versions of spdlog and fmt, which conflict
with the versions provided by Conan. Therefore, it is necessary to
isolate the build environment for knowhere to avoid these conflicts.
issue: #33991

Signed-off-by: yusheng.ma <yusheng.ma@zilliz.com>
 2024-06-20 09:20:00 +08:00
nico
27905de192
test: update test cases (#33971) 
Signed-off-by: nico <cheng.yuan@zilliz.com>
 2024-06-19 16:48:01 +08:00
elstic
caa1b0f06e
test: [cherry-pick] add more sparse test cases (#33918) 
issue: https://github.com/milvus-io/milvus/issues/31483
pr: https://github.com/milvus-io/milvus/pull/33916

Signed-off-by: elstic <hao.wang@zilliz.com>
 2024-06-19 15:24:09 +08:00
wei liu
2d3c5191b8
enhance: Refine name rule check error msg (#33815) (#33900) 
pr: #33815

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-06-19 14:26:01 +08:00
congqixia
60695bdb44
enhance: [2.4] Set maxPartitionNum default value to 1024 (#33950) 
Cherry-pick from master
pr: #33949
See also #30059

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-06-18 18:42:00 +08:00
presburger
79546a6c25
fix: fix not find arrow::parquet lib (#33941) 
Upgrade the Arrow version and remove unnecessary library link
dependencies.
issue: #33942
pr: https://github.com/milvus-io/milvus/pull/33928

Signed-off-by: yusheng.ma <yusheng.ma@zilliz.com>
 2024-06-18 13:09:59 +08:00
nico
530756fe42
test: update test cases (#33899) 
pr: #33096

Signed-off-by: nico <cheng.yuan@zilliz.com>
 2024-06-18 11:35:59 +08:00
wei liu
aafa86095f
enhance: Force to reset coord connection for unavailable error (#33910) 
pr: #33908

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-06-18 11:15:58 +08:00
wei liu
4513569207
enhance: add restful api to trigger component stop (#32076) (#33799) 
issue: #32698
pr: #32076
This PR add two rest api for component stop and status check:
1. `/management/stop?role=querynode` can stop the specified component
2. `/management/check/ready?role=rootcoord` can check whether the target
component is serviceable

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-06-17 21:53:59 +08:00
Gao
08c096cf55
enhance: Use primitive type for vectorType (#33911) 
issue: #22837 
pr: #33868 

Use primitive type instead of proto enum type for queryHook to recognize

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2024-06-17 19:05:58 +08:00
congqixia
e8071830fa
fix: [2.4] Prevent use captured iteration variable partitionID (#33907) 
Cherry-pick from master
pr: #33906 
See also #33902

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-06-17 19:02:00 +08:00
Gao
5fc1370f6f
enhance: [2.4] autoindex for multi data type (#33867) 
issue: #22837 
pr: https://github.com/milvus-io/milvus/pull/33868

- opensource autoindex support
- metric type check for different data types
- autoindex data type for search param

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2024-06-14 23:26:00 +08:00
congqixia
59144d8246
enhance: Bump milvus & proto version to v2.4.5 (#33852) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-06-14 22:27:59 +08:00
cqy123456
562369627d
enhance: [cherry-pick]check index with data type in knowhere api (#33878) 
issue: https://github.com/milvus-io/milvus/issues/22837
related: https://github.com/milvus-io/milvus/pull/33880

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2024-06-14 19:45:58 +08:00
jaime
fd1c7b1a1c
enhance: enable flush rate limiter of collection level (#33864) 
pr: #33837

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-06-14 16:27:57 +08:00
zhuwenxing
5b847e93fc
test: [cherry-pick]update import test case to support different dim (#33709) (#33866) 
pr: #33709

add test case for https://github.com/milvus-io/milvus/issues/33681

---------

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
(cherry picked from commit ca1f7ab019)
 2024-06-14 16:23:57 +08:00
zhuwenxing
f06be79a4d
test: [cherry-pick]refine load state check in restful test (#33862) 
pr: https://github.com/milvus-io/milvus/pull/33861

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2024-06-14 16:22:10 +08:00
zhagnlu
b071ebe49b
fix: readd timestamp index because segment timestamp not ordered (#33857) 
pr: #33856

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-06-14 14:34:01 +08:00
chyezh
dd6c982bdb
fix: load operation when segment is on releasing (#33699) 
issue: #30857
pr: #31340

---------

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-06-14 11:47:57 +08:00
XuanYang-cn
dff7ec7bb2
enhance: Donot mark compaction tasks to be timeouted (#33833) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-06-14 10:45:58 +08:00
congqixia
62bd51ec6d
fix: [2.4] Compare config value then swap when caching param value (#33785) (#33797) 
Cherry-pick from master
pr: #33785
See also #33784

This PR change the behavior of `SetCacheValue` of config manager:

- Use mutex and map instead of concurrent map for `configCache`
- Compare config raw value before set cache value

With this implementation, concurrent caching & eviction shall always
have current output:

|time|caching |eviction|config   |cached   |
|----|--------|------- |---------|---------|
|t0  |get     |        |old value|null     |
|t1  |CAS OK  |        |old value|old value|
|t2  |        |update  |new value|old value|
|t3  |        |eviction|new value|null     |

|time|caching |eviction|config   |cached   |
|----|--------|------- |---------|---------|
|t0  |get     |        |old value|null     |
|t1  |        |update  |new value|null     |
|t2  |CAS fail|        |old value|null     |
|t3  |        |eviction|new value|null     |

|time|caching |eviction|config   |cached   |
|----|--------|------- |---------|---------|
|t0  |        |update  |new value|null     |
|t1  |get     |        |new value|null     |
|t2  |CAS OK  |        |new value|new value|
|t3  |        |eviction|new value|null     |

|time|caching |eviction|config   |cached   |
|----|--------|------- |---------|---------|
|t0  |        |update  |new value|null     |
|t1  |get     |        |new value|null     |
|t2  |        |eviction|new value|null     |
|t3  |CAS OK  |        |new value|new value|

|time|caching |eviction|config   |cached   |
|----|--------|------- |---------|---------|
|t0  |        |update  |new value|null     |
|t1  |        |eviction|new value|null     |
|t2  |get     |        |new value|null     |
|t3  |CAS OK  |        |new value|new value|

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-06-13 17:51:57 +08:00
XuanYang-cn
a446e754b4
fix: [2.4]DeleteData merge wrong data casuing data loss (#33821) 
See also: #33819
pr: #33820

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-06-13 16:07:56 +08:00
wei liu
c9fa167a38
enhance: Add rbac support on describe/alter database api (#33804) 
issue: #32707
pr: #33803

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-06-13 14:15:56 +08:00
wei liu
25d8b74f71
enhance: Execute bloom filter apply in parallel to speed up segment predict (#33793) 
issue: #33610
pr: #33792

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-06-13 14:14:04 +08:00
Cai Yudong
ebd0af14f4
enhance: Handle Float16Vector/BFloat16Vector numpy bulk insert as same as BinaryVector (#33760) (#33788) 
pr: #33760
Issue: #22837

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2024-06-13 10:49:57 +08:00
congqixia
86f3433053
enhance: [2.4]Use fastjson lib for unmarshal delete log (#33787) (#33802) 
Cherry-pick from master
pr: #33878
```
goos: linux
goarch: amd64
GOMAXPROC=1
cpu: Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz
BenchmarkJsonSerdeStd             343872              3568 ns/op            1335 B/op         25 allocs/op
BenchmarkJsonSerdeFastjson       5124177               234.9 ns/op            16 B/op          1 allocs/op
```

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-06-13 10:27:57 +08:00
congqixia
efd1fa8b8a
fix: [2.4] Prevent restart timetick sender creating ut datanode (#33790) (#33801) 
Cherry-pick from master
pr: #33790
See also #33789

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-06-13 10:03:57 +08:00
sammy.huang
5a14f65ee4
fix:[skip e2e] make more retries on codecov uploader (#33700) 
pr: #33683

Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
 2024-06-12 20:40:01 +08:00
chyezh
74b4bd1c77
fix: lost DroppedAt field when DropCollection (#33767) 
issue: #33097
pr: #33766

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-06-12 20:19:56 +08:00
Gao
7ef28927e6
fix: [2.4] fix binary vector data size (#33751) 
issue #22837 
pr: #33750

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2024-06-12 20:11:55 +08:00
wei liu
54feef30e7
enhance: Use BatchPkExist to reduce bloom filter func call cost (#33752) 
issue: #33610
pr: #33611

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-06-12 17:45:58 +08:00
edward.zeng
eeba85118e
test: [skip e2e] fix the push order for milvus image (#33762) (#33768) 
/cc @yanliang567

Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2024-06-12 09:55:55 +08:00
Aldrin
ff9bb17cc5
fix: Masked kafka credentials getting printed in logs (#33747) 
Cherry-pick from master
pr: #33730 
issue: https://github.com/milvus-io/milvus/issues/33727

Signed-off-by: Aldrin <imagesai32@gmail.com>
 2024-06-11 15:59:54 +08:00
SimFG
f664b51ebe
enhance: [2.4] try to speed up the loading of small collections (#33746) 
- issue: #33569
- pr: #33570

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-06-11 15:07:55 +08:00
SimFG
c331aa4ad3
enhance: [2.4] add the includeCurrentMsg param for the Seek method (#33743) 
/kind improvement

- issue: #33325
- pr: #33326

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-06-11 15:01:55 +08:00
congqixia
ee22750104
fix: [2.4] Write back dbid modification for nonDB id collection (#33641) (#33694) 
Cherry-pick from master
pr: #33641
See also #33608

Make `fixDefaultDBIDConsistency` also write back collection dbid
modification when nonDB id collection is found.

This fix shall prevent dropped collections of this kind show up again
after dropping and restart.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-06-11 11:25:55 +08:00
yihao.dai
396f8608dd
fix: Fix multiple vector fields import (#33723) (#33724) 
1. Fix dim mismatch with multi-vector fields and JSON import
2. Enhance: do not display file ID in GetImportResponse.

issue: https://github.com/milvus-io/milvus/issues/33681,
https://github.com/milvus-io/milvus/issues/33682

pr: https://github.com/milvus-io/milvus/pull/33723

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-06-10 21:55:55 +08:00
yihao.dai
b71a404776
fix: Check if the import job exists (#33672) (#33673) 
issue: https://github.com/milvus-io/milvus/issues/33671

pr: https://github.com/milvus-io/milvus/pull/33672

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-06-10 21:50:29 +08:00
yihao.dai
ed1dee9e38
enhance: Support L0 import (#33514) (#33712) 
issue: https://github.com/milvus-io/milvus/issues/33157

pr: https://github.com/milvus-io/milvus/pull/33514

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-06-08 11:17:52 +08:00
Buqian Zheng
39e341e83a
fix: [2.4] update check for sparse hnsw index (#33714) 
issue: #29419
pr: #33713

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
 2024-06-07 18:36:07 +08:00
zhagnlu
729001c3a8
fix: clean vector memory (#33692) (#33708) 
pr: #33692

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-06-07 18:17:53 +08:00
chyezh
93348af5c0
fix: async warmup will be blocked by state lock (#33687) 
issue: #33685
pr: #33686

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-06-07 14:23:54 +08:00
XuanYang-cn
26db471968
enhance: [2.4]Add consts of MsgDispatcher to configs (#33680) 
See also: #33676
pr: #33679

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-06-07 14:21:58 +08:00
zhuwenxing
29efd69857
test: refine import test (#33691) 
pr: #33600 #33356
 2024-06-07 09:21:53 +08:00