test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-11-30 19:08:30 +08:00
Code Issues Actions 11 Packages Projects Releases Wiki Activity
19,504 Commits 94 Branches 105 Tags 184 MiB
420baacb6d
Commit Graph

19504 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
SimFG
420baacb6d
enhance: use the rate log in the privilege interceptor (#32037) 
/kind improvement

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-04-09 16:19:17 +08:00
Cai Yudong
a0a4ec8b67
enhance: make range search param check message more meaningful (#32006) 
Issue: #31970

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2024-04-09 16:17:26 +08:00
wei liu
177ddda47f
fix: Check stale should check leader task's leader id (#31962) 
issue: #30816

check stale rules for leader task:
1. for reduce leader task, it should keep executing until leader's node
become offline.
2. for grow leader task,it should keep executing until leader's node
become stopping.

This PR check leader node's stopping state for grow leader task

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-04-09 15:33:25 +08:00
aoiasd
5b693c466d
fix: delegator filter out all partition's delete msg when loading segment (#31585) 
May cause deleted data queryable a period of time.
relate: https://github.com/milvus-io/milvus/issues/31484
https://github.com/milvus-io/milvus/issues/31548

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-04-09 15:21:24 +08:00
zhenshan.cao
089c805e0a
enhance:Refactor hybrid search (#32020) 
issue: https://github.com/milvus-io/milvus/issues/25639
https://github.com/milvus-io/milvus/issues/31368

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2024-04-09 14:21:18 +08:00
cqy123456
8fda3cbeda
enhance: check fp16/bf16 nan or inf value (#31840) 
issue:https://github.com/milvus-io/milvus/issues/22837

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2024-04-09 14:19:27 +08:00
cai.zhang
1b767669a4
enhance: Throw error instead of crash when index cannot be built (#31844) 
issue: #27589

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-04-09 11:51:18 +08:00
yiwangdr
1cd15d9322
test: support segment release in integration test (#31190) 
issue: #29507

Notice that api_testonly.go files should be guarded by compiler tag
`test`, so that production build rules don't compile them and these APIs
don't get misused.

Signed-off-by: yiwangdr <yiwangdr@gmail.com>
 2024-04-09 11:39:17 +08:00
congqixia
1f7f3993a1
fix: Validate PlaceholderGroups before combine them (#32016) 
See also #32015

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-09 11:33:17 +08:00
sammy.huang
08bfb431b7
enhance: [skip e2e]add prefix to differentiate (#32025) 
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
 2024-04-09 10:57:24 +08:00
congqixia
a860bf66f9
enhance: Bump google.golang.org/grpc to 1.57.1 (#31985) 
See also #31986
See dependency alert for "gRPC-Go HTTP/2 Rapid Reset vulnerability"

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-09 10:53:17 +08:00
congqixia
0fa8c2efd6
test: Use Strong consistency level verifying count(*) (#32031) 
See also #32030

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-09 10:05:17 +08:00
congqixia
7e0d03420e
enhance: Remove duplicated err check code (#32001) 
The `err != nil` check is duplicated here and shall be removed

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-08 21:59:17 +08:00
yihao.dai
49d109de18
enhance: Use an individual buffer size parameter for imports (#31833) 
Use an individual buffer size parameter for imports and set buffer size
to 64MB.

issue: https://github.com/milvus-io/milvus/issues/28521

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-04-08 21:07:18 +08:00
Ted Xu
9901958288
fix: deadlock in cache concurrent visit (#31938) 
See #31944

There is deadlock in concurrent invocation on `cache.DoWait()`. Suppose
2 callers are calling `DoWait` concurrently, the notification for cache
wait queue may be limited to just 1 caller's occupation instead of them
combined.

To fix this issue, this patch is trying to notify all waiters in queue.

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2024-04-08 18:55:16 +08:00
sammy.huang
a22dffd68d
enhance: [skip e2e] fetch value of GPU_DATE_VERSION from .env (#32004) 
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
 2024-04-08 17:49:23 +08:00
chyezh
73adf2a5cc
fix: use stateful lock to avoid load and release on LocalSegment concurrently (#31606) 
issue: #31605

---------

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-04-08 17:09:16 +08:00
sammy.huang
2aa6f3d3ec
enhance: bump codecod-action to v4 (#31984) 
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
 2024-04-08 16:57:17 +08:00
chyezh
7b400252ff
fix: add configuration disk capacity config for lru and fix some bug (#31977) 
issue: #30361

- Add configurable disk capacity limit

- fix bitset reset logic

- make insert record reinsert after clear

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-04-08 15:55:16 +08:00
sammy.huang
703fcac819
enhance: add openblas in runtime env (#31974) 
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
 2024-04-08 14:57:16 +08:00
congqixia
1c2ae59ece
fix: [skip e2e] Dedup available ports and retry for integration setup (#31902) 
See also #31901

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-08 10:35:17 +08:00
yanliang567
019d76c79d
test: Enable group by tests in e2e (#31903) 
enable groupby tests in e2e

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
 2024-04-08 10:25:16 +08:00
nico
14a9785380
test: update pymilvus version and test cases (#31827) 
Signed-off-by: nico <cheng.yuan@zilliz.com>
 2024-04-07 18:49:16 +08:00
nico
ddf759abb2
enhance: [skip e2e] update chart version in nightly test (#31960) 
Signed-off-by: nico <cheng.yuan@zilliz.com>
 2024-04-07 17:31:23 +08:00
Cai Yudong
00438f408f
enhance: Unify data type check APIs for go (#31887) 
Issue: #22837

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2024-04-07 14:27:22 +08:00
cqy123456
aba4993c6c
fix: fix some fp16/bf16 code miss in segcore. (#31771) 
issue：https://github.com/milvus-io/milvus/issues/22837

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2024-04-07 14:13:16 +08:00
sre-ci-robot
5429c353c5
[automated] Update gpu Builder image changes (#31929) 
Update gpu Builder image changes
See changes:
db698756dc
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-04-07 09:51:21 +08:00
sre-ci-robot
ea624e143d Update all contributors 
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
 2024-04-07 00:01:27 +00:00
yihao.dai
d6cdcf74db
fix: Return err for conc.Future in sync manager (#31790) 
Should not return `err, nil` when using conc.Future, as the error will
be lost/ignored when using `AwaitAll` to wait for the future.

issue: https://github.com/milvus-io/milvus/issues/31788

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-04-06 11:36:57 -07:00
yihao.dai
23ba2a5388
enhance: Remove deprecated DataCoordNumStoredRowsCounter metric (#31778) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-04-07 00:25:15 +08:00
Alexander Guzhva
cae5722229
enhance: performance improvements for the bitset (#31753) 
Issue: #31752

This PR improves the performance for bitset utilities (introduced in PR
#30454), including varchar filtering

Signed-off-by: Alexandr Guzhva <alexanderguzhva@gmail.com>
 2024-04-06 05:19:22 +08:00
sammy.huang
db698756dc
enhance: make gpu builder both support amd64 and gpu64 (#31888) 
the purpose of this pr is to build arm-based and x86-based
milvusdb/milvus-env for milvus gpu. milvusdb/milvus-env would be based
on ubuntu22.04

Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
 2024-04-05 10:09:22 +08:00
zhagnlu
b2669e26dc
fix:reduce thread pool test time (#31893) 
#31877

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-04-05 10:05:12 +08:00
chyezh
a2502bde75
enhance: replica manager enhancement (#31496) 
issue: #30647 

- ReplicaManager manage read only node now, and always do persistent of
node distribution of replica.

- All segment/channel checker using ReplicaManager to get read-only node
or read-write node, but not ResourceManager.

- ReplicaManager promise that only apply unique querynode to one replica
in same collection now (replicas in same collection never hold same
querynode at same time).

- ReplicaManager promise that fairly node count assignment policy if
multi replicas of collection is assigned to one resource group.

- Move some parameters check into ReplicaManager to avoid data race.

- Allow transfer replica to resource group that already load replica of
same collection

- Allow transfer node between resource groups that load replica of same
collection

---------

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-04-05 04:57:16 +08:00
zhagnlu
d6d3b01a04
fix:remove thread pool timeout test because of high load cpu (#31879) 
#31877

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-04-03 15:55:38 +08:00
Cai Yudong
0d1abb081b
enhance: Disable HNSW Binary (#31825) 
Issue: #31494

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2024-04-03 14:53:14 +08:00
Jiquan Long
03e0db109e
fix: udpate Cargo.lock (#31859) 
issue: #31681

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2024-04-03 14:18:23 +08:00
shaoyue
be53e028a8
doc: [skip e2e] add config-docs-generator (#22134) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>

Part of issue: #21856

Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2024-04-03 14:13:13 +08:00
sammy.huang
b11c2da930
enhance: for supporting ARM-based GPU images (#31757) 
This PR is for supporting ARM-based GPU images.
issue: #31862

Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
 2024-04-03 11:49:14 +08:00
congqixia
49b8ee4339
fix: Make FlushTs Sync Policy apply to all buffers (#31839) 
See also #30552

FlushTS policy was orignally designed to flushed/L0 segments only, but
in some edge case, new growing segment buffer would by-pass flush
request and hold a buffer before flush ts, which caused flush timeout

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-03 11:47:13 +08:00
sammy.huang
ae307af19e
enhance: [skip e2e]enable gpu e2e pipeline (#31821) 
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
 2024-04-03 11:41:21 +08:00
congqixia
1af2ee903c
fix: Check parse QueryInfo error before using it (#31836) 
See also milvus-io/milvus-sdk-go#693 #31735

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-03 11:29:14 +08:00
binbin
3b5209c460
test: support enable or disable multiple vector fields in all data type tests (#31201) 
issue: #29799

Signed-off-by: binbin lv <binbin.lv@zilliz.com>
 2024-04-03 11:11:23 +08:00
congqixia
c2aad513c0
fix: Check collection nil before check load status (#31850) 
See also #31849

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-03 10:07:13 +08:00
congqixia
56e371c478
fix: Check replica exists before get latest leader (#31848) 
See also #31847

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-03 10:05:22 +08:00
wei liu
7471a8005f
fix: querycoord panic after node down (#31831) 
issue: #30519

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-04-03 10:03:22 +08:00
congqixia
0feee53631
enhance: Add back unit test for compactor and fix some TODOs (#31829) 
This PR adds back compactor "Unhandled" data type unit test and fixes
some TODOs behvaior

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-02 20:35:14 +08:00
Cai Yudong
246586be27
enhance: Unify data type check APIs under internal/core (#31800) 
Issue: #22837 

Move and rename following C++ APIs:
datatype_sizeof() ==> GetDataTypeSize()
datatype_name() ==> GetDataTypeName()
datatype_is_vector() / IsVectorType() ==> IsVectorDataType()
datatype_is_variable() ==> IsVariableDataType()
datatype_is_sparse_vector() ==> IsSparseFloatVectorDataType()
datatype_is_string() / IsString() ==> IsDataTypeString()
datatype_is_floating() / IsFloat() ==> IsDataTypeFloat()
datatype_is_binary() ==> IsDataTypeBinary()
datatype_is_json() ==> IsDataTypeJson()
datatype_is_array() ==> IsDataTypeArray()
datatype_is_variable() == IsDataTypeVariable()
datatype_is_integer() / IsIntegral() ==> IsDataTypeInteger()

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2024-04-02 19:15:14 +08:00
Chun Han
92c84ccc3f
enhance: ban groupby on hybrid-search(#29968) (#31810) 
related: #29968

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2024-04-02 15:47:18 +08:00
Bingyi Sun
91cb529ba6
fix: get latest collection info when checking index (#31744) 
issue: https://github.com/milvus-io/milvus/issues/31727

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2024-04-02 14:43:13 +08:00