test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-02 20:09:57 +08:00
Code Issues Actions 12 Packages Projects Releases Wiki Activity
19,559 Commits 94 Branches 105 Tags 184 MiB
14977b84d1
Commit Graph

19559 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
wei liu
7471a8005f
fix: querycoord panic after node down (#31831) 
issue: #30519

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-04-03 10:03:22 +08:00
congqixia
0feee53631
enhance: Add back unit test for compactor and fix some TODOs (#31829) 
This PR adds back compactor "Unhandled" data type unit test and fixes
some TODOs behvaior

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-02 20:35:14 +08:00
Cai Yudong
246586be27
enhance: Unify data type check APIs under internal/core (#31800) 
Issue: #22837 

Move and rename following C++ APIs:
datatype_sizeof() ==> GetDataTypeSize()
datatype_name() ==> GetDataTypeName()
datatype_is_vector() / IsVectorType() ==> IsVectorDataType()
datatype_is_variable() ==> IsVariableDataType()
datatype_is_sparse_vector() ==> IsSparseFloatVectorDataType()
datatype_is_string() / IsString() ==> IsDataTypeString()
datatype_is_floating() / IsFloat() ==> IsDataTypeFloat()
datatype_is_binary() ==> IsDataTypeBinary()
datatype_is_json() ==> IsDataTypeJson()
datatype_is_array() ==> IsDataTypeArray()
datatype_is_variable() == IsDataTypeVariable()
datatype_is_integer() / IsIntegral() ==> IsDataTypeInteger()

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2024-04-02 19:15:14 +08:00
Chun Han
92c84ccc3f
enhance: ban groupby on hybrid-search(#29968) (#31810) 
related: #29968

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2024-04-02 15:47:18 +08:00
Bingyi Sun
91cb529ba6
fix: get latest collection info when checking index (#31744) 
issue: https://github.com/milvus-io/milvus/issues/31727

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2024-04-02 14:43:13 +08:00
jaime
bd853be8c7
enhance: Add db label for some usual metrics (#30956) 
issue: #31782

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-04-02 14:27:13 +08:00
SimFG
ac26908cc4
enhance: Remove the storage info report (#31772) 
issue: #30436
origin pr: #30438

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-04-01 20:50:59 -07:00
XuanYang-cn
5d97693bcd
enhance: [skip e2e]Add it for channel balance (#30973) 
See also: #30993

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-04-02 10:25:15 +08:00
wei liu
0944a1f790
enhance: Refactor channel dist manager interface (#31119) 
issue: #31091
This PR add GetByFilter interface in channel dist manager, instead of
all kind of get func

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-04-02 10:23:14 +08:00
congqixia
16d869c57e
enhance: Add EmbedEtcd testutil and remove etcd dep of task pkg (#31802) 
See also #20478

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-02 09:59:14 +08:00
congqixia
e191827a87
fix: Clone child segment info before decompress its deltalog (#31792) 
Related to #31791

This segment meta is implemented in COW pattern. All modification on
segment info shall happen on the copied version of it.

This PR clones the child segment info for `GetSegmentInfo` in case data
race problem.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-01 21:41:15 +08:00
wei liu
9111ecab82
enhance: Rootcoord's stop may block in quota_center's stop (#31447) 
this PR fixed that rootcoord's stop may block in quota_center' stop

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-04-01 21:09:18 +08:00
yihao.dai
4e264003bf
enhance: Ensure ImportV2 waits for the index to be built and refine some logic (#31629) 
Feature Introduced:
1. Ensure ImportV2 waits for the index to be built

Enhancements Introduced:
1. Utilization of local time for timeout ts instead of allocating ts
from rootcoord.
3. Enhanced input file length check for binlog import.
4. Removal of duplicated manager in datanode.
5. Renaming of executor to scheduler in datanode.
6. Utilization of a thread pool in the scheduler in datanode.

issue: https://github.com/milvus-io/milvus/issues/28521

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-04-01 20:09:13 +08:00
XuanYang-cn
39337e09b8
fix: Using zero serverID for metrics (#31518) 
Fixes: #31516

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-04-01 16:55:19 +08:00
wei liu
4c8cc6ceff
fix: Avoid acquire index meta's lock for each segment (#31723) 
issue: #31662 #31409 

during FilterIndexedSegment in GetRecoveryInfo, it try to acquire index
meta's read lock for every segment. when a collection has thousands of
segments, which may blocked for more than 10 seconds and even longer.
cause `AddSegmentIndex` may also triggered frequently, which try to get
the write lock.

This PR avoid acquire index meta's lock for each segment

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-04-01 15:49:13 +08:00
Xiaofan
b6fefee0cf
fix: etcd not connectable when auth enabled (#31633) 
Fix etcd config source didn't respect auth enabled
Also removed pulsar recoverable error when pulsar return ConsumerBusy.
It could happen that pulsar didn't find the original consumer is dead
and recover takes some time.
fix #31631

Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
 2024-04-01 15:23:19 +08:00
PowderLi
d299fa502e
fix: use milvus-io/vcpkg (#31770) 
issue: #31769
GitHub Disables The XZ Repository because of CVE-2024-3094

Signed-off-by: PowderLi <min.li@zilliz.com>
 2024-04-01 15:01:13 +08:00
chyezh
5655ec4fc0
enhance: add mmap usage metrics (#31708) 
issue: #31707

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-04-01 11:35:12 +08:00
zhuwenxing
571f0bc054
test: fix bulk insert case (#31738) 
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2024-04-01 10:49:18 +08:00
wei liu
bb500d66c7
fix: Remove segment from leader view can't be executed (#31663) 
issue: #31664

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-04-01 10:39:12 +08:00
wei liu
c311932d5f
fix: Update segment's version in leader task (#31643) 
issue: #31468

1. when segment's version in leader view doesn't match segment's version
in dist, should update leader view
2. after call loadDeltalog, should update segment's load version with
latest ts
3. change leader task's priority from high to low, to avoid leader task
replace segment task and balance task

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-04-01 10:37:21 +08:00
sammy.huang
f7a59766df
enhance: support rockylinux8 (#31751) 
Signed-off-by: Sammy Huang <sammy.huang@zilliz.com>
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
 2024-04-01 10:23:12 +08:00
chyezh
1ad5ccc50f
enhance: add rg and db interface for segment and db/rg metric label (#31715) 
issue: #30931

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-04-01 10:21:21 +08:00
congqixia
3ffe126dc7
enhance: Refine error message when search vector type not matched (#31725) 
Previously the error message only reports the case happened without
field name and vector type.

This PR add field name and vector type information in the error
messages.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-01 10:11:12 +08:00
sammy.huang
b0a941c2fb
enhance: to reduce container image size (#31576) 
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
 2024-04-01 09:39:12 +08:00
yihao.dai
417c678754
enhance: Retry on inconsistent requery result (#31713) 
Make retry on ErrInConsistentRequery in proxy rather than in every SDK.

issue: https://github.com/milvus-io/milvus/issues/31642

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-03-30 20:53:19 +08:00
zhuwenxing
88d426f390
enhance:[skip e2e]add some custom deploy config back (#31716) 
This reverts commit ede5269221.

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2024-03-29 18:15:17 +08:00
Cai Yudong
675a5dc822
fix: Save traceID and spanID as std::vector into search config (#31278) 
Issue: #30961

Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
 2024-03-29 14:29:11 +08:00
congqixia
357fe814ce
fix: Remove unnecessary deleteSession operation (#31647) 
See also #31628

The `Revoke` operation shall delete all keys related to the lease
attaching to. This `deleteSession` operation may also remove the session
key in next epoch by mistake and cause chaos session status

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-03-29 13:57:11 +08:00
binbin
c15beecb96
test: modify the hybrid search test case (#31702) 
issue: #31339

Signed-off-by: binbin lv <binbin.lv@zilliz.com>
 2024-03-29 10:47:17 +08:00
congqixia
fe2f34d76f
fix: Use server ctx instead of loopCtx for datacoord LivenessCheck (#31691) 
See also #31689

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-03-29 10:37:11 +08:00
Jiquan Long
9750e78f1d
enhance: lock tantivy dependencies (#31688) 
issue: https://github.com/milvus-io/milvus/issues/31681

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2024-03-29 10:15:17 +08:00
cai.zhang
be6b468a3b
fix: Check whether the DataType of the column is legal (#31697) 
issue: #31696

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-03-29 09:33:16 +08:00
binbin
6c6764bcad
test: modify the order function in test cases (#31680) 
issue: #30694

Signed-off-by: binbin lv <binbin.lv@zilliz.com>
 2024-03-28 17:37:10 +08:00
Chun Han
b99c46246c
enhance: ban groupby on binary vector(#31134) (#31659) 
related: #31134

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2024-03-28 15:19:10 +08:00
Jiquan Long
e33dba8afe
fix: [skip-e2e] use zstd-sys 2.0.9 (#31682) 
fix: #31681 
/kind improvement

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2024-03-28 15:14:10 +08:00
cqy123456
976928ecd1
fix: fix fp16/bf16 some code missing and add more fp16/bf16 test (#31612) 
issue: #31534

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2024-03-28 14:11:10 +08:00
Bingyi Sun
3d66670619
fix: add lazy load field to mark segment load type (#31591) 
issue: https://github.com/milvus-io/milvus/issues/31673

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2024-03-28 11:23:10 +08:00
zhuwenxing
c42492c0fd
test: enable import job test in ci (#31530) 
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2024-03-28 10:31:10 +08:00
yihao.dai
78fbb87b3a
enhance: Release blobs in sync task once sync is completed (#31661) 
Once the synchronization of the sync task is completed, it's necessary
to release the blob within the sync task, as the caller may continue to
reference it.

issue: https://github.com/milvus-io/milvus/issues/31545

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-03-28 10:23:11 +08:00
SimFG
8f3e0b6b41
enhance: the return result of list db api (#31544) 
issue: #31543

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-03-28 07:13:10 +08:00
SimFG
b1a1cca10b
feat: add more operation detail info for better allocation (#30438) 
issue: #30436

---------

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-03-28 06:33:11 +08:00
Bingyi Sun
fbff46a005
enhance: add lazyload global config (#31610) 
Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2024-03-27 20:23:10 +08:00
Jiquan Long
4eb4df1e81
fix: predict inverted index resource usage more reasonably (#31615) 
/kind improvement
issue: #31617

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2024-03-27 19:33:09 +08:00
congqixia
655097f171
fix: Verify PlaceHolderValue type before search (#31626) 
See also #31625

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-03-27 17:49:10 +08:00
sammy.huang
d88026bd73
enhance: enable docker buildkit (#31600) 
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
 2024-03-27 16:49:15 +08:00
Ted Xu
efa645db6f
feat: adding the Dowait primitive in cache (#31578) 
See: #31577

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2024-03-27 16:17:15 +08:00
wei liu
92971707de
enhance: Add restful api for devops to execute rolling upgrade (#29998) 
issue: #29261
This PR Add restful api for devops to execute rolling upgrade, including
suspend/resume balance and manual transfer segments/channels.

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-03-27 16:15:19 +08:00
Chun Han
bd44bd5ae2
enhance: add default value config for segment prune filterRatio(#31003) (#31580) 
related: #31003

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2024-03-27 16:13:10 +08:00
congqixia
f399416b92
enhance: Use gotestsum to run go unit test (#31622) 
See also #31490

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-03-27 15:29:10 +08:00