zhagnlu
d6d3b01a04
fix:remove thread pool timeout test because of high load cpu ( #31879 )
...
#31877
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-04-03 15:55:38 +08:00
Cai Yudong
0d1abb081b
enhance: Disable HNSW Binary ( #31825 )
...
Issue: #31494
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
2024-04-03 14:53:14 +08:00
Jiquan Long
03e0db109e
fix: udpate Cargo.lock ( #31859 )
...
issue: #31681
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-04-03 14:18:23 +08:00
shaoyue
be53e028a8
doc: [skip e2e] add config-docs-generator ( #22134 )
...
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
Part of issue: #21856
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2024-04-03 14:13:13 +08:00
sammy.huang
b11c2da930
enhance: for supporting ARM-based GPU images ( #31757 )
...
This PR is for supporting ARM-based GPU images.
issue: #31862
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-04-03 11:49:14 +08:00
congqixia
49b8ee4339
fix: Make FlushTs Sync Policy apply to all buffers ( #31839 )
...
See also #30552
FlushTS policy was orignally designed to flushed/L0 segments only, but
in some edge case, new growing segment buffer would by-pass flush
request and hold a buffer before flush ts, which caused flush timeout
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-04-03 11:47:13 +08:00
sammy.huang
ae307af19e
enhance: [skip e2e]enable gpu e2e pipeline ( #31821 )
...
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-04-03 11:41:21 +08:00
congqixia
1af2ee903c
fix: Check parse QueryInfo error before using it ( #31836 )
...
See also milvus-io/milvus-sdk-go#693 #31735
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-04-03 11:29:14 +08:00
binbin
3b5209c460
test: support enable or disable multiple vector fields in all data type tests ( #31201 )
...
issue: #29799
Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2024-04-03 11:11:23 +08:00
congqixia
c2aad513c0
fix: Check collection nil before check load status ( #31850 )
...
See also #31849
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-04-03 10:07:13 +08:00
congqixia
56e371c478
fix: Check replica exists before get latest leader ( #31848 )
...
See also #31847
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-04-03 10:05:22 +08:00
wei liu
7471a8005f
fix: querycoord panic after node down ( #31831 )
...
issue: #30519
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-04-03 10:03:22 +08:00
congqixia
0feee53631
enhance: Add back unit test for compactor and fix some TODOs ( #31829 )
...
This PR adds back compactor "Unhandled" data type unit test and fixes
some TODOs behvaior
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-04-02 20:35:14 +08:00
Cai Yudong
246586be27
enhance: Unify data type check APIs under internal/core ( #31800 )
...
Issue: #22837
Move and rename following C++ APIs:
datatype_sizeof() ==> GetDataTypeSize()
datatype_name() ==> GetDataTypeName()
datatype_is_vector() / IsVectorType() ==> IsVectorDataType()
datatype_is_variable() ==> IsVariableDataType()
datatype_is_sparse_vector() ==> IsSparseFloatVectorDataType()
datatype_is_string() / IsString() ==> IsDataTypeString()
datatype_is_floating() / IsFloat() ==> IsDataTypeFloat()
datatype_is_binary() ==> IsDataTypeBinary()
datatype_is_json() ==> IsDataTypeJson()
datatype_is_array() ==> IsDataTypeArray()
datatype_is_variable() == IsDataTypeVariable()
datatype_is_integer() / IsIntegral() ==> IsDataTypeInteger()
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
2024-04-02 19:15:14 +08:00
Chun Han
92c84ccc3f
enhance: ban groupby on hybrid-search( #29968 ) ( #31810 )
...
related: #29968
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2024-04-02 15:47:18 +08:00
Bingyi Sun
91cb529ba6
fix: get latest collection info when checking index ( #31744 )
...
issue: https://github.com/milvus-io/milvus/issues/31727
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-04-02 14:43:13 +08:00
jaime
bd853be8c7
enhance: Add db label for some usual metrics ( #30956 )
...
issue: #31782
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-04-02 14:27:13 +08:00
SimFG
ac26908cc4
enhance: Remove the storage info report ( #31772 )
...
issue: #30436
origin pr: #30438
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-04-01 20:50:59 -07:00
XuanYang-cn
5d97693bcd
enhance: [skip e2e]Add it for channel balance ( #30973 )
...
See also: #30993
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-04-02 10:25:15 +08:00
wei liu
0944a1f790
enhance: Refactor channel dist manager interface ( #31119 )
...
issue: #31091
This PR add GetByFilter interface in channel dist manager, instead of
all kind of get func
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-04-02 10:23:14 +08:00
congqixia
16d869c57e
enhance: Add EmbedEtcd testutil and remove etcd dep of task pkg ( #31802 )
...
See also #20478
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-04-02 09:59:14 +08:00
congqixia
e191827a87
fix: Clone child segment info before decompress its deltalog ( #31792 )
...
Related to #31791
This segment meta is implemented in COW pattern. All modification on
segment info shall happen on the copied version of it.
This PR clones the child segment info for `GetSegmentInfo` in case data
race problem.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-04-01 21:41:15 +08:00
wei liu
9111ecab82
enhance: Rootcoord's stop may block in quota_center's stop ( #31447 )
...
this PR fixed that rootcoord's stop may block in quota_center' stop
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-04-01 21:09:18 +08:00
yihao.dai
4e264003bf
enhance: Ensure ImportV2 waits for the index to be built and refine some logic ( #31629 )
...
Feature Introduced:
1. Ensure ImportV2 waits for the index to be built
Enhancements Introduced:
1. Utilization of local time for timeout ts instead of allocating ts
from rootcoord.
3. Enhanced input file length check for binlog import.
4. Removal of duplicated manager in datanode.
5. Renaming of executor to scheduler in datanode.
6. Utilization of a thread pool in the scheduler in datanode.
issue: https://github.com/milvus-io/milvus/issues/28521
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-04-01 20:09:13 +08:00
XuanYang-cn
39337e09b8
fix: Using zero serverID for metrics ( #31518 )
...
Fixes : #31516
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-04-01 16:55:19 +08:00
wei liu
4c8cc6ceff
fix: Avoid acquire index meta's lock for each segment ( #31723 )
...
issue: #31662 #31409
during FilterIndexedSegment in GetRecoveryInfo, it try to acquire index
meta's read lock for every segment. when a collection has thousands of
segments, which may blocked for more than 10 seconds and even longer.
cause `AddSegmentIndex` may also triggered frequently, which try to get
the write lock.
This PR avoid acquire index meta's lock for each segment
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-04-01 15:49:13 +08:00
Xiaofan
b6fefee0cf
fix: etcd not connectable when auth enabled ( #31633 )
...
Fix etcd config source didn't respect auth enabled
Also removed pulsar recoverable error when pulsar return ConsumerBusy.
It could happen that pulsar didn't find the original consumer is dead
and recover takes some time.
fix #31631
Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
2024-04-01 15:23:19 +08:00
PowderLi
d299fa502e
fix: use milvus-io/vcpkg ( #31770 )
...
issue: #31769
GitHub Disables The XZ Repository because of CVE-2024-3094
Signed-off-by: PowderLi <min.li@zilliz.com>
2024-04-01 15:01:13 +08:00
chyezh
5655ec4fc0
enhance: add mmap usage metrics ( #31708 )
...
issue: #31707
Signed-off-by: chyezh <chyezh@outlook.com>
2024-04-01 11:35:12 +08:00
zhuwenxing
571f0bc054
test: fix bulk insert case ( #31738 )
...
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-04-01 10:49:18 +08:00
wei liu
bb500d66c7
fix: Remove segment from leader view can't be executed ( #31663 )
...
issue: #31664
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-04-01 10:39:12 +08:00
wei liu
c311932d5f
fix: Update segment's version in leader task ( #31643 )
...
issue: #31468
1. when segment's version in leader view doesn't match segment's version
in dist, should update leader view
2. after call loadDeltalog, should update segment's load version with
latest ts
3. change leader task's priority from high to low, to avoid leader task
replace segment task and balance task
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-04-01 10:37:21 +08:00
sammy.huang
f7a59766df
enhance: support rockylinux8 ( #31751 )
...
Signed-off-by: Sammy Huang <sammy.huang@zilliz.com>
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-04-01 10:23:12 +08:00
chyezh
1ad5ccc50f
enhance: add rg and db interface for segment and db/rg metric label ( #31715 )
...
issue: #30931
Signed-off-by: chyezh <chyezh@outlook.com>
2024-04-01 10:21:21 +08:00
congqixia
3ffe126dc7
enhance: Refine error message when search vector type not matched ( #31725 )
...
Previously the error message only reports the case happened without
field name and vector type.
This PR add field name and vector type information in the error
messages.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-04-01 10:11:12 +08:00
sammy.huang
b0a941c2fb
enhance: to reduce container image size ( #31576 )
...
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-04-01 09:39:12 +08:00
yihao.dai
417c678754
enhance: Retry on inconsistent requery result ( #31713 )
...
Make retry on ErrInConsistentRequery in proxy rather than in every SDK.
issue: https://github.com/milvus-io/milvus/issues/31642
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-30 20:53:19 +08:00
zhuwenxing
88d426f390
enhance:[skip e2e]add some custom deploy config back ( #31716 )
...
This reverts commit ede5269221
2024-03-29 18:15:17 +08:00
Cai Yudong
675a5dc822
fix: Save traceID and spanID as std::vector into search config ( #31278 )
...
Issue: #30961
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
2024-03-29 14:29:11 +08:00
congqixia
357fe814ce
fix: Remove unnecessary deleteSession operation ( #31647 )
...
See also #31628
The `Revoke` operation shall delete all keys related to the lease
attaching to. This `deleteSession` operation may also remove the session
key in next epoch by mistake and cause chaos session status
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-03-29 13:57:11 +08:00
binbin
c15beecb96
test: modify the hybrid search test case ( #31702 )
...
issue: #31339
Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2024-03-29 10:47:17 +08:00
congqixia
fe2f34d76f
fix: Use server ctx instead of loopCtx for datacoord LivenessCheck ( #31691 )
...
See also #31689
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-03-29 10:37:11 +08:00
Jiquan Long
9750e78f1d
enhance: lock tantivy dependencies ( #31688 )
...
issue: https://github.com/milvus-io/milvus/issues/31681
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-03-29 10:15:17 +08:00
cai.zhang
be6b468a3b
fix: Check whether the DataType of the column is legal ( #31697 )
...
issue: #31696
---------
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-03-29 09:33:16 +08:00
binbin
6c6764bcad
test: modify the order function in test cases ( #31680 )
...
issue: #30694
Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2024-03-28 17:37:10 +08:00
Chun Han
b99c46246c
enhance: ban groupby on binary vector( #31134 ) ( #31659 )
...
related: #31134
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2024-03-28 15:19:10 +08:00
Jiquan Long
e33dba8afe
fix: [skip-e2e] use zstd-sys 2.0.9 ( #31682 )
...
fix : #31681
/kind improvement
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-03-28 15:14:10 +08:00
cqy123456
976928ecd1
fix: fix fp16/bf16 some code missing and add more fp16/bf16 test ( #31612 )
...
issue: #31534
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2024-03-28 14:11:10 +08:00
Bingyi Sun
3d66670619
fix: add lazy load field to mark segment load type ( #31591 )
...
issue: https://github.com/milvus-io/milvus/issues/31673
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-03-28 11:23:10 +08:00
zhuwenxing
c42492c0fd
test: enable import job test in ci ( #31530 )
...
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-03-28 10:31:10 +08:00
