test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-01 11:29:48 +08:00
Code Issues Actions 11 Packages Projects Releases Wiki Activity
20,108 Commits 94 Branches 105 Tags 184 MiB
419a4d22dd
Commit Graph

20108 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
wayblink
419a4d22dd
fix: [cherry-pick] fix bug if datacoord restart when l0 compaction saveSegmentMeta (#34714) 
issue: #34681
pr: #34679

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-07-16 17:27:38 +08:00
XuanYang-cn
5909a62ca6
fix: Fix accidentlly exit MixCompaction task loop (#34689) 
See also: #33431, #34460
pr: #34688

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-07-16 16:21:38 +08:00
cai.zhang
0c5aafd2d1
fix:[cherry-pick] Reset flushed row num after pack segment for clustering compaction (#34704) 
issue: #34703 
master pr: #34702

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-07-16 15:57:43 +08:00
SimFG
13f129aa9c
enhance: [2.4] format the sdk code (#34699) 
- issue: #31293
- pr: #34033

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-07-16 14:15:38 +08:00
SimFG
d607a43ea6
fix: [2.4] metaCache cleanup issue when listPolicy failed (#34697) 
- issue:#34667
- pr: #34449

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-07-16 14:13:45 +08:00
Xiaofan
66b5cbe27c
fix: CVE by upgrading some dependencies. (#34693) 
fix #34434 and #34456
upgrade otelgrpc to fix CVE

Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
 2024-07-16 14:05:37 +08:00
elstic
40f6febf9b
test: [cherry-pick] Fix index creation mismatch for sparse vector columns in multivector case (#34620) 
pr: https://github.com/milvus-io/milvus/pull/34618

Signed-off-by: elstic <hao.wang@zilliz.com>
 2024-07-16 09:39:38 +08:00
congqixia
274790b187
enhance: [2.4] Print reason when seal segments (#34575) (#34687) 
Cherry-pick from master
pr: #34575
See also #34574

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-15 22:49:39 +08:00
smellthemoon
0fdb288de7
enhance: upsert support autoid(#30342) (#34633) 
pr: #30342 
issue: #29258

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
 2024-07-15 20:53:39 +08:00
wei liu
cf701a9bf0
enhance: Preserve fixed-size memory in delegator node for growing segment (#34600) 
issue: #34595
pr: #34596
When consuming insert data on the delegator node, QueryCoord will move
out some sealed segments to manage its memory usage. After the growing
segment gets flushed, some sealed segments from other workers will be
moved back to the delegator node. To avoid the frequent movement of
segments, we estimate the maximum growing row count and preserve a
fixed-size memory in the delegator node.

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-07-15 20:51:46 +08:00
congqixia
10c04f33c7
enhance: [2.4] Add param item for segmentFlushInterval (#34629) (#34663) 
Cherry-pick from master
pr: #34629
See also #28817

Add paramitem for segment flush interval

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-15 14:31:40 +08:00
wayblink
92afcb7a39
fix: [cherry-pick] fix meta revert bug when clustering compaction task fail (#34606) 
issue: #34603
pr: #34604

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-07-14 22:23:39 +08:00
zhagnlu
1cf0c67af6
fix: fix auto merge error (#34662) 
pr: #34661

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-07-13 10:39:38 +08:00
cai.zhang
e1686d096f
fix: [cherry-pick] Remove flushed segment in segment manager generated through import (#34651) 
issue: #34648 

master pr: #34649

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-07-12 23:15:38 +08:00
congqixia
6a3a14affb
enhance: [2.4] Add lint rule to forbid gogo protobuf (#34594) (#34630) 
Cherry pick from master
pr: #34594
github.com/gogo/protobuf is deprecated and could be error prune after
upgrade protobuf message to v2.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-12 18:13:36 +08:00
zhagnlu
91952e860a
fix: fix bug that set incorrect info to columnbase (#34621) 
pr: #34428

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-07-12 17:07:36 +08:00
SimFG
9bcab2ebee
enhance: [2.4] use the insert rate limit for the upsert request (#34616) 
- issue: #34614
- pr: #34615

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-07-12 16:19:36 +08:00
zhagnlu
4e02e57044
enhance: mark duplicated pk as deleted (#34619) 
pr: #34586

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-07-12 10:27:37 +08:00
wayblink
86b57b7827
enhance: [cherry-pick] Alter compactTo segments before compactFrom to avoid data loss if crash (#34532) 
issue: #34512 
pr: #34513

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-07-12 00:47:37 +08:00
jaime
1d2062a68a
fix: encountering orphan channel-cp meta after GC (#34609) 
issue: #34545
pr: #34612

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-07-11 22:51:37 +08:00
XuanYang-cn
be41892c48
fix: Compaction task l0 state transfer wrong (#34599) 
This bug caused failed L0 compaction tasks never end.

See also: #34460
pr: #34597

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-07-11 20:21:37 +08:00
wayblink
9dc98ac76d
enhance: [cherry-pick] Refine clustering compaction latency metrics (#34539) 
issue: #34527
pr: #34528

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-07-11 17:43:43 +08:00
edward.zeng
3d29ebfffb
enhance: Refine default histogram metric bucket (#34592) 
/kind improvement
pr: https://github.com/milvus-io/milvus/pull/34580

Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2024-07-11 16:51:35 +08:00
XuanYang-cn
7e2a9d6bca
fix: Milvus panic when compaction disabled and dropping a collection (#34206) 
See also: #31059
pr: #34103

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-07-11 14:20:52 +08:00
aoiasd
d6fc6a9f41
fix: [Cherry-pick] Restful API use deprecate error code cause access log panic. (#34579) 
Avoid panic when use deprecate error code and use merr code replace
deprecate common code.
relate: https://github.com/milvus-io/milvus/issues/34578
pr： https://github.com/milvus-io/milvus/pull/34576

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-07-11 11:58:51 +08:00
jaime
bfd386aad7
fix: unstable UT for level0 deletion (#34525) 
issue: #34533
pr: #34524

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-07-11 10:02:56 +08:00
jaime
073a501fc7
enhance: skip orphan channel cp meta when checking cp lag (#34556) 
issue: #34545
pr: #34555

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-07-11 09:36:56 +08:00
XuanYang-cn
2bef9c38c6
enhance: Add back compactionTaskNum metrics (#34534) 
Fix L0 compaction task recover unable to set segment
not isCompacting

See also: milvus-io#34460
pr: milvus-io#34583

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-07-11 00:08:52 +08:00
yihao.dai
cbba5481f8
fix: Prevent dispatcher merging if curTs is 0 (#34562) (#34563) 
When the main dispatcher has not yet consumed data, curTs is 0. During
this time, merging dispatchers should not be allowed; otherwise, the
data of the solo dispatcher will be skipped.

issue: https://github.com/milvus-io/milvus/issues/34255

pr: https://github.com/milvus-io/milvus/pull/34562

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-07-10 22:46:51 +08:00
SimFG
15adb2feac
enhance: [2.4] add the seal segment when dispatch delete msgs (#34566) 
/kind improvement
- pr: #34565

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-07-10 19:18:51 +08:00
wei liu
a7190b042d
fix: Indexnode stuck at stopping progress cause by wrong lifetime control (#34559) 
issue: #34557
pr: #34558

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-07-10 15:14:13 +08:00
yihao.dai
1589d68413
fix: Fix no L0 compaction tasks are executed (#34537) (#34538) 
The nodeID for compaction task initialization is 0. This PR adjusts the
task reassignment conditions to allow new compaction tasks to be
reassigned and executed.

issue: https://github.com/milvus-io/milvus/issues/34460

pr: https://github.com/milvus-io/milvus/pull/34537

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-07-10 11:08:12 +08:00
wei liu
f0d12aa68d
fix: RootCoord panic during check health (#34542) 
issue: #34540
pr: #34541

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-07-10 09:50:38 +08:00
congqixia
0d8defb1f9
fix: [2.4] Write padding at end of mmap file not chunk (#34530) 
Cherry-pick from master
pr: #34529
Related to #34508

The padding bytes shall be written only at the end of the mmap file not
the chunk of each field data file.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-09 20:32:14 +08:00
jaime
60aab15ef4
enhance: support setting properties in create database request (#34511) 
issue: #34493
pr: #34510

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-07-09 18:16:18 +08:00
congqixia
ddb1015563
fix: [2.4] Check err is ErrKeyNotFound when CASCachedValue (#34489) 
Cherry-pick from master
pr: #34488
See also #33785

When config item is not present in paramtable, CAS fails due to
GetConfig returns error.

This PR make this returned err instance of ErrKeyNotFound and check
error type in \`CASCachedValue\` methods.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-09 18:12:13 +08:00
SimFG
737bd7c734
enhance: [2.4] release the record in delete codec and add some log for compaction (#34506) 
/kind improvement
- pr: #34454

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-07-09 15:40:17 +08:00
wayblink
7788decb37
fix: [cherry-pick] Fix compaction start time (#34505) 
bug: #34497 #34498
pr: #34496

---------

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-07-09 13:30:14 +08:00
yihao.dai
42b38fba00
fix: Fix unstable ut of msgdispatcher (#34485) (#34486) 
issue: https://github.com/milvus-io/milvus/issues/34484

pr: https://github.com/milvus-io/milvus/pull/34485

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-07-09 10:06:24 +08:00
wei liu
7034260721
fix: Query may return deleted records (#34502) 
issue: #34500
pr: #34501

cause the sort in `GetLevel0Deletions` will broken the corresponed order
between pks and tss, then the pks and tss will be sorted in
segment.Delete() interface.

This PR remove this uncessary and incorrect sort progress to avoid query
may return deleted records.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-07-09 10:00:14 +08:00
Gao
f6cd84161c
enhance: ensure autoindex default metric type compatibility (#34479) 
issue: #34304 
pr: #34261

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2024-07-08 19:48:14 +08:00
Chun Han
2f38483418
fix: lose partitionIDs when scalar pruning and refine segment prune ratio metrics(#30376) (#34475) 
related: #30376
pr: https://github.com/milvus-io/milvus/pull/34477

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2024-07-08 19:44:13 +08:00
wayblink
56a74e72f7
fix: [cherry-pick]fix can't enqueue when compaction queue is full(#34445) (#34469) 
issue: #30633
pr: #34445

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-07-08 19:20:13 +08:00
yihao.dai
0d7ba810b3
enhance: Check segment existence when FlushSegments and add some key logs (#34438) (#34472) 
Check if the segment exists during FlushSegments and add some key logs
in write path.

issue: https://github.com/milvus-io/milvus/issues/34255

pr: https://github.com/milvus-io/milvus/pull/34438

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-07-08 19:00:13 +08:00
yihao.dai
0732167c87
fix: Fix incorrect segment num rows (#34441) (#34474) 
Repeated calls to UpdateStatistics, this PR correct it.

issue: https://github.com/milvus-io/milvus/issues/34440

pr: https://github.com/milvus-io/milvus/pull/34441

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-07-08 17:30:12 +08:00
XuanYang-cn
26ac76944b
fix: Accidently exit the check loop (#34480) 
See also: #34460
pr: #34481

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-07-08 17:14:15 +08:00
congqixia
7c3c9c2ed4
fix: [2.4] Add nbits parameter check for IVF_PQ (#34451) (#34473) 
Cherry-pick from master
pr: #34451
See also #34426

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-08 17:10:12 +08:00
zhuwenxing
f2d0517f96
test: [cherry-pick]add testcase for count query (#34471) 
pr: https://github.com/milvus-io/milvus/pull/34453

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2024-07-08 12:54:12 +08:00
Gao
a60e2a65ff
enhance: change autoindex default metric type (#34277) 
issue: #34304 
pr: #34261

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2024-07-08 10:52:14 +08:00
congqixia
014820e9d2
fix: [2.4] Write padding into mmap file in case of SIGBUS (#34443) (#34455) 
Cherry-pick from master
pr: #34443
See also #34442

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-08 10:06:19 +08:00