test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-01 03:18:29 +08:00
Code Issues Actions 11 Packages Projects Releases Wiki Activity
20,168 Commits 94 Branches 105 Tags 184 MiB
a2c19af3ed
Commit Graph

20168 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
SimFG
0e226502e4
enhance: [2.4] pick default root password and log level pr (#34777) 
default root password
- issue: #33058
- pr: #34752

set log level
- issue: #34756
- pr: #34757

---------

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-07-18 13:45:43 +08:00
wayblink
a26e965e6a
enhance:[cherry-pick] Add compaction task slot usage logic (#34625) 
issue: #34544
pr: #34581

---------

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-07-18 09:55:43 +08:00
cai.zhang
d74a5bdc1d
fix: [cherry-pick] Fix bug where binlogs already flushed with new segment during pack (#34760) 
issue: #34703

master pr: #34762

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-07-18 09:15:47 +08:00
wayblink
83fc26c31a
fix: [cherry-pick] compaction task not be cleaned correctly (#34766) 
1.fix compaction task not be cleaned correctly
2.add a new parameter to control compaction gc loop interval
3.remove some useless configs of clustering compaction

bug: #34764
pr: #34765

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-07-17 22:17:42 +08:00
Patrick Weizhi Xu
3cf2e1831a
feat: [cherry-pick][2.4] support partition key isolation (#34336) (#34613) 
issue: #34332
pr: #34336

---------

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
(cherry picked from commit 104d0966b7)
 2024-07-17 14:03:43 +08:00
cai.zhang
19d3606e0d
fix: [cherry-pick] Fix the bug that caused small segment flush frequently (#34727) 
issue: #34703 

master pr: #34725

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-07-17 09:47:43 +08:00
Buqian Zheng
dfb41582cd
correctly set search params when using knowhere iterator (#34732) 
issue: #34730
pr: #34731

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
 2024-07-16 21:17:41 +08:00
sre-ci-robot
cdb4979c29
[automated] Bump milvus version to v2.4.6 (#34723) 
Bump milvus version to v2.4.6
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-07-16 20:05:38 +08:00
XuanYang-cn
edefc3cbb5
enhance: [skip e2e]Enable compaction it test (#34526) (#34720) 
pr: #34526

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-07-16 18:19:38 +08:00
SimFG
9cf1f03e89
fix: [2.4] the lint-fix error (#34717) 
- issue: #31293
- pr: #34033
- related pr: #34699

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-07-16 17:37:39 +08:00
wayblink
419a4d22dd
fix: [cherry-pick] fix bug if datacoord restart when l0 compaction saveSegmentMeta (#34714) 
issue: #34681
pr: #34679

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-07-16 17:27:38 +08:00
XuanYang-cn
5909a62ca6
fix: Fix accidentlly exit MixCompaction task loop (#34689) 
See also: #33431, #34460
pr: #34688

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-07-16 16:21:38 +08:00
cai.zhang
0c5aafd2d1
fix:[cherry-pick] Reset flushed row num after pack segment for clustering compaction (#34704) 
issue: #34703 
master pr: #34702

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-07-16 15:57:43 +08:00
SimFG
13f129aa9c
enhance: [2.4] format the sdk code (#34699) 
- issue: #31293
- pr: #34033

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-07-16 14:15:38 +08:00
SimFG
d607a43ea6
fix: [2.4] metaCache cleanup issue when listPolicy failed (#34697) 
- issue:#34667
- pr: #34449

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-07-16 14:13:45 +08:00
Xiaofan
66b5cbe27c
fix: CVE by upgrading some dependencies. (#34693) 
fix #34434 and #34456
upgrade otelgrpc to fix CVE

Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
 2024-07-16 14:05:37 +08:00
elstic
40f6febf9b
test: [cherry-pick] Fix index creation mismatch for sparse vector columns in multivector case (#34620) 
pr: https://github.com/milvus-io/milvus/pull/34618

Signed-off-by: elstic <hao.wang@zilliz.com>
 2024-07-16 09:39:38 +08:00
congqixia
274790b187
enhance: [2.4] Print reason when seal segments (#34575) (#34687) 
Cherry-pick from master
pr: #34575
See also #34574

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-15 22:49:39 +08:00
smellthemoon
0fdb288de7
enhance: upsert support autoid(#30342) (#34633) 
pr: #30342 
issue: #29258

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
 2024-07-15 20:53:39 +08:00
wei liu
cf701a9bf0
enhance: Preserve fixed-size memory in delegator node for growing segment (#34600) 
issue: #34595
pr: #34596
When consuming insert data on the delegator node, QueryCoord will move
out some sealed segments to manage its memory usage. After the growing
segment gets flushed, some sealed segments from other workers will be
moved back to the delegator node. To avoid the frequent movement of
segments, we estimate the maximum growing row count and preserve a
fixed-size memory in the delegator node.

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-07-15 20:51:46 +08:00
congqixia
10c04f33c7
enhance: [2.4] Add param item for segmentFlushInterval (#34629) (#34663) 
Cherry-pick from master
pr: #34629
See also #28817

Add paramitem for segment flush interval

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-15 14:31:40 +08:00
wayblink
92afcb7a39
fix: [cherry-pick] fix meta revert bug when clustering compaction task fail (#34606) 
issue: #34603
pr: #34604

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-07-14 22:23:39 +08:00
zhagnlu
1cf0c67af6
fix: fix auto merge error (#34662) 
pr: #34661

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-07-13 10:39:38 +08:00
cai.zhang
e1686d096f
fix: [cherry-pick] Remove flushed segment in segment manager generated through import (#34651) 
issue: #34648 

master pr: #34649

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-07-12 23:15:38 +08:00
congqixia
6a3a14affb
enhance: [2.4] Add lint rule to forbid gogo protobuf (#34594) (#34630) 
Cherry pick from master
pr: #34594
github.com/gogo/protobuf is deprecated and could be error prune after
upgrade protobuf message to v2.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-12 18:13:36 +08:00
zhagnlu
91952e860a
fix: fix bug that set incorrect info to columnbase (#34621) 
pr: #34428

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-07-12 17:07:36 +08:00
SimFG
9bcab2ebee
enhance: [2.4] use the insert rate limit for the upsert request (#34616) 
- issue: #34614
- pr: #34615

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-07-12 16:19:36 +08:00
zhagnlu
4e02e57044
enhance: mark duplicated pk as deleted (#34619) 
pr: #34586

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-07-12 10:27:37 +08:00
wayblink
86b57b7827
enhance: [cherry-pick] Alter compactTo segments before compactFrom to avoid data loss if crash (#34532) 
issue: #34512 
pr: #34513

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-07-12 00:47:37 +08:00
jaime
1d2062a68a
fix: encountering orphan channel-cp meta after GC (#34609) 
issue: #34545
pr: #34612

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-07-11 22:51:37 +08:00
XuanYang-cn
be41892c48
fix: Compaction task l0 state transfer wrong (#34599) 
This bug caused failed L0 compaction tasks never end.

See also: #34460
pr: #34597

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-07-11 20:21:37 +08:00
wayblink
9dc98ac76d
enhance: [cherry-pick] Refine clustering compaction latency metrics (#34539) 
issue: #34527
pr: #34528

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-07-11 17:43:43 +08:00
edward.zeng
3d29ebfffb
enhance: Refine default histogram metric bucket (#34592) 
/kind improvement
pr: https://github.com/milvus-io/milvus/pull/34580

Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2024-07-11 16:51:35 +08:00
XuanYang-cn
7e2a9d6bca
fix: Milvus panic when compaction disabled and dropping a collection (#34206) 
See also: #31059
pr: #34103

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-07-11 14:20:52 +08:00
aoiasd
d6fc6a9f41
fix: [Cherry-pick] Restful API use deprecate error code cause access log panic. (#34579) 
Avoid panic when use deprecate error code and use merr code replace
deprecate common code.
relate: https://github.com/milvus-io/milvus/issues/34578
pr： https://github.com/milvus-io/milvus/pull/34576

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-07-11 11:58:51 +08:00
jaime
bfd386aad7
fix: unstable UT for level0 deletion (#34525) 
issue: #34533
pr: #34524

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-07-11 10:02:56 +08:00
jaime
073a501fc7
enhance: skip orphan channel cp meta when checking cp lag (#34556) 
issue: #34545
pr: #34555

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-07-11 09:36:56 +08:00
XuanYang-cn
2bef9c38c6
enhance: Add back compactionTaskNum metrics (#34534) 
Fix L0 compaction task recover unable to set segment
not isCompacting

See also: milvus-io#34460
pr: milvus-io#34583

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-07-11 00:08:52 +08:00
yihao.dai
cbba5481f8
fix: Prevent dispatcher merging if curTs is 0 (#34562) (#34563) 
When the main dispatcher has not yet consumed data, curTs is 0. During
this time, merging dispatchers should not be allowed; otherwise, the
data of the solo dispatcher will be skipped.

issue: https://github.com/milvus-io/milvus/issues/34255

pr: https://github.com/milvus-io/milvus/pull/34562

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-07-10 22:46:51 +08:00
SimFG
15adb2feac
enhance: [2.4] add the seal segment when dispatch delete msgs (#34566) 
/kind improvement
- pr: #34565

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-07-10 19:18:51 +08:00
wei liu
a7190b042d
fix: Indexnode stuck at stopping progress cause by wrong lifetime control (#34559) 
issue: #34557
pr: #34558

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-07-10 15:14:13 +08:00
yihao.dai
1589d68413
fix: Fix no L0 compaction tasks are executed (#34537) (#34538) 
The nodeID for compaction task initialization is 0. This PR adjusts the
task reassignment conditions to allow new compaction tasks to be
reassigned and executed.

issue: https://github.com/milvus-io/milvus/issues/34460

pr: https://github.com/milvus-io/milvus/pull/34537

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-07-10 11:08:12 +08:00
wei liu
f0d12aa68d
fix: RootCoord panic during check health (#34542) 
issue: #34540
pr: #34541

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-07-10 09:50:38 +08:00
congqixia
0d8defb1f9
fix: [2.4] Write padding at end of mmap file not chunk (#34530) 
Cherry-pick from master
pr: #34529
Related to #34508

The padding bytes shall be written only at the end of the mmap file not
the chunk of each field data file.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-09 20:32:14 +08:00
jaime
60aab15ef4
enhance: support setting properties in create database request (#34511) 
issue: #34493
pr: #34510

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-07-09 18:16:18 +08:00
congqixia
ddb1015563
fix: [2.4] Check err is ErrKeyNotFound when CASCachedValue (#34489) 
Cherry-pick from master
pr: #34488
See also #33785

When config item is not present in paramtable, CAS fails due to
GetConfig returns error.

This PR make this returned err instance of ErrKeyNotFound and check
error type in \`CASCachedValue\` methods.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-09 18:12:13 +08:00
SimFG
737bd7c734
enhance: [2.4] release the record in delete codec and add some log for compaction (#34506) 
/kind improvement
- pr: #34454

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-07-09 15:40:17 +08:00
wayblink
7788decb37
fix: [cherry-pick] Fix compaction start time (#34505) 
bug: #34497 #34498
pr: #34496

---------

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-07-09 13:30:14 +08:00
yihao.dai
42b38fba00
fix: Fix unstable ut of msgdispatcher (#34485) (#34486) 
issue: https://github.com/milvus-io/milvus/issues/34484

pr: https://github.com/milvus-io/milvus/pull/34485

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-07-09 10:06:24 +08:00
wei liu
7034260721
fix: Query may return deleted records (#34502) 
issue: #34500
pr: #34501

cause the sort in `GetLevel0Deletions` will broken the corresponed order
between pks and tss, then the pks and tss will be sorted in
segment.Delete() interface.

This PR remove this uncessary and incorrect sort progress to avoid query
may return deleted records.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-07-09 10:00:14 +08:00