test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-02 11:59:00 +08:00
Code Issues Actions 12 Packages Projects Releases Wiki Activity
20,309 Commits 94 Branches 105 Tags 184 MiB
c75f556769
Commit Graph

20309 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
jaime
c75f556769
fix: inconsistent meta view causes rate limit invalid (#35664) 
issue: #35663
pr: #35665

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-08-28 11:21:04 +08:00
yihao.dai
8e6ec58652
fix: Fix rate wasn't limited to the expected value (#35699) (#35700) 
Each time the rate is reset, the token bucket is fully refilled, causing
the rate wasn't limited to the expected value. This PR addresses the
issue by preventing the token reset.

issue: https://github.com/milvus-io/milvus/issues/35675,
https://github.com/milvus-io/milvus/issues/35702

pr: https://github.com/milvus-io/milvus/pull/35699

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-08-28 10:29:00 +08:00
jaime
eecc598786
fix: mistaken deletions may occur during GC channel checkpoints (#35708) 
issue: #35706
pr: #35707

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-08-28 10:11:05 +08:00
Zhen Ye
a4533f1b8a
enhance: optimize milvus core building (#35660) 
issue: #35549,#35611,#35633
pr: #35610

- remove milvus_segcore milvus_indexbuilder..., add libmilvus_core
- core building only link once
- move opendal compilation into cmake
- fix odr

---------

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-08-27 18:55:00 +08:00
yihao.dai
be17cf6ff1
fix: Fix panic due to empty candidate import segments (#35673) (#35674) 
issue: https://github.com/milvus-io/milvus/issues/35662

pr: https://github.com/milvus-io/milvus/pull/35673

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-08-27 17:34:59 +08:00
Zhen Ye
69e8d5ef46
fix: munmap deallocate too much memory (#35726) 
issue: #35693
pr: #35725

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-08-27 17:22:59 +08:00
SimFG
fc324b4254
feat: [2.4] add the rbac msg and send them to the replicate channel (#35562) 
- issue: #35391
- pr: #35392

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-08-27 14:45:00 +08:00
yellow-shine
03627150d3
enhance new jenkinsfile for nightly and e2e (#35715) 
https://github.com/milvus-io/milvus/pull/35672

Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
 2024-08-27 14:23:05 +08:00
congqixia
ab261d0f8b
feat: [2.4] Support field partial load collection (#35416) (#35696) 
Cherry-pick from master
pr: #35416
Related to #35415

---------

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-27 14:07:00 +08:00
yellow-shine
33fbc1fb96
enhance: turnoff USE_ASAN in ci (#35723) 
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
 2024-08-27 14:01:11 +08:00
XuanYang-cn
f2caecc577
fix: [cp]Fill nil schema so that Milvus can watch channel for those upgraded from 2.2 to 2.4 (#35695) 
See also: #35701 
pr: #35694

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-08-27 10:38:59 +08:00
XuanYang-cn
ff68eb53d7
fix: [skip e2e]unstable l0 it (#35613) 
See also: #35617 
pr: #35612

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-08-26 18:53:04 +08:00
Zhen Ye
7062e6fcbf
enhance: enable asan for milvus (#35687) 
issue: #35626
pr: #35627

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-08-26 09:54:58 +08:00
Xiaofan
7269d5eda2
enhance: [2.4] reduce the log level of frequent log (#35653) 
pr: #35651

Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
 2024-08-25 17:48:57 +08:00
jaime
5c1a380d9d
fix: the incorrect number of partitions in rootcoord meta (#35601) 
pr: #35600

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-08-25 16:30:59 +08:00
Patrick Weizhi Xu
4b5bd07971
enhance: [2.4] enable MV on binary vectors (#35669) 
pr: #35668

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
(cherry picked from commit 5404228fca58d4ad4a788ec6d58d2693019132b6)
 2024-08-25 16:27:08 +08:00
wei liu
35d2f9b210
fix: Fix index memory estimation (#35225) (#35670) 
issue: https://github.com/milvus-io/milvus/issues/35229
pr: #32525

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Co-authored-by: Bingyi Sun <sunbingyi1992@gmail.com>
 2024-08-24 10:28:57 +08:00
XuanYang-cn
81c4cae2c1
enhance: Enable to write multiple segments in mix compactor (#35648) 
Prevent segments to be written larger than maxSize * expansionRate

See also: #35584

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-08-24 10:00:58 +08:00
XuanYang-cn
eb18e114a1
fix: Clear channelcp meta and metrics ASAP (#35659) 
See also: #35588
pr: #35658

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-08-24 09:28:58 +08:00
nish112022
79d093257c
fix: Added check for validating varchar,array max length (#35499) (#35581) 
issue : https://github.com/milvus-io/milvus/issues/34150
pr: https://github.com/milvus-io/milvus/pull/35499
This is for numpy,parquet,json readers.

cherry-picked from master branch

Signed-off-by: Nischay Yadav <nischay.yadav@ibm.com>
 2024-08-23 17:14:58 +08:00
congqixia
f3fb37655b
fix: [2.4] Use correct release func when datanode unwatch a channel (#35657) 
Cherry-pick from master
pr: #35655 
See also #35654

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-23 14:09:03 +08:00
Gao
1687d64c46
enhance: [2.4] add hit segment num metrics for queryHook (#35619) 
issue: #35576 
pr: #35577

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2024-08-23 12:49:02 +08:00
SimFG
5b5119a51f
feat: [2.4] provide more general configuration to control mmap behavior (#35609) 
- issue: #35273
- pr: #35359

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-08-23 12:35:02 +08:00
zhagnlu
90a5bafe17
fix:fix bitmap log error (#35644) 
pr: #32902

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-08-23 08:56:58 +08:00
sre-ci-robot
216f33c529
[automated] Update Pytest image changes (#35639) 
Update Pytest image changes
See changes:
d7d4525c2e
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-08-22 19:18:58 +08:00
wei liu
421a00bfe8
enhance: Refresh proxy cache after restore rbac meta (#35636) 
issue: #35443
pr: #35635

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-22 19:09:01 +08:00
congqixia
9c67965d6c
fix: [2.4] Re-read value after once initialization (#35643) 
Cherry-pick from master
pr: #35642
Related to #35641
See also #35271

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-22 16:30:57 +08:00
wei liu
e014ad9280
fix: fix dynamic update config doesn't works for some param (#35572) (#35637) 
issue: #35570
pr: #35572
milvus support config cache to spped up config access, but only evict
param's cache when param has been updated. but milvus's param may rely
on other param's value, let's say ParamsA relys on paramsB, when paramsB
updated, it will evict paramB's cache, but the paramA's cache still keep
the old value.

This PR evict all config cache to solve the above issue, cause dynamic
update config won't be much frequetly.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-22 16:00:58 +08:00
zhagnlu
946881e123
enhance: optimize retrieve on dynamic field (#35602) 
pr: #35580

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
Co-authored-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2024-08-22 14:18:57 +08:00
yanliang567
d7d4525c2e
test:[cherry-pick] Add upsert with autoid and update init collection method (#35624) 
Related issue: https://github.com/milvus-io/milvus/issues/32653
pr:  #35596

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
 2024-08-22 14:00:57 +08:00
wei liu
e2542a1bf5
enhance: Update protobuf-go to protobuf-go v2 (#34394) (#35555) 
issue: #34252
pr: #34394 #35072 #35084

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-21 18:50:58 +08:00
yellow-shine
bfd8758152
feat: [skip e2e] effective way to use cache (#35620) 
issue: https://github.com/milvus-io/milvus/issues/34876
https://github.com/milvus-io/milvus/pull/35598

Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
 2024-08-21 18:44:57 +08:00
congqixia
f108ce0fc5
fix: [2.4] Match int8_t and int16_t in Array::get_data (#35579) (#35597) 
Cherry-pick from master
pr: #35579
Related to #35578

Previously int16/int8 bitmap index may read int32 array as int16, which
may cause build index with half of the data(if array is full) and half
zeros. This causes BITMAP index lost information.

This PR matches int8_t & int16_t while `get_data` when building index.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-21 17:47:01 +08:00
wei liu
4bf4cbad85
enhance: Mark query node as read only after suspend (#35492) (#35586) 
issue: #34985 #35493
pr: #35492
after querynode has been suspended, it's not allow to load
segment/channel on it, which means the node is read only. to be
compatible with resource group design, after query node has been
suspend, we remove it from it's original resource group, make it a read
only query node in replica. then two things will happens:
1. it's original resource group will be lacking of query nodes, query
coord will assign new node to it.
2. querycoord will try to move out all segments/channels after querynode
has been suspended

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-20 19:00:56 +08:00
wei liu
4610dafb2e
enhance: make configure load param feature be compatible with old sdk(#35520) (#35573) 
issue: #31570 #35521
pr: #35520 #35546

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-20 18:20:57 +08:00
Chun Han
cf8494ef45
enhance: support httpv1/v2 throttle and add it for httpV2(#35350) (#35504) 
related: #35350
pr: https://github.com/milvus-io/milvus/pull/35470

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2024-08-20 16:32:56 +08:00
wei liu
fc344d1eae
fix: Fix proxy panic after Import related api failed (#35540) (#35559) 
issue: #35539
pr: #35540

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-19 19:42:55 +08:00
wei liu
14ec3dc357
enhance: Enable ReadOnly/ReadWrite/Admin Privilege Group to simplify RBAC grant progress (#35472) (#35543) 
issue: #35471
pr: #35472 #35515

---------

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-19 16:24:54 +08:00
XuanYang-cn
4a5e6bc6f6
enhance: Add integration tests for l0 (#35430) 
pr: #35429
See also: #34796

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-08-19 10:54:55 +08:00
sre-ci-robot
5da1a36180
[automated] Bump milvus version to v2.4.9 (#35537) 
Bump milvus version to v2.4.9
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-08-19 10:32:55 +08:00
wei liu
248a6ea401
enhance: Add BackupRBAC/RestoreRBAC API to enable rbac backup (#35444) (#35513) 
issue: #35443
pr: #35444

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-18 13:42:55 +08:00
Alexander Guzhva
8ddc3fd463
enhance: [2.4] Improve bitset performance for AVX512 (#35480) 
see #35478
pr: #35479

Signed-off-by: Alexandr Guzhva <alexanderguzhva@gmail.com>
 2024-08-17 03:58:56 +08:00
sre-ci-robot
9717185f56
[automated] Update cpu Builder image changes (#35509) 
Update cpu Builder image changes
See changes:
90127b948a
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-08-16 20:34:58 +08:00
congqixia
4e098f1b23
enhance: Bump milvus & proto to v2.4.9 (#35516) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-16 16:24:54 +08:00
congqixia
bd222e58eb
enhance: [2.4] Exclude L0 segment from readable snapshot (#35510) 
Cherry-pick from master
pr: #35507

L0 segments now do not contain insert data and may cause confusion for
query hook optimizer if counted as sealed segment number.

This PR add segment level flag in segment entry and exclude L0 segments
while get readable segment snapshot

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-16 15:26:54 +08:00
congqixia
555b7a6aa7
enhance: [2.4] Remove duplicated schema helper creation in proxy (#35489) (#35502) 
Cherry-pick from master
pr: #35489
Related to PRs of #35415

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-16 14:32:54 +08:00
yellow-shine
90127b948a
enhance: [skip e2e] enable ubuntu20.04 os (#35457) 
https://github.com/milvus-io/milvus/pull/35482

Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
 2024-08-16 10:06:58 +08:00
cai.zhang
09aea3fbf1
enhance: [cherry-pick] Optimize the use of locks and avoid double flush clustering buffer writer (#35490) 
issue: #35436 

master pr: #35486

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-08-16 02:24:59 +08:00
SimFG
4fae5407dd
enhance: [2.4] remove the invalid log (#35473) 
/kind improvement
pr: #35465

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-08-15 14:14:53 +08:00
wayblink
8c29c5bb23
enhance: [cherry-pick] Add clustering compaction user guide doc (#35428) 
issue: #30633
master pr: #35427

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-08-15 10:30:58 +08:00