Commit Graph

20279 Commits

Author SHA1 Message Date
wei liu
e2542a1bf5
enhance: Update protobuf-go to protobuf-go v2 (#34394) (#35555)
issue: #34252
pr: #34394 #35072 #35084

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-21 18:50:58 +08:00
yellow-shine
bfd8758152
feat: [skip e2e] effective way to use cache (#35620)
issue: https://github.com/milvus-io/milvus/issues/34876
https://github.com/milvus-io/milvus/pull/35598

Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-08-21 18:44:57 +08:00
congqixia
f108ce0fc5
fix: [2.4] Match int8_t and int16_t in Array::get_data (#35579) (#35597)
Cherry-pick from master
pr: #35579
Related to #35578

Previously int16/int8 bitmap index may read int32 array as int16, which
may cause build index with half of the data(if array is full) and half
zeros. This causes BITMAP index lost information.

This PR matches int8_t & int16_t while `get_data` when building index.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-21 17:47:01 +08:00
wei liu
4bf4cbad85
enhance: Mark query node as read only after suspend (#35492) (#35586)
issue: #34985 #35493
pr: #35492
after querynode has been suspended, it's not allow to load
segment/channel on it, which means the node is read only. to be
compatible with resource group design, after query node has been
suspend, we remove it from it's original resource group, make it a read
only query node in replica. then two things will happens:
1. it's original resource group will be lacking of query nodes, query
coord will assign new node to it.
2. querycoord will try to move out all segments/channels after querynode
has been suspended

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-20 19:00:56 +08:00
wei liu
4610dafb2e
enhance: make configure load param feature be compatible with old sdk(#35520) (#35573)
issue: #31570 #35521
pr: #35520 #35546

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-20 18:20:57 +08:00
Chun Han
cf8494ef45
enhance: support httpv1/v2 throttle and add it for httpV2(#35350) (#35504)
related: #35350
pr: https://github.com/milvus-io/milvus/pull/35470

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-08-20 16:32:56 +08:00
wei liu
fc344d1eae
fix: Fix proxy panic after Import related api failed (#35540) (#35559)
issue: #35539
pr: #35540

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-19 19:42:55 +08:00
wei liu
14ec3dc357
enhance: Enable ReadOnly/ReadWrite/Admin Privilege Group to simplify RBAC grant progress (#35472) (#35543)
issue: #35471
pr: #35472 #35515

---------

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-19 16:24:54 +08:00
XuanYang-cn
4a5e6bc6f6
enhance: Add integration tests for l0 (#35430)
pr: #35429
See also: #34796

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-08-19 10:54:55 +08:00
sre-ci-robot
5da1a36180
[automated] Bump milvus version to v2.4.9 (#35537)
Bump milvus version to v2.4.9
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-08-19 10:32:55 +08:00
wei liu
248a6ea401
enhance: Add BackupRBAC/RestoreRBAC API to enable rbac backup (#35444) (#35513)
issue: #35443
pr: #35444

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-18 13:42:55 +08:00
Alexander Guzhva
8ddc3fd463
enhance: [2.4] Improve bitset performance for AVX512 (#35480)
see #35478
pr: #35479

Signed-off-by: Alexandr Guzhva <alexanderguzhva@gmail.com>
2024-08-17 03:58:56 +08:00
sre-ci-robot
9717185f56
[automated] Update cpu Builder image changes (#35509)
Update cpu Builder image changes
See changes:
90127b948a
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-08-16 20:34:58 +08:00
congqixia
4e098f1b23
enhance: Bump milvus & proto to v2.4.9 (#35516)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-16 16:24:54 +08:00
congqixia
bd222e58eb
enhance: [2.4] Exclude L0 segment from readable snapshot (#35510)
Cherry-pick from master
pr: #35507

L0 segments now do not contain insert data and may cause confusion for
query hook optimizer if counted as sealed segment number.

This PR add segment level flag in segment entry and exclude L0 segments
while get readable segment snapshot

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-16 15:26:54 +08:00
congqixia
555b7a6aa7
enhance: [2.4] Remove duplicated schema helper creation in proxy (#35489) (#35502)
Cherry-pick from master
pr: #35489
Related to PRs of #35415

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-16 14:32:54 +08:00
yellow-shine
90127b948a
enhance: [skip e2e] enable ubuntu20.04 os (#35457)
https://github.com/milvus-io/milvus/pull/35482

Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-08-16 10:06:58 +08:00
cai.zhang
09aea3fbf1
enhance: [cherry-pick] Optimize the use of locks and avoid double flush clustering buffer writer (#35490)
issue: #35436 

master pr: #35486

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-08-16 02:24:59 +08:00
SimFG
4fae5407dd
enhance: [2.4] remove the invalid log (#35473)
/kind improvement
pr: #35465

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-08-15 14:14:53 +08:00
wayblink
8c29c5bb23
enhance: [cherry-pick] Add clustering compaction user guide doc (#35428)
issue: #30633
master pr: #35427

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-08-15 10:30:58 +08:00
congqixia
c8926aee23
enhance: [2.4] Support dynamic field in SchemaHelper (#35461) (#35469)
Cherry-pick from master
pr: #35461
Related to #35415

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-15 10:02:54 +08:00
Ted Xu
57d4bcbf15
enhance: adding the msgchannel section in generated yaml (#35466)
See #32168

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-08-14 19:03:11 +08:00
sre-ci-robot
573de289ce
[automated] Bump milvus version to v2.4.8 (#35468)
Bump milvus version to v2.4.8
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-08-14 18:04:59 +08:00
congqixia
e44c63883b
enhance: Bump milvus & proto to v2.4.8 (#35460)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-14 12:52:52 +08:00
XuanYang-cn
0fb3bf43cd
enhance: Init ChannelCP when creating a channel (#35388)
pr: #35387

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-08-14 10:16:58 +08:00
Ted Xu
ce53e79f12
fix: enable milvus.yaml check (#34567) (#35446)
See #32168

pr: #34567 #35152

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-08-13 19:00:23 +08:00
Bingyi Sun
acd3c53bda
fix: fix sparse float vector column warmup (#35437)
https://github.com/milvus-io/milvus/issues/35438

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-08-13 15:18:20 +08:00
aoiasd
a20cb727eb
enhance:[Cherry-pick] Check by proxy rate limiter when delete get data by query. (#30891) (#35262)
relate: https://github.com/milvus-io/milvus/issues/30927
pr: https://github.com/milvus-io/milvus/pull/30891

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-08-13 14:32:21 +08:00
wei liu
8cd6718672
enhance: limit getSegmentInfo batch size to avoid excced grpc message limit (#35432)
issue: #35395 
pr: #35394

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-13 11:42:19 +08:00
wei liu
b316040634
fix: force update next target if target can't be loaded (#35366)
issue: #35361
pr: #35365

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-13 10:24:20 +08:00
yihao.dai
b371467604
enhance: Remove the check for channel cp lag when generating compaction plan (#35383) (#35384)
issue: https://github.com/milvus-io/milvus/issues/35382

pr: https://github.com/milvus-io/milvus/pull/35383

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-08-13 10:02:20 +08:00
smellthemoon
23052f6ac2
fix: upsert use the previous pk in insert when autoid(#34672) (#35130)
issue: #34668 
pr: #34672

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-08-12 18:58:25 +08:00
congqixia
bafc5a831a
enhance: [2.4] Remove stackerror imported unexpected (#35256) (#35330)
Cherry-pick from master
pr: #35256
`stackerror` pkg is imported by accident and could be replaced by
cockroachdb errors lib. This PR removes import line and fix this
problem.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-12 11:12:20 +08:00
zhagnlu
36e6cba771
enhance: optimize index interface (#35363)
pr: #35314 
cherry-pick part  code from commit

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-08-10 20:18:19 +08:00
Bingyi Sun
c47d039f1e
fix: fix empty indices of sparse float (#35400)
https://github.com/milvus-io/milvus/issues/35401

Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-08-09 20:52:18 +08:00
jaime
623f9d3c53
fix: incorrect memory size and delta count in SegmentInfo (#35370)
issue: #35368
pr: #35369

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-08-09 20:34:19 +08:00
congqixia
e3cb994ca3
enhance: [2.4] Fix progress always zero due to integer divide (#35393)
Cherry-pick from master
pr: #35374
See also #35183

Signed-off-by: Congqi.Xia <congqi.xia@zilliz.com>
2024-08-09 17:20:18 +08:00
wei liu
a4fa9ced16
fix: delete by expr failed at retry progress (#35241) (#35268)
issue: #35240
pr: #35241
delete by expr shard the same err object between channels, so if one
channel's query failed, it will fail all channel, which will break
channel level retry policy, and make delete operation failed.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-09 16:12:18 +08:00
wayblink
f7a3fd8bbf
enhance:[cherry-pick] improve ut cov of clustering compaction task (#35243)
issue: #34792
master pr: #35242

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-08-09 15:26:18 +08:00
Patrick Weizhi Xu
96c53bcf2b
fix: [2.4] rollback grpc to 1.50.1 (#35386)
issue: #35298 
pr: #35385

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
(cherry picked from commit faf151050a50f1565b9e8cc48c754e30ec5c65d8)
2024-08-09 14:06:18 +08:00
congqixia
537a817be9
fix: [2.4] Use k locations only for basic BF test location (#35381)
Cherry-pick from master
pr: #35380
Related to #35379

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-09 07:52:22 +08:00
XuanYang-cn
5cd462bc9e
enhance: Init channel cp metric when DataCoord recover (#35352)
pr: #35353

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-08-08 12:22:18 +08:00
sre-ci-robot
6c0ca98644
[automated] Bump milvus version to v2.4.7 (#35362)
Bump milvus version to v2.4.7
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-08-08 11:42:18 +08:00
yellow-shine
ddb52e71cb
enhance: pinpoint revision of 3thirdparty package (#35316)
https://github.com/milvus-io/milvus/pull/35325

---------

Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-08-08 09:46:22 +08:00
zhagnlu
db9e4b898a
enhance: support bitmap index (#35336)
pr: #32902
cherry-pick bitmap index from master

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-08-07 20:12:18 +08:00
wei liu
b22f3a6276
fix: datacoord binlog size metrics include dropped segment (#35339)
issue: #35337
pr: #35338

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-07 15:12:17 +08:00
Patrick Weizhi Xu
5369ada091
fix: [2.4] init telemetry fails when set otlp as trace exporter (#35299)
issue: #35298
pr: #35301
2024-08-07 14:20:21 +08:00
wei liu
0201e00a2f
enhance: enable to set load config in cluster level (#35293)
issue: #35170
pr: #35169
This PR enable to set load configs in cluster level, such as replicas
and resource groups. then when load collections will use the load
config.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-07 12:38:21 +08:00
cai.zhang
4f0c1982d3
fix: [cherry-pick] Fix the issue of missing stats log after clustering compaction (#35267)
issue: #35265 

master pr: #35266

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-08-07 12:08:16 +08:00
Jiquan Long
a00688c00a
fix: comparision operations between incompatible operands (#35264) (#35307)
fix: https://github.com/milvus-io/milvus/issues/34139
pr: https://github.com/milvus-io/milvus/pull/35264

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-08-07 11:24:16 +08:00