test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-04 21:09:06 +08:00
Code Issues Actions 14 Packages Projects Releases Wiki Activity
20,695 Commits 96 Branches 105 Tags 186 MiB
c0c12c6c5b
Commit Graph

1533 Commits

Author SHA1 Message Date
congqixia
51870aef08
enhance: Use stats Handler to record request/response size metrics (#36107) 
Related to #36102

This PR use newly added `grpcSizeStatsHandler` to reduce calling
`proto.Size` since the request & response size info is recorded by grpc
framework.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-09-09 17:13:05 +08:00
jaime
91d23ecbe1
fix: memory leak in proxy meta cache (#36075) 
issue: #36074

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-09-08 17:33:05 +08:00
Chun Han
e480b103bd
feat: supporing hybrid search group_by (#35982) 
related: #35096

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2024-09-08 17:09:04 +08:00
smellthemoon
21b135c7c2
fix: not append valid data when transfer to insert record (#36027) 
fix not append valid data when transfer to insert record and add a tiny
check when in groupBy field.
#35924

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-09-06 14:53:04 +08:00
yihao.dai
8b043f58dc
fix: Fix DB limiter nodes are mistakenly cleaned up (#35991) 
This issue only occurs for a short time right after a table is created.
To avoid this, we simply reduce the frequency of cleaning up invalid
limiter nodes.

issue: https://github.com/milvus-io/milvus/issues/35933

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-09-05 17:35:04 +08:00
wei liu
32e55a02ea
fix: Fix privilege group hasn't been register for validate (#35937) 
issue: #35471

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-09-05 15:35:04 +08:00
congqixia
66ed289a85
enhance: Fix typo of clustering key not loaded msg (#35948) 
Related to #35415

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-09-05 10:49:03 +08:00
jaime
24fb10114b
enhance: remove cooling off in rate limiter for read requests (#35935) 
issue: #35934

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-09-04 14:39:10 +08:00
zhagnlu
325f1987d9
enhance: rewrite index params for compatibility (#35788) 
#32900

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-09-02 19:41:03 +08:00
Chun Han
4641fd9195
enhance: make search groupby stop when reaching topk groups (#35814) 
related: #33544

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2024-09-02 18:25:03 +08:00
congqixia
9d80137698
fix: Check clustering key skip load behavior (#35865) 
feature issue: #35415
See also #35861

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-09-02 11:17:02 +08:00
SimFG
311f860676
enhance: support to drop the role which is related the privilege list (#35727) 
- issue: #35545

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-08-30 15:17:00 +08:00
congqixia
3c8941a88e
enhance: Print log only when rate limit updates (#35806) 
The debug log for "RateLimiter register for rateType" is too frequent
and in e2e cases, the may print 18M times in one run.

This PR make the log be printed only when rate limit is updated.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-29 15:07:00 +08:00
congqixia
985d84d3ee
enhance: Add skip load validation for create collection task (#35737) 
Related to #35415

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-29 10:05:08 +08:00
Zhen Ye
99dff06391
enhance: using streaming service in insert/upsert/flush/delete/querynode (#35406) 
issue: #33285

- using streaming service in insert/upsert/flush/delete/querynode
- fixup flusher bugs and refactor the flush operation
- enable streaming service for dml and ddl
- pass the e2e when enabling streaming service
- pass the integration tst when enabling streaming service

---------

Signed-off-by: chyezh <chyezh@outlook.com>
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-08-29 10:03:08 +08:00
yihao.dai
56b3490a98
fix: Fix rate wasn't limited to the expected value (#35699) 
Each time the rate is reset, the token bucket is fully refilled, causing
the rate wasn't limited to the expected value. This PR addresses the
issue by preventing the token reset.

issue: https://github.com/milvus-io/milvus/issues/35675,
https://github.com/milvus-io/milvus/issues/35702

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-08-28 14:13:00 +08:00
Chun Han
bfd9d86fe9
feat: support groupby size on go-layer(#33544) (#33845) 
related: #33544

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2024-08-27 14:21:00 +08:00
SimFG
3e1052f889
enhance: use the msg position obj when getting replicate channel position (#35606) 
/kind improvement

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-08-27 10:28:59 +08:00
Zhen Ye
a773836b89
enhance: optimize milvus core building (#35610) 
issue: #35549,#35611,#35633

- remove milvus_segcore milvus_indexbuilder..., add libmilvus_core
- core building only link once
- move opendal compilation into cmake
- fix odr

---------

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-08-23 12:35:02 +08:00
congqixia
9fff07936a
fix: Re-read value after once initialization (#35642) 
Related to #35641
See also #35271

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-22 17:24:58 +08:00
zhagnlu
3107701fe8
enhance: optimize retrieve on dynamic field (#35580) 
#35514

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
Co-authored-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2024-08-22 14:24:56 +08:00
SimFG
731d45abbe
enhance: provide more general configuration to control mmap behavior (#35359) 
- issue: #35273

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-08-21 00:22:54 +08:00
aoiasd
bc2bbdc82f
enhance: Forbid access log print error message with line break (#34491) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-08-20 17:36:55 +08:00
congqixia
2fbc628994
feat: Support field partial load collection (#35416) 
Related to #35415

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-20 16:49:02 +08:00
Chun Han
031ee6f155
enhance: support httpv1/v2 throttle and add it for httpV2(#35350) (#35470) 
related: #35350

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2024-08-20 16:16:55 +08:00
wei liu
e09dc3be58
enhance: Mark query node as read only after suspend (#35492) 
issue: #34985 #35493
after querynode has been suspended, it's not allow to load
segment/channel on it, which means the node is read only. to be
compatible with resource group design, after query node has been
suspend, we remove it from it's original resource group, make it a read
only query node in replica. then two things will happens:
1. it's original resource group will be lacking of query nodes, query
coord will assign new node to it.
2. querycoord will try to move out all segments/channels after querynode
has been suspended

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-20 14:02:54 +08:00
wei liu
14ef88f423
fix: Fix proxy panic after Import related api failed (#35540) 
issue: #35539

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-19 15:22:55 +08:00
wei liu
a570567644
enhance: Enable ReadOnly/ReadWrite/Admin Privilege Group to simplify RBAC grant progress (#35472) 
issue: #35471

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-16 14:18:54 +08:00
wei liu
1d49358f82
enhance: Add BackupRBAC/RestoreRBAC API to enable rbac backup (#35444) 
issue: #35443

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-16 10:10:53 +08:00
congqixia
fe20366b5c
enhance: Remove duplicated schema helper creation in proxy (#35489) 
Related to PRs of #35415

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-15 19:18:53 +08:00
SimFG
b2cc4b0776
feat: add the rbac msg and send them to the replicate channel (#35392) 
- issue: #35391

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-08-15 12:06:52 +08:00
SimFG
0400eb91c1
enhance: remove the invalid log (#35465) 
/kind improvement

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-08-14 18:00:53 +08:00
Jiquan Long
f13b21c041
fix: error message when input is not aligned (#35322) 
fix: https://github.com/milvus-io/milvus/issues/35321

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2024-08-07 09:56:28 +08:00
congqixia
a2b517523d
enhance: Add in-memory cache for casbin enforcer result (#35271) 
See also #35270

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-05 18:48:15 +08:00
wei liu
c22b0a636e
fix: delete by expr failed at retry progress (#35241) 
issue: #35240
delete by expr shard the same err object between channels, so if one
channel's query failed, it will fail all channel, which will break
channel level retry policy, and make delete operation failed.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-05 11:18:16 +08:00
congqixia
c64a078458
enhance: Support proxy/delegator qn client pooling (#35194) 
See also #35196

Add param item for proxy/delegator query node client pooling and
implement pooling logic

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-02 11:24:19 +08:00
smellthemoon
6106a48acb
fix: upsert result use the previous pk (#34672) 
#34668

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-07-31 15:25:51 +08:00
congqixia
de8a266d8a
enhance: Enable linux code checker (#35084) 
See also #34483

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-30 15:53:51 +08:00
Patrick Weizhi Xu
3c7f73137e
fix: disallow expr when partition key isolation is enabled (#35031) 
issue: #34336 

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
(cherry picked from commit 2889481ce9a14230e9c7f1c8f9c3c1decde77e03)
 2024-07-29 14:21:50 +08:00
wei liu
c45f38aa61
enhance: Update protobuf-go to protobuf-go v2 (#34394) 
issue: #34252

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-07-29 11:31:51 +08:00
congqixia
783f9d9c33
fix: Unify hook singleton implementation in proxy (#34887) 
Related to #34885

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-26 18:07:53 +08:00
congqixia
94d8f48b85
enhance: Use auto detection proto interface to fetch grpc size (#35024) 
Related to #34763

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-26 14:23:47 +08:00
congqixia
4ee6c69217
enhance: Add Segment Level in milvus segment info APIs (#34763) 
See also #34746

This PR add segment level field in response of
`GetPersistentSegmentInfo` and `GetQuerySegmentInfo`

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-26 10:01:46 +08:00
chyezh
d93c51d5f6
enhance: change GetVChannels rpc into GetPChannelInfo (#34940) 
issue: #33285

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-07-25 13:29:45 +08:00
congqixia
e2f40fc2a8
fix: Check legacy guarantee ts when skipping alloc ts (#34981) 
See also #34980

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-25 10:17:45 +08:00
Jiquan Long
a2ac84bd64
feat: record the duration waiting in the proxy queue (#34744) 
fix: https://github.com/milvus-io/milvus/issues/34743

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2024-07-23 14:23:52 +08:00
yihao.dai
8aab6cbfac
enhance: Organize the common modules of streamingNode and dataNode (#34773) 
1. Move the common modules of streamingNode and dataNode to flushcommon
2. Add new GetVChannels interface for rootcoord

issue: https://github.com/milvus-io/milvus/issues/33285

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-07-22 11:33:51 +08:00
SimFG
88c38a3756
fix: execute method should return an error when result is a failure (#34813) 
- issue: #34812

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-07-20 23:13:40 +08:00
zhagnlu
fbad77f22a
fix: fix ut test fail (#34832) 
#34831

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-07-19 17:51:40 +08:00
congqixia
e4e18cb8c3
enhance: Add dynamic cgo pool for proxy CGO call (#34768) 
Related to #34705

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-19 15:11:41 +08:00