test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-04 21:09:06 +08:00
Code Issues Actions 14 Packages Projects Releases Wiki Activity
20,121 Commits 96 Branches 105 Tags 186 MiB
34873a7f76
Commit Graph

50 Commits

Author SHA1 Message Date
wei liu
ba02d54a30
enhance: update shard leader cache when leader location changed (#32470) 
issue: #32466

this PR enhance that when shard location changed, update proxy's shard
leader cache. in case of query node failover case, proxy can find
replica recover

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-05-08 10:05:29 +08:00
SimFG
c012e6786f
feat: support rate limiter based on db and partition levels (#31070) 
issue: https://github.com/milvus-io/milvus/issues/30577
co-author: @jaime0815

---------

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
Signed-off-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
 2024-04-12 16:01:19 +08:00
SimFG
b1a1cca10b
feat: add more operation detail info for better allocation (#30438) 
issue: #30436

---------

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-03-28 06:33:11 +08:00
cai.zhang
40ca98f57f
enhance: Skip timestamp allocation when search/query consistency level is eventually (#29773) 
issue: #29772 
1. Skip timestamp allocation when search/query consistency level is
eventually.

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-02-21 09:52:59 +08:00
cai.zhang
d87726e4c7
enhance: Don't expire ShardLeaders cache actively (#29879) 
issue: #29772 

The shardLeaders cache does not actively expire, update the cache when
search/query fails.

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-01-26 14:13:01 +08:00
aoiasd
38746dfc2b
fix: Remove useless lock which cause porxy meta cache recursive lock (#30203) 
relate: https://github.com/milvus-io/milvus/issues/30193

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-01-23 23:56:55 +08:00
aoiasd
a81d2b4780
enhance: refine proxy metacache for concurrent safe (#29872) 
relate: https://github.com/milvus-io/milvus/issues/29675

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-01-22 14:28:55 +08:00
congqixia
4f8c540c77
enhance: cache collection schema attributes to reduce proxy cpu (#29668) 
See also #29113

The collection schema is crucial when performing search/query but some
of the information is calculated for every request.

This PR change schema field of cached collection info into a utility
`schemaInfo` type to store some stable result, say pk field,
partitionKeyEnabled, etc. And provided field name to id map for
search/query services.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-01-04 17:28:46 +08:00
congqixia
bcf8f27aa7
enhance: refine proxy meta cache partition logic (#29315) 
See also #29113

- Unify partition info refresh logic
- Prevent parse partition names for each partition key search request

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-20 10:02:43 +08:00
PowderLi
20fc90c591
enhance: find collection schema from cache (#28782) 
issue: #28781 #28329

1. There is no need to call `DescribeCollection`, if the collection's
schema is found in the globalMetaCache
2. did `GetProperties` to check the access to Azure Blob Service while
construct the ChunkManager

Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-12-03 19:22:33 +08:00
yah01
be980fbc38
Refine state check (#27541) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-11 21:01:35 +08:00
jaime
7f7c71ea7d
Decoupling client and server API in types interface (#27186) 
Co-authored-by:: aoiasd <zhicheng.yue@zilliz.com>

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-09-26 09:57:25 +08:00
yah01
b4f86ea55e
Construct all success status with merr (#27226) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-09-20 10:57:23 +08:00
yah01
3349db4aa7
Refine errors to remove changes breaking design (#26521) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-09-04 09:57:09 +08:00
congqixia
e8f1b1736e
Remove log.Error(err.error())-style log (#26783) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-01 13:09:01 +08:00
wei liu
f9823e039f
fix describe rg with non exist collection (#26227) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-08-10 18:43:16 +08:00
SimFG
4e1b65d38f
Refine the rbac cache update process (#26150) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-08-07 11:59:07 +08:00
chyezh
f915f1739a
[Fixup] Short-term fix metacache data race (#25802) 
Signed-off-by: chyezh <ye.zhen@zilliz.com>
 2023-08-04 17:47:07 +08:00
smellthemoon
9614e61f14
Fix collection and channel not match (#25859) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-08-01 17:33:06 +08:00
yiwangdr
b9189b9f41
Organize mocks from types.go (#25466) 
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
 2023-07-14 10:12:31 +08:00
jaime
18df2ba6fd
[Cherry-Pick] Support Database (#24769) 
Support Database(#23742)
Fix db nonexists error for FlushAll (#24222)
Fix check collection limits fails (#24235)
backward compatibility with empty DB name (#24317)
Fix GetFlushAllState with DB (#24347)
Remove db from global meta cache after drop database (#24474)
Fix db name is empty for describe collection response (#24603)
Add RBAC for Database API (#24653)
Fix miss load the same name collection during recover stage (#24941)

RBAC supports Database validation (#23609)
Fix to list grant with db return empty (#23922)
Optimize PrivilegeAll permission check (#23972)
Add the default db value for the rbac request (#24307)

Signed-off-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
 2023-06-25 17:20:43 +08:00
congqixia
41af0a98fa
Use go-api/v2 for milvus-proto (#24770) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-06-09 01:28:37 +08:00
yah01
ebd0279d3f
Check error by Error() and NoError() for better report message (#24736) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-06-08 15:36:36 +08:00
yihao.dai
3827ac30bc
Remove load cache (#23287) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-05-09 10:36:41 +08:00
congqixia
32be374c33
Fix MetaCache ut internal not changed (#23276) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-04-07 13:20:28 +08:00
jaime
c9d0c157ec
Move some modules from internal to public package (#22572) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-04-06 19:14:32 +08:00
wei liu
ed60b434fa
fix unstable ut (#23070) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-03-28 19:06:01 +08:00
wei liu
6b5dfa6db2
deprecate shard cache immediately if query failed (#22779) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-03-17 16:25:55 +08:00
wei liu
69cc68caff
support first or random strategy (#22766) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-03-16 14:25:55 +08:00
congqixia
3ffa6d5302
Mark cache deprecated instead of removing it (#22674) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-03-10 10:19:53 +08:00
congqixia
d9908ff231
Fix TestMetaCache_ExpireShardLeaderCache unit test (#22651) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-03-09 18:25:52 +08:00
wei liu
1a1a5caafa
update shard leader cache periodically (#22595) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-03-08 15:13:51 +08:00
Enwei Jiao
697dedac7e
Use cockroachdb/errors to replace other error pkg (#22390) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-02-26 11:31:49 +08:00
wei liu
13b508103e
refine mock querycoord (#22198) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-02-16 15:38:34 +08:00
wei liu
73c44d4b29
resource group impl (#21609) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-01-30 10:19:48 +08:00
zhenshan.cao
ce3e0fcf9e
Fixbug: proxy meta_cache partially update (#21232) (#21242) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-12-14 20:45:23 +08:00
SimFG
a55f739608
Separate public proto files (#19782) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-10-16 20:49:27 +08:00
SimFG
d7f38a803d
Separate some proto files (#19218) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-09-16 16:56:49 +08:00
Zach
0c9a10e8f8
1. refine logging interfaces (#18692) 
2. adjust logs for query/search requests

Signed-off-by: Zach41 <zongmei.zhang@zilliz.com>

Signed-off-by: Zach41 <zongmei.zhang@zilliz.com>
 2022-08-23 10:44:52 +08:00
SimFG
ff0200210a
Support Role-Based Access Control (#18425) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-08-04 11:04:34 +08:00
zhenshan.cao
2763efc9b0
Add cache of grpc client of ShardLeader in proxy (#17301) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-06-02 12:16:03 +08:00
bigsheeper
9eeec4a2d5
Add collection load cache and InvalidateCollMetaCache by collID (#16882) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-05-19 10:13:56 +08:00
XuanYang-cn
127dd34b37
Make proxy use roundrobin to choose replica (#17063) 
Fixes: #17055

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-05-17 22:35:57 +08:00
XuanYang-cn
a1480aaf76
Clear shard leaders cache when releasing (#16942) 
See also: #16926

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-05-17 11:11:56 +08:00
congqixia
69252f812d
Implement memory replica in Proxy, QueryNode and QueryCoord (#16470) 
Related to #16298 #16291 #16154
Co-authored-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Letian Jiang <letian.jiang@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-20 16:15:41 +08:00
codeman
4e12271d79
Support login with username and password (#15656) (#16341) 
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
 2022-04-11 19:49:34 +08:00
XuanYang-cn
d4f77253e6
[skip ci]Update license for proxy meta (#11607) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-11-10 23:56:35 +08:00
Xiaofan
619c4393f3
Refine logs in proxy search path(#7357) (#7753) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-09-12 17:54:01 +08:00
紫晴
4c7b57ffc5
Upload codecov report in GitHub Actions (#6565) 
* Upload codecov report in GitHub Actions

Signed-off-by: wangting0128 <ting.wang@zilliz.com>
 2021-07-16 14:52:21 +08:00
Cai Yudong
2926a78968
Rename proxynode to proxy (#5961) 
* rename package name from proxynode to proxy

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move distributed/proxynode to distributed/proxy

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move internal/proxynode to internal/proxy

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename proxynode to proxy

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-06-22 14:40:07 +08:00