test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-03 12:29:36 +08:00
Code Issues Actions 13 Packages Projects Releases Wiki Activity
4682bba586
milvus/internal/proxy
History
wei liu 2a4c00de9d
enhance: Decouple shard client manager from shard cache (#37371) 
issue: #37115
the old implementation update shard cache and shard client manager at
same time, which causes lots of conor case due to concurrent issue
without lock.

This PR decouple shard client manager from shard cache, so only shard
cache will be updated if delegator changes. and make sure shard client
manager will always return the right client, and create a new client if
not exist. in case of client leak, shard client manager will purge
client in async for every 10 minutes.

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-11-12 10:30:28 +08:00
..
accesslog enhance: Forbid access log print error message with line break (#34491) 2024-08-20 17:36:55 +08:00
connection enhance: timer instead of ticker (#36133) 2024-09-10 15:23:06 +08:00
authentication_interceptor_test.go fix: Unify hook singleton implementation in proxy (#34887) 2024-07-26 18:07:53 +08:00
authentication_interceptor.go enhance: make the auth error message more suitable (#32253) 2024-04-16 16:47:19 +08:00
cgo_util_test.go enhance: add vector index mgr to remove vector index type dependency (#36843) 2024-10-17 22:15:25 +08:00
cgo_util.go enhance: optimize milvus core building (#35610) 2024-08-23 12:35:02 +08:00
channels_mgr_test.go enhance: make sure stream closed (#29456) 2023-12-26 19:56:47 +08:00
channels_mgr.go feat: add more operation detail info for better allocation (#30438) 2024-03-28 06:33:11 +08:00
channels_time_ticker_test.go Format the code (#27275) 2023-09-21 09:45:27 +08:00
channels_time_ticker.go Format the code (#27275) 2023-09-21 09:45:27 +08:00
condition_test.go Format the code (#27275) 2023-09-21 09:45:27 +08:00
condition.go Propagate task ctx done err for proxy (#27892) 2023-10-26 19:06:13 +08:00
count_reducer_test.go Format the code (#27275) 2023-09-21 09:45:27 +08:00
count_reducer.go feat: add more operation detail info for better allocation (#30438) 2024-03-28 06:33:11 +08:00
data_coord_mock_test.go enhance: Remove import v1 (#31403) 2024-03-22 15:29:09 +08:00
database_interceptor_test.go enhance: Update protobuf-go to protobuf-go v2 (#34394) 2024-07-29 11:31:51 +08:00
database_interceptor.go fix: add more requests to the database interceptor (#30452) 2024-02-02 16:49:06 +08:00
default_limit_reducer.go Improve the error message for getting all indexes of collection (#27389) 2023-10-08 21:23:32 +08:00
dummyreq_test.go Format the code (#27275) 2023-09-21 09:45:27 +08:00
dummyreq.go [skip ci]Update license for proxy dummy (#11605) 2021-11-10 23:43:27 +08:00
hook_interceptor_test.go enhance: set the rpc error code to avoid the invalid retry (#37024) 2024-10-22 10:33:26 +08:00
hook_interceptor.go enhance: set the rpc error code to avoid the invalid retry (#37024) 2024-10-22 10:33:26 +08:00
http_req_impl_test.go feat: add tasks page into management WebUI (#37002) 2024-10-28 10:13:29 +08:00
http_req_impl.go feat: add tasks page into management WebUI (#37002) 2024-10-28 10:13:29 +08:00
impl_test.go fix: Revert "enhance: Support db for bulkinsert (#37012)" (#37420) 2024-11-07 17:02:25 +08:00
impl.go enhance: Invalidate collection cache when release collection (#37577) 2024-11-12 10:16:29 +08:00
interface_def.go Decoupling client and server API in types interface (#27186) 2023-09-26 09:57:25 +08:00
lb_balancer.go enhance: Decouple shard client manager from shard cache (#37371) 2024-11-12 10:30:28 +08:00
lb_policy_test.go enhance: Decouple shard client manager from shard cache (#37371) 2024-11-12 10:30:28 +08:00
lb_policy.go enhance: Decouple shard client manager from shard cache (#37371) 2024-11-12 10:30:28 +08:00
look_aside_balancer_test.go enhance: Decouple shard client manager from shard cache (#37371) 2024-11-12 10:30:28 +08:00
look_aside_balancer.go enhance: Decouple shard client manager from shard cache (#37371) 2024-11-12 10:30:28 +08:00
management_test.go enhance: add restful api to trigger component stop (#32076) 2024-06-07 10:35:54 +08:00
management.go enhance: Mark query node as read only after suspend (#35492) 2024-08-20 14:02:54 +08:00
meta_cache_adapter_test.go enhance: Avoid initializing casbin enforcer for each request (#29117) 2023-12-12 10:36:43 +08:00
meta_cache_adapter.go fix: the panic when db isn't existed in the rate limit interceptor (#33244) 2024-05-22 15:57:39 +08:00
meta_cache_test.go enhance: Decouple shard client manager from shard cache (#37371) 2024-11-12 10:30:28 +08:00
meta_cache.go enhance: Decouple shard client manager from shard cache (#37371) 2024-11-12 10:30:28 +08:00
metrics_info_test.go feat: management WebUI homepage (#36822) 2024-10-23 11:29:28 +08:00
metrics_info.go enhance: remove cooling off in rate limiter for read requests (#35935) 2024-09-04 14:39:10 +08:00
mock_cache.go enhance: Decouple shard client manager from shard cache (#37371) 2024-11-12 10:30:28 +08:00
mock_channels_manager.go fix: fix make generate-mockery panic with go1.22 (#36830) 2024-10-17 12:11:31 +08:00
mock_lb_balancer.go enhance: Decouple shard client manager from shard cache (#37371) 2024-11-12 10:30:28 +08:00
mock_lb_policy.go fix: fix make generate-mockery panic with go1.22 (#36830) 2024-10-17 12:11:31 +08:00
mock_msgstream_test.go Support to replicate the mq message (#27240) 2023-10-20 14:26:09 +08:00
mock_shardclient_manager.go enhance: Decouple shard client manager from shard cache (#37371) 2024-11-12 10:30:28 +08:00
mock_test.go feat: record the duration waiting in the proxy queue (#34744) 2024-07-23 14:23:52 +08:00
mock_tso_test.go fix: fix make generate-mockery panic with go1.22 (#36830) 2024-10-17 12:11:31 +08:00
msg_pack_test.go feat: Support field partial load collection (#35416) 2024-08-20 16:49:02 +08:00
msg_pack.go enhance: Enable linux code checker (#35084) 2024-07-30 15:53:51 +08:00
OWNERS [skip ci]Update OWNERS files (#11898) 2021-11-16 15:41:11 +08:00
privilege_cache_test.go enhance: Add in-memory cache for casbin enforcer result (#35271) 2024-08-05 18:48:15 +08:00
privilege_cache.go fix: Re-read value after once initialization (#35642) 2024-08-22 17:24:58 +08:00
privilege_interceptor_test.go fix: fix ReadWrite privilege group deny all global API (#36144) 2024-09-13 10:33:08 +08:00
privilege_interceptor.go fix: fix ReadWrite privilege group deny all global API (#36144) 2024-09-13 10:33:08 +08:00
proxy_rpc_test.go Refine Proxy errors (#27499) 2023-10-09 10:09:33 +08:00
proxy_test.go fix: Revert "enhance: Support db for bulkinsert (#37012)" (#37420) 2024-11-07 17:02:25 +08:00
proxy.go enhance: remove cooling off in rate limiter for read requests (#35935) 2024-09-04 14:39:10 +08:00
rate_limit_interceptor_test.go enhance: support httpv1/v2 throttle and add it for httpV2(#35350) (#35470) 2024-08-20 16:16:55 +08:00
rate_limit_interceptor.go enhance: support httpv1/v2 throttle and add it for httpV2(#35350) (#35470) 2024-08-20 16:16:55 +08:00
reducer_test.go Format the code (#27275) 2023-09-21 09:45:27 +08:00
reducer.go Fill output fields for count request (#27967) 2023-10-30 10:00:12 +08:00
repack_func_test.go Format the code (#27275) 2023-09-21 09:45:27 +08:00
repack_func.go Support to replicate the mq message (#27240) 2023-10-20 14:26:09 +08:00
replicate_stream_manager_test.go fix: Fix the unstable unit test TestReplicateManager (#28718) 2023-11-24 17:50:30 +08:00
replicate_stream_manager.go Support to replicate the mq message (#27240) 2023-10-20 14:26:09 +08:00
reScorer_test.go enhance:Refactor hybrid search (#32020) 2024-04-09 14:21:18 +08:00
reScorer.go fix: add rescorer activation function for BM25 (#37481) 2024-11-07 12:08:25 +08:00
rootcoord_mock_test.go enhance: Support custom privilege group for RBAC (#37087) 2024-11-09 08:44:28 +08:00
roundrobin_balancer_test.go enhance: Enable dynamic update replica selection policy (#35860) 2024-09-13 17:05:15 +08:00
roundrobin_balancer.go enhance: Decouple shard client manager from shard cache (#37371) 2024-11-12 10:30:28 +08:00
rpc_msg.go Add slow log for read (#23272) 2023-04-14 15:30:29 +08:00
search_reduce_util_test.go enhance: simplify reduction on single search result (#36334) 2024-09-20 11:59:10 +08:00
search_reduce_util.go fix: change pymilvus version for hybridsearch-groupby(#36407) (#36451) 2024-09-24 14:29:13 +08:00
search_util.go enhance:refine group_strict_size parameter(#37482) (#37483) 2024-11-12 09:56:28 +08:00
segment_test.go fix: wrong context passing into NewClient, error handling lost in session_util (#30817) 2024-02-28 10:40:09 +08:00
segment.go enhance:change some logs (#29579) 2024-01-05 16:12:48 +08:00
shard_client_test.go enhance: Decouple shard client manager from shard cache (#37371) 2024-11-12 10:30:28 +08:00
shard_client.go enhance: Decouple shard client manager from shard cache (#37371) 2024-11-12 10:30:28 +08:00
simple_rate_limiter_test.go enhance: enable flush rate limiter of collection level (#33837) 2024-06-24 10:52:03 +08:00
simple_rate_limiter.go fix: Fix DB limiter nodes are mistakenly cleaned up (#35991) 2024-09-05 17:35:04 +08:00
task_alias_test.go fix: execute method should return an error when result is a failure (#34813) 2024-07-20 23:13:40 +08:00
task_alias.go enhance: Enable linux code checker (#35084) 2024-07-30 15:53:51 +08:00
task_database_test.go enhance: expose DescribeDatabase api in proxy (#32732) 2024-05-09 22:51:30 +08:00
task_database.go feat: add the rbac msg and send them to the replicate channel (#35392) 2024-08-15 12:06:52 +08:00
task_delete_streaming.go fix: split delete task msg to MaxMessageSize to avoid mq message too large error (#36197) 2024-09-27 18:15:19 +08:00
task_delete_test.go feat: The expression supports filling elements through templates (#37033) 2024-10-31 14:20:22 +08:00
task_delete.go feat: The expression supports filling elements through templates (#37033) 2024-10-31 14:20:22 +08:00
task_flush_streaming.go enhance: using streaming service in insert/upsert/flush/delete/querynode (#35406) 2024-08-29 10:03:08 +08:00
task_flush.go enhance: using streaming service in insert/upsert/flush/delete/querynode (#35406) 2024-08-29 10:03:08 +08:00
task_index_test.go enhance: add unify vector index config management (#36846) 2024-11-01 06:18:21 +08:00
task_index.go fix: optimize invalid datatype error msg (#37376) 2024-11-04 11:46:28 +08:00
task_insert_streaming.go enhance: add create segment message, enable empty segment flush (#37407) 2024-11-08 10:16:34 +08:00
task_insert_test.go fix: Fix conan gdal building problem (#37338) 2024-10-31 21:04:16 +08:00
task_insert.go fix: iaccurate size estimation for encoded array data (#36373) 2024-09-24 14:51:14 +08:00
task_policies_test.go enhance: Decouple shard client manager from shard cache (#37371) 2024-11-12 10:30:28 +08:00
task_policies.go enhance: Decouple shard client manager from shard cache (#37371) 2024-11-12 10:30:28 +08:00
task_query_test.go enhance: Decouple shard client manager from shard cache (#37371) 2024-11-12 10:30:28 +08:00
task_query.go feat: Tokenizer support build with params and clone for concurrency (#37048) 2024-11-06 17:48:24 +08:00
task_scheduler_test.go enhance: Enable linux code checker (#35084) 2024-07-30 15:53:51 +08:00
task_scheduler.go feat: record the duration waiting in the proxy queue (#34744) 2024-07-23 14:23:52 +08:00
task_search_test.go enhance: Decouple shard client manager from shard cache (#37371) 2024-11-12 10:30:28 +08:00
task_search.go feat: Tokenizer support build with params and clone for concurrency (#37048) 2024-11-06 17:48:24 +08:00
task_statistic_test.go enhance: Decouple shard client manager from shard cache (#37371) 2024-11-12 10:30:28 +08:00
task_statistic.go enhance: Update protobuf-go to protobuf-go v2 (#34394) 2024-07-29 11:31:51 +08:00
task_test.go enhance: rename tokenizer to analyzer and check analyzer params (#37478) 2024-11-10 16:12:26 +08:00
task_upsert_streaming.go fix: split delete task msg to MaxMessageSize to avoid mq message too large error (#36197) 2024-09-27 18:15:19 +08:00
task_upsert_test.go enhance: Enable linux code checker (#35084) 2024-07-30 15:53:51 +08:00
task_upsert.go fix: unhandle error in upsert task (#36604) 2024-09-30 12:59:16 +08:00
task.go enhance:refine group_strict_size parameter(#37482) (#37483) 2024-11-12 09:56:28 +08:00
timestamp_test.go Format the code (#27275) 2023-09-21 09:45:27 +08:00
timestamp.go enhance:Remove msgbase unnecessary assignments (#28511) 2023-11-24 15:02:39 +08:00
trace_log_interceptor_test.go enhance: Support to trace restful request and request error (#28685) 2023-11-27 20:14:26 +08:00
trace_log_interceptor.go enhance: refine access log to support format access log by yaml and print name info. (#28319) 2023-11-28 15:32:31 +08:00
type_def.go [skip e2e] Add comment for pChan (#14098) 2021-12-23 21:44:12 +08:00
util_test.go enhance: rename tokenizer to analyzer and check analyzer params (#37478) 2024-11-10 16:12:26 +08:00
util.go enhance: rename tokenizer to analyzer and check analyzer params (#37478) 2024-11-10 16:12:26 +08:00
validate_util_test.go fix: not check json data when set nullable or default_value (#36575) 2024-09-29 20:39:15 +08:00
validate_util.go fix: Fix conan gdal building problem (#37338) 2024-10-31 21:04:16 +08:00