milvus/internal/util
yiwangdr 32cff25f97
enhance: decrease coordinator init time (#29822)
This PR mainly improve two items:
1. Target observer should refresh loading status during init time. An
uninitialized loading status blocks search/query. Currently, the target
observer refreshes every 10 seconds, i.e. we'd need to wait for 10s for
no reason. That's also the reason why we constantly see false log
"collection unloaded" upon mixcoord restarts.
2. Delete session when service is stopped. So that the new service
doesn't need to wait for the previous session to expire (~10s).

Item 1 is the major improvement of this PR, which should speed up init
time by 10s.
Item 2 is not a big concern in most cases as coordinators usually shut
down after stop(). In those cases, coordinator restart triggers serverID
change which further triggers an existing logic that deletes expired
session. This PR only fixes rare cases where serverID doesn't change.

integration test:
`go test -tags dynamic -v -coverprofile=profile.out -covermode=atomic
tests/integration/coordrecovery/coord_recovery_test.go -timeout=20m`
Performance after the change:
Average init time of coordinators: 10s
Hardware: M2 Pro
Test setup: 1000 collections with 1000 rows (dim=128) per collection.


issue: #29409

Signed-off-by: yiwangdr <yiwangdr@gmail.com>
2024-02-05 14:00:12 +08:00
..
cgoconverter enhance: move the cgo code in the pkg dir to interal dir (#30261) 2024-01-25 15:15:01 +08:00
componentutil enhance:add some log when create client and get component states (#28160) 2023-11-22 09:12:22 +08:00
dependency enhance: add basic information of milvus into metrics (#29665) 2024-01-29 15:47:02 +08:00
flowgraph fix: Use determined order to lock in BlockAll to avoid deadlock (#29246) 2024-01-22 14:50:56 +08:00
funcutil Format the code (#27275) 2023-09-21 09:45:27 +08:00
grpcclient fix: grpc client check session skipped due to role not match (#29356) 2023-12-21 10:12:51 +08:00
importutil enhance: support varchar autoid when bulkinsert (#30377) 2024-02-01 19:45:09 +08:00
importutilv2 feat: Add import executor and manager for datanode (#29438) 2024-01-31 20:45:04 +08:00
indexcgowrapper enhance: pass partition key scalar info if enabled when build vector index (#29931) 2024-01-24 00:04:55 +08:00
initcore enhance: free CString in InitTraceConfig (#30055) 2024-01-17 15:15:03 +08:00
metrics fix: symbol 'GetStorageMetrics' and 'enableDynamicField' (#28580) 2023-11-21 10:20:22 +08:00
mock fix: Support mvcc with hybrid serach (#30114) 2024-02-01 16:03:03 +08:00
pipeline Format the code (#27275) 2023-09-21 09:45:27 +08:00
proxyutil enhance: Break down the granularity of collection info cache expired (#29977) 2024-01-30 16:45:02 +08:00
segmentutil Format the code (#27275) 2023-09-21 09:45:27 +08:00
sessionutil enhance: decrease coordinator init time (#29822) 2024-02-05 14:00:12 +08:00
streamrpc Add querynode client wrapper and avoid grpc in standalone mode (#27781) 2023-10-19 11:10:07 +08:00
tsoutil tikv integration (#26246) 2023-09-07 07:25:14 +08:00
typeutil feat: integrate storagev2 into loading segment (#29336) 2024-01-12 18:10:51 +08:00
wrappers fix: Support mvcc with hybrid serach (#30114) 2024-02-01 16:03:03 +08:00