test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-04 04:49:08 +08:00
Code Issues Actions 14 Packages Projects Releases Wiki Activity
20,382 Commits 96 Branches 105 Tags 185 MiB
134422eeaf
Commit Graph

808 Commits

Author SHA1 Message Date
congqixia
1cd8d1bd80
enhance: [2.4] Use stats Handler to record request/response size metrics (#36107) (#36118) 
Cherry-pick from master
pr: #36107 
Related to #36102

This PR use newly added `grpcSizeStatsHandler` to reduce calling
`proto.Size` since the request & response size info is recorded by grpc
framework.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-09-10 17:13:08 +08:00
XuanYang-cn
835c9d5c65
fix: Change l0SegmentsRowCount limits to a reasonable value (#36015) 
pr: #36014
See also: #36028

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-09-08 16:55:05 +08:00
congqixia
9d0378ae84
enhance: Bump milvus & proto version to v2.4.11 (#36069) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-09-06 23:39:06 +08:00
congqixia
55b33cd3cf
fix: [2.4] Fix tracing config update logic (#35928) (#35998) 
Cherry-pick from master
pr: #35928 
Related to #35927

There are serveral issue this PR addresses:
- Use `ResetTraceConfig` method instead init one in update event handler
- Implement dynamic stats.Handler to receive tracing config update event
- Update `enable_trace` flag when `ResetTraceConfig` is invoked
- Change `enable_trace` to `std::atomic<bool>` in case of data race

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-09-06 11:19:05 +08:00
Ted Xu
45b2049d5d
fix: fallback params may be overridden (#35972) (#36006) 
See #35756

---------

pr: #35972

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2024-09-05 19:05:05 +08:00
wei liu
ceca666e2a
fix: Fix privilege group hasn't been register for validate (#35938) 
issue: #35471
pr: #35937

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-09-05 18:09:11 +08:00
congqixia
da0bc22a5f
enhance: [2.4] Add delete buffer related quota logic (#35918) (#35997) 
Cherry pick from master
pr: #35128 #35918
See also #35303

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: aoiasd <45024769+aoiasd@users.noreply.github.com>
 2024-09-05 16:43:06 +08:00
jaime
2c1fa50412
enhance: remove cooling off in rate limiter for read requests (#35936) 
issue: #35934
pr: #35935

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-09-04 14:39:10 +08:00
congqixia
268c60c301
fix: [2.4] Check clustering key skip load behavior (#35865) (#35899) 
Cherry-pick from master
pr: #35865
feature issue: #35415
See also #35861

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-09-02 16:03:04 +08:00
SimFG
8b706122a8
enhance: [2.4] support to drop the role which is related the privilege list (#35863) 
- issue: #35545
- pr: #35727

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-08-31 21:57:02 +08:00
congqixia
fedf3a7fe1
enhance: Bump milvus & proto version to v2.4.10 (#35798) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-29 14:55:01 +08:00
jaime
2fc111eb10
enhance: set database properties to restrict read access (#35754) 
issue: #35744
pr: #35745

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-08-29 11:41:01 +08:00
SimFG
fc324b4254
feat: [2.4] add the rbac msg and send them to the replicate channel (#35562) 
- issue: #35391
- pr: #35392

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-08-27 14:45:00 +08:00
congqixia
ab261d0f8b
feat: [2.4] Support field partial load collection (#35416) (#35696) 
Cherry-pick from master
pr: #35416
Related to #35415

---------

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-27 14:07:00 +08:00
Gao
1687d64c46
enhance: [2.4] add hit segment num metrics for queryHook (#35619) 
issue: #35576 
pr: #35577

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2024-08-23 12:49:02 +08:00
SimFG
5b5119a51f
feat: [2.4] provide more general configuration to control mmap behavior (#35609) 
- issue: #35273
- pr: #35359

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-08-23 12:35:02 +08:00
wei liu
e014ad9280
fix: fix dynamic update config doesn't works for some param (#35572) (#35637) 
issue: #35570
pr: #35572
milvus support config cache to spped up config access, but only evict
param's cache when param has been updated. but milvus's param may rely
on other param's value, let's say ParamsA relys on paramsB, when paramsB
updated, it will evict paramB's cache, but the paramA's cache still keep
the old value.

This PR evict all config cache to solve the above issue, cause dynamic
update config won't be much frequetly.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-22 16:00:58 +08:00
wei liu
e2542a1bf5
enhance: Update protobuf-go to protobuf-go v2 (#34394) (#35555) 
issue: #34252
pr: #34394 #35072 #35084

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-21 18:50:58 +08:00
Chun Han
cf8494ef45
enhance: support httpv1/v2 throttle and add it for httpV2(#35350) (#35504) 
related: #35350
pr: https://github.com/milvus-io/milvus/pull/35470

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2024-08-20 16:32:56 +08:00
wei liu
14ec3dc357
enhance: Enable ReadOnly/ReadWrite/Admin Privilege Group to simplify RBAC grant progress (#35472) (#35543) 
issue: #35471
pr: #35472 #35515

---------

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-19 16:24:54 +08:00
wei liu
248a6ea401
enhance: Add BackupRBAC/RestoreRBAC API to enable rbac backup (#35444) (#35513) 
issue: #35443
pr: #35444

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-18 13:42:55 +08:00
congqixia
4e098f1b23
enhance: Bump milvus & proto to v2.4.9 (#35516) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-16 16:24:54 +08:00
congqixia
c8926aee23
enhance: [2.4] Support dynamic field in SchemaHelper (#35461) (#35469) 
Cherry-pick from master
pr: #35461
Related to #35415

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-15 10:02:54 +08:00
Ted Xu
57d4bcbf15
enhance: adding the msgchannel section in generated yaml (#35466) 
See #32168

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2024-08-14 19:03:11 +08:00
congqixia
e44c63883b
enhance: Bump milvus & proto to v2.4.8 (#35460) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-14 12:52:52 +08:00
Ted Xu
ce53e79f12
fix: enable milvus.yaml check (#34567) (#35446) 
See #32168

pr: #34567 #35152

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2024-08-13 19:00:23 +08:00
aoiasd
a20cb727eb
enhance:[Cherry-pick] Check by proxy rate limiter when delete get data by query. (#30891) (#35262) 
relate: https://github.com/milvus-io/milvus/issues/30927
pr: https://github.com/milvus-io/milvus/pull/30891

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-08-13 14:32:21 +08:00
congqixia
bafc5a831a
enhance: [2.4] Remove stackerror imported unexpected (#35256) (#35330) 
Cherry-pick from master
pr: #35256
`stackerror` pkg is imported by accident and could be replaced by
cockroachdb errors lib. This PR removes import line and fix this
problem.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-12 11:12:20 +08:00
zhagnlu
db9e4b898a
enhance: support bitmap index (#35336) 
pr: #32902
cherry-pick bitmap index from master

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-08-07 20:12:18 +08:00
wei liu
0201e00a2f
enhance: enable to set load config in cluster level (#35293) 
issue: #35170
pr: #35169
This PR enable to set load configs in cluster level, such as replicas
and resource groups. then when load collections will use the load
config.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-07 12:38:21 +08:00
Jiquan Long
d14d00b0cc
enhance: add metric to record disk quota (#35305) (#35306) 
https://github.com/milvus-io/milvus/issues/35177
pr: https://github.com/milvus-io/milvus/pull/35305

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2024-08-07 10:24:27 +08:00
wei liu
2ac1bf7532
enhance: Enable setting the replica number and resource group during collection creation (#34403) (#34561) 
issue: #30040
pr: #34403

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-06 15:06:17 +08:00
congqixia
eebd9804d6
enhance: Bump milvus & proto version to v2.4.7 (#35034) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-06 12:40:16 +08:00
cai.zhang
2534b30e39
enhance: [cherry-pick] Add monitoring metrics for task execution time in datacoord (#35141) 
issue: #35138 

master pr: #35139

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-08-02 19:46:16 +08:00
wei liu
2609e95cb5
enhance: add segment state on data coord bin log size metrics (#35153) (#35235) 
pr: #35153

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-02 18:28:14 +08:00
wei liu
d767f8977a
enhance: Refine param init for MmapDirPath (#35181) (#35214) 
pr: #35181

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-02 16:30:15 +08:00
congqixia
f8444b900f
enhance: [2.4] Support proxy/delegator qn client pooling (#35195) 
Cherry pick from master
pr: #35194
See also #35196
Add param item for proxy/delegator query node client pooling and
implement pooling logic

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-02 11:24:19 +08:00
Jiquan Long
ae3657b23e
enhance: add metric to record maxInsertRate and queryNodeMemoryHighWaterLevel (#35185) (#35188) 
pr: https://github.com/milvus-io/milvus/pull/35185
fix: https://github.com/milvus-io/milvus/issues/35177

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2024-08-02 10:16:15 +08:00
wei liu
5f601fcc50
enhance: Reduce delegator memory overloaded factor to 0.1 (#35092) (#35164) 
pr: #35092

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-01 14:20:13 +08:00
Gao
be0123863f
enhance: add channel num for queryHook optimization (#35105) 
pr: #35104

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2024-07-31 18:23:51 +08:00
cai.zhang
c340f387cf
enhance: [cherry-pick] Change the fixed value to a ratio for clustering segment size (#35075) 
issue: #34495 

master pr: #35076

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-07-31 10:32:00 +08:00
congqixia
935a117396
enhance: [2.4] Support otlp http exporter (#35053) (#35073) 
Cherry-pick from master
pr: #35053
See also #35052

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-30 21:00:08 +08:00
Jiquan Long
86edca8c1b
fix: support auto index for array (#35095) 
/kind branch-feature
pr: #34450

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
Co-authored-by: Zhagnlu <lu.zhang@zilliz.com>
 2024-07-30 17:57:50 +08:00
PowderLi
fa9d606aa8
enhance: add metrics for restful apis (#35083) 
issue: #34968
pr: #34969

Signed-off-by: PowderLi <min.li@zilliz.com>
 2024-07-30 14:13:49 +08:00
yihao.dai
cc188fb322
enhance: Avoid panic due to nil schema (#35063) (#35064) 
/kind improvement

issue: https://github.com/milvus-io/milvus/discussions/25620

pr: https://github.com/milvus-io/milvus/pull/35063

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-07-29 20:07:49 +08:00
congqixia
d16320705e
enhance: [2.4] Add Segment Level in milvus segment info APIs (#34763) (#35023) 
Cherry-pick from master
pr: #34763
See also #34746

This PR add segment level field in response of
`GetPersistentSegmentInfo` and `GetQuerySegmentInfo`

---------

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-29 10:11:52 +08:00
wei liu
b3bc7f3985
enhance: Limit collection's normal balance speed (#34810) (#34987) 
issue: #34798
pr: #34810

after we remove the task priority on query coord, to avoid load/release
segment blocked by too much balance task, we limit the balance task size
in each round. at same time, we reduce the balance interval to trigger
balance more frequently.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-07-26 10:13:46 +08:00
Jiquan Long
2822d8729c
feat: record the number of times milvus turns into force-deny-writing states (#34960) (#34989) 
/kind improvement
fix: #34961
pr: #34960 

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2024-07-25 15:43:09 +08:00
jaime
77ae127a62
fix: check collection health(queryable) fail for releasing collection (#34948) 
issue: #34946
pr: #34947

---------

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-07-25 10:25:57 +08:00
cai.zhang
12a24c3957
enhance: [cherry-pick] Refine code for GetRecoveryInfo (#34974) 
issue: #34495 

master pr: #34973

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-07-24 22:01:44 +08:00