Commit Graph

23 Commits

Author SHA1 Message Date
jaime
f48a7ff8ff
enhance: use Delete instead of DeletePartialMatch to remove metrics (#33029)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-05-14 18:49:33 +08:00
jaime
ba625835bc
enhance: Add metrics for segment index files size (#32979)
issue:#32980

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-05-13 17:59:32 +08:00
jaime
3b5cbed86a
enhance: add db label for deletion and bulkinsert operation (#32611)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-04-26 15:53:29 +08:00
chyezh
2586c2f1b3
enhance: use WalkWithPrefix api for oss, enable piplined file gc (#31740)
issue: #19095,#29655,#31718

- Change `ListWithPrefix` to `WalkWithPrefix` of OOS into a pipeline
mode.

- File garbage collection is performed in other goroutine.

- Segment Index Recycle clean index file too.

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2024-04-25 20:41:27 +08:00
jaime
d4fd6c7283
enhance: add db label on binlog size metrics (#32003)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-04-10 21:01:20 +08:00
yihao.dai
23ba2a5388
enhance: Remove deprecated DataCoordNumStoredRowsCounter metric (#31778)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-04-07 00:25:15 +08:00
jaime
bd853be8c7
enhance: Add db label for some usual metrics (#30956)
issue: #31782

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-04-02 14:27:13 +08:00
Jiquan Long
dc2cdbe387
enhance: add more metrics (#31271)
/kind improvement
fix: #31272 

This pr add more metrics, which are:
- Slow query count, which the duration considered as slow can be
configurable;
- Number of deleted entities;
- Number of entities imported;
- Number of entities per collection;
- Number of loaded entities per collection;
- Number of indexed entities;
- Number of indexed entities, per collection, per index and whether it's
a vetor index;
- Quota states (LongTimeTickDelay, MemoryExhuasted, DiskQuotaExhuasted)
per database;

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-03-19 15:23:06 +08:00
yihao.dai
a434d33e75
feat: Add import scheduler and manager (#29367)
This PR introduces novel managerial roles for importv2:
1. ImportMeta: To manage all the import tasks;
2. ImportScheduler: To process tasks and modify their states;
3. ImportChecker: To ascertain the completion of all tasks and instigate
relevant operations.

issue: https://github.com/milvus-io/milvus/issues/28521

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-01 18:31:02 +08:00
XuanYang-cn
fd19e419f9
fix: Use size bucket for compacted segment size metric (#30028)
See also: #29204

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-01-26 10:53:02 +08:00
XuanYang-cn
bebe1baf18
fix: Reigster CompactionTaskNum metrics (#29640)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-01-04 17:16:45 +08:00
XuanYang-cn
4b406e5973
enhance: Add CompactionTaskNum metrics (#29518)
See also: #27606

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-12-28 15:46:55 +08:00
aoiasd
010d8362ad
fix: datanode L0 segment num wrong (#29050)
relate:  https://github.com/milvus-io/milvus/issues/27675

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-12-20 11:20:44 +08:00
congqixia
cbf0f9c527
enhance: change cp metric to absolute unix ts (#29328)
See also #29327

Change channel checkpoint metrics to unix seconds instead of checkpoint
timestamp lag value

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-20 11:04:45 +08:00
congqixia
dd682658da
enhance: Add channel checkpoint ts lag metric (#28767)
See also #28765

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-11-28 07:00:28 +08:00
aoiasd
8a4cfb7d6a
enhance: add l0 metric and fix datacoord no need drop l0 segment when flush (#28373)
relate: https://github.com/milvus-io/milvus/issues/27675

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-11-24 15:58:24 +08:00
congqixia
1d76565894
Add metrics for garbage collection (#27303)
Also fix second metrics usage in compaction

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-22 18:47:25 +08:00
SimFG
26f06dd732
Format the code (#27275)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-21 09:45:27 +08:00
yihao.dai
3c62acde05
Add coll and segState lebels to DataCoordNumStoredRows monitoring (#26299)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-08-17 15:52:17 +08:00
xige-16
33c2012675
Add more metrics (#25081)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-06-26 17:52:44 +08:00
yihao.dai
19164ea753
Make binlog size metric to collect healthy binlog only (#24557)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-31 16:45:29 +08:00
yihao.dai
fc010ee86b
Improve the metric of stored binlog size to the level of collection and segment (#23859)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-07 20:52:38 +08:00
jaime
c9d0c157ec
Move some modules from internal to public package (#22572)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-04-06 19:14:32 +08:00