test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-02 03:48:37 +08:00
Code Issues Actions 12 Packages Projects Releases Wiki Activity
21,318 Commits 94 Branches 105 Tags 184 MiB
e5775a71af
Commit Graph

58 Commits

Author SHA1 Message Date
Zhen Ye
fca946dee1
enhance: move the search utilities from querynode into new module (#37531) 
issue: #33285

- The search utilities will be shared between query node and streaming
node.

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-11-11 11:10:27 +08:00
congqixia
ba25320aea
fix: Unify loaded partition check to delegator (#36879) 
Related to #36370

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-10-15 19:15:23 +08:00
aoiasd
c22a2cebb6
fix: split stream query result to avoid grpc response too large error (#36090) 
relate: https://github.com/milvus-io/milvus/issues/36089

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-09-13 15:07:09 +08:00
jaime
24fb10114b
enhance: remove cooling off in rate limiter for read requests (#35935) 
issue: #35934

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-09-04 14:39:10 +08:00
wei liu
c45f38aa61
enhance: Update protobuf-go to protobuf-go v2 (#34394) 
issue: #34252

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-07-29 11:31:51 +08:00
Chun Han
416a2cf507
fix: query iterator lack results(#33137) (#33422) 
related: #33137 
adding has_more_result_tag for various level's reduce to rectify
reduce_stop_for_best

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2024-05-30 17:51:44 +08:00
aoiasd
59a7a46904
enhance: Merge query stream result for reduce delete task (#32855) 
relate: https://github.com/milvus-io/milvus/issues/32854

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-05-27 18:15:43 +08:00
Jiquan Long
dd9919a7dc
fix: two-phase retrieval on lru-segment (#32945) 
issue: #31822

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2024-05-15 17:53:34 +08:00
SimFG
1d48d0aeb2
enhance: use different value to get related data size according to segment type (#33017) 
issue: #30436

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-05-14 14:59:33 +08:00
chyezh
1c84a1c9b6
fix: lru related issue fixup patch (#32916) 
issue: #32206, #32801

- search failure with some assertion, segment not loaded and resource
insufficient.

- segment leak when query segments

---------

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-05-10 19:17:30 +08:00
Bingyi Sun
fecd9c21ba
feat: LRU cache implementation (#32567) 
issue: https://github.com/milvus-io/milvus/issues/32783
This pr is the implementation of lru cache on branch lru-dev.

Signed-off-by: sunby <sunbingyi1992@gmail.com>
Co-authored-by: chyezh <chyezh@outlook.com>
Co-authored-by: MrPresent-Han <chun.han@zilliz.com>
Co-authored-by: Ted Xu <ted.xu@zilliz.com>
Co-authored-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: wayblink <anyang.wang@zilliz.com>
 2024-05-06 20:29:30 +08:00
Jiquan Long
c002745902
enhance: retrieve output fields after local reduce (#32346) 
issue: #31822

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2024-04-25 09:49:26 +08:00
SimFG
90bed1caf9
enhance: add the related data size for the read apis (#31816) 
issue: #30436
origin pr: #30438
related pr: #31772

---------

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-04-10 15:07:17 +08:00
chyezh
c9faa6d936
enhance: add more metrics for cache and search (#31777) 
issue: #30931

---------

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-04-10 10:55:17 +08:00
congqixia
1f7f3993a1
fix: Validate PlaceholderGroups before combine them (#32016) 
See also #32015

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-09 11:33:17 +08:00
SimFG
b1a1cca10b
feat: add more operation detail info for better allocation (#30438) 
issue: #30436

---------

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-03-28 06:33:11 +08:00
yiwangdr
c6665c2a4c
test: support multiple data/querynodes in integration test (#30618) 
issue: https://github.com/milvus-io/milvus/issues/29507

Signed-off-by: yiwangdr <yiwangdr@gmail.com>
 2024-02-21 11:54:53 +08:00
PowderLi
8fc4ebfa11
fix: empty MetricType (#30216) 
issue: #30102 #30225

we should read MetricType from SearchResult,
because query node never
1. read metricType from LoadMeta
2. store to collection
3. set SearchRequest.MetricType

Signed-off-by: PowderLi <min.li@zilliz.com>
 2024-01-28 15:33:02 +08:00
congqixia
f2c0ead51a
enhance: Add trace span for scheduling read tasks in QueryNode (#30265) 
This PR adds a trace span for search/query task scheduling duration

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-01-25 16:59:00 +08:00
yah01
0d4e781f69
fix: the system rejects all queries and never recovers if enabled read rate limit (#30061) 
fix #30060

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2024-01-17 23:30:55 +08:00
zhenshan.cao
60e88fb833
fix: Restore the MVCC functionality. (#29749) 
When the TimeTravel functionality was previously removed, it
inadvertently affected the MVCC functionality within the system. This PR
aims to reintroduce the internal MVCC functionality as follows:

1. Add MvccTimestamp to the requests of Search/Query and the results of
Search internally.
2. When the delegator receives a Query/Search request and there is no
MVCC timestamp set in the request, set the delegator's current tsafe as
the MVCC timestamp of the request. If the request already has an MVCC
timestamp, do not modify it.
3. When the Proxy handles Search and triggers the second phase ReQuery,
divide the ReQuery into different shards and pass the MVCC timestamp to
the corresponding Query requests.

issue: #29656

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2024-01-09 11:38:48 +08:00
yah01
9e0163e12f
enhance: use GPU pool for gpu tasks (#29678) 
- this much improve the performance for GPU index

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2024-01-04 17:50:46 +08:00
congqixia
b251c3a682
enhance: add ctx for HandleCStatus and callers (#29517) 
See also #29516

Make `HandleCStatus` print trace id for better logging

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-27 16:10:47 +08:00
yah01
be980fbc38
Refine state check (#27541) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-11 21:01:35 +08:00
congqixia
6ff4ffa6c2
Fix scheduler may panic during close (#27431) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-28 16:53:26 +08:00
congqixia
258e1ccd66
Refine querynode scheduler lifetime (#26915) 
This PR refines scheduler lifetime control:
- Move private tri-state into lifetime package
- Make scheduler block incoming "Add" task
- Make scheduler Stop wait until all previously accepted task done

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-28 10:21:26 +08:00
yah01
6539a5ae2c
Refine DataCoord status (#27262) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-09-26 17:15:27 +08:00
SimFG
26f06dd732
Format the code (#27275) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-09-21 09:45:27 +08:00
MrPresent-Han
33e3e78937
add metrics for query reduce (#27201) 
/kind improvement
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-09-19 14:53:22 +08:00
aoiasd
e107d0794c
support complex delete expression (#25752) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-09-12 10:19:17 +08:00
yah01
3349db4aa7
Refine errors to remove changes breaking design (#26521) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-09-04 09:57:09 +08:00
yah01
5dae6a654a
Protect segment from being released while query/search (#26322) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-08-16 18:38:17 +08:00
Enwei Jiao
ca1349708b
Remove time travel ralted testcase (#26119) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-08-10 18:53:17 +08:00
wei liu
f33a89387f
enable config shard level cost metrics whether contains worker's cost (#26132) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-08-09 11:37:15 +08:00
wei liu
302897f866
refine look aside balance logic (#25837) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-07-25 18:51:01 +08:00
congqixia
bcd3abf644
Separate segments/shards reduce latency metrics (#25475) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-07-11 18:18:28 +08:00
chyezh
dc1cb54404
[Fixup] unexpected read_task_ready_len metric (#25349) 
Signed-off-by: chyezh <ye.zhen@zilliz.com>
 2023-07-06 20:02:29 +08:00
chyezh
d7d61f529c
[Feature|Pick] enable scheduler policy and add user-task-polling policy (#24839) 
Signed-off-by: chyezh <ye.zhen@zilliz.com>
 2023-07-03 18:24:25 +08:00
yah01
f7ecbff986
Make merged task channel buffered (#25242) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-06-30 14:58:22 +08:00
wei liu
9b91519c88
fix search cost metrics on replica selection (#25059) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-06-27 19:02:44 +08:00
yah01
9250423288
Fix QueryNode metrics (#24916) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-06-16 16:02:39 +08:00
wei liu
75adf1df98
fix group nq metrics (#24918) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-06-15 19:50:39 +08:00
wei liu
e40d95e093
add search/query request cost metrics on qn (#24413) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-06-15 14:24:38 +08:00
congqixia
41af0a98fa
Use go-api/v2 for milvus-proto (#24770) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-06-09 01:28:37 +08:00
aoiasd
44e5daae3a
Fix querynode read quota (#24412) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-06-08 17:24:36 +08:00
congqixia
31880ab427
Use conc.PoolOption instead of ants.Option (#24585) 
- Add conc.PoolOption to setup conc.Pool
- Change panic default behavior
- Make future has error when job panicks

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-06-01 15:52:31 +08:00
yah01
1c64d0618c
Revert "Consume threads of the number of nq (#24410)" (#24554) 
This reverts commit 71a7fef5c5.

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-05-31 16:41:33 +08:00
yah01
71a7fef5c5
Consume threads of the number of nq (#24410) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-05-26 17:33:27 +08:00
yah01
eab94489ba
Refine the merge algorithm (#23767) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-04-27 18:26:35 +08:00
smellthemoon
f0ababb420
Use merr in querynodev2 (#23652) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-04-26 17:06:34 +08:00