test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-03 04:19:18 +08:00
Code Issues Actions 13 Packages Projects Releases Wiki Activity
19,918 Commits 95 Branches 105 Tags 184 MiB
066c8ea175
Commit Graph

45 Commits

Author SHA1 Message Date
Bingyi Sun
fecd9c21ba
feat: LRU cache implementation (#32567) 
issue: https://github.com/milvus-io/milvus/issues/32783
This pr is the implementation of lru cache on branch lru-dev.

Signed-off-by: sunby <sunbingyi1992@gmail.com>
Co-authored-by: chyezh <chyezh@outlook.com>
Co-authored-by: MrPresent-Han <chun.han@zilliz.com>
Co-authored-by: Ted Xu <ted.xu@zilliz.com>
Co-authored-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: wayblink <anyang.wang@zilliz.com>
 2024-05-06 20:29:30 +08:00
wei liu
1a98ce39f5
enhance: Remove useless logic about FromShardLeader (#32029) 
issue: #32047

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-04-10 20:11:19 +08:00
zhenshan.cao
089c805e0a
enhance:Refactor hybrid search (#32020) 
issue: https://github.com/milvus-io/milvus/issues/25639
https://github.com/milvus-io/milvus/issues/31368

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2024-04-09 14:21:18 +08:00
wei liu
c311932d5f
fix: Update segment's version in leader task (#31643) 
issue: #31468

1. when segment's version in leader view doesn't match segment's version
in dist, should update leader view
2. after call loadDeltalog, should update segment's load version with
latest ts
3. change leader task's priority from high to low, to avoid leader task
replace segment task and balance task

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-04-01 10:37:21 +08:00
yiwangdr
c6665c2a4c
test: support multiple data/querynodes in integration test (#30618) 
issue: https://github.com/milvus-io/milvus/issues/29507

Signed-off-by: yiwangdr <yiwangdr@gmail.com>
 2024-02-21 11:54:53 +08:00
xige-16
060c8603a3
fix: Support mvcc with hybrid serach (#30114) 
issue: https://github.com/milvus-io/milvus/issues/29656
/kind bug

Signed-off-by: xige-16 <xi.ge@zilliz.com>

---------

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2024-02-01 16:03:03 +08:00
zhenshan.cao
60e88fb833
fix: Restore the MVCC functionality. (#29749) 
When the TimeTravel functionality was previously removed, it
inadvertently affected the MVCC functionality within the system. This PR
aims to reintroduce the internal MVCC functionality as follows:

1. Add MvccTimestamp to the requests of Search/Query and the results of
Search internally.
2. When the delegator receives a Query/Search request and there is no
MVCC timestamp set in the request, set the delegator's current tsafe as
the MVCC timestamp of the request. If the request already has an MVCC
timestamp, do not modify it.
3. When the Proxy handles Search and triggers the second phase ReQuery,
divide the ReQuery into different shards and pass the MVCC timestamp to
the corresponding Query requests.

issue: #29656

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2024-01-09 11:38:48 +08:00
yah01
c4dda3c50f
fix: panic caused by type assert LocalSegment on Segment (#29018) 
related #29017

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-12-08 01:50:38 +08:00
yihao.dai
4b8bc2798e
enhance: Print nq (#28507) 
Log nq in search path.

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-11-30 10:42:27 +08:00
Gao
3e77365de5
fix: correct autoindex segment num (#28387) 
Fix #28386 
Current code snippet 
```
// get delegator
sd, ok := node.delegators.Get(channel)
if !ok {
err := merr.WrapErrChannelNotFound(channel)
log.Warn("Query failed, failed to get shard delegator for search", zap.Error(err))
return nil, err
}
req, err = node.optimizeSearchParams(ctx, req, sd)
if err != nil {
log.Warn("failed to optimize search params", zap.Error(err))
return nil, err
}
// do search
results, err := sd.Search(searchCtx, req)
```

We could move these into `ShardDelegator`, and directly use sealed
segment num in `Search` methods, also segment num got outside could be
wrong when we specify partitions.

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2023-11-22 11:12:22 +08:00
yah01
ece592a42f
Deliver L0 segments delete records (#27722) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-11-07 01:44:18 +08:00
wei liu
9b737b77a7
fix fromShardLeader typo (#28061) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-11-02 15:50:16 +08:00
aoiasd
9ba79a97e4
Add some log for delete task when it need query (#27317) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-10-24 14:54:12 +08:00
yah01
be980fbc38
Refine state check (#27541) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-11 21:01:35 +08:00
yah01
6539a5ae2c
Refine DataCoord status (#27262) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-09-26 17:15:27 +08:00
yah01
00c65fa0d7
Refine QueryNode errors (#27013) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-09-12 16:07:18 +08:00
aoiasd
e107d0794c
support complex delete expression (#25752) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-09-12 10:19:17 +08:00
SimFG
28681276e2
Improve the retry of the rpc client (#26795) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-09-06 17:43:14 +08:00
yah01
3349db4aa7
Refine errors to remove changes breaking design (#26521) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-09-04 09:57:09 +08:00
yah01
5dae6a654a
Protect segment from being released while query/search (#26322) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-08-16 18:38:17 +08:00
wei liu
b47a72bfcf
fix set dirty segment distribution to leader view (#26180) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-08-11 11:21:32 +08:00
smellthemoon
3fc18c53a3
Fix wrong log (#26022) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-08-02 11:01:07 +08:00
wei liu
302897f866
refine look aside balance logic (#25837) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-07-25 18:51:01 +08:00
congqixia
cc71e065e3
Support load index after segment loaded (#25567) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-07-18 10:51:19 +08:00
Enwei Jiao
66fdc71479
Refactor logs in DataCoord & DataNode (#25574) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-07-14 15:56:31 +08:00
yah01
205a7c430a
Fix QueryNode panic while inserting to released segment (#25495) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-07-12 19:48:29 +08:00
wei liu
b7740835fe
fix metric label for sq count metrics (#25399) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-07-07 14:06:25 +08:00
wei liu
d5ec7dda80
fix metric label in search/query count (#25377) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-07-06 17:48:26 +08:00
wei liu
c84496a1ba
fix sq count metrics (#25287) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-07-06 08:46:26 +08:00
chyezh
d7d61f529c
[Feature|Pick] enable scheduler policy and add user-task-polling policy (#24839) 
Signed-off-by: chyezh <ye.zhen@zilliz.com>
 2023-07-03 18:24:25 +08:00
Gao
de6e4817a2
Support dynamic tuning config (#25152) 
Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2023-07-03 15:18:24 +08:00
wei liu
68ae199a9f
load segment with target version, avoid read redundant segment (#24929) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-06-27 11:48:45 +08:00
Gao
1f8af0596e
Fix autoindex filter bug (#24775) 
Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2023-06-09 18:18:36 +08:00
congqixia
41af0a98fa
Use go-api/v2 for milvus-proto (#24770) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-06-09 01:28:37 +08:00
aoiasd
44e5daae3a
Fix querynode read quota (#24412) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-06-08 17:24:36 +08:00
Gao
ea62922829
Disable search params rewrite while searching with filter (#24625) 
Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2023-06-06 14:28:35 +08:00
yihao.dai
014387fd94
Forbid to get quantized vector from ChunkManager (#24334) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-05-24 23:03:27 +08:00
wei liu
ce03248b1a
simplify logic circle in search (#24110) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-05-23 16:01:26 +08:00
yihao.dai
426ed30d2d
Log optimized search params (#23930) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-05-09 10:12:42 +08:00
smellthemoon
f0ababb420
Use merr in querynodev2 (#23652) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-04-26 17:06:34 +08:00
LeoReeYang
a2ab94ec3d
remove unused code. (#23572) 
Signed-off-by: LeoReeYang <LeoReeYang@outlook.com>
 2023-04-20 17:56:32 +08:00
Jiquan Long
ad7f3d4c3e
Support Count(*) on querynodev2 (#23321) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-04-11 15:40:31 +08:00
jaime
c9d0c157ec
Move some modules from internal to public package (#22572) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-04-06 19:14:32 +08:00
congqixia
8db24e0a83
Support queryHook in querynodev2 (#23140) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-04-06 14:32:29 +08:00
yah01
081572d31c
Refactor QueryNode (#21625) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-03-27 00:42:00 +08:00