test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-11-30 02:48:45 +08:00
Code Issues Actions 11 Packages Projects Releases Wiki Activity
20,599 Commits 94 Branches 105 Tags 184 MiB
1cd9123a6c
Commit Graph

20599 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
ThreadDao
a62118ca33
test: update cases for index describe and upsert with autoID (#35191) 
issue: #33419

---------

Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
 2024-08-05 18:00:16 +08:00
shaoyue
1db099973f
enhance: Gin & restfulv1 handler use sonic json (#35020) 
/cc @czs007 @congqixia 
/kind enhancement

Signed-off-by: haorenfsa <haorenfsa@gmail.com>
 2024-08-05 16:44:17 +08:00
yihao.dai
b71e058bc5
enhance: Add import option to skip disk quota check (#35274) 
Add an option to skip the disk quota check for backup-restore import.

issue: https://github.com/milvus-io/milvus/issues/33775

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-08-05 16:40:16 +08:00
cai.zhang
6542c1ab0e
enhance: Add monitoring metrics for task execution time in datacoord (#35139) 
issue: #35138

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-08-05 16:26:17 +08:00
cai.zhang
d905713724
fix: Maintain load idempotency even when building new indexes (#35178) 
issue: #34404

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-08-05 16:22:17 +08:00
sthuang
d8668fe405
enhance: upgrade go version to 1.21.11 (#35257) 
issue: #35142

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
 2024-08-05 16:12:15 +08:00
yihao.dai
678c0a708b
enhance: Refine import error log (#35067) 
issue: https://github.com/milvus-io/milvus/issues/35060

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-08-05 15:30:16 +08:00
congqixia
a6903701a4
enhance: Remove stackerror imported unexpected (#35256) 
`stackerror` pkg is imported by accident and could be replaced by
cockroachdb errors lib. This PR removes import line and fix this
problem.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-05 14:48:16 +08:00
wei liu
c22b0a636e
fix: delete by expr failed at retry progress (#35241) 
issue: #35240
delete by expr shard the same err object between channels, so if one
channel's query failed, it will fail all channel, which will break
channel level retry policy, and make delete operation failed.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-05 11:18:16 +08:00
zhagnlu
4b553b0333
enhance: revert remove duplicated pk function (#35103) 
issue: #34778
 Revert "fix: fix query count(*) concurrently"
 Revert "enhance: mark duplicated pk as deleted "

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-08-05 10:48:17 +08:00
sre-ci-robot
e6640346d0
[automated] Update gpu Builder image changes (#35246) 
Update gpu Builder image changes
See changes:
7bf68d8657
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-08-05 09:26:15 +08:00
sre-ci-robot
eb5461905b
[automated] Update Pytest image changes (#35244) 
Update Pytest image changes
See changes:
241c71fdde
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-08-05 09:24:33 +08:00
sre-ci-robot
e201110fbd
[automated] Update cpu Builder image changes (#35248) 
Update cpu Builder image changes
See changes:
7bf68d8657
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-08-05 09:22:21 +08:00
chyezh
9871966415
enhance: segment alloc interceptor (#34996) 
#33285

- add segment alloc interceptor for streamingnode.
- add add manual alloc segment rpc for datacoord.

---------

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-08-04 07:40:15 +08:00
sre-ci-robot
a7f7d911cf Update all contributors 
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
 2024-08-02 12:00:48 +00:00
Chun Han
3faef63a25
enhance: add log for partition stats( #30376) (#35219) 
related:  #30376

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2024-08-02 19:34:22 +08:00
zhagnlu
16dd53e7cf
enhance: remove timestamp_filter after retrieve (#35207) 
#35226

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-08-02 19:32:46 +08:00
yellow-shine
241c71fdde
enhance: use docker compose instead of docker-compose (#35208) 
https://github.com/milvus-io/milvus/issues/35209

---------

Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
 2024-08-02 19:32:32 +08:00
cqy123456
b99a554827
enhance:update mmap_chunk_manager config in milvus.yaml (#35180) 
issue: https://github.com/milvus-io/milvus/issues/32984
related pr: https://github.com/milvus-io/milvus/pull/35186

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2024-08-02 19:28:15 +08:00
yellow-shine
7bf68d8657
enhance: upgrade conan version (#35215) 
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
 2024-08-02 19:22:15 +08:00
yihao.dai
a4439cc911
enhance: Implement flusher in streamingNode (#34942) 
- Implement flusher to:
  - Manage the pipelines (creation, deletion, etc.)
  - Manage the segment write buffer
  - Manage sync operation (including receive flushMsg and execute flush)
- Add a new `GetChannelRecoveryInfo` RPC in DataCoord.
- Reorganize packages: `flushcommon` and `datanode`.

issue: https://github.com/milvus-io/milvus/issues/33285

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-08-02 18:30:23 +08:00
jaime
fcec4c21b9
fix: check collection health(queryable) fail for releasing collection (#34947) 
issue: #34946

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-08-02 17:20:15 +08:00
Bingyi Sun
3641ae6611
fix: Fix index memory estimation (#35225) 
issue: https://github.com/milvus-io/milvus/issues/35229

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2024-08-02 16:24:15 +08:00
congqixia
db06b86594
enhance: Sync otlp dependency version to fix security issue (#35192) 
Related to #34434

otelgrpc DoS vulnerability due to unbound cardinality metrics

https://github.com/milvus-io/milvus/security/dependabot/91

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-02 16:22:20 +08:00
wayblink
81773bfadf
enhance: add commit time in partitionStats proto (#35125) 
fix: #35110

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-08-02 16:16:14 +08:00
smellthemoon
475c333fa2
enhance: add valid_data in span (#35030) 
#31728

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-08-02 15:40:14 +08:00
wei liu
f466129924
enhance: add segment state on data coord bin log size metrics (#35153) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-02 14:48:14 +08:00
wei liu
3dd3749f0b
enhance: Avoid unnecesary syncTargetVersion func call after querycoord recover (#34954) 
before querycoord stop gracefully, we will save the current target to
meta store and recover it after querycoord start up, to speed the
querycoord's recovery time. but the target version hasn't been recovered
as expected, and it use latest timestamp as current target's version,
which has no effect to querycoord but an unnecessary syncTargetVersion
func call.

This PR recover the correct target version to avoid unnecessary
syncTargetVersion func call

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-02 14:46:13 +08:00
congqixia
095d77269b
enhance: Set WatchProgress for channel operation progress response (#35183) 
The watch progress is always zero in CheckChannelOperationProgress
response, which is meaningless and confusing. This PR set progress value
in rpc response to fix this problem.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-02 12:06:14 +08:00
aoiasd
3655ab10b2
fix: evict paramtable cache miss (#34771) 
relate: https://github.com/milvus-io/milvus/issues/33461

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-08-02 11:52:14 +08:00
congqixia
c64a078458
enhance: Support proxy/delegator qn client pooling (#35194) 
See also #35196

Add param item for proxy/delegator query node client pooling and
implement pooling logic

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-02 11:24:19 +08:00
wei liu
3b735b4b02
enhance: Refine param init for MmapDirPath (#35181) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-02 11:12:14 +08:00
wayblink
95462668ca
enhance: unify time in clustering compaction task to unix (#35167) 
#34495

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-08-02 10:30:19 +08:00
wei liu
27b6d58981
fix: Set legacy level to l0 segment after qc restart (#35197) 
issue: #35087
after qc restarts, and target is not ready yet, if dist_handler try to
update segment dist, it will set legacy level to l0 segment, which may
cause l0 segment be moved to other node, cause search/query failed.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-02 10:18:13 +08:00
Jiquan Long
4dbd023402
enhance: add metric to record maxInsertRate and queryNodeMemoryHighWaterLevel (#35185) 
fix: https://github.com/milvus-io/milvus/issues/35177

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2024-08-02 10:14:13 +08:00
cai.zhang
196a7986b3
enhance: Change the fixed value to a ratio for clustering segment size (#35076) 
issue: #34495

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-08-01 22:04:14 +08:00
congqixia
a642a26ed4
enhance: Resolve ChunkFileWriter lint issue (#35166) 
See also #34483

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-01 16:52:13 +08:00
yihao.dai
7721a28584
fix: Fix unstable TestDispatchToVchannels ut (#35163) 
issue: https://github.com/milvus-io/milvus/issues/35150

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-08-01 14:56:13 +08:00
wayblink
5bbb1c201c
enhance:support l2 single compaction (#34935) 
#34928

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-08-01 14:36:13 +08:00
zhagnlu
f8c1b138a8
fix:fix get array error for int type (#35154) 
#35055

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-08-01 14:30:12 +08:00
Ted Xu
b4d0f4df0a
enhance: update param documents (#35152) 
See: #35161

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2024-08-01 12:38:13 +08:00
zhenshan.cao
aa247f192d
enhance: remove unused code for StorageV2 (#35132) 
issue: https://github.com/milvus-io/milvus/issues/34168

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2024-08-01 12:08:13 +08:00
cai.zhang
9412002d7d
fix: Fix data race for clustering buffer writer (#35145) 
issue: #34495

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-08-01 11:20:13 +08:00
cai.zhang
089fb2b5fb
enhance: Skip updating the level that are already at the specified level (#35094) 
issue: #34495

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-08-01 11:18:12 +08:00
Bingyi Sun
f229f244d2
enhance: add chunk basic impl (#34634) 
https://github.com/milvus-io/milvus/issues/35112
This pr would not affect milvus functionality by now.
It implments a Chunk memory layout that looks like 
```
VariableColumn
|offset|offset|offset|
|data|data|data|

```
We maybe move offsets to the beginning and add null bitmaps later but
not in this PR.
And mmap test will also be added in another PR.

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2024-08-01 10:29:51 +08:00
wei liu
e9d61daa3f
enhance: Reduce delegator memory overloaded factor to 0.1 (#35092) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-01 10:21:50 +08:00
congqixia
dfda9f0478
enhance: Add depguard rules to ban deprecated proto lib (#35140) 
See also #34394 #34252

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-01 10:01:49 +08:00
sre-ci-robot
c15d074b21
[automated] Update Knowhere Commit (#35157) 
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-08-01 01:17:49 +08:00
Gao
6695c6d0a3
enhance: add channel num for queryHook optimization (#35104) 
At most cases, data in each channel is almost evenly distributed, we
could utilize the channel num info to optimize searh param in queryHook

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2024-07-31 18:59:49 +08:00
wei liu
03912a8788
enhance: Avoid balance stuck after segment list become stable (#34728) 
issue: #34715
if collection's segment list doesn't changes anymore, then the next
target will be empty at most time, and balance segment will check
whether segment exist in both current and next target, so the balance
cloud be blocked due to next target is empty.

This PR permit segment to be moved if next target is empty, to avoid
balance stuck.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-07-31 18:09:48 +08:00