test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-11-30 02:48:45 +08:00
Code Issues Actions 11 Packages Projects Releases Wiki Activity
20,589 Commits 94 Branches 105 Tags 184 MiB
98866205fa
Commit Graph

2948 Commits

Author SHA1 Message Date
yellow-shine
bb9bed64a3
enhance: new nightly ci to support testing distributed-streaming (#35672) 
issue: https://github.com/milvus-io/milvus/issues/35704

Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
 2024-08-27 14:24:59 +08:00
XuanYang-cn
0e7877d413
fix: [skip e2e]unstable l0 it (#35612) 
See also: #35617

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-08-26 18:53:04 +08:00
ThreadDao
a90133cdf5
test: Optimize sparse vector case (#35661) 
/kind improvement

Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
 2024-08-26 14:54:57 +08:00
nico
f756f01445
test: update test cases (#35640) 
Signed-off-by: nico <cheng.yuan@zilliz.com>
 2024-08-26 12:00:58 +08:00
wei liu
5c245d51c4
enhance: Refresh proxy cache after restore rbac meta (#35635) 
issue: #35443

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-22 19:09:01 +08:00
ThreadDao
570a8879e7
test: update gosdk case for valid empty sparse vector (#35621) 
/kind improvement

Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
 2024-08-22 10:54:56 +08:00
OxalisCu
ed4eaffc9d
enhance: add csv support for bulkinsert (#34938) 
See this issue for details: #34937

---------

Signed-off-by: OxalisCu <2127298698@qq.com>
 2024-08-21 17:47:01 +08:00
smellthemoon
ba6db117e3
enhance: add some integration test in null (#35599) 
#31728

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-08-21 17:44:56 +08:00
yanliang567
7ac339ac64
test: Update init collection method (#35596) 
Related issue: #32653

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
 2024-08-21 09:22:56 +08:00
smellthemoon
80a7c78f28
enhance: import supports null in parquet and json formats (#35558) 
#31728

---------

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-08-20 16:50:55 +08:00
Chun Han
031ee6f155
enhance: support httpv1/v2 throttle and add it for httpV2(#35350) (#35470) 
related: #35350

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2024-08-20 16:16:55 +08:00
yanliang567
249dc4d9eb
test: Add tests for upsert with auto id (#35556) 
Related issue: #34668

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
 2024-08-20 14:20:56 +08:00
wei liu
e09dc3be58
enhance: Mark query node as read only after suspend (#35492) 
issue: #34985 #35493
after querynode has been suspended, it's not allow to load
segment/channel on it, which means the node is read only. to be
compatible with resource group design, after query node has been
suspend, we remove it from it's original resource group, make it a read
only query node in replica. then two things will happens:
1. it's original resource group will be lacking of query nodes, query
coord will assign new node to it.
2. querycoord will try to move out all segments/channels after querynode
has been suspended

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-20 14:02:54 +08:00
wei liu
22ced010cd
enhance: make configure load param feature be compatible with old sdk (#35520) 
issue: #31570 #35521

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-20 10:30:55 +08:00
yellow-shine
6e29d713b3
enhance: [skip e2e]dockerfile for milvus-sdk-go test (#35426) 
https://github.com/milvus-io/milvus/issues/35567

---------

Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
 2024-08-20 09:48:55 +08:00
congqixia
b4743b4ca8
enhance: [GoSDK] Sync latest milvus proto and mockery (#35511) 
Related to #35443

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-19 16:22:55 +08:00
XuanYang-cn
967f38672a
enhance: Add integration tests for l0 (#35429) 
See also: #34796

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-08-19 10:56:54 +08:00
ThreadDao
635e9ddb0e
test: fix go sdk cases parallel context deadline (#35519) 
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
 2024-08-16 20:32:53 +08:00
Buqian Zheng
f4a91e135b
enhance: Allow empty sparse row (#34700) 
issue: #29419

* If a sparse vector with 0 non-zero value is inserted, no ANN search on
this sparse vector field will return it as a result. User may retrieve
this row via scalar query or ANN search on another vector field though.
* If the user uses an empty sparse vector as the query vector for a ANN
search, no neighbor will be returned.

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
 2024-08-16 14:14:54 +08:00
wei liu
1d49358f82
enhance: Add BackupRBAC/RestoreRBAC API to enable rbac backup (#35444) 
issue: #35443

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-16 10:10:53 +08:00
wei liu
344dc6a9f8
enhance: enable to set load config in cluster level (#35169) 
issue: #35170
This PR enable to set load configs in cluster level, such as replicas
and resource groups. then when load collections will use the load
config.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-07 12:38:21 +08:00
zhagnlu
c19fe95154
fix: support string match for hybrid and bitmap index (#35294) 
#34841

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-08-07 09:54:22 +08:00
ThreadDao
a62118ca33
test: update cases for index describe and upsert with autoID (#35191) 
issue: #33419

---------

Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
 2024-08-05 18:00:16 +08:00
sthuang
d8668fe405
enhance: upgrade go version to 1.21.11 (#35257) 
issue: #35142

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
 2024-08-05 16:12:15 +08:00
zhagnlu
4b553b0333
enhance: revert remove duplicated pk function (#35103) 
issue: #34778
 Revert "fix: fix query count(*) concurrently"
 Revert "enhance: mark duplicated pk as deleted "

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-08-05 10:48:17 +08:00
sre-ci-robot
eb5461905b
[automated] Update Pytest image changes (#35244) 
Update Pytest image changes
See changes:
241c71fdde
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-08-05 09:24:33 +08:00
yellow-shine
241c71fdde
enhance: use docker compose instead of docker-compose (#35208) 
https://github.com/milvus-io/milvus/issues/35209

---------

Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
 2024-08-02 19:32:32 +08:00
congqixia
db06b86594
enhance: Sync otlp dependency version to fix security issue (#35192) 
Related to #34434

otelgrpc DoS vulnerability due to unbound cardinality metrics

https://github.com/milvus-io/milvus/security/dependabot/91

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-02 16:22:20 +08:00
cai.zhang
196a7986b3
enhance: Change the fixed value to a ratio for clustering segment size (#35076) 
issue: #34495

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-08-01 22:04:14 +08:00
congqixia
a642a26ed4
enhance: Resolve ChunkFileWriter lint issue (#35166) 
See also #34483

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-01 16:52:13 +08:00
wayblink
5bbb1c201c
enhance:support l2 single compaction (#34935) 
#34928

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-08-01 14:36:13 +08:00
congqixia
dfda9f0478
enhance: Add depguard rules to ban deprecated proto lib (#35140) 
See also #34394 #34252

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-01 10:01:49 +08:00
smellthemoon
6106a48acb
fix: upsert result use the previous pk (#34672) 
#34668

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-07-31 15:25:51 +08:00
congqixia
de8a266d8a
enhance: Enable linux code checker (#35084) 
See also #34483

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-30 15:53:51 +08:00
zhagnlu
a8a4779749
fix: fix parse plan proto failed for search type (#34944) 
#25848

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-07-29 21:19:49 +08:00
zhagnlu
86322e0468
fix: fix query count(*) concurrently (#35007) 
#34778
#34849
fix two problems:
1. count(*) incorrect, if growing insert duplicated (pk, timestamp)
pairs that pk and timestamp all same, need to keep just one pair.
2. count(*) may core dump, if get_real_count interface get snapshot and
do mvcc at not consistency status, mainly happens under concurrency.

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-07-29 19:53:50 +08:00
wei liu
c45f38aa61
enhance: Update protobuf-go to protobuf-go v2 (#34394) 
issue: #34252

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-07-29 11:31:51 +08:00
cai.zhang
2372452fac
enhance: Optimized the GC logic to ensure that memory is released in time (#34949) 
issue: #34703

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-07-28 23:53:47 +08:00
congqixia
783f9d9c33
fix: Unify hook singleton implementation in proxy (#34887) 
Related to #34885

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-26 18:07:53 +08:00
congqixia
4ee6c69217
enhance: Add Segment Level in milvus segment info APIs (#34763) 
See also #34746

This PR add segment level field in response of
`GetPersistentSegmentInfo` and `GetQuerySegmentInfo`

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-26 10:01:46 +08:00
congqixia
482373b3d3
enhance: Sync go e2e package proto dependency (#34913) 
Go e2e package point to a lower verion of milvus-proto and may cause
compile error locally.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-23 16:09:43 +08:00
cai.zhang
4c45bc412f
enhance: Add integration test for clustering compaction (#34881) 
issue: #34792

---------

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-07-23 10:13:43 +08:00
wayblink
c339df26fc
enhance: refine clustering compaction basic it (#34793) 
#34792

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-07-22 11:27:51 +08:00
yihao.dai
b22e549844
enhance: Rename config of sealing by growing segmetns size (#34787) 
/kind improvement

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-07-19 20:27:41 +08:00
yihao.dai
4939f82d4f
enhance: Seal by total growing segments size (#34692) 
Seals the largest growing segment if the total size of growing segments
of each shard exceeds the size threshold(default 4GB). Introducing this
policy can help keep the size of growing segments within a suitable
level, alleviating the pressure on the delegator.

issue: https://github.com/milvus-io/milvus/issues/34554

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-07-17 21:45:41 +08:00
wei liu
acb33bba4d
enhance: Preserve fixed-size memory in delegator node for growing segment. (#34596) 
issue: #34595
When consuming insert data on the delegator node, QueryCoord will move
out some sealed segments to manage its memory usage. After the growing
segment gets flushed, some sealed segments from other workers will be
moved back to the delegator node. To avoid the frequent movement of
segments, we estimate the maximum growing row count and preserve a
fixed-size memory in the delegator node.

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-07-15 20:51:46 +08:00
XuanYang-cn
eb472b7f08
enhance: [skip e2e]Enable compaction it test (#34526) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-07-15 20:45:39 +08:00
elstic
2c462d387b
test: Fix index creation mismatch for sparse vector columns in multivector case (#34618) 
If `is_all_data_type` is true, the case will add float32, f16, bf16, and
sparse vectors, but the created indexes are all `flat` indexes by
default. The sparse type cannot create a flat index. Fix the test code
to create a `SPARSE_INVERTED_INDEX` index for the sparse vector when
is_all_data_type is true

Signed-off-by: elstic <hao.wang@zilliz.com>
 2024-07-13 10:45:37 +08:00
smellthemoon
07b94b4615
enhance: support upsert autoid==true (#30342) 
related with: #29258

---------

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-07-11 16:53:35 +08:00
ThreadDao
e4115485b6
test: add cases for gosdk v2 partitions (#34593) 
issue: #33419 #34426

Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
 2024-07-11 16:49:36 +08:00