congqixia
b4743b4ca8
enhance: [GoSDK] Sync latest milvus proto and mockery ( #35511 )
...
Related to #35443
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-19 16:22:55 +08:00
XuanYang-cn
967f38672a
enhance: Add integration tests for l0 ( #35429 )
...
See also: #34796
---------
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-08-19 10:56:54 +08:00
ThreadDao
635e9ddb0e
test: fix go sdk cases parallel context deadline ( #35519 )
...
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2024-08-16 20:32:53 +08:00
Buqian Zheng
f4a91e135b
enhance: Allow empty sparse row ( #34700 )
...
issue: #29419
* If a sparse vector with 0 non-zero value is inserted, no ANN search on
this sparse vector field will return it as a result. User may retrieve
this row via scalar query or ANN search on another vector field though.
* If the user uses an empty sparse vector as the query vector for a ANN
search, no neighbor will be returned.
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-08-16 14:14:54 +08:00
wei liu
1d49358f82
enhance: Add BackupRBAC/RestoreRBAC API to enable rbac backup ( #35444 )
...
issue: #35443
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-16 10:10:53 +08:00
wei liu
344dc6a9f8
enhance: enable to set load config in cluster level ( #35169 )
...
issue: #35170
This PR enable to set load configs in cluster level, such as replicas
and resource groups. then when load collections will use the load
config.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-07 12:38:21 +08:00
zhagnlu
c19fe95154
fix: support string match for hybrid and bitmap index ( #35294 )
...
#34841
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-08-07 09:54:22 +08:00
ThreadDao
a62118ca33
test: update cases for index describe and upsert with autoID ( #35191 )
...
issue: #33419
---------
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2024-08-05 18:00:16 +08:00
sthuang
d8668fe405
enhance: upgrade go version to 1.21.11 ( #35257 )
...
issue: #35142
Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-08-05 16:12:15 +08:00
zhagnlu
4b553b0333
enhance: revert remove duplicated pk function ( #35103 )
...
issue: #34778
Revert "fix: fix query count(*) concurrently"
Revert "enhance: mark duplicated pk as deleted "
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-08-05 10:48:17 +08:00
sre-ci-robot
eb5461905b
[automated] Update Pytest image changes ( #35244 )
...
Update Pytest image changes
See changes:
241c71fddesre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-08-05 09:24:33 +08:00
yellow-shine
241c71fdde
enhance: use docker compose instead of docker-compose ( #35208 )
...
https://github.com/milvus-io/milvus/issues/35209
---------
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-08-02 19:32:32 +08:00
congqixia
db06b86594
enhance: Sync otlp dependency version to fix security issue ( #35192 )
...
Related to #34434
otelgrpc DoS vulnerability due to unbound cardinality metrics
https://github.com/milvus-io/milvus/security/dependabot/91
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-02 16:22:20 +08:00
cai.zhang
196a7986b3
enhance: Change the fixed value to a ratio for clustering segment size ( #35076 )
...
issue: #34495
---------
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-08-01 22:04:14 +08:00
congqixia
a642a26ed4
enhance: Resolve ChunkFileWriter lint issue ( #35166 )
...
See also #34483
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-01 16:52:13 +08:00
wayblink
5bbb1c201c
enhance:support l2 single compaction ( #34935 )
...
#34928
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-08-01 14:36:13 +08:00
congqixia
dfda9f0478
enhance: Add depguard rules to ban deprecated proto lib ( #35140 )
...
See also #34394 #34252
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-01 10:01:49 +08:00
smellthemoon
6106a48acb
fix: upsert result use the previous pk ( #34672 )
...
#34668
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-07-31 15:25:51 +08:00
congqixia
de8a266d8a
enhance: Enable linux code checker ( #35084 )
...
See also #34483
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-30 15:53:51 +08:00
zhagnlu
a8a4779749
fix: fix parse plan proto failed for search type ( #34944 )
...
#25848
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-07-29 21:19:49 +08:00
zhagnlu
86322e0468
fix: fix query count(*) concurrently ( #35007 )
...
#34778
#34849
fix two problems:
1. count(*) incorrect, if growing insert duplicated (pk, timestamp)
pairs that pk and timestamp all same, need to keep just one pair.
2. count(*) may core dump, if get_real_count interface get snapshot and
do mvcc at not consistency status, mainly happens under concurrency.
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-07-29 19:53:50 +08:00
wei liu
c45f38aa61
enhance: Update protobuf-go to protobuf-go v2 ( #34394 )
...
issue: #34252
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-07-29 11:31:51 +08:00
cai.zhang
2372452fac
enhance: Optimized the GC logic to ensure that memory is released in time ( #34949 )
...
issue: #34703
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-07-28 23:53:47 +08:00
congqixia
783f9d9c33
fix: Unify hook singleton implementation in proxy ( #34887 )
...
Related to #34885
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-26 18:07:53 +08:00
congqixia
4ee6c69217
enhance: Add Segment Level in milvus segment info APIs ( #34763 )
...
See also #34746
This PR add segment level field in response of
`GetPersistentSegmentInfo` and `GetQuerySegmentInfo`
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-26 10:01:46 +08:00
congqixia
482373b3d3
enhance: Sync go e2e package proto dependency ( #34913 )
...
Go e2e package point to a lower verion of milvus-proto and may cause
compile error locally.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-23 16:09:43 +08:00
cai.zhang
4c45bc412f
enhance: Add integration test for clustering compaction ( #34881 )
...
issue: #34792
---------
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-07-23 10:13:43 +08:00
wayblink
c339df26fc
enhance: refine clustering compaction basic it ( #34793 )
...
#34792
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-07-22 11:27:51 +08:00
yihao.dai
b22e549844
enhance: Rename config of sealing by growing segmetns size ( #34787 )
...
/kind improvement
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-07-19 20:27:41 +08:00
yihao.dai
4939f82d4f
enhance: Seal by total growing segments size ( #34692 )
...
Seals the largest growing segment if the total size of growing segments
of each shard exceeds the size threshold(default 4GB). Introducing this
policy can help keep the size of growing segments within a suitable
level, alleviating the pressure on the delegator.
issue: https://github.com/milvus-io/milvus/issues/34554
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-07-17 21:45:41 +08:00
wei liu
acb33bba4d
enhance: Preserve fixed-size memory in delegator node for growing segment. ( #34596 )
...
issue: #34595
When consuming insert data on the delegator node, QueryCoord will move
out some sealed segments to manage its memory usage. After the growing
segment gets flushed, some sealed segments from other workers will be
moved back to the delegator node. To avoid the frequent movement of
segments, we estimate the maximum growing row count and preserve a
fixed-size memory in the delegator node.
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-07-15 20:51:46 +08:00
XuanYang-cn
eb472b7f08
enhance: [skip e2e]Enable compaction it test ( #34526 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-07-15 20:45:39 +08:00
elstic
2c462d387b
test: Fix index creation mismatch for sparse vector columns in multivector case ( #34618 )
...
If `is_all_data_type` is true, the case will add float32, f16, bf16, and
sparse vectors, but the created indexes are all `flat` indexes by
default. The sparse type cannot create a flat index. Fix the test code
to create a `SPARSE_INVERTED_INDEX` index for the sparse vector when
is_all_data_type is true
Signed-off-by: elstic <hao.wang@zilliz.com>
2024-07-13 10:45:37 +08:00
smellthemoon
07b94b4615
enhance: support upsert autoid==true ( #30342 )
...
related with: #29258
---------
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-07-11 16:53:35 +08:00
ThreadDao
e4115485b6
test: add cases for gosdk v2 partitions ( #34593 )
...
issue: #33419 #34426
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2024-07-11 16:49:36 +08:00
wei liu
9b37d3f517
enhance: Enable setting the replica number and resource group during collection creation ( #34403 )
...
issue: #30040
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-07-10 10:20:13 +08:00
congqixia
3333160b8d
enhance: Fix lint issues from recent PRs ( #34482 )
...
See also #34483
Some lint issues are introduced due to lack of static check run. This PR
fixes these problems.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-09 10:06:24 +08:00
Gao
ae6d6f91e6
enhance: change autoindex default metric type ( #34261 )
...
issue: #34304
cosine is more widely used in float vectors, and cosine and hamming
distance are 'metrics' which have good geometric properties
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2024-07-08 19:52:24 +08:00
elstic
f7898a3ad6
test: update test case ( #34108 )
...
Signed-off-by: elstic <hao.wang@zilliz.com>
2024-07-08 11:46:10 +08:00
ThreadDao
67747245f4
test: add test cases for gosdk v2 index ( #34431 )
...
issue: #33419
- Add test cases for gosdk v2 index
- Add sparse index for go client
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2024-07-05 09:10:09 +08:00
ThreadDao
e4cece8de8
test: add cases for gosdk v2 upsert ( #34400 )
...
- test: add cases for gosdk v2 upsert
- test: update clientv2 to client reference
issue: #33419
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2024-07-04 14:26:09 +08:00
congqixia
0fd0fcfe1d
enhance: Fix lint issues & sdk testcase ( #34399 )
...
Some lint issue is not detect due to recent static check pipeline issue.
This PR fixes these problem and Go milvusclient testcases.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-03 19:42:10 +08:00
jaime
d6afb31b94
enhance: make subfunctions of datanode component modular ( #33992 )
...
issue: #33994
also remove deprecated channel manager based on the etcd implementation
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-07-01 14:46:07 +08:00
zhuwenxing
00e5a406d4
test: remove dup code ( #34242 )
...
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-06-28 10:46:04 +08:00
nico
f2107ad263
test: update test cases about autoindex ( #34224 )
...
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-06-27 17:06:05 +08:00
ThreadDao
be23495233
test: add cases got gosdk v2 search and databases ( #34204 )
...
issue: #33952 #33627 #33419
---------
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2024-06-27 09:40:08 +08:00
zhuwenxing
a08000cfbf
enhance:[skip e2e]update one pod mode resource ( #34196 )
...
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-06-26 16:54:05 +08:00
ThreadDao
a77e2951af
test: add cases for go milvusclient query ( #33884 )
...
issue: #33419
---------
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2024-06-26 09:52:04 +08:00
sammy.huang
2cc9df5c50
[automated] Update cpu Builder image changes ( #34078 )
...
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-06-25 10:56:10 +08:00
zhuwenxing
4c6f6c54dc
test: refine restful testcases trace ( #34066 )
...
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-06-24 10:36:25 +08:00
