test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-04 21:09:06 +08:00
Code Issues Actions 14 Packages Projects Releases Wiki Activity
20,310 Commits 96 Branches 105 Tags 186 MiB
c61592dcea
Commit Graph

258 Commits

Author SHA1 Message Date
Xiaofan
be7760a9ab
fix: CVE by upgrading some dependencies. (#34462) 
fix #34434 and #34456
upgrade otelgrpc to fix CVE

Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
 2024-07-16 11:55:36 +08:00
wei liu
9b37d3f517
enhance: Enable setting the replica number and resource group during collection creation (#34403) 
issue: #30040

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-07-10 10:20:13 +08:00
Patrick Weizhi Xu
df5ba3fae3
enhance: update milvus proto (#34490) 
issue: #34336

Revert CollectionSchema changes

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
 2024-07-09 10:04:11 +08:00
Aldrin
686a212d8b
fix: Upgraded Azidentity Package to v1.6.0 (#34464) 
issue : https://github.com/milvus-io/milvus/issues/34456

Signed-off-by: Ald392 <imagesai32@gmail.com>
 2024-07-08 17:51:32 +08:00
chyezh
b9237280c2
enhance: wal interface definition (#33745) 
issue: #33285

---------

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-06-24 10:34:12 +08:00
congqixia
b39dfc25dc
enhance: Use fastjson lib for unmarshal delete log (#33787) 
```
goos: linux
goarch: amd64
GOMAXPROC=1
cpu: Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz
BenchmarkJsonSerdeStd             343872              3568 ns/op            1335 B/op         25 allocs/op
BenchmarkJsonSerdeFastjson       5124177               234.9 ns/op            16 B/op          1 allocs/op
```

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-06-12 20:41:57 +08:00
wei liu
22a059d4de
enhance: update dependency for blobloom (#33565) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-06-03 21:59:46 +08:00
wei liu
c6a1c49e02
enhance: Use Blocked Bloom Filter instead of basic bloom fitler impl. (#33405) 
issue: #32995
To speed up the construction and querying of Bloom filters, we chose a
blocked Bloom filter instead of a basic Bloom filter implementation.

WARN: This PR is compatible with old version bf impl, but if fall back
to old milvus version, it may causes bloom filter deserialize failed.

In single Bloom filter test cases with a capacity of 1,000,000 and a
false positive rate (FPR) of 0.001, the blocked Bloom filter is 5 times
faster than the basic Bloom filter in both querying and construction, at
the cost of a 30% increase in memory usage.

- Block BF construct time	{"time": "54.128131ms"}
- Block BF size	                {"size": 3021578}
- Block BF Test cost	        {"time": "55.407352ms"}
- Basic BF construct time	{"time": "210.262183ms"}
- Basic BF size	                {"size": 2396308}
- Basic BF Test cost	        {"time": "192.596229ms"}

In multi Bloom filter test cases with a capacity of 100,000, an FPR of
0.001, and 100 Bloom filters, we reuse the primary key locations for all
Bloom filters to avoid repeated hash computations. As a result, the
blocked Bloom filter is also 5 times faster than the basic Bloom filter
in querying.

- Block BF TestLocation cost    {"time": "529.97183ms"}
- Basic BF TestLocation cost	{"time": "3.197430181s"}

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-05-31 17:49:45 +08:00
yihao.dai
7730b910b9
enhance: Decouple compaction from shard (#33138) 
Decouple compaction from shard, remove dependencies on shards (e.g.
SyncSegments, injection).

issue: https://github.com/milvus-io/milvus/issues/32809

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-05-24 09:07:41 +08:00
smellthemoon
1671c7898a
enhance: sync milvus proto version (#33094) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-05-16 18:51:34 +08:00
congqixia
8cf2cf5c94
enhance: Add go-deadlock as unittest only dependency (#33063) 
See also #33062

This PR:

- Add `lock.RWMutex` & `lock.Mutex` alias to switch implementation based
  on build flags
- When build flags has `test` in it, use `go-deadlock` to detect
  possible deadlocks
- Replace all `sync.RWMutex` & `sync.Mutex` in datacoord pkg

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-05-15 16:33:34 +08:00
congqixia
b2d83d3354
enhance: Bump milvus version to v2.4.2 (#33048) 
Bumping version to v2.4.2. Also bump milvus-proto version to v2.4.3.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-05-15 13:57:35 +08:00
Xiaofan
36f1ea93a5
enhance: optimize plan parser pool to avoid unnessary recycle (#32869) 
fix #32868
plan parser takes too much cpu on high qps,this pr try to avoid create
lexer and parser too freequent

Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
 2024-05-11 10:51:31 +08:00
wei liu
c35797c399
enhance: expose DescribeDatabase api in proxy (#32732) 
issue: #32707

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-05-09 22:51:30 +08:00
wayblink
42d0412e93
enhance: Add channelCPs in FlushResponse (#32044) 
#32609

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-04-30 09:45:27 +08:00
congqixia
f07e78ec91
enhance: Bump Milvus & proto version to v2.4.1 (#32693) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-29 14:37:26 +08:00
wei liu
07720f1a95
enhance: expose alter database api in proxy (#32639) 
issue: #30040

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-04-28 18:15:31 +08:00
SimFG
bed6363feb
enhance: update the go-api version for the list api (#32605) 
issue: https://github.com/milvus-io/milvus/issues/32550
/kind improvement

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-04-26 17:49:33 +08:00
jaime
2c63f848bf
fix: upgrade nats server to fix security vulnerabilities (#32021) 
issue: #32022

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-04-11 16:07:18 +08:00
SimFG
789e014c74
enhance: add the db id for the describe collection response (#32114) 
/kind improvement
issue: #32110

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-04-10 21:23:20 +08:00
congqixia
9d16aa0bd3
enhance: Bump jose2go for security alerts (#32040) 
See also #31986

- jose2go vulnerable to denial of service via large p2c value

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-10 10:05:19 +08:00
congqixia
a860bf66f9
enhance: Bump google.golang.org/grpc to 1.57.1 (#31985) 
See also #31986
See dependency alert for "gRPC-Go HTTP/2 Rapid Reset vulnerability"

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-09 10:53:17 +08:00
cqy123456
976928ecd1
fix: fix fp16/bf16 some code missing and add more fp16/bf16 test (#31612) 
issue: #31534

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2024-03-28 14:11:10 +08:00
Bingyi Sun
8e661f791a
fix: lazy load index data in cache (#31094) 
issue: https://github.com/milvus-io/milvus/issues/31571

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2024-03-25 15:43:07 +08:00
yihao.dai
2b7df47aa9
feat: Add RBAC functionality to ImportV2 (#31298) 
issue: https://github.com/milvus-io/milvus/issues/31282,
https://github.com/milvus-io/milvus/issues/28521

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-03-19 20:01:07 +08:00
cai.zhang
72ac2b1e93
fix:Update tencent cloud version (#30847) 
Update tencent cloud version for updating token when expired.
issue: #30846

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-03-01 11:09:00 +08:00
chyezh
e59bc1e526
enhance: use latest milvus proto, add unimplemented server (#30889) 
issue: #30647

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-02-29 10:13:00 +08:00
Buqian Zheng
1632668b64
enhance: update milvus-proto dep to add sparse vector related protos (#30400) 
issue: https://github.com/milvus-io/milvus/issues/29419

this PR solely adds proto definition. sparse float vector support will
be in subsequent PRs.

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
 2024-02-02 12:41:04 +08:00
Bingyi Sun
adaf270697
enhance: roll back apimachinery to v0.28.6 (#30398) 
apimachinery use go 1.21 in v0.29.0
reference issue: golang/go#62409

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2024-02-01 20:31:03 +08:00
SimFG
463765922e
enhance: support related privilege for grant api (#30153) 
/kind improvement

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-01-23 15:56:54 +08:00
cai.zhang
6cf2f09b60
feat: Support tencent cloud object storage for milvus (#30163) 
issue: #30162

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-01-23 11:28:56 +08:00
jaime
ee26561b5d
fix: remove checking if running inside container (#29940) 
issue: https://github.com/milvus-io/milvus/issues/29846

Since checking if Milvus running inside the container is difficult,
which is decided by the system version and cgroup version, we try to get
the memory limit of the container each time instead of checking whether
inside the container first.

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-01-14 10:24:52 +08:00
Bingyi Sun
e1258b8cad
feat: integrate storagev2 into loading segment (#29336) 
issue: #29335

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2024-01-12 18:10:51 +08:00
jaime
90984a2cab
enhance: Support read hardware metrics for cgroupv2 (#29850) 
issue: https://github.com/milvus-io/milvus/issues/29846

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-01-12 10:30:51 +08:00
zhenshan.cao
2f702ad316
enhance: update milvus-proto to support rbac for alias (#29801) 
issue: https://github.com/milvus-io/milvus/issues/29781
issue: https://github.com/milvus-io/milvus-proto/issues/237
Update the protocol to support the RBAC functionality for aliases

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2024-01-09 22:02:49 +08:00
wayblink
635a7f777c
feat: add clustering key in create/describe collection (#29506) 
#28410
/kind feature

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-01-07 19:56:48 +08:00
wayblink
05d735c322
enhance: Rename SearchV2 to HybridSearch (#29592) 
related: https://github.com/milvus-io/milvus-proto/pull/233
issue: #29593
/kind enhancement

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-01-04 19:22:46 +08:00
aoiasd
a76e3b2813
Refine delete by expression for forbid proxy dml task scheduler hang (#29340) 
relate: https://github.com/milvus-io/milvus/issues/29146

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-12-26 19:52:48 +08:00
SimFG
dd9c61831d
enhance: Support to get the param value in the runtime (#29297) 
/kind improvement
issue: #29299

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-12-22 18:36:44 +08:00
yah01
a0e1a1eb31
feat: support enable/disable mmap for index (#29005) 
support enable/disable mmap for index, the user could alter the index's
mode by `AlterIndex` method
related: https://github.com/milvus-io/milvus/issues/21866

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-12-21 18:07:24 +08:00
PowderLi
bcd6865b29
enhance: add 3 builtin roles (#28961) 
issue: #28960 [milvus-proto
#212](https://github.com/milvus-io/milvus-proto/issues/212)

add new configuration: builtinRoles
user can define roles in config file: `milvus.yaml`

there is an example:
1. db_ro, only have read privileges, include load
2. db_rw, read and write privileges, include create/drop/rename
collection
3. db_admin, not only read and write privileges, but also user
administration

Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-12-18 14:28:41 +08:00
Bingyi Sun
4fedff6d47
feat: integrate storage v2 into the write path (#28440) 
#28378

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2023-11-23 17:26:24 +08:00
yah01
bfccfcd0ca
enhance: refine error messages (#28424) 
- Split the simple reason and full detail
- Refine existing error messages
related: #28422

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-11-21 17:02:24 +08:00
Bingyi Sun
59355cb3dc
Update arrow version to v12 (#28425) 
issue: https://github.com/milvus-io/milvus/issues/28423

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2023-11-15 10:36:19 +08:00
Jovan
bc85422a04
fix: bitSet panic: runtime error: makeslice: len out of range (#27984) 
Signed-off-by: Jovan <ulovecode@gmail.com>
 2023-10-27 18:02:12 +08:00
congqixia
ee184a709e
Bump Milvus Version & milvus-proto/go-api to v2.3.2 (#27893) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-10-24 21:02:11 +08:00
SimFG
9b0ecbdca7
Support to replicate the mq message (#27240) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-10-20 14:26:09 +08:00
SimFG
84d05b939b
Support the apikey authentication (#27723) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-10-17 21:00:11 +08:00
Xiaofan
333a0c8a53
Fix RocksMQ seek speed (#27646) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-10-16 10:18:08 +08:00
dependabot[bot]
f0e358fc56
Bump golang.org/x/net from 0.10.0 to 0.17.0 (#27656) 
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.10.0 to 0.17.0.
- [Commits](https://github.com/golang/net/compare/v0.10.0...v0.17.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-10-13 17:19:34 +08:00