test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-11-30 02:48:45 +08:00
Code Issues Actions 11 Packages Projects Releases Wiki Activity
20,220 Commits 94 Branches 105 Tags 184 MiB
a1c65dec89
Commit Graph

20220 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Gao
0d20303e54
fix: fix binary vector data size (#33750) 
issue: https://github.com/milvus-io/milvus/issues/22837

- fix byte size wrong for binary vectors
- fix the expect/actual error msg

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2024-06-18 21:39:59 +08:00
Gao
a789c60380
enhance: autoindex for multi data type (#33868) 
issue: #22837 

contain https://github.com/milvus-io/milvus/pull/33625
https://github.com/milvus-io/milvus/pull/33867
https://github.com/milvus-io/milvus/pull/33911 which already merged to
2.4 branch

Signed-off-by: chasingegg <chao.gao@zilliz.com>
Co-authored-by: foxspy <xianliang.li@zilliz.com>
 2024-06-18 21:34:01 +08:00
sre-ci-robot
74186091d4 Update all contributors 
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
 2024-06-18 12:00:57 +00:00
cqy123456
b460862537
fix: can't find Chunk struct after growing support mmap (#33951) 
issue: https://github.com/milvus-io/milvus/issues/32984

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2024-06-18 18:37:58 +08:00
wayblink
5cb0760187
fix: Small fixs of major compaction (#33929) 
#30633

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-06-18 16:53:58 +08:00
congqixia
3fdaae8792
fix: Return record with largest timestamp for entires with same PK (#33936) 
See also #33883

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-06-18 15:55:59 +08:00
congqixia
5772123ca2
fix: Check the correct return error in MultiRemove (#33926) 
See also #33925

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-06-18 15:23:59 +08:00
wei liu
5e906fd17f
enhance: Force to reset coord connection for unavailable error (#33908) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-06-18 14:53:59 +08:00
cqy123456
32f685ff12
enhance: growing segment support mmap (#32633) 
issue: https://github.com/milvus-io/milvus/issues/32984

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2024-06-18 14:42:00 +08:00
congqixia
ec64499536
fix: Check nodeID wildcard when removing pkOracle (#33895) 
See also #33894

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-06-18 14:11:58 +08:00
shaoting-huang
8cdc0e6233
fix: fix data codec writer close (#33818) 
issue:#33813

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
 2024-06-18 13:59:57 +08:00
presburger
e83ecd5074
enhance: fix not find arrow::parquet lib (#33928) 
Signed-off-by: yusheng.ma <yusheng.ma@zilliz.com>
 2024-06-18 00:22:01 +08:00
sre-ci-robot
88cca9d90f Update all contributors 
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
 2024-06-17 12:00:53 +00:00
congqixia
2a04b0929a
fix: Prevent use captured iteration variable partitionID (#33906) 
See also #33902

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-06-17 19:11:59 +08:00
ThreadDao
2da289c5b5
enhance: add Slice func for go milvusclient column (#33836) 
issue: #33419

Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
 2024-06-17 19:07:58 +08:00
congqixia
f993b2913b
enhance: Reserve space of payload writer when serialize data (#33817) 
See also #33561 #33562

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-06-17 12:06:04 +08:00
wei liu
188ee433b9
enhance: Refine name rule check error msg (#33815) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-06-17 10:37:58 +08:00
cai.zhang
95148866ed
fix: Don't remove growing L0 segment in datanode metacache (#33829) 
issue: #33540 
1. gorwing L0 segments is invisible to datacoord.
2. flushed L0 segments need to clean by datacoord.

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-06-17 10:09:57 +08:00
wei liu
b72026c859
enhance: Add rbac support on describe/alter database api (#33803) 
issue: #32707

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-06-17 10:07:57 +08:00
yihao.dai
1a9ab52f66
enhance: Ensure the idempotency of compaction task (#33872) 
/kind enhancement

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-06-16 22:09:57 +08:00
yihao.dai
8537f3daeb
enhance: Rename Compaction to CompactionV2 (#33858) 
Due to the removal of injection and syncSegments from the compaction, we
need to ensure that no compaction is successfully executed during the
rolling upgrade. This PR renames Compaction to CompactionV2, with the
following effects:
- New datacoord + old datanode: Utilizes the CompactionV2 interface,
resulting in the datanode error "CompactionV2 not implemented," causing
compaction to fail;
- Old datacoord + new datanode: Utilizes the CompactionV1 interface,
resulting in the datanode error "CompactionV1 not implemented," causing
compaction to fail.

issue: https://github.com/milvus-io/milvus/issues/32809

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-06-16 22:07:57 +08:00
zhagnlu
d43ec4db0b
enhance: support array bitmap index (#33527) 
#32900

---------

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-06-16 21:51:58 +08:00
zhagnlu
e422168f09
fix: readd timestamp index because segment timestamp not ordered (#33856) 
#33533

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-06-16 21:48:01 +08:00
XuanYang-cn
f67b6dc2b0
fix: DeleteData merge wrong data casuing data loss (#33820) 
See also: #33819

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-06-14 17:57:56 +08:00
nico
b748d8af5d
test: add different datatype for some functions (#33869) 
Signed-off-by: nico <cheng.yuan@zilliz.com>
 2024-06-14 17:51:57 +08:00
zhuwenxing
a963afa0d3
test: refine load state check in restful test (#33861) 
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2024-06-14 16:22:10 +08:00
chyezh
9b69601dfb
fix: load operation when segment is on releasing (#31340) 
issue: #30857

---------

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-06-14 15:35:56 +08:00
yihao.dai
86a36b105a
enhance: Tidy compaction executor (#33778) 
Move compaction executor to compaction pacakge.

issue: https://github.com/milvus-io/milvus/issues/32451

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-06-14 14:34:01 +08:00
shaoting-huang
0ecd694305
enhance: legacy code clean up (#33838) 
issue: #33839

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
 2024-06-14 14:25:56 +08:00
wei liu
4987067375
enhance: Execute bloom filter apply in parallel to speed up segment predict (#33792) 
issue: #33610

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-06-14 11:37:56 +08:00
cai.zhang
b69e9093c8
fix: Fallback field type when it isn't in request (#33832) 
issue: #33432

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-06-14 09:55:56 +08:00
sre-ci-robot
f3e4cb0de9 Update all contributors 
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
 2024-06-14 00:01:27 +00:00
congqixia
c689ef4822
fix: Remove eviction operations out of lock (#33834) 
See also #33823

`EvictCacheValueByFormat` may be block by on going `CASCacheValue` and
cause possible deadlock

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-06-13 21:15:55 +08:00
wayblink
08fcf3f62b
fix: Fix meta prefix overlap bug (#33830) 
#30633

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-06-13 19:27:57 +08:00
wei liu
ab93d9c23d
enhance: Use BatchPkExist to reduce bloom filter func call cost (#33611) 
issue:#33610

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-06-13 17:57:56 +08:00
ArenaSu
2dfa752527
doc: [skip-e2e] add comments for event manager (#33444) 
Add comments for event manager(internal/datanode/event_manager.go).

Signed-off-by: ArenaSu <704427617@qq.com>
 2024-06-13 17:56:06 +08:00
ArenaSu
db59249776
doc: [skip-e2e] add comments for meta table (#33403) 
Add commnets for meta table in internal/rootcoord/meta_table.go.

Signed-off-by: ArenaSu <704427617@qq.com>
 2024-06-13 17:54:09 +08:00
zhuwenxing
ca1f7ab019
test: update import test case to support different dim (#33709) 
add test case for https://github.com/milvus-io/milvus/issues/33681

---------

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2024-06-13 17:11:55 +08:00
congqixia
144ee269f2
fix: [skip e2e] Skip unstable integration test for master (#33824) 
See also #33716 #33823

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-06-13 16:53:55 +08:00
Yinzuo Jiang
7d74edd6dd
fix: update clang-tidy and clang-format from 10 to 12 (#33141) 
Default llvm toolchain version in Ubuntu 20.04 is 10, while Ubuntu 22.04
does not have `clang-tidy-10` or `clang-format-10` by default.

issue: #33142

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
Signed-off-by: Yinzuo Jiang <jiangyinzuo@foxmail.com>
 2024-06-13 15:27:58 +08:00
ThreadDao
9e99f960ab
test: add cases for go milvusclient delete (#33777) 
issue: #33419

Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
 2024-06-13 14:43:57 +08:00
congqixia
512ea6be5f
enhance: Avoid merging insert data when buffering insert msgs (#33562) 
See also #33561

This PR:
- Use zero copy when buffering insert messages
- Make `storage.InsertCodec` support serialize multiple insert data
chunk into same batch binlog files

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-06-13 11:15:56 +08:00
congqixia
9ab3058da2
fix: Prevent restart timetick sender creating ut datanode (#33790) 
See also #33789

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-06-12 22:29:58 +08:00
congqixia
b39dfc25dc
enhance: Use fastjson lib for unmarshal delete log (#33787) 
```
goos: linux
goarch: amd64
GOMAXPROC=1
cpu: Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz
BenchmarkJsonSerdeStd             343872              3568 ns/op            1335 B/op         25 allocs/op
BenchmarkJsonSerdeFastjson       5124177               234.9 ns/op            16 B/op          1 allocs/op
```

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-06-12 20:41:57 +08:00
chyezh
b3ee1a4bd5
fix: lost DroppedAt field when DropCollection (#33766) 
issue: #33097

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-06-12 20:21:56 +08:00
cqy123456
be3559e022
enhance: proxy check hnsw with sparse is legal (#33697) 
issue:https://github.com/milvus-io/milvus/issues/22837
related:https://github.com/milvus-io/milvus/pull/33653

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2024-06-12 20:13:55 +08:00
yihao.dai
9a3e4080f1
enhance: Add comment for channel cp updater (#33759) 
/kind enhancement

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-06-12 20:01:55 +08:00
congqixia
b90999b741
fix: Compare config value then swap when caching param value (#33785) 
See also #33784

This PR change the behavior of `SetCacheValue` of config manager:

- Use mutex and map instead of concurrent map for `configCache`
- Compare config raw value before set cache value

With this implementation, concurrent caching & eviction shall always
have current output:

|time|caching |eviction|config   |cached   |
|----|--------|------- |---------|---------|
|t0  |get     |        |old value|null     |
|t1  |CAS OK  |        |old value|old value|
|t2  |        |update  |new value|old value|
|t3  |        |eviction|new value|null     |

|time|caching |eviction|config   |cached   |
|----|--------|------- |---------|---------|
|t0  |get     |        |old value|null     |
|t1  |        |update  |new value|null     |
|t2  |CAS fail|        |old value|null     |
|t3  |        |eviction|new value|null     |

|time|caching |eviction|config   |cached   |
|----|--------|------- |---------|---------|
|t0  |        |update  |new value|null     |
|t1  |get     |        |new value|null     |
|t2  |CAS OK  |        |new value|new value|
|t3  |        |eviction|new value|null     |

|time|caching |eviction|config   |cached   |
|----|--------|------- |---------|---------|
|t0  |        |update  |new value|null     |
|t1  |get     |        |new value|null     |
|t2  |        |eviction|new value|null     |
|t3  |CAS OK  |        |new value|new value|

|time|caching |eviction|config   |cached   |
|----|--------|------- |---------|---------|
|t0  |        |update  |new value|null     |
|t1  |        |eviction|new value|null     |
|t2  |get     |        |new value|null     |
|t3  |CAS OK  |        |new value|new value|

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-06-12 18:51:56 +08:00
Cai Yudong
9d4535ce0b
enhance: Handle Float16Vector/BFloat16Vector numpy bulk insert as same as BinaryVector (#33760) 
Issue: #22837

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2024-06-12 17:17:55 +08:00
Buqian Zheng
47b04ea167
enhance: support sparse cardinal hnsw index (#33656) 
issue: #29419

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
 2024-06-12 16:57:55 +08:00