test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-11-30 02:48:45 +08:00
Code Issues Actions 11 Packages Projects Releases Wiki Activity
20,042 Commits 94 Branches 105 Tags 184 MiB
0ecd694305
Commit Graph

8897 Commits

Author SHA1 Message Date
shaoting-huang
0ecd694305
enhance: legacy code clean up (#33838) 
issue: #33839

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
 2024-06-14 14:25:56 +08:00
wei liu
4987067375
enhance: Execute bloom filter apply in parallel to speed up segment predict (#33792) 
issue: #33610

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-06-14 11:37:56 +08:00
cai.zhang
b69e9093c8
fix: Fallback field type when it isn't in request (#33832) 
issue: #33432

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-06-14 09:55:56 +08:00
wayblink
08fcf3f62b
fix: Fix meta prefix overlap bug (#33830) 
#30633

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-06-13 19:27:57 +08:00
wei liu
ab93d9c23d
enhance: Use BatchPkExist to reduce bloom filter func call cost (#33611) 
issue:#33610

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-06-13 17:57:56 +08:00
ArenaSu
2dfa752527
doc: [skip-e2e] add comments for event manager (#33444) 
Add comments for event manager(internal/datanode/event_manager.go).

Signed-off-by: ArenaSu <704427617@qq.com>
 2024-06-13 17:56:06 +08:00
ArenaSu
db59249776
doc: [skip-e2e] add comments for meta table (#33403) 
Add commnets for meta table in internal/rootcoord/meta_table.go.

Signed-off-by: ArenaSu <704427617@qq.com>
 2024-06-13 17:54:09 +08:00
Yinzuo Jiang
7d74edd6dd
fix: update clang-tidy and clang-format from 10 to 12 (#33141) 
Default llvm toolchain version in Ubuntu 20.04 is 10, while Ubuntu 22.04
does not have `clang-tidy-10` or `clang-format-10` by default.

issue: #33142

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
Signed-off-by: Yinzuo Jiang <jiangyinzuo@foxmail.com>
 2024-06-13 15:27:58 +08:00
congqixia
512ea6be5f
enhance: Avoid merging insert data when buffering insert msgs (#33562) 
See also #33561

This PR:
- Use zero copy when buffering insert messages
- Make `storage.InsertCodec` support serialize multiple insert data
chunk into same batch binlog files

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-06-13 11:15:56 +08:00
congqixia
9ab3058da2
fix: Prevent restart timetick sender creating ut datanode (#33790) 
See also #33789

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-06-12 22:29:58 +08:00
congqixia
b39dfc25dc
enhance: Use fastjson lib for unmarshal delete log (#33787) 
```
goos: linux
goarch: amd64
GOMAXPROC=1
cpu: Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz
BenchmarkJsonSerdeStd             343872              3568 ns/op            1335 B/op         25 allocs/op
BenchmarkJsonSerdeFastjson       5124177               234.9 ns/op            16 B/op          1 allocs/op
```

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-06-12 20:41:57 +08:00
chyezh
b3ee1a4bd5
fix: lost DroppedAt field when DropCollection (#33766) 
issue: #33097

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-06-12 20:21:56 +08:00
yihao.dai
9a3e4080f1
enhance: Add comment for channel cp updater (#33759) 
/kind enhancement

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-06-12 20:01:55 +08:00
Cai Yudong
9d4535ce0b
enhance: Handle Float16Vector/BFloat16Vector numpy bulk insert as same as BinaryVector (#33760) 
Issue: #22837

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2024-06-12 17:17:55 +08:00
Buqian Zheng
47b04ea167
enhance: support sparse cardinal hnsw index (#33656) 
issue: #29419

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
 2024-06-12 16:57:55 +08:00
Buqian Zheng
8cb350598c
enhance: Improve GetVectorById of Sparse Float Vector (#33209) 
issue: #29419

* sparse float vector to support raw data mmap

For get vector from chunk cache, I added a unit test but marking it as
skipped due to a known issue. I have tested it locally.

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
 2024-06-12 10:09:55 +08:00
coldWater
6b9901c59f
enhance: add a semaphore for CompactionExecutor (#33558) 
#33182

---------

Signed-off-by: coldWater <254244460@qq.com>
 2024-06-11 17:25:55 +08:00
Chun Han
f7af323d1e
fix: sync partitiion stats blocking balance task(#33741) (#33742) 
related: #33741

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2024-06-11 14:21:56 +08:00
Jiquan Long
ecf2bcee42
enhance: speed up array-equal operator via inverted index (#33633) 
fix: #33632

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2024-06-11 14:13:54 +08:00
chyezh
2b7ee1968f
enhance: new messsage interface for log service (#33286) 
issue: #33285

---------

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-06-11 10:38:01 +08:00
chyezh
8ca5ced821
fix: async warmup will be blocked by state lock (#33686) 
issue: #33685

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-06-10 21:59:53 +08:00
yihao.dai
b1d46eb34b
fix: Fix multiple vector fields import (#33723) 
1. Fix dim mismatch with multi-vector fields and JSON import
2. Enhance: do not display file ID in GetImportResponse.

issue: https://github.com/milvus-io/milvus/issues/33681,
https://github.com/milvus-io/milvus/issues/33682

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-06-10 21:57:54 +08:00
yihao.dai
eb5d4de390
fix: Check if the import job exists (#33672) 
issue: https://github.com/milvus-io/milvus/issues/33671

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-06-10 21:51:55 +08:00
wayblink
a1232fafda
feat: Major compaction (#33620) 
#30633

Signed-off-by: wayblink <anyang.wang@zilliz.com>
Co-authored-by: MrPresent-Han <chun.han@zilliz.com>
 2024-06-10 21:34:08 +08:00
chyezh
f53ab54c5d
enhance: async cgo utility (#33133) 
issue: #30926, #33132

- implement future-based cgo utility.

---------

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-06-09 22:55:53 +08:00
yihao.dai
3540eee977
enhance: Support L0 import (#33514) 
issue: https://github.com/milvus-io/milvus/issues/33157

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-06-07 14:17:20 +08:00
zhagnlu
6ce9df913f
fix: clean vector memory (#33692) 
#33533

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-06-07 10:37:54 +08:00
wei liu
0a3d456688
enhance: add restful api to trigger component stop (#32076) 
issue: #32698
This PR add two rest api for component stop and status check:
1. `/management/stop?role=querynode` can stop the specified component
2. `/management/check/ready?role=rootcoord` can check whether the target
component is serviceable

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-06-07 10:35:54 +08:00
smellthemoon
c61fb1eff5
enhance: do check when add not empty logpath (#33640) 
meta only store logid

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-06-07 10:19:51 +08:00
SimFG
ecee7d90d4
enhance: try to speed up the loading of small collections (#33570) 
- issue: #33569

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-06-07 08:25:53 +08:00
cai.zhang
27cc9f2630
enhance: Support analyze data (#33651) 
issue: #30633

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
Co-authored-by: chasingegg <chao.gao@zilliz.com>
 2024-06-06 17:37:51 +08:00
cai.zhang
cfea3f43cf
fix: Don't sync L0 segments to channel watcher (#33664) 
issue: #33540

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-06-06 15:59:50 +08:00
XuanYang-cn
4dd0c54ca0
fix: Fix l0 compactor may cause DN from OOM (#33554) 
See also: #33547

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-06-06 14:33:52 +08:00
congqixia
f6e251514f
fix: Write back dbid modification for nonDB id collection (#33641) 
See also #33608

Make `fixDefaultDBIDConsistency` also write back collection dbid
modification when nonDB id collection is found.

This fix shall prevent dropped collections of this kind show up again
after dropping and restart.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-06-06 14:29:53 +08:00
wei liu
b69740c8f3
enhance: Remove unnecessary log info during load segment (#33663) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-06-06 14:13:50 +08:00
XuanYang-cn
68c9e7db8c
fix: Sync dropped segment for dropped partition (#33331) 
See also: #33330

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-06-06 10:25:52 +08:00
sre-ci-robot
fd191dd7db
[automated] Update Knowhere Commit (#33655) 
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-06-06 01:53:50 +08:00
cai.zhang
feeb869ff9
enhance: Remove compaction plans on the datanode (#33548) 
issue: #33546

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-06-05 20:27:51 +08:00
cqy123456
703fc73f71
enhance: disk index support binary vector (#33631) 
issue:https://github.com/milvus-io/milvus/issues/22837
related https://github.com/milvus-io/milvus/pull/33575

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2024-06-05 19:37:57 +08:00
cai.zhang
412ccfbb20
enhance: Refine IndexNode code and ensure compatibility (#33458) 
issue: #33432 , #33183

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-06-05 19:35:52 +08:00
zhagnlu
8ad26093ba
fix: fix load failure (#33599) 
issue: #33533

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-06-05 19:19:51 +08:00
aoiasd
b9bafc76b4
enhance: access log support print with consistency level (#33503) 
relate: https://github.com/milvus-io/milvus/issues/33563

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-06-05 17:13:50 +08:00
yihao.dai
01ce32caa1
enhance: Print more disk quota info (#33596) 
/kind enhancement

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-06-05 16:51:56 +08:00
yihao.dai
bbdf99a45e
fix: Fix import segment size is uneven (#33605) 
The data coordinator computed the appropriate number of import segments,
thus when importing in the data node, one can randomly select a segment.

issue: https://github.com/milvus-io/milvus/issues/33604

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-06-05 15:41:51 +08:00
Gao
545d4725fb
fix: correct get vector data size for bf16/fp16/binary vector (#33377) 
related #22837

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2024-06-05 14:31:57 +08:00
congqixia
597f4c5e03
enhance: Make hasMoreResult accurate when hit number larger than limit (#33609) 
See also milvus-io/milvus-sdk-go#756

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-06-05 11:51:51 +08:00
yihao.dai
35532a3e7d
fix: Fill stats log id and check validity (#33477) 
1. Fill log ID of stats log from import
2. Add a check to validate the log ID before writing to meta

issue: https://github.com/milvus-io/milvus/issues/33476

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-06-05 11:17:56 +08:00
congqixia
25080bb455
enhance: Print UseDefaultConsistency param in read requests (#33617) 
`UseDefaultConsistency` param is crucial for debugging slow query
problems. It could be confusing when guarantee timestamp is 1 while this
param is not logged

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-06-05 10:27:50 +08:00
zhenshan.cao
ac4f3997ce
enhance: Reconstructing Compaction to possess persistence capability (#33265) 
issue #33586

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2024-06-05 10:17:50 +08:00
aoiasd
387b7cd7f4
enhance:avoid maintain checkpoint info in sync manager (#33413) 
relate: https://github.com/milvus-io/milvus/issues/32915

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-06-05 10:05:50 +08:00