test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-03 12:29:36 +08:00
Code Issues Actions 13 Packages Projects Releases Wiki Activity
14,582 Commits 96 Branches 105 Tags 184 MiB
2205b878b1
Commit Graph

1010 Commits

Author SHA1 Message Date
jaime
2205b878b1
Refine log in query_shard.go (#16693) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-04-28 17:13:48 +08:00
zhenshan.cao
0a953948af
Simplify monitoring metrics (#16687) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-04-27 23:03:47 +08:00
congqixia
3a6db2faeb
Fix handling segment change logic (#16695) 
Dispatch segmentChangeInfo to ShardCluster leader
Hold segment remove before search is done

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-27 22:23:46 +08:00
Letian Jiang
2879796b42
Validate partitionIDs & segmentIDs in query request (#16684) 
* check partitions if they are released or unloaded
    * check segments if their collection/partition(s) are released or unloaded

Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
 2022-04-27 19:23:48 +08:00
congqixia
0bf176a7e1
Add SyncReplicaSegments interface for QueryNode (#16672) 
Add SyncReplicaSegments for LoadCollection/LoadPartition child task
This API notifies related ShardCluster to record segment Loaded
Preventing the Etcd events is delayed, which will cause some segment not search

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-27 10:41:46 +08:00
Xiaofan
b657b10eb5
Remove useless log (#16624) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-04-27 10:01:47 +08:00
Letian Jiang
f75dedb317
Validate partitionIDs & segmentIDs in search request (#16639) 
* check partitions if they are released or unloaded
* check segments if their collection/partition(s) are released or unloaded

Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
 2022-04-26 14:19:45 +08:00
yah01
80ae6de323
Add comments for concurrency package (#16654) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-04-26 11:33:45 +08:00
congqixia
bb69e09736
Use fmt.Errorf instead of string concat in querynode/segment.go (#16422) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-25 16:11:48 +08:00
yah01
7af02fa531
Improve load performance, load binlogs concurrently per file, deserialize binlogs concurrently per field/segment (#16514) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-04-25 15:57:47 +08:00
congqixia
b99b65c26e
Add segment reference count and handles change info in ShardCluster (#16620) 
Resolves #16619
Add reference count for each search/query request
For SegmentChangeInfo
- Wait all segments in OnlineList to be loaded
- Add handoff event into pending list
- Wait all segments in OfflineList is not used (reference count = 0)

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-25 11:51:46 +08:00
Ten Thousand Leaves
4ef2df8cb9
Fix an issue where query coord calls MinioChunkManager during init (#16629) 
It causes Milvus not runnable with local storage.

issue: #15604

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-04-25 11:11:46 +08:00
Xiaofan
89b4a34892
Fix Data race in NodeID (#16603) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-04-24 22:03:44 +08:00
congqixia
bd29518b6e
Remove debug log with no meaning (#16583) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-22 13:11:42 +08:00
congqixia
6de1d223a6
Remove debug log in querynode segment/node detector (#16572) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-22 11:07:42 +08:00
congqixia
69252f812d
Implement memory replica in Proxy, QueryNode and QueryCoord (#16470) 
Related to #16298 #16291 #16154
Co-authored-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Letian Jiang <letian.jiang@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-20 16:15:41 +08:00
codeman
4e12271d79
Support login with username and password (#15656) (#16341) 
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
 2022-04-11 19:49:34 +08:00
Cai Yudong
71cd7ba67a
Add configuration common.indexSliceSize (#16438) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-04-08 20:29:33 +08:00
xige-16
27b4cbc098
Cherry pick remove translateHits commit to mater (#16436) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Co-authored-by: bigsheeper <yihao.dai@zilliz.com>
 2022-04-08 20:27:31 +08:00
godchen
bb7a0766fe
Add dependency factory (#16204) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-04-07 22:05:32 +08:00
congqixia
4cb7c28bfa
Fix ReplicaInfo package name (#16400) 
Fix #16399
PR 16373 breaks the build after ReplicaInfo moved from querypb to milvuspb
Also fix  problem in Makefile
Limit action/setup-go version to ~1.17.1(>=1.17.1 <1.18.0)

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-06 19:47:31 +08:00
congqixia
18d76261a0
Add ShardNodeDetector implementation (#16373) 
Use etcd as a ShardNodeDetector
Unit tests is based on embedded etcd

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-06 15:43:31 +08:00
bigsheeper
2b27fa4609
Print numTimestamps instead of Timestamps (#16369) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-04-06 14:47:31 +08:00
bigsheeper
4f4a417c8f
Unify in log print (#16368) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-04-06 14:09:35 +08:00
Letian Jiang
09b90fd8bf
Fix QueryCollection retrieve unit test (#16376) 
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
 2022-04-04 21:07:30 +08:00
xige-16
99984b88e1
Support delete varChar value (#16229) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-04-02 17:43:29 +08:00
congqixia
aa1c26de77
Add ShardCluster implementation (#16360) 
ShardCluster maintains shard replica meta information
It watches node & segment change events
Provides shard replica search/query services

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-02 17:35:29 +08:00
Letian Jiang
028c5cb882
Modify grpc interface for replica Search/Query in QueryNode (#16326) 
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
 2022-04-02 14:15:31 +08:00
congqixia
7a44fff8cd
Use fmt.Errorf instead of string concat in tsafe_replica (#16344) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-01 18:37:28 +08:00
Letian Jiang
cb65860e22
Add implementation for QueryShardService (#16317) 
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
 2022-04-01 14:19:29 +08:00
Letian Jiang
ad9a2217ae
Add interface for QueryShardService and QueryShard (#16309) 
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
 2022-03-31 14:05:28 +08:00
congqixia
899638810a
Add error log when tsafe register watcher more than once (#16294) 
Add some test case to cover the error case
Add log in error branch

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-03-30 22:07:28 +08:00
Jiquan Long
ba37531456
Add support for loading multiple indexes (#16138) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-03-30 21:11:28 +08:00
Cai Yudong
5a9e1af3bb
Update create plan CGO interface (#16271) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-03-30 17:13:27 +08:00
congqixia
18a3e9f265
Add grpc interface for replica Search/Query in QueryNode (#16197) 
Resolves #16195
Add Search and Query grpc interface in query proto and types/types.go

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-03-30 12:03:27 +08:00
jaime
d5d6e115bd
Refine delete segment data log (#16241) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-03-29 18:41:27 +08:00
congqixia
91d5436807
Improve log level and param function in QueryCollection (#16254) 
Use debug level for all debug information
Use specific parameter function for appliable ones

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-03-29 12:31:27 +08:00
bigsheeper
9d673006e0
Improve primary keys print (#16232) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-03-28 23:15:27 +08:00
xige-16
d2424fc50a
Print msgID instead of loadInfo when load segment (#16238) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-03-28 23:13:27 +08:00
godchen
478f6ca11e
Add cache for vector chunk (#15912) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-03-26 22:05:26 +08:00
xige-16
205c92e54b
Support insert string data (#15993) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-03-25 14:27:25 +08:00
bigsheeper
7fee4743f5
Improve log print in OOM error info (#16170) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-03-24 20:17:25 +08:00
yah01
63a71e247a
Loading segments concurrency level fallbacks to the value that won't cause OOM (#16172) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-03-24 13:39:25 +08:00
xiyichan
ca129d4308
Support configurable msgstream (#16131) 
Signed-off-by: xiyichan <2863768433@qq.com>
 2022-03-24 10:15:25 +08:00
Jiquan Long
f8d9bc919d
Unify interface of vector index & scalar index. (#15959) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-03-21 14:23:24 +08:00
yah01
31c5b286da
Fix wrong unittest, check whether loading collection will cause OOM (#16068) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-03-18 13:17:22 +08:00
godchen
78557ca6d7
Replace minio kv with minio chunk manager (#15936) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-03-17 18:03:23 +08:00
Ji Bin
3cd28420f1
Support compile under windows (#15786) 
This patch makes compile milvus under windows(MSYS), including:
- some cpp adaptation for compile under msys/gcc-10.3
- install toolchain scripts for setup from MinGW/MSYS `scripts/install_deps_msys.sh`
- adaptation for POSIX API use in golang
  * using gofrs/flock instead of syscall.Flock
  * using x/exp/mmap instead of syscall.Mmap
- introducing github actions for build milvus.exe under windows/MSYS
- rocksdb's patch for MSYS
- adaptation for compile knowhere under windows
- a windows package script for pack zip file, `scripts/package_windows.sh`

issue #7706

Signed-off-by: Ji Bin <matrixji@live.com>
 2022-03-17 17:17:22 +08:00
Jiquan Long
ffdad82b9c
Set load info in reschedule tasks (#16065) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-03-16 11:43:21 +08:00
zhenshan.cao
f3eeecf146
Reduce the number of metrics (#16030) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-03-15 21:51:21 +08:00