test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-01 11:29:48 +08:00
Code Issues Actions 12 Packages Projects Releases Wiki Activity
14,674 Commits 94 Branches 105 Tags 184 MiB
3579ae8240
Commit Graph

5222 Commits

Author SHA1 Message Date
yah01
93416d5524
Fix LoadBalance loads the same segment multiple times (#16734) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-04-29 15:19:47 +08:00
shaoyue
2f49db9d3c
Implements remaining rest APIs (#16387) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2022-04-29 15:15:47 +08:00
yah01
415a0e5978
Fix miss segments/DmChannel when query by nodeID (#16719) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-04-29 14:11:47 +08:00
Letian Jiang
a38c84e7e7
Lock meta-replica during search/query (#16703) 
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
 2022-04-29 13:37:47 +08:00
xige-16
515d0369de
Support string type in segcore (#16546) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Co-authored-by: dragondriver <jiquan.long@zilliz.com>

Co-authored-by: dragondriver <jiquan.long@zilliz.com>
 2022-04-29 13:35:49 +08:00
jaime
68b1b82faf
Remove DataKV interface (#16692) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-04-28 21:03:47 +08:00
groot
21b5973253
Refine list import task api (#16690) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-04-28 17:21:47 +08:00
jaime
2205b878b1
Refine log in query_shard.go (#16693) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-04-28 17:13:48 +08:00
yah01
206b554d49
Sync the distribution of segments to shard leaders (#16685) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-04-28 16:01:48 +08:00
Xiaofan
ed51ffd50f
Fix Compact Entity ttl param (#16665) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-04-28 11:51:47 +08:00
jaime
c0a73dbe98
Using a temporary dir instead of a specified dir within embed etcd test (#16694) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-04-28 11:17:47 +08:00
yah01
7017756601
Validate the number of replicas for load collection/partitions request (#16697) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-04-28 10:54:00 +08:00
zhenshan.cao
0a953948af
Simplify monitoring metrics (#16687) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-04-27 23:03:47 +08:00
congqixia
3a6db2faeb
Fix handling segment change logic (#16695) 
Dispatch segmentChangeInfo to ShardCluster leader
Hold segment remove before search is done

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-27 22:23:46 +08:00
yah01
bb6cd4b484
Disable the unit test for the case of filter out unavailable leaders of GetShardLeaders (#16688) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-04-27 22:15:47 +08:00
Xiaofan
be5d1f1243
Fix Datacoord ut when etcd is slow (#16689) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-04-27 21:49:47 +08:00
xiyichan
056fd9aa17
Correct wrong words (#16490) 
Signed-off-by: chenlifeng <chenlifeng0995@wezhuiyi.com>

Co-authored-by: chenlifeng <chenlifeng0995@wezhuiyi.com>
 2022-04-27 19:51:46 +08:00
Letian Jiang
2879796b42
Validate partitionIDs & segmentIDs in query request (#16684) 
* check partitions if they are released or unloaded
    * check segments if their collection/partition(s) are released or unloaded

Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
 2022-04-27 19:23:48 +08:00
XuanYang-cn
35b7267edb
Add ut for task_policies (#16663) 
See also: #16652

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-04-27 16:55:46 +08:00
yah01
68ef720d6b
Filter out the unavailable shard leaders (#16677) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-04-27 16:27:46 +08:00
congqixia
0bf176a7e1
Add SyncReplicaSegments interface for QueryNode (#16672) 
Add SyncReplicaSegments for LoadCollection/LoadPartition child task
This API notifies related ShardCluster to record segment Loaded
Preventing the Etcd events is delayed, which will cause some segment not search

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-27 10:41:46 +08:00
Xiaofan
b657b10eb5
Remove useless log (#16624) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-04-27 10:01:47 +08:00
yah01
60f7fef3b3
Remove offline nodes from replica info after LoadBalanceTask done (#16653) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-04-26 18:23:46 +08:00
zhenshan.cao
a3f7bb5f2d
Remove the collectionID label from metrics (#16571) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-04-26 17:29:45 +08:00
Ten Thousand Leaves
4855ba6fa3
Optimize Mac M1 support some tweaks to embedded Milvus (#16650) 
issue: #16500
issue: #16621

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-04-26 14:21:46 +08:00
Letian Jiang
f75dedb317
Validate partitionIDs & segmentIDs in search request (#16639) 
* check partitions if they are released or unloaded
* check segments if their collection/partition(s) are released or unloaded

Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
 2022-04-26 14:19:45 +08:00
yah01
80ae6de323
Add comments for concurrency package (#16654) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-04-26 11:33:45 +08:00
Xiaofan
c1ff0cec8f
Change QueryCoord Log level (#16590) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-04-26 11:29:54 +08:00
XuanYang-cn
fbc7fe1cdc
Fix proxy unable to update cache (#16646) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-04-26 11:27:53 +08:00
groot
bc5e9ec929
Add list import tasks api (#16605) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-04-25 17:37:46 +08:00
congqixia
bb69e09736
Use fmt.Errorf instead of string concat in querynode/segment.go (#16422) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-25 16:11:48 +08:00
yah01
7af02fa531
Improve load performance, load binlogs concurrently per file, deserialize binlogs concurrently per field/segment (#16514) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-04-25 15:57:47 +08:00
Bingyi Sun
1682dc49fc
Fix unit test timeout (#16537) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2022-04-25 14:49:46 +08:00
congqixia
b99b65c26e
Add segment reference count and handles change info in ShardCluster (#16620) 
Resolves #16619
Add reference count for each search/query request
For SegmentChangeInfo
- Wait all segments in OnlineList to be loaded
- Add handoff event into pending list
- Wait all segments in OfflineList is not used (reference count = 0)

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-25 11:51:46 +08:00
Ten Thousand Leaves
4ef2df8cb9
Fix an issue where query coord calls MinioChunkManager during init (#16629) 
It causes Milvus not runnable with local storage.

issue: #15604

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-04-25 11:11:46 +08:00
Ten Thousand Leaves
e66ac6a77c
Resolve an issue with bulk load where segments in flow graph replica are not updated (#16609) 
issue: #15604

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-04-25 11:07:47 +08:00
zhenshan.cao
65a9e01a8f
Use AssertInfo instead of assert to ensure compiling (#16533) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-04-25 10:27:46 +08:00
Xiaofan
89b4a34892
Fix Data race in NodeID (#16603) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-04-24 22:03:44 +08:00
yah01
5c66300408
Always add the shard leader into the nodes list of shard (#16616) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-04-24 17:37:46 +08:00
Aivin V. Solatorio
04fffb08ef
Support arithmetic operations on numerical fields for scalar filtering (#16520) 
Signed-off-by: Aivin V. Solatorio <avsolatorio@gmail.com>
 2022-04-24 16:43:45 +08:00
yah01
b82f6a1a0a
Fix GetReplicas() returns all nodes in ShardReplicas (#16597) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-04-24 11:35:45 +08:00
Ten Thousand Leaves
242f08fb67
Refine import task related logic (#16601) 
issue: #16600

/kind bug

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-04-24 11:29:45 +08:00
yah01
2022a361c0
Enable ZSTD compression for pulsar (#16014) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-04-24 10:25:43 +08:00
groot
0cec1b9448
Fix bulkload bug (#16591) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-04-22 18:43:43 +08:00
yah01
3f5733b738
Return the local variable directly, make RVO enabled (#16201) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-04-22 17:43:42 +08:00
yah01
804fbb6e27
Fix panic: Set Base of GetReplicas request (#16578) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-04-22 14:47:42 +08:00
congqixia
bd29518b6e
Remove debug log with no meaning (#16583) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-22 13:11:42 +08:00
congqixia
6de1d223a6
Remove debug log in querynode segment/node detector (#16572) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-22 11:07:42 +08:00
groot
a6a3b69d91
Report bulk load state (#16555) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-04-21 21:37:42 +08:00
codeman
91e84ffedf
fix: not create user root if exists, optimize log & imports (#16554) 
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
 2022-04-21 19:57:42 +08:00
Ten Thousand Leaves
6f75d02c65
Disable knowhere logging for embedded Milvus and some other tweaks (#16496) 
/kind enhancement

issue: #15711
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-04-20 17:23:46 +08:00
XuanYang-cn
93777e0c9d
Handle flowgraph releasing properly (#16169) 
See also: #15846

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-04-20 16:45:41 +08:00
congqixia
69252f812d
Implement memory replica in Proxy, QueryNode and QueryCoord (#16470) 
Related to #16298 #16291 #16154
Co-authored-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Letian Jiang <letian.jiang@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-20 16:15:41 +08:00
Ten Thousand Leaves
289e468a7a
Make bulk load fully work (#16512) 
issue: #15604

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-04-20 14:03:40 +08:00
codeman
3a1b2cedd2
create default root user for authentication (#16545) (#16549) 
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
 2022-04-20 13:03:40 +08:00
congqixia
9769426e84
Use fmt.Errorf instead of string concat in local_chunk_manager.go (#16442) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-19 18:39:39 +08:00
Xiaofan
f367249b9e
Fix storage type config key (#16542) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-04-19 17:47:39 +08:00
codeman
f5e1ded854
fix #16504 (#16523) 
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
 2022-04-19 16:35:39 +08:00
Xiaofan
fce8f6cfab
Remove Timetick channel (#16449) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-04-17 23:47:36 +08:00
Cai Yudong
35e7ed45b3
Upgrade to knowhere v1.1.5 to fully support MinGW build (#16498) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-04-15 09:53:34 +08:00
zhenshan.cao
58ea38142f
Use boost dynamic_bitset in segcore (#16476) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-04-14 22:37:34 +08:00
Ten Thousand Leaves
cf9b9aba79
Fix a bug where RPC calls before proxy finishes initialization crashes Milvus (#16474) 
/kind bug

issue: #16473
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-04-14 20:01:34 +08:00
Ten Thousand Leaves
e76a8c5ec2
Update compile options and config for embedded Milvus (#16472) 
/kind enhancement

issue: #15711
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-04-14 19:57:34 +08:00
jaime
c7e98b922a
Fix mq factory ut (#16483) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-04-14 10:45:34 +08:00
zhenshan.cao
eae0747f14
Fix bug: data race caused by read and write Params.DataNodeCfg.NodeID (#16478) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-04-13 20:09:34 +08:00
Ji Bin
a5ac637ba5
Adaptation windows for Knowhere 1.1.3 (#16444) 
fixes #16443

Signed-off-by: Ji Bin <matrixji@live.com>
 2022-04-13 14:39:34 +08:00
godchen
4781db8a2a
Add datanode import (#16414) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-04-12 22:19:34 +08:00
zwd1208
a2011c1f25
Add env variable for metrics port (#16465) 
Signed-off-by: Weida Zhu <weida.zhu@zilliz.com>
 2022-04-12 20:11:34 +08:00
jaime
ed3c29eae7
Support Kafka (#16149) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-04-12 19:47:33 +08:00
Ten Thousand Leaves
442c73a544
Support passing channel names to DataNode (#16424) 
/kind feature

issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-04-12 13:25:34 +08:00
Cai Yudong
70bd6b6e6d
Remove dependency on knowhere::scalar (#16458) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-04-12 09:49:33 +08:00
codeman
4e12271d79
Support login with username and password (#15656) (#16341) 
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
 2022-04-11 19:49:34 +08:00
congqixia
994bb829d3
Use fmt.Errorf instead of string concat in schema.go (#16404) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-11 09:47:37 +08:00
Cai Yudong
fa697d0d48
[skip e2e] Remove useless OpenMP settings in CMakeLists.txt (#16441) 
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2022-04-09 14:15:32 +08:00
Cai Yudong
71cd7ba67a
Add configuration common.indexSliceSize (#16438) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-04-08 20:29:33 +08:00
xige-16
27b4cbc098
Cherry pick remove translateHits commit to mater (#16436) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Co-authored-by: bigsheeper <yihao.dai@zilliz.com>
 2022-04-08 20:27:31 +08:00
Cai Yudong
a37479d728
Upgrade to knowhere-v1.1.2 to support all index types for mac (#16416) 
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2022-04-08 15:29:31 +08:00
Cai Yudong
b09e56ff81
Remove unused directory (#16419) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-04-08 00:55:31 +08:00
godchen
bb7a0766fe
Add dependency factory (#16204) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-04-07 22:05:32 +08:00
Ten Thousand Leaves
768bd4035f
Drop log level in delete node showDelBuf as they confuse our users. (#16410) 
issue: 16349

/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-04-07 17:59:31 +08:00
Cai Yudong
6c4d497d73
Fix mac build error (#16415) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-04-07 17:29:31 +08:00
XuanYang-cn
909dd73741
Avoid using specific port in ut (#16398) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-04-06 20:13:32 +08:00
congqixia
4cb7c28bfa
Fix ReplicaInfo package name (#16400) 
Fix #16399
PR 16373 breaks the build after ReplicaInfo moved from querypb to milvuspb
Also fix  problem in Makefile
Limit action/setup-go version to ~1.17.1(>=1.17.1 <1.18.0)

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-06 19:47:31 +08:00
congqixia
18d76261a0
Add ShardNodeDetector implementation (#16373) 
Use etcd as a ShardNodeDetector
Unit tests is based on embedded etcd

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-06 15:43:31 +08:00
Ten Thousand Leaves
1e6f6bd072
Implement set segment state call from RootCoord to DataCoord. (#16374) 
/kind feature

issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-04-06 15:33:32 +08:00
XuanYang-cn
78200009a3
Impl GetReplicas in Proxy (#16390) 
See also: #16298

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-04-06 14:57:31 +08:00
bigsheeper
2b27fa4609
Print numTimestamps instead of Timestamps (#16369) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-04-06 14:47:31 +08:00
cai.zhang
4267dc2efc
Handle error and oragnize log (#16367) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-04-06 14:33:30 +08:00
bigsheeper
7161a7760c
Fix segIDAssigner spelling mistake (#16370) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-04-06 14:11:31 +08:00
bigsheeper
4f4a417c8f
Unify in log print (#16368) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-04-06 14:09:35 +08:00
cai.zhang
e73139deeb
Compressed IndexNode log (#16365) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-04-06 11:33:31 +08:00
cai.zhang
04076393c5
Fix log error (#16363) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-04-06 10:13:32 +08:00
cai.zhang
f9c6b3980f
Compressed log (#16354) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-04-06 10:11:36 +08:00
Cai Yudong
543ec4a209
Rename log to milvus_log (#16383) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-04-05 22:55:30 +08:00
Cai Yudong
7323180861
Move bench to directory unittest (#16386) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-04-05 12:43:30 +08:00
Cai Yudong
0c53f06a33
Fix build error on mac (#16380) 
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2022-04-04 21:09:32 +08:00
Letian Jiang
09b90fd8bf
Fix QueryCollection retrieve unit test (#16376) 
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
 2022-04-04 21:07:30 +08:00
Ten Thousand Leaves
2bd0a9526a
Implement task expiring logic and make all deadlines configurable. (#16355) 
/kind feature

issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-04-03 11:37:29 +08:00
groot
be8d9a8b6b
Support import numpy file (#16348) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-04-03 11:27:29 +08:00
xige-16
99984b88e1
Support delete varChar value (#16229) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-04-02 17:43:29 +08:00