test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-03 04:19:18 +08:00
Code Issues Actions 13 Packages Projects Releases Wiki Activity
15,442 Commits 95 Branches 105 Tags 184 MiB
2bafdf8c53
Commit Graph

830 Commits

Author SHA1 Message Date
bigsheeper
2d9a52206d
Use jemalloc in QueryNode, DataNode and IndexNode (#17470) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-06-10 11:24:08 +08:00
xige-16
6d078ebe7a
Print more information when loading fields with different lines (#17448) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-06-08 20:16:06 +08:00
bigsheeper
af108b5bda
Add malloc_trim after reduce search result (#17423) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-06-08 00:12:05 +08:00
xige-16
b5c11a216d
Alter varChar type params's name to max_length (#17409) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-06-07 15:58:06 +08:00
bigsheeper
f38637c227
Pass PlaceholderGroup pointer to prevent memory copy in SegCore (#17389) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-06-06 21:34:05 +08:00
bigsheeper
9f786dd752
Remove unnecessary bitset flip in Search and Query (#17387) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-06-06 17:38:05 +08:00
Jiquan Long
e88ffb8a57
Enable marisa trie ut on MacOS (#17316) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-06-02 10:48:03 +08:00
Xiaofan
9579a645c6
Support compile marisa on Macos (#17261) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-05-31 13:28:02 +08:00
SimFG
731870211a
Support role-based access control (only pb) (#16987) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-05-28 00:04:01 +08:00
Letian Jiang
f2a27e0e64
Retry GetShardLeaders until service available or timeout (#17183) 
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
 2022-05-26 20:28:02 +08:00
xige-16
56778787be
Reverse data from scalar index (#17145) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-05-26 14:58:01 +08:00
xige-16
7d810ac159
Fix query empty when insert same pk after deletion (#17222) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-05-25 22:06:00 +08:00
Ten Thousand Leaves
5e1e7a6896
Fix an issue that causes flush hanging (#17169) 
issue: #16628
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-05-25 14:34:00 +08:00
xige-16
a8829554eb
Merge multiple sets of expr's optypes in diff dirs (#17192) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-05-24 21:56:00 +08:00
zhenshan.cao
d05c24a0ff
Delete unused code (#17182) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-05-24 14:49:59 +08:00
zhenshan.cao
08a4dff41d
Add logic for search merging and a simple task scheduler for read tasks (#17022) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Co-authored-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: bigsheeper <yihao.dai@zilliz.com>

Co-authored-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: bigsheeper <yihao.dai@zilliz.com>
 2022-05-23 16:41:58 +08:00
Enwei Jiao
d28a2db46c
move arrow from storage to core (#17061) 
Signed-off-by: Enwei Jiao <jiaoew2011@gmail.com>
 2022-05-22 20:03:58 +08:00
yah01
7746a5b742
Add NodeIds field for QuerySegmentInfo (#17121) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-05-20 18:03:58 +08:00
Jiquan Long
99b4042f8c
Disable segment small index (#16935) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-05-18 15:27:56 +08:00
Jiquan Long
77b0f290a8
Fix prefix match on sealed segment (#17050) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-05-17 20:15:55 +08:00
xige-16
935b729a0c
Fix multiple deletes not taking effect (#17033) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-05-16 18:39:56 +08:00
XuanYang-cn
43b36b4e5f
Make proxy update cache with NotShardLeader status code (#16940) 
See also: #16926

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-05-13 11:35:53 +08:00
Jiquan Long
c5149c7e85
Replace text codec of pb with binary codec (#16955) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-05-13 09:53:53 +08:00
Ten Thousand Leaves
63385b7c12
Make data queryable and data indexed states as independent flags (#16837) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-05-12 19:23:53 +08:00
xige-16
08ad77c71b
Delete all repeated primary keys (#16863) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-05-12 14:09:53 +08:00
Letian Jiang
b3eb2b1d0d
Support deltaLog loading on growing segment (#16903) 
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
 2022-05-12 11:59:53 +08:00
bigsheeper
64d2793fe9
Fix wrong result topks when reduce search result (#16815) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-05-06 21:17:51 +08:00
Jiquan Long
98ceb162aa
Support antlr as plan parser (#16696) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>

Co-authored-by: xaxys <tpnnghd@163.com>

Co-authored-by: xaxys <tpnnghd@163.com>
 2022-05-06 17:43:51 +08:00
groot
8736372fd2
Fix bulkload bugs (#16760) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-05-06 11:21:50 +08:00
Ten Thousand Leaves
1acd256481
Add DataQueryable and DataIndexed states for bulk load tasks (#16725) 
issue: #16607

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-05-05 21:17:50 +08:00
Xiaofan
337ad53613
Support config small index nprobe and nlist (#16731) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-05-05 09:49:50 +08:00
Cai Yudong
21a1311f66
Merge utils/Utils.h into common/Utils.h (#16762) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-05-03 12:05:50 +08:00
Cai Yudong
6a62ff18bf
Support easylogging config for segcore and knowhere (#16751) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-05-03 08:39:49 +08:00
Cai Yudong
d5db4ae463
Merge utils/Types.h with common/Types.h (#16445) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-05-02 22:21:51 +08:00
xige-16
515d0369de
Support string type in segcore (#16546) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Co-authored-by: dragondriver <jiquan.long@zilliz.com>

Co-authored-by: dragondriver <jiquan.long@zilliz.com>
 2022-04-29 13:35:49 +08:00
groot
21b5973253
Refine list import task api (#16690) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-04-28 17:21:47 +08:00
Ten Thousand Leaves
4855ba6fa3
Optimize Mac M1 support some tweaks to embedded Milvus (#16650) 
issue: #16500
issue: #16621

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-04-26 14:21:46 +08:00
groot
bc5e9ec929
Add list import tasks api (#16605) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-04-25 17:37:46 +08:00
zhenshan.cao
65a9e01a8f
Use AssertInfo instead of assert to ensure compiling (#16533) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-04-25 10:27:46 +08:00
Aivin V. Solatorio
04fffb08ef
Support arithmetic operations on numerical fields for scalar filtering (#16520) 
Signed-off-by: Aivin V. Solatorio <avsolatorio@gmail.com>
 2022-04-24 16:43:45 +08:00
yah01
3f5733b738
Return the local variable directly, make RVO enabled (#16201) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-04-22 17:43:42 +08:00
yah01
804fbb6e27
Fix panic: Set Base of GetReplicas request (#16578) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-04-22 14:47:42 +08:00
Ten Thousand Leaves
6f75d02c65
Disable knowhere logging for embedded Milvus and some other tweaks (#16496) 
/kind enhancement

issue: #15711
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-04-20 17:23:46 +08:00
congqixia
69252f812d
Implement memory replica in Proxy, QueryNode and QueryCoord (#16470) 
Related to #16298 #16291 #16154
Co-authored-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Letian Jiang <letian.jiang@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-20 16:15:41 +08:00
codeman
f5e1ded854
fix #16504 (#16523) 
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
 2022-04-19 16:35:39 +08:00
Cai Yudong
35e7ed45b3
Upgrade to knowhere v1.1.5 to fully support MinGW build (#16498) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-04-15 09:53:34 +08:00
zhenshan.cao
58ea38142f
Use boost dynamic_bitset in segcore (#16476) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-04-14 22:37:34 +08:00
Ji Bin
a5ac637ba5
Adaptation windows for Knowhere 1.1.3 (#16444) 
fixes #16443

Signed-off-by: Ji Bin <matrixji@live.com>
 2022-04-13 14:39:34 +08:00
Ten Thousand Leaves
442c73a544
Support passing channel names to DataNode (#16424) 
/kind feature

issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-04-12 13:25:34 +08:00
Cai Yudong
70bd6b6e6d
Remove dependency on knowhere::scalar (#16458) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-04-12 09:49:33 +08:00
codeman
4e12271d79
Support login with username and password (#15656) (#16341) 
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
 2022-04-11 19:49:34 +08:00
Cai Yudong
fa697d0d48
[skip e2e] Remove useless OpenMP settings in CMakeLists.txt (#16441) 
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2022-04-09 14:15:32 +08:00
Cai Yudong
71cd7ba67a
Add configuration common.indexSliceSize (#16438) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-04-08 20:29:33 +08:00
xige-16
27b4cbc098
Cherry pick remove translateHits commit to mater (#16436) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Co-authored-by: bigsheeper <yihao.dai@zilliz.com>
 2022-04-08 20:27:31 +08:00
Cai Yudong
a37479d728
Upgrade to knowhere-v1.1.2 to support all index types for mac (#16416) 
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2022-04-08 15:29:31 +08:00
Cai Yudong
b09e56ff81
Remove unused directory (#16419) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-04-08 00:55:31 +08:00
Cai Yudong
6c4d497d73
Fix mac build error (#16415) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-04-07 17:29:31 +08:00
XuanYang-cn
78200009a3
Impl GetReplicas in Proxy (#16390) 
See also: #16298

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-04-06 14:57:31 +08:00
Cai Yudong
543ec4a209
Rename log to milvus_log (#16383) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-04-05 22:55:30 +08:00
Cai Yudong
7323180861
Move bench to directory unittest (#16386) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-04-05 12:43:30 +08:00
Cai Yudong
0c53f06a33
Fix build error on mac (#16380) 
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2022-04-04 21:09:32 +08:00
xige-16
99984b88e1
Support delete varChar value (#16229) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-04-02 17:43:29 +08:00
Jiquan Long
fd589baca7
Integrates marisa trie index (#16192) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-04-01 15:31:29 +08:00
zhenshan.cao
b03da87df5
Accelerate query speed in sealed segment (#16287) 
Fix delete duplicate key

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-03-31 17:33:28 +08:00
Ten Thousand Leaves
0d54697d42
Implement ReportImport related logic. (#16312) 
/kind feature

issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-03-31 13:51:28 +08:00
Jiquan Long
ba37531456
Add support for loading multiple indexes (#16138) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-03-30 21:11:28 +08:00
Cai Yudong
5a9e1af3bb
Update create plan CGO interface (#16271) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-03-30 17:13:27 +08:00
Cai Yudong
0baa8c60a5
Fix MinGW build error (#16267) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-03-30 00:15:27 +08:00
Cai Yudong
1160693e4c
Fix centos build error (#16251) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-03-29 15:13:30 +08:00
Cai Yudong
2ac53c89e9
Revert #16242 (#16247) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-03-29 00:53:27 +08:00
Cai Yudong
9a6b2b8806
Fix centos build error (#16242) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-03-28 23:23:27 +08:00
Ji Bin
4f08f6d8c6
Workaround for knowhere under windows (#16214) 
issue #16205

Signed-off-by: Ji Bin <matrixji@live.com>
 2022-03-28 17:41:27 +08:00
Cai Yudong
3e3756d3dc
Upgrade to knowhere v1.1.1 to support Mac M1 build (#16210) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-03-28 14:33:26 +08:00
Cai Yudong
f4ebd3a9ce
Upgrade to knowhere v1.1.0 (#16186) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-03-25 13:49:25 +08:00
yah01
0d16ecf0dd
Add protobuf messages to support multi memory replica (#16153) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-03-25 11:13:25 +08:00
Jiquan Long
48706f416f
Migrate scalar index from knowhere (#16174) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-03-24 14:57:26 +08:00
congqixia
661469445c
Copy index file content to c memory (#16162) 
Might be the root cause of #15896
The index content bytes were passed via unsafe.Pointer,
which shall be copy to cpp managed memory inmediately

This solution might cause a lot of memory consumption.
Maybe we need to store the index bytes in cpp memory in the first place.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-03-24 14:55:25 +08:00
groot
478890a700
modify import proto (#16155) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-03-23 19:29:25 +08:00
Letian Jiang
321105cc01
Add column-based insert interface in segcore (#16100) 
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
 2022-03-23 19:05:25 +08:00
groot
1c4b949a1d
modify import rpc interface (#16129) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-03-22 15:11:24 +08:00
groot
2078b24dca
Prepare for import (#16083) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-03-21 15:47:23 +08:00
Jiquan Long
f8d9bc919d
Unify interface of vector index & scalar index. (#15959) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-03-21 14:23:24 +08:00
Ji Bin
3cd28420f1
Support compile under windows (#15786) 
This patch makes compile milvus under windows(MSYS), including:
- some cpp adaptation for compile under msys/gcc-10.3
- install toolchain scripts for setup from MinGW/MSYS `scripts/install_deps_msys.sh`
- adaptation for POSIX API use in golang
  * using gofrs/flock instead of syscall.Flock
  * using x/exp/mmap instead of syscall.Mmap
- introducing github actions for build milvus.exe under windows/MSYS
- rocksdb's patch for MSYS
- adaptation for compile knowhere under windows
- a windows package script for pack zip file, `scripts/package_windows.sh`

issue #7706

Signed-off-by: Ji Bin <matrixji@live.com>
 2022-03-17 17:17:22 +08:00
Cai Yudong
d65cf5309f
Remove useless LogMgr (#16080) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-03-16 21:19:21 +08:00
Ten Thousand Leaves
a3a29fed06
Add entry point for embedded Milvus. (#16008) 
/kind feature

issue: #15711
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-03-16 16:51:22 +08:00
xige-16
09b2c46b61
Support create collection with VarChar field (#15904) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-03-14 23:20:02 +08:00
groot
89356ebcb9
Modify import rpc interfaces (#16006) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2022-03-11 17:13:59 +08:00
Letian Jiang
8f52e5b6c7
Add unit tests in segcore (#15960) 
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
 2022-03-10 16:33:59 +08:00
groot
5fdef607d5
Add rpc interfaces for import (#15930) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2022-03-09 18:33:59 +08:00
Cai Yudong
529098fd87
[skip ci] Update knowhere/CMakeLists.txt (#15804) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-03-01 17:55:55 +08:00
Cai Yudong
503724be19
Optimize CMakeLists.txt under internal/core (#15770) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-03-01 10:31:55 +08:00
Nemo
a77e0562b2
Make C++ log level configurable and add log configuration while building embedded milvus. (#15743) 
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-03-01 10:15:55 +08:00
Cai Yudong
adca79facc
Fix centos build error (#15760) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-02-25 18:49:54 +08:00
Cai Yudong
54b8b24151
Rename variable names for better readibility (#15700) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-02-22 22:15:52 +08:00
Cai Yudong
9a6ee61731
Change all fields in Expr to const to avoid modification (#15663) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-02-22 14:45:51 +08:00
zhenshan.cao
142848fcc3
Abandon using protobuf to pass binaryset parameter (#15626) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-02-18 18:39:50 +08:00
Nemo
dc65b1aeea
Switch to knowhere v1.0.1. Also switch to forked gorocksdb v0.0.1 (#15603) 
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-02-17 19:05:49 +08:00
jaime
cb9c63d039
Remove index dir (#15467) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-02-09 16:07:45 +08:00
jaime
307a8ce535
Support compile and run on Mac (#15491) 
Co-authored-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: Cai Yudong <yudong.cai@zilliz.com>
Co-authored-by: Jenny Li <jing.li@zilliz.com>
Co-authored-by: Nemo <yuchen.gao@zilliz.com>
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: Cai Yudong <yudong.cai@zilliz.com>
Co-authored-by: Jenny Li <jing.li@zilliz.com>
Co-authored-by: Nemo <yuchen.gao@zilliz.com>
 2022-02-09 14:27:46 +08:00
Ji Bin
d83f69bb21
Support windows compliation (#15448) 
- remove dl dependency while compiling some lib under MSYS
- int types aligement in golang
- core/src/utils adaptation for compile under MSYS

issue: #7706

Signed-off-by: Ji Bin <matrixji@live.com>
 2022-02-07 23:43:46 +08:00