test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-09 23:40:06 +08:00
Code Issues Actions 15 Packages Projects Releases Wiki Activity
5,593 Commits 96 Branches 105 Tags 189 MiB
6bc7e6d372
Commit Graph

1205 Commits

Author SHA1 Message Date
FluorineDog
08ee90e1de Add gtest dependencies 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2020-08-27 10:08:42 +08:00
become-nice
22a44c995d Add serialzation function 
Signed-off-by: become-nice <995581097@qq.com>
 2020-08-26 18:29:42 +08:00
FluorineDog
fc6e31c8b0 Add test for unique_ptr manual release 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2020-08-26 17:38:45 +08:00
FluorineDog
3c4a9f38b0 Fix compatibility of Gtest 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2020-08-26 17:30:32 +08:00
FluorineDog
996c591088 Enable Segment Naive and Base, support Collection 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2020-08-26 16:49:03 +08:00
become-nice
b8f012d8f0 Add interface of writenode 
Signed-off-by: become-nice <995581097@qq.com>
 2020-08-24 11:58:51 +08:00
yefu.chen
49058d5dfb start 2.0 
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-04-21 11:37:21 +08:00
shengjun.li
83cbe0f490
fix hamming (#3338) 
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-08-20 09:42:04 +08:00
MXDA
1962b6e78b
Specifying distance type at runtime for Flat index (#3273) 
* modfied IndexIDMAP.cpp && SearchTask.cpp

Signed-off-by: Zhengda Wu <wuzhengda55@gmail.com>

* Specifying distance type at runtime for Flat index(CPU)

Signed-off-by: Zhengda Wu <wuzhengda55@gmail.com>

* Specifying distance type at runtime for Flat index(GPU)

Signed-off-by: Zhengda Wu <wuzhengda55@gmail.com>

Co-authored-by: Zhengda Wu <wuzhengda55@gmail.com>
Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
 2020-08-17 17:05:59 +08:00
shengjun.li
5f830293f9
rm find faiss (#3269) 
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-08-15 19:18:03 +08:00
jinhai
f59c240457 Update to 0.10.3 
Signed-off-by: jinhai <hai.jin@zilliz.com>
 2020-08-15 16:21:43 +08:00
shengjun.li
43c272fd13
fix annoy l2 (#3232) 
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-08-12 19:11:13 +08:00
Xiaohai Xu
058cdf03bb
Add ivf index nprobe logging (#3070) 
* add nprobe logging

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* add all cases

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
 2020-08-01 09:46:53 +08:00
shengjun.li
09cf1b323e
fix crash when GPU search (#3074) 
* fix crash when GPU search

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-07-31 09:46:55 +08:00
shengjun.li
b1b1165c56
Opt faiss builder (#3067) 
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-07-31 09:35:31 +08:00
shengjun.li
1c36e6d83d
optimizer parallel policy (#3014) 
* optimizer parallel policy

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* modify parallel policy

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-07-26 01:37:23 +08:00
BossZou
c394608d4f
[skip ci] Up to v0.10.2 (#3006) 
Signed-off-by: yhz <413554850@qq.com>
 2020-07-25 10:02:09 +08:00
cqy123456
91d3fe5cbc
Improve IVF search performance when NQ and nProbe are both large (#2984) 
* fix indexflat  search

Signed-off-by: cqy <yaya645@126.com>

* add parallel_policy_threshold

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
 2020-07-24 15:56:47 +08:00
Tinkerrr
dd938878ea
fix (#2981) 
Signed-off-by: Tinkerrr <xiaojun.lin@zilliz.com>
 2020-07-23 17:02:28 +08:00
shengjun.li
3908d10498
Fix the result merging of IVF_PQ IP (#2953) 
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-07-22 10:08:05 +08:00
shengjun.li
a88f5b6cbb
Fix index size (#2941) 
* add virtual IF UpdateIndexSize

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* update ivf index size

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix PQ logic

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix index size of index hnsw, annoy and nsg

Signed-off-by: cmli <chengming.li@zilliz.com>

* add GetSize() interface 4 SPTAG

Signed-off-by: cmli <chengming.li@zilliz.com>

* fix binary ivf

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

Co-authored-by: sahuang <xiaohai.xu@zilliz.com>
Co-authored-by: cmli <chengming.li@zilliz.com>
 2020-07-21 14:26:08 +08:00
groot
f31a81ab16
add error log for empty result (#2923) 
* add error log for empty result

Signed-off-by: groot <yihua.mo@zilliz.com>

* add error log for empty result

Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-07-18 19:30:29 +08:00
godchen0212
c083556189
add web_impl util unittest (#2886) 
* add testcase of empty vector insert

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* format python code

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* add web_impl util unittest

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* format code

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* format code

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2020-07-18 10:38:46 +08:00
Tinkerrr
8046f2946e
Hotfix 2487 (#2892) 
* update.

Signed-off-by: Tinkerrr <linxiaojun.cn@outlook.com>

* update.

Signed-off-by: Tinkerrr <xiaojun.lin@zilliz.com>

* add fd check

Signed-off-by: Tinkerrr <xiaojun.lin@zilliz.com>

* fix

Signed-off-by: Tinkerrr <xiaojun.lin@zilliz.com>
 2020-07-17 21:00:42 +08:00
Wang XiangYu
283e087a78
add config value gpu.cache.enable (#2899) 
* add config value gpu.cache.enable

Signed-off-by: wxyu <xy.wang@zilliz.com>

* clang-format

Signed-off-by: wxyu <xy.wang@zilliz.com>

* clang-format

Signed-off-by: wxyu <xy.wang@zilliz.com>

* fix ut

Signed-off-by: wxyu <xy.wang@zilliz.com>
 2020-07-17 20:59:31 +08:00
XuanYang-cn
8135cee414
Add some cpp unittests about scheduler algorithm (#2888) 
* add some testcases

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* reformat code

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Co-authored-by: yangxuan <xuan.yang@zilliz.com>
 2020-07-16 19:46:21 +08:00
XuanYang-cn
ea58edc128
Add python functional tests about timeout (#2793) 
* Add python tests about param timeout

Signed-off-by: yangxuan <jumpthepig@gmail.com>

* add faiss configure args msg print

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* change assertion conditions

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Co-authored-by: yangxuan <xuan.yang@zilliz.com>
 2020-07-15 14:32:40 +08:00
yukun
a474ff269a
Server down during loading data (#2807) 
* Server down during loading data

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix test_config.cpp

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* ci retry

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Change limit_in_bytes init value to max

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

Co-authored-by: Jin Hai <hai.jin@zilliz.com>
 2020-07-14 00:40:20 +08:00
cqy123456
585c8ea018
fix rnsg ip (#2827) 
Signed-off-by: cqy <yaya645@126.com>
 2020-07-13 14:05:28 +08:00
groot
5447abec5b
fix #2768 (#2809) 
* fix #2752

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix #2768

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: groot <yihua.mo@zilliz.com>

* typo again

Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-07-12 15:41:45 +08:00
cqy123456
bdebffa73c
fix search error (#2806) 
* fix search error

Signed-off-by: cqy <yaya645@126.com>

* fix search in test_web

Signed-off-by: cqy <yaya645@126.com>
 2020-07-11 13:34:15 +08:00
groot
d7c47a7905
fix #2752 (#2789) 
* fix #2752

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2020-07-10 16:49:47 +08:00
BossZou
a970c7f2fd
Add log to check search result structure (#2786) 
* Add log to check search result structure

Signed-off-by: yhz <413554850@qq.com>

* CI retry

Signed-off-by: yhz <413554850@qq.com>

* Retry

Signed-off-by: yhz <413554850@qq.com>
 2020-07-10 09:46:21 +08:00
op-hunter
4922c9532f
fix a bug of getting wrong nprobe limitation in knowhere on GPU version (#2788) 
* fix a bug of getting wrong nprobe limitation in knowhere on GPU version

Signed-off-by: cmli <chengming.li@zilliz.com>

* update changelog and fix build error

Signed-off-by: cmli <chengming.li@zilliz.com>

Co-authored-by: cmli <chengming.li@zilliz.com>
 2020-07-09 23:15:07 +08:00
shengjun.li
4038b0653e
fix too many data copies (#2784) 
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-07-09 09:23:49 +08:00
Wang XiangYu
cf47007545
update config gpu_search_threshold description (#2748) 
Signed-off-by: Xiangyu Wang <xy.wang@zilliz.com>
 2020-07-06 12:59:44 +08:00
BossZou
5bd75f9381 Web fix 2690 (#2691) 
* Remove body parser in show partitions endpoints (fix #2690)

Signed-off-by: yhz <413554850@qq.com>

* Remove surplus output and comments

Signed-off-by: yhz <413554850@qq.com>

* Remove surplus output and comments

Signed-off-by: yhz <413554850@qq.com>

* Remove space line

Signed-off-by: yhz <413554850@qq.com>

* Remove surplus ut

Signed-off-by: yhz <413554850@qq.com>

* laten sleep time in case test_delete_vector_collection_count_no_flush

Signed-off-by: yhz <413554850@qq.com>
 2020-07-04 16:31:03 +08:00
groot
367c72675c
#2692 Milvus hangs during multi-thread concurrent search (#2715) 
* #2642 Create index failed caused by server crashed.

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* changelog

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* changelog

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* #2692 Milvus hangs during multi-thread concurrent search

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2020-07-02 19:03:24 +08:00
groot
714b777d05
#2642 Create index failed caused by server crashed. (#2701) 
* #2642 Create index failed caused by server crashed.

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* changelog

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* changelog

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2020-07-02 09:37:15 +08:00
shengjun.li
36db2f4da4
fix vectors format (#2677) 
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-06-28 17:50:55 +08:00
Jin Hai
0b16c92ccc
Increase GRPC concurrency (#2671) 
* Increase GRPC concurrency

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix lint

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Add limitation

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix compilation error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix compilation error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix compilation error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
 2020-06-26 12:16:44 +08:00
shengjun.li
0fb88f733a
fix too many copies (#2660) 
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-06-24 09:46:48 +08:00
groot
ed012f2980
Flush data when insert buffer is full (#2659) 
* fix potential problem

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* insert buffer full trigger flush

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* cleanup files if no collection flushed

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* format

Signed-off-by: yhmo <yihua.mo@zilliz.com>

Co-authored-by: Jin Hai <hai.jin@zilliz.com>
 2020-06-24 09:23:20 +08:00
op-hunter
3b38f7f198
#2649 Remove annoy search parameter validation (#2650) 
* remove annoy search parameter validation

Signed-off-by: cmli <chengming.li@zilliz.com>

* fix test failed

Signed-off-by: cmli <chengming.li@zilliz.com>

* fix lint error

Signed-off-by: cmli <chengming.li@zilliz.com>

* fix lint error again

Signed-off-by: cmli <chengming.li@zilliz.com>

Co-authored-by: cmli <chengming.li@zilliz.com>
 2020-06-23 23:41:04 +08:00
Xiaohai Xu
d4fc48bfa7
Fix naming inconsistency (#2654) 
* fix naming

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix clang-format

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
 2020-06-23 20:23:14 +08:00
groot
d15487656c
fix potential problem (#2657) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2020-06-23 15:30:21 +08:00
shengjun.li
cd04f3fc9f
Suit the range of HNSW parameters (#2638) 
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-06-22 16:42:15 +08:00
groot
0500d26ce2
Merge pull request #2644 from yhmo/v101 
Add internal switch to enable/disable search combine
 2020-06-22 14:36:53 +08:00
groot
cb09380c6c typo 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-06-21 21:33:05 +08:00
groot
ac7403f00b search combine configable 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-06-21 19:03:21 +08:00
groot
6e8fc976f3
#2640 (#2641) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-06-20 20:31:35 +08:00
groot
69eb1d254e #2640 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-06-20 17:47:10 +08:00
shengjun.li
48e8cd36c2
Fix HNSW and RNSG index files size (#2618) 
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-06-20 14:06:05 +08:00
Cai Yudong
6ac3d2bd08 fix Milvus docker image report illegal instruction (#2598) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2020-06-19 10:44:23 +08:00
shengjun.li
290ff2e439 Support IVF_PQ on GPU with using metric_type IP (#2593) 
* Support IVF_PQ on GPU with using metric_type IP

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* remove useless code

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-06-19 10:42:23 +08:00
groot
508da260ca fix #2578 (#2591) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2020-06-19 10:35:07 +08:00
groot
36d2214fc9 fix random crash of INSERT_DUPLICATE_ID case (#2557) 
* fix random crash of INSERT_DUPLICATE_ID case

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* cahnge ver

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* sdk readme

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix ut failure

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2020-06-19 10:33:08 +08:00
JinHai-CN
bd7e0ee5c9 Update code due to 0.10.1 
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
 2020-06-19 10:07:55 +08:00
Wang XiangYu
5f3c005247
fix cache.cache_size range check error (#2565) 
* fix cache.cache_size range check error

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update ci

Signed-off-by: wxyu <xy.wang@zilliz.com>
 2020-06-15 22:44:58 +08:00
BossZou
1f443acb3b
Mishards 010 upgrade & Add config unit in demo config (#2550) 
* Update mishards config

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Update mishards config

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Upgrade mishards for milvus v0.10.0 (fix #2548)

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Launch server fail using demo config (#2549)

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* [skip ci] Remove surplus config yaml

Signed-off-by: Yhz <yinghao.zou@zilliz.com>
 2020-06-13 19:45:31 +08:00
Wang XiangYu
bcd3de2b41
[skip ci]update comments in config file (#2542) 
* update comments in config file

Signed-off-by: wxyu <xy.wang@zilliz.com>

* Update config

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
 2020-06-13 14:06:10 +08:00
Wang XiangYu
83cc812935
Upgrade config (#2511) 
* enable cron test on 0.10.0

Signed-off-by: zw <zw@milvus.io>

* fix merge result

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* import test_during_creating_index_restart

Signed-off-by: zw <zw@milvus.io>

* config cluster

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config general

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config network

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config storage

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config wal

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config cache

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config gpu and metrics

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config logs

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update server_config.template

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update changelog

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config with unit

Signed-off-by: wxyu <xy.wang@zilliz.com>

* fix clang-format

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update unittests

Signed-off-by: wxyu <xy.wang@zilliz.com>

* disable restart case

Signed-off-by: zw <zw@milvus.io>

* fix gpu compile failed

Signed-off-by: wxyu <xy.wang@zilliz.com>

* Fix lint and comments

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* fix wal unittest failed

Signed-off-by: wxyu <xy.wang@zilliz.com>

* fix some unittests

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update config test cases

Signed-off-by: zw <zw@milvus.io>

* Fix config error

Signed-off-by: yhz <413554850@qq.com>

* remove unused code

Signed-off-by: wxyu <xy.wang@zilliz.com>

* Update CHANGELOG

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* [skip-ci] fix test cases

Signed-off-by: zw <zw@milvus.io>

* fix config storage

Signed-off-by: wxyu <xy.wang@zilliz.com>

* make -j3 on ci

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update python test

Signed-off-by: wxyu <xy.wang@zilliz.com>

* remove unused code

Signed-off-by: wxyu <xy.wang@zilliz.com>

* fix ut

Signed-off-by: wxyu <xy.wang@zilliz.com>

* fix ut

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update status message

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update

Signed-off-by: wxyu <xy.wang@zilliz.com>

* fix set config bug

Signed-off-by: wxyu <xy.wang@zilliz.com>

Co-authored-by: zw <zw@milvus.io>
Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
Co-authored-by: yhz <413554850@qq.com>
 2020-06-12 09:22:26 +08:00
yukun
a9629951e4
Improve ut coverage (#2516) (#2522) 
* Improve ut coverage

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Delete unused code

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add fiu in HybridSearchRequest

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Update helm config

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Change BinaryQuery validation check

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* code format

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* code format

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* code format

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
 2020-06-11 18:45:14 +08:00
BossZou
551be85d83
Improve ut coverage (#2516) (#2517) 
* Improve ut coverage (#2516)

Signed-off-by: yhz <413554850@qq.com>

* Improve ut coverage

Signed-off-by: yhz <413554850@qq.com>

* CI retry

Signed-off-by: yhz <413554850@qq.com>

* Update helm config

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* CI retry

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* CI retry

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
 2020-06-11 01:25:55 +00:00
groot
19a9d0d1f3
revert code (#2525) 
* revert code

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* changelog

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2020-06-10 19:27:36 +08:00
Jin Hai
34b4d72517
Add exception throw on mysql meta error (#2490) 
* Add exception throw on mysql meta error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix lint

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* [skip ci] update changelog

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Update

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Update

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix Unit test

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
 2020-06-08 21:16:22 +08:00
groot
bd0049ff14
return hnsw name (#2501) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-06-07 12:03:01 +00:00
Cai Yudong
8254e9ed67
fix performance (#2499) 
* optimize sq_get_distance_computer

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add sq_select_inverted_list_scanner_ref

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add sq_select_inverted_list_scanner_avx

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add sq_select_inverted_list_scanner_avx512

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize Codec

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize ScalarQuantizerCodec_avx.h

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize ScalarQuantizerCodec_avx512.h

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* clean code

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2020-06-06 07:41:57 +00:00
groot
9f7995cbba
default nlist 2048 (#2498) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-06-06 05:03:06 +00:00
Jin Hai
4b5eff3672
Fix 2495: Add more reason of creating lock file failed (#2496) 
* Fix 2495: Add more reason of creating lock file failed

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix 2495: Update changelog

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix lint

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix lint

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix compile error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
 2020-06-05 01:58:17 +00:00
BossZou
167743c993
read_only node detect delete vector operation using mishards #2368 (#2473) 
* Add new request ReLoadSegments

Signed-off-by: yhz <413554850@qq.com>

* Finish load segments functionality

Signed-off-by: yhz <413554850@qq.com>

* Add api in grpc

Signed-off-by: yhz <413554850@qq.com>

* update Reloadsegments

Signed-off-by: yhz <413554850@qq.com>

* .

Signed-off-by: yhz <413554850@qq.com>

* create new blacklist if not exists

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update api names

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Finish mishard for support api reloadsegments

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update changlog

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Add more details when failed in search task

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Fix compile issue

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update mishards requirements

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Code format

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* modify docker images in mysql all_in_one

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update shards code

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Move updatedeldocs function to dbimpl

Signed-off-by: yhz <413554850@qq.com>

* Move reload segment del docs function to dbimpl

Signed-off-by: yhz <413554850@qq.com>

* [skip ci] correct shards requirements

Signed-off-by: yhz <413554850@qq.com>
 2020-06-02 14:34:19 +08:00
groot
0710ececae
#2373 Build index for small segment waste time on waiting background index thread finish (#2482) 
* #2349 Drop collection timeout if too many partitions created on collection

Signed-off-by: groot <yihua.mo@zilliz.com>

* changelog

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* #2373 Build index for small segment waste time on waiting background index thread finish

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2020-06-01 10:16:37 +00:00
shengjun.li
257ea7782f
fix merge result (#2463) 
* fix merge result

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* fix tests

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-06-01 07:31:58 +00:00
groot
1daf00dcf7
#2394 Drop collection timeout if too many partitions created on colle… (#2477) 
* #2349 Drop collection timeout if too many partitions created on collection

Signed-off-by: groot <yihua.mo@zilliz.com>

* changelog

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2020-06-01 11:19:50 +08:00
op-hunter
9de7f710e6
optimize the implemention of k-selection on faiss gpu version (#2467) 
* optimize the implemention of k-selection on faiss gpu version

Signed-off-by: cmli <chengming.li@zilliz.com>

* update change log, re-classified the pr from feature to improvement

Signed-off-by: cmli <chengming.li@zilliz.com>

* update change log

Signed-off-by: cmli <chengming.li@zilliz.com>

Co-authored-by: cmli <chengming.li@zilliz.com>
 2020-05-30 04:27:59 +00:00
groot
ae98a92fc9 merge before index 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-05-29 20:00:57 +08:00
op-hunter
cab9e5c2a6 fix bug of openblas library install fail (#2457) 
* fix bug of openblas library install fail

Signed-off-by: cmli <chengming.li@zilliz.com>

* fix openblas install cmakelist error

Signed-off-by: cmli <chengming.li@zilliz.com>

Co-authored-by: cmli <chengming.li@zilliz.com>
 2020-05-29 20:00:24 +08:00
BossZou
0e25bab619
Fix inconsistent reading and writing when using mishards (# 2367) (#2442) 
* Fix inconsistent reading and writing when using mishards (# 2367)

Signed-off-by: yhz <413554850@qq.com>

* CI retry

Signed-off-by: yhz <413554850@qq.com>
 2020-05-29 13:32:49 +08:00
shengjun.li
7c72dcf4d3
fix search by GPU (#2453) 
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-05-28 15:39:16 +00:00
Cai Yudong
f4aeca2988
improve knowhere coverage (#2444) 
* increase nb for NSG code coverage

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* test more APIs in test_annoy

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* disable get_vector_by_id and search_by_id

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* improve code coverage

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* improve code coverage

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* install test_instructionset

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2020-05-28 10:22:14 +08:00
groot
dcb60f3021
performance issues (#2438) 
* performance issues

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix bugs

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* preload collection issue

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* uncomment boring log

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* reduce unittest time

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* reduce metric test time cost

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2020-05-28 00:19:08 +08:00
Cai Yudong
ce4dcf69ae
add debug APIs and update GPU temp memory size (#2420) 
* add debug APIs and update GPU temp memory size

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update thread name

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update DESIGN.md

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* retry ci

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2020-05-26 14:39:44 +08:00
groot
2d9b358e16
optimize merge process (#2419) 
* optimize merge process

Signed-off-by: groot <yihua.mo@zilliz.com>

* typo

Signed-off-by: groot <yihua.mo@zilliz.com>

* refine code

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* drop collecion issue

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2020-05-25 14:19:25 +08:00
shengjun.li
2982004e2b
midify Binary IVF in faiss 1.6.3 (#2416) 
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-05-23 11:01:54 +00:00
Xiaohai Xu
6809150b1d
#2395 Fix large nq cudaMalloc (#2409) 
* fix large nq cuda malloc issue

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix clang format

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* Change log

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
 2020-05-23 08:23:05 +00:00
groot
077dfa4092
Wal lsn issue (#2412) 
* set wal lsn after all mem files serialized

Signed-off-by: groot <yihua.mo@zilliz.com>

* typo

Signed-off-by: groot <yihua.mo@zilliz.com>

* return status

Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-05-23 06:51:18 +00:00
groot
4ce3e2c03c
#2366 Reduce SQL execution times (#2383) 
* #2366

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix ut

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix python test

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix ut

Signed-off-by: groot <yihua.mo@zilliz.com>

* changelog

Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-05-23 11:13:09 +08:00
groot
4f6fe96550
mysql meta connection pool issue (#2413) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-05-23 01:26:15 +00:00
Wang XiangYu
e40f0b3939
logging build index progress (#2411) 
Signed-off-by: wxyu <xy.wang@zilliz.com>
 2020-05-22 13:10:24 +00:00
Cai Yudong
344ae823b2
#2353 remove log_config (#2406) 
* remove log_config from code and script

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update script

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2020-05-22 10:57:16 +00:00
Jin Hai
2c2e1bb8f1
#2403 MySQL max_idle_time is 10 by default (#2405) 
* #2403 MySQL max_idle_time is 10 by default

Signed-off-by: jinhai <hai.jin@zilliz.com>

* #2403 MySQL max_idle_time is 10 by default

Signed-off-by: jinhai <hai.jin@zilliz.com>
 2020-05-22 14:35:11 +08:00
shengjun.li
34fdc00d19
fix MatchNlist (#2401) 
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-05-22 06:20:26 +00:00
Cai Yudong
386e58ce0d
upgrade faiss 1.6.3 (#2400) 
* roll back to original faiss 1.6.0

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update to faiss_1.6.3

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* patch all change to faiss 1.6.3

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* faiss CPU version build pass

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* faiss GPU version build pass

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2020-05-22 01:27:16 +00:00
groot
4c9684366c
#2378 (#2388) 
* return partition lsn

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix wal lsn

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* fix wal issue

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* changelog

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* all collection include partition

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix build error

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix flush

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
 2020-05-21 02:13:46 +00:00
Cai Yudong
b532a69c8b
clean compile warning (#2380) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2020-05-19 17:58:23 +00:00
JinHai-CN
b616dcb761 Update branch 
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
 2020-05-19 11:27:57 +08:00
groot
014a0e860c
QPS decrease (#2372) 
* optimize merge strategy

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* #2365

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix typo

Signed-off-by: groot <yihua.mo@zilliz.com>

* optimize search

Signed-off-by: groot <yihua.mo@zilliz.com>

* code format

Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-05-19 11:21:53 +08:00
Cai Yudong
b9e8acb5b2
#2370 clean compile warning (#2371) 
* #2370 clean compile warning

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2020-05-18 02:35:14 +00:00
Cai Yudong
42f751047e
update branch version to 0.9.1 (#2364) 
* update branch version to 0.9.1

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* retry ci

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* retry ci

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update test version

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2020-05-17 04:36:42 +00:00
Cai Yudong
eba33cc46d
Caiyd 090 rename config (#2340) 
* rename config items

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update config unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang-format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update config unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update log rotate default value

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update parameter usage in LogUtil.cpp (#3)

Signed-off-by: wxyu <xy.wang@zilliz.com>

* retry ci

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix ut

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
Co-authored-by: Wang XiangYu <scsven@qq.com>
 2020-05-14 17:57:18 +00:00