test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-05 05:18:52 +08:00
Code Issues Actions 14 Packages Projects Releases Wiki Activity
5,420 Commits 96 Branches 105 Tags 186 MiB
cd04f3fc9f
Commit Graph

1109 Commits

Author SHA1 Message Date
shengjun.li
cd04f3fc9f
Suit the range of HNSW parameters (#2638) 
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-06-22 16:42:15 +08:00
groot
0500d26ce2
Merge pull request #2644 from yhmo/v101 
Add internal switch to enable/disable search combine
 2020-06-22 14:36:53 +08:00
groot
cb09380c6c typo 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-06-21 21:33:05 +08:00
groot
ac7403f00b search combine configable 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-06-21 19:03:21 +08:00
groot
6e8fc976f3
#2640 (#2641) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-06-20 20:31:35 +08:00
groot
69eb1d254e #2640 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-06-20 17:47:10 +08:00
shengjun.li
48e8cd36c2
Fix HNSW and RNSG index files size (#2618) 
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-06-20 14:06:05 +08:00
Cai Yudong
6ac3d2bd08 fix Milvus docker image report illegal instruction (#2598) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2020-06-19 10:44:23 +08:00
shengjun.li
290ff2e439 Support IVF_PQ on GPU with using metric_type IP (#2593) 
* Support IVF_PQ on GPU with using metric_type IP

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* remove useless code

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-06-19 10:42:23 +08:00
groot
508da260ca fix #2578 (#2591) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2020-06-19 10:35:07 +08:00
groot
36d2214fc9 fix random crash of INSERT_DUPLICATE_ID case (#2557) 
* fix random crash of INSERT_DUPLICATE_ID case

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* cahnge ver

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* sdk readme

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix ut failure

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2020-06-19 10:33:08 +08:00
JinHai-CN
bd7e0ee5c9 Update code due to 0.10.1 
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
 2020-06-19 10:07:55 +08:00
Wang XiangYu
5f3c005247
fix cache.cache_size range check error (#2565) 
* fix cache.cache_size range check error

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update ci

Signed-off-by: wxyu <xy.wang@zilliz.com>
 2020-06-15 22:44:58 +08:00
BossZou
1f443acb3b
Mishards 010 upgrade & Add config unit in demo config (#2550) 
* Update mishards config

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Update mishards config

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Upgrade mishards for milvus v0.10.0 (fix #2548)

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Launch server fail using demo config (#2549)

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* [skip ci] Remove surplus config yaml

Signed-off-by: Yhz <yinghao.zou@zilliz.com>
 2020-06-13 19:45:31 +08:00
Wang XiangYu
bcd3de2b41
[skip ci]update comments in config file (#2542) 
* update comments in config file

Signed-off-by: wxyu <xy.wang@zilliz.com>

* Update config

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
 2020-06-13 14:06:10 +08:00
Wang XiangYu
83cc812935
Upgrade config (#2511) 
* enable cron test on 0.10.0

Signed-off-by: zw <zw@milvus.io>

* fix merge result

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* import test_during_creating_index_restart

Signed-off-by: zw <zw@milvus.io>

* config cluster

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config general

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config network

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config storage

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config wal

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config cache

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config gpu and metrics

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config logs

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update server_config.template

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update changelog

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config with unit

Signed-off-by: wxyu <xy.wang@zilliz.com>

* fix clang-format

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update unittests

Signed-off-by: wxyu <xy.wang@zilliz.com>

* disable restart case

Signed-off-by: zw <zw@milvus.io>

* fix gpu compile failed

Signed-off-by: wxyu <xy.wang@zilliz.com>

* Fix lint and comments

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* fix wal unittest failed

Signed-off-by: wxyu <xy.wang@zilliz.com>

* fix some unittests

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update config test cases

Signed-off-by: zw <zw@milvus.io>

* Fix config error

Signed-off-by: yhz <413554850@qq.com>

* remove unused code

Signed-off-by: wxyu <xy.wang@zilliz.com>

* Update CHANGELOG

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* [skip-ci] fix test cases

Signed-off-by: zw <zw@milvus.io>

* fix config storage

Signed-off-by: wxyu <xy.wang@zilliz.com>

* make -j3 on ci

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update python test

Signed-off-by: wxyu <xy.wang@zilliz.com>

* remove unused code

Signed-off-by: wxyu <xy.wang@zilliz.com>

* fix ut

Signed-off-by: wxyu <xy.wang@zilliz.com>

* fix ut

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update status message

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update

Signed-off-by: wxyu <xy.wang@zilliz.com>

* fix set config bug

Signed-off-by: wxyu <xy.wang@zilliz.com>

Co-authored-by: zw <zw@milvus.io>
Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
Co-authored-by: yhz <413554850@qq.com>
 2020-06-12 09:22:26 +08:00
yukun
a9629951e4
Improve ut coverage (#2516) (#2522) 
* Improve ut coverage

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Delete unused code

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add fiu in HybridSearchRequest

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Update helm config

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Change BinaryQuery validation check

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* code format

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* code format

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* code format

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
 2020-06-11 18:45:14 +08:00
BossZou
551be85d83
Improve ut coverage (#2516) (#2517) 
* Improve ut coverage (#2516)

Signed-off-by: yhz <413554850@qq.com>

* Improve ut coverage

Signed-off-by: yhz <413554850@qq.com>

* CI retry

Signed-off-by: yhz <413554850@qq.com>

* Update helm config

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* CI retry

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* CI retry

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
 2020-06-11 01:25:55 +00:00
groot
19a9d0d1f3
revert code (#2525) 
* revert code

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* changelog

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2020-06-10 19:27:36 +08:00
Jin Hai
34b4d72517
Add exception throw on mysql meta error (#2490) 
* Add exception throw on mysql meta error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix lint

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* [skip ci] update changelog

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Update

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Update

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix Unit test

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
 2020-06-08 21:16:22 +08:00
groot
bd0049ff14
return hnsw name (#2501) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-06-07 12:03:01 +00:00
Cai Yudong
8254e9ed67
fix performance (#2499) 
* optimize sq_get_distance_computer

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add sq_select_inverted_list_scanner_ref

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add sq_select_inverted_list_scanner_avx

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add sq_select_inverted_list_scanner_avx512

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize Codec

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize ScalarQuantizerCodec_avx.h

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize ScalarQuantizerCodec_avx512.h

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* clean code

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2020-06-06 07:41:57 +00:00
groot
9f7995cbba
default nlist 2048 (#2498) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-06-06 05:03:06 +00:00
Jin Hai
4b5eff3672
Fix 2495: Add more reason of creating lock file failed (#2496) 
* Fix 2495: Add more reason of creating lock file failed

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix 2495: Update changelog

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix lint

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix lint

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix compile error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
 2020-06-05 01:58:17 +00:00
BossZou
167743c993
read_only node detect delete vector operation using mishards #2368 (#2473) 
* Add new request ReLoadSegments

Signed-off-by: yhz <413554850@qq.com>

* Finish load segments functionality

Signed-off-by: yhz <413554850@qq.com>

* Add api in grpc

Signed-off-by: yhz <413554850@qq.com>

* update Reloadsegments

Signed-off-by: yhz <413554850@qq.com>

* .

Signed-off-by: yhz <413554850@qq.com>

* create new blacklist if not exists

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update api names

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Finish mishard for support api reloadsegments

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update changlog

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Add more details when failed in search task

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Fix compile issue

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update mishards requirements

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Code format

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* modify docker images in mysql all_in_one

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update shards code

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Move updatedeldocs function to dbimpl

Signed-off-by: yhz <413554850@qq.com>

* Move reload segment del docs function to dbimpl

Signed-off-by: yhz <413554850@qq.com>

* [skip ci] correct shards requirements

Signed-off-by: yhz <413554850@qq.com>
 2020-06-02 14:34:19 +08:00
groot
0710ececae
#2373 Build index for small segment waste time on waiting background index thread finish (#2482) 
* #2349 Drop collection timeout if too many partitions created on collection

Signed-off-by: groot <yihua.mo@zilliz.com>

* changelog

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* #2373 Build index for small segment waste time on waiting background index thread finish

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2020-06-01 10:16:37 +00:00
shengjun.li
257ea7782f
fix merge result (#2463) 
* fix merge result

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* fix tests

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-06-01 07:31:58 +00:00
groot
1daf00dcf7
#2394 Drop collection timeout if too many partitions created on colle… (#2477) 
* #2349 Drop collection timeout if too many partitions created on collection

Signed-off-by: groot <yihua.mo@zilliz.com>

* changelog

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2020-06-01 11:19:50 +08:00
op-hunter
9de7f710e6
optimize the implemention of k-selection on faiss gpu version (#2467) 
* optimize the implemention of k-selection on faiss gpu version

Signed-off-by: cmli <chengming.li@zilliz.com>

* update change log, re-classified the pr from feature to improvement

Signed-off-by: cmli <chengming.li@zilliz.com>

* update change log

Signed-off-by: cmli <chengming.li@zilliz.com>

Co-authored-by: cmli <chengming.li@zilliz.com>
 2020-05-30 04:27:59 +00:00
groot
ae98a92fc9 merge before index 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-05-29 20:00:57 +08:00
op-hunter
cab9e5c2a6 fix bug of openblas library install fail (#2457) 
* fix bug of openblas library install fail

Signed-off-by: cmli <chengming.li@zilliz.com>

* fix openblas install cmakelist error

Signed-off-by: cmli <chengming.li@zilliz.com>

Co-authored-by: cmli <chengming.li@zilliz.com>
 2020-05-29 20:00:24 +08:00
BossZou
0e25bab619
Fix inconsistent reading and writing when using mishards (# 2367) (#2442) 
* Fix inconsistent reading and writing when using mishards (# 2367)

Signed-off-by: yhz <413554850@qq.com>

* CI retry

Signed-off-by: yhz <413554850@qq.com>
 2020-05-29 13:32:49 +08:00
shengjun.li
7c72dcf4d3
fix search by GPU (#2453) 
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-05-28 15:39:16 +00:00
Cai Yudong
f4aeca2988
improve knowhere coverage (#2444) 
* increase nb for NSG code coverage

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* test more APIs in test_annoy

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* disable get_vector_by_id and search_by_id

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* improve code coverage

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* improve code coverage

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* install test_instructionset

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2020-05-28 10:22:14 +08:00
groot
dcb60f3021
performance issues (#2438) 
* performance issues

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix bugs

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* preload collection issue

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* uncomment boring log

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* reduce unittest time

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* reduce metric test time cost

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2020-05-28 00:19:08 +08:00
Cai Yudong
ce4dcf69ae
add debug APIs and update GPU temp memory size (#2420) 
* add debug APIs and update GPU temp memory size

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update thread name

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update DESIGN.md

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* retry ci

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2020-05-26 14:39:44 +08:00
groot
2d9b358e16
optimize merge process (#2419) 
* optimize merge process

Signed-off-by: groot <yihua.mo@zilliz.com>

* typo

Signed-off-by: groot <yihua.mo@zilliz.com>

* refine code

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* drop collecion issue

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2020-05-25 14:19:25 +08:00
shengjun.li
2982004e2b
midify Binary IVF in faiss 1.6.3 (#2416) 
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-05-23 11:01:54 +00:00
Xiaohai Xu
6809150b1d
#2395 Fix large nq cudaMalloc (#2409) 
* fix large nq cuda malloc issue

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix clang format

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* Change log

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
 2020-05-23 08:23:05 +00:00
groot
077dfa4092
Wal lsn issue (#2412) 
* set wal lsn after all mem files serialized

Signed-off-by: groot <yihua.mo@zilliz.com>

* typo

Signed-off-by: groot <yihua.mo@zilliz.com>

* return status

Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-05-23 06:51:18 +00:00
groot
4ce3e2c03c
#2366 Reduce SQL execution times (#2383) 
* #2366

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix ut

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix python test

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix ut

Signed-off-by: groot <yihua.mo@zilliz.com>

* changelog

Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-05-23 11:13:09 +08:00
groot
4f6fe96550
mysql meta connection pool issue (#2413) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-05-23 01:26:15 +00:00
Wang XiangYu
e40f0b3939
logging build index progress (#2411) 
Signed-off-by: wxyu <xy.wang@zilliz.com>
 2020-05-22 13:10:24 +00:00
Cai Yudong
344ae823b2
#2353 remove log_config (#2406) 
* remove log_config from code and script

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update script

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2020-05-22 10:57:16 +00:00
Jin Hai
2c2e1bb8f1
#2403 MySQL max_idle_time is 10 by default (#2405) 
* #2403 MySQL max_idle_time is 10 by default

Signed-off-by: jinhai <hai.jin@zilliz.com>

* #2403 MySQL max_idle_time is 10 by default

Signed-off-by: jinhai <hai.jin@zilliz.com>
 2020-05-22 14:35:11 +08:00
shengjun.li
34fdc00d19
fix MatchNlist (#2401) 
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-05-22 06:20:26 +00:00
Cai Yudong
386e58ce0d
upgrade faiss 1.6.3 (#2400) 
* roll back to original faiss 1.6.0

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update to faiss_1.6.3

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* patch all change to faiss 1.6.3

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* faiss CPU version build pass

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* faiss GPU version build pass

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2020-05-22 01:27:16 +00:00
groot
4c9684366c
#2378 (#2388) 
* return partition lsn

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix wal lsn

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* fix wal issue

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* changelog

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* all collection include partition

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix build error

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix flush

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
 2020-05-21 02:13:46 +00:00
Cai Yudong
b532a69c8b
clean compile warning (#2380) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2020-05-19 17:58:23 +00:00
JinHai-CN
b616dcb761 Update branch 
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
 2020-05-19 11:27:57 +08:00