Commit Graph

447 Commits

Author SHA1 Message Date
Cai Yudong
1384484b7d
Caiyd 2039 avx sse (#2136)
* #2039 clean redundant compile options

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #2039 add distances_avx.h and distances_avx.cpp

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #2039 add ScalarQuantizerDC_avx.cpp, ScalarQuantizerDC_avx.h and ScalarQuantizerCodec_avx.h

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #2039 remove -mavx2 from default compile option

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #2039 clean code

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #2039 clean code

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-04-27 15:01:47 +08:00
Wang XiangYu
12b3bf0693
Add instance lock (#2060)
* Add instance lock

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update message

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update unittest CMakeLists.txt

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update

Signed-off-by: wxyu <xy.wang@zilliz.com>

* fix clang-format

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update

Signed-off-by: wxyu <xy.wang@zilliz.com>

Co-authored-by: Jin Hai <hai.jin@zilliz.com>
2020-04-26 08:31:18 +00:00
BossZou
8edb7b6a09
add cpu instruction sets check (#2055)
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
2020-04-26 00:59:25 +08:00
groot
777c36a0ac
#1997 index file missed after compact (#2108)
* #1997 index file missed after compact

Signed-off-by: groot <yihua.mo@zilliz.com>

* changelog

Signed-off-by: groot <yihua.mo@zilliz.com>

* typo

Signed-off-by: groot <yihua.mo@zilliz.com>
2020-04-25 22:06:26 +08:00
groot
82f4c289eb
New api HasPartition, SearchByID, GetVectorsByID (#2056)
* modify proto

Signed-off-by: groot <yihua.mo@zilliz.com>

* modify c sdk

Signed-off-by: groot <yihua.mo@zilliz.com>

* rewrite search by id

Signed-off-by: groot <yihua.mo@zilliz.com>

* change cases

Signed-off-by: zw <zw@zilliz.com>

* fix cases

Signed-off-by: zw <zw@zilliz.com>

* fix wal case

Signed-off-by: zw <zw@zilliz.com>

* fix search by id

Signed-off-by: groot <yihua.mo@zilliz.com>

* modify sdk

Signed-off-by: groot <yihua.mo@zilliz.com>

* refine code

Signed-off-by: groot <yihua.mo@zilliz.com>

* refine code

Signed-off-by: groot <yihua.mo@zilliz.com>

* changelog

Signed-off-by: groot <yihua.mo@zilliz.com>

* add search_by_ids case

Signed-off-by: zw <zw@zilliz.com>

* disable test_search)by_ids

Signed-off-by: zw <zw@zilliz.com>

Co-authored-by: zw <zw@zilliz.com>
Co-authored-by: del-zhenwu <56623710+del-zhenwu@users.noreply.github.com>
2020-04-25 14:49:32 +08:00
Jin Hai
54c9386bdc
#2076 CheckMetricConfigAddress error message (#2077)
Signed-off-by: jinhai <hai.jin@zilliz.com>
2020-04-25 10:04:04 +08:00
Jin Hai
723b10600b
#2073 Fix CheckDBConfigBackendUrl error message (#2074)
Signed-off-by: jinhai <hai.jin@zilliz.com>
2020-04-25 09:31:25 +08:00
Wang XiangYu
67819be29b
warn when use sqlite as metadata management (#2065)
Signed-off-by: wxyu <xy.wang@zilliz.com>
2020-04-24 15:23:46 +08:00
Cai Yudong
f039032c8c
Caiyd 1965 unify metric cal (#2008)
* #1965 add metric algorithm benchmark for FAISS

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #1965 add metric algorithm benchmark for NSG

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #1965 add metric algorithm benchmark for ANNOY

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #1965 add metric algorithm benchmark for HNSW

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code opt

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* calculate average time

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #1965 annoy/nsg/hnsw all use faiss distance algorithm

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #1965 support AVX512/AVX2/SSE42

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #1965 update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #1965 fix hnsw ip calculation error

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-04-21 14:22:33 +08:00
Jin Hai
315dac3338
Automerge1 (#1984)
* [skip ci] remove auto-merge git-action

Signed-off-by: jinhai <hai.jin@zilliz.com>

* [skip ci] update CHANGELOG.md

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Stop strip the binary

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Improve code

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix compiling error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix warning

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-04-20 20:39:22 +08:00
mergify[bot]
b97682529a
Merge pull request #1963 from JinHai-CN/test1
[skip ci] update change log
2020-04-18 11:31:57 +00:00
groot
85de706d22
#1929 Skip mysql meta schema field width check (#1937)
* #1929 Skip mysql meta schema field width check

Signed-off-by: groot <yihua.mo@zilliz.com>

* typo

Signed-off-by: groot <yihua.mo@zilliz.com>

* merge master

Signed-off-by: groot <yihua.mo@zilliz.com>
2020-04-18 10:26:22 +08:00
Jin Hai
59035d9892
#1454 Improve code quality (#1948)
* Improve code quality

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Improve code quality

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Improve code quality

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix lint

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix compiling error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix compiling error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix compiling error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix issue

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-04-17 21:05:17 +08:00
jinhai
b6a8477629 [skip ci] update change log
Signed-off-by: jinhai <hai.jin@zilliz.com>
2020-04-17 12:20:18 +08:00
jinhai
35ab255f59 [skip ci] update change log
Signed-off-by: jinhai <hai.jin@zilliz.com>
2020-04-17 11:46:28 +08:00
groot
93244a2a79
#1705 Limit the insert data batch size (#1936)
* #1705 Limit the insert data batch size

Signed-off-by: groot <yihua.mo@zilliz.com>

* typo

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix unittest

Signed-off-by: groot <yihua.mo@zilliz.com>
2020-04-16 00:18:23 +08:00
Jin Hai
0323aa1aad
Merge 080 (#1940)
* #1910 C++ SDK GetIDsInSegment could not work for large dataset (#1911)

Signed-off-by: groot <yihua.mo@zilliz.com>

* #1903 Fix invalid annoy result (#1912)

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* #1914: Partition max size should be 4096 (#1915)

Signed-off-by: jinhai <hai.jin@zilliz.com>

* add log (#1913)

* add log

Signed-off-by: groot <yihua.mo@zilliz.com>

* add log

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix ut

Signed-off-by: groot <yihua.mo@zilliz.com>

* partition limit 4096

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix py test

Signed-off-by: groot <yihua.mo@zilliz.com>

* update server version (#1916)

Signed-off-by: zw <zw@zilliz.com>

* Update to 0.8.0 (#1918)

* Create new branch 0.8.0 and change preload_table to preload_collection

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix format

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Update CHANGELOG

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Update CHANGELOG

Signed-off-by: jinhai <hai.jin@zilliz.com>

* update helm version

Signed-off-by: zw <zw@zilliz.com>

* Update CHANGELOG

Signed-off-by: jinhai <hai.jin@zilliz.com>

Co-authored-by: zw <zw@zilliz.com>

* fix issue 1901 (#1920)

* fix issue 1901

Signed-off-by: cmli <chengming.li@zilliz.com>

* update change log

Signed-off-by: cmli <chengming.li@zilliz.com>

Co-authored-by: cmli <chengming.li@zilliz.com>

* #1900 (#1923)

* add log

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix #1900

Signed-off-by: groot <yihua.mo@zilliz.com>

* Upgrade mishards to 0.8.0 (#1933)

* update grpc server of milvus & rename table name to collection

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update changlog

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* [skip ci] Skip CI

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* [skip ci] Update changlog

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Caiyd 1883 fix rw (#1926)

* #1883 use DiskIO

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix logic error

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* retry CI

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* Update CHANGELOG

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Co-authored-by: JinHai-CN <hai.jin@zilliz.com>

* #1928 Too many data and uid copies when loading files (#1931)

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

Co-authored-by: Jin Hai <hai.jin@zilliz.com>

* Update mishards configure files (#1938)

* Update web readme

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* [skip ci] update configure files

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* [skip ci] rename table to collection

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Update test.groovy

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Update test.groovy

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix lint

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix compiling error

Signed-off-by: jinhai <hai.jin@zilliz.com>

Co-authored-by: groot <yhmo@zeronedata.com>
Co-authored-by: shengjun.li <49774184+shengjun1985@users.noreply.github.com>
Co-authored-by: del-zhenwu <56623710+del-zhenwu@users.noreply.github.com>
Co-authored-by: zw <zw@zilliz.com>
Co-authored-by: op-hunter <ophunter52@gmail.com>
Co-authored-by: cmli <chengming.li@zilliz.com>
Co-authored-by: BossZou <40255591+BossZou@users.noreply.github.com>
Co-authored-by: Cai Yudong <yudong.cai@zilliz.com>
2020-04-15 21:32:20 +08:00
Wang XiangYu
890fe08e7c
Refactor LOG macro and improve log format (#1927)
* update log format

Signed-off-by: wxyu <xy.wang@zilliz.com>

* add new log macro

Signed-off-by: wxyu <xy.wang@zilliz.com>

* use new log macro instead

Signed-off-by: Xiangyu Wang <xy.wang@zilliz.com>

* add SetThreadName function

Signed-off-by: wxyu <xy.wang@zilliz.com>

* clang-format

Signed-off-by: wxyu <xy.wang@zilliz.com>

* set thread name

Signed-off-by: Xiangyu Wang <xy.wang@zilliz.com>

* add changelog

Signed-off-by: wxyu <xy.wang@zilliz.com>

* add git ignore .swp file

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update log level in LogUtil.cpp

Signed-off-by: wxyu <xy.wang@zilliz.com>
2020-04-15 14:44:00 +08:00
Cai Yudong
a63d832772
#1921 use TimeRecorder instead of chrono (#1922)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-04-13 18:42:31 +08:00
Jin Hai
f8a28699b4
Change to 0.8.0 and rename from preload_table to preload_collection (#1904)
* Create new branch 0.8.0 and change preload_table to preload_collection

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix format

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Update CHANGELOG

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Update CHANGELOG

Signed-off-by: jinhai <hai.jin@zilliz.com>

* update helm version

Signed-off-by: zw <zw@zilliz.com>

Co-authored-by: zw <zw@zilliz.com>
2020-04-11 19:01:29 +08:00
shengjun.li
b04b3e942c
#1897 add heap_swap_top (#1898)
* add heap_swap_top

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* fix wrong code

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-04-11 12:56:32 +08:00
Wang XiangYu
3bc17d8cc2
Fix multi client search crash in tracing module fix #1789 fix #1832 (#1899)
* Fix multi client search crash in tracing module fix #1789 fix #1832

Signed-off-by: wxyu <xy.wang@zilliz.com>

* add lock for every context_map_ access

Signed-off-by: wxyu <xy.wang@zilliz.com>

* remove never used variable

Signed-off-by: wxyu <xy.wang@zilliz.com>
2020-04-11 12:12:38 +08:00
groot
c8a59b273c
#1276 SQLite throw exception when create 50000+ partitions in a table (#1879)
* #1276 SQLite throw exception when create 50000+ partitions in a table

Signed-off-by: groot <yihua.mo@zilliz.com>

* #1276 SQLite throw exception when create 50000+ partitions in a table

Signed-off-by: groot <yihua.mo@zilliz.com>

* remove some log

Signed-off-by: groot <yihua.mo@zilliz.com>

* remove some log

Signed-off-by: groot <yihua.mo@zilliz.com>

* typo

Signed-off-by: groot <yihua.mo@zilliz.com>

* typo

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix unittest

Signed-off-by: groot <yihua.mo@zilliz.com>

Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
2020-04-10 22:51:01 +08:00
groot
e25c5af2d7
rename table to collection (#1888)
* rename table to collection

Signed-off-by: groot <yihua.mo@zilliz.com>

* rename collection

Signed-off-by: groot <yihua.mo@zilliz.com>

* rename

Signed-off-by: groot <yihua.mo@zilliz.com>

* Update change log

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
2020-04-10 19:17:33 +08:00
Cai Yudong
a05ec6a972
Caiyd 1885 optimize knowhere ut (#1889)
* optimize knowhere unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code clean

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize knowhere util

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang-format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix unittest build error

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* retry CI

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Co-authored-by: Jin Hai <hai.jin@zilliz.com>
2020-04-09 23:37:13 +08:00
BossZou
354f29a058
Refactor log on search and insert request (#1887)
* fix stash conflict

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Log config information and device information (fix #1893)

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Add index annoy into http module (fix #1882)

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Refactor log on search and insert request (fix #1886)

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* use vsnprintf to avoid oom

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* fix PR Quality problem & remove surplus comments

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* code format

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* replace strlen with strnlen & remove some surplus comments

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* format

Signed-off-by: Yhz <yinghao.zou@zilliz.com>
2020-04-09 21:04:18 +08:00
Cai Yudong
66e38d20cd
Caiyd 1881 bad alloc (#1892)
* #1881 update storage APIs, use int64_t for size

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-04-09 17:13:15 +08:00
BossZou
a4b31cb02f
[skip ci] Fix incorrect information in http readme (fix #1812) (#1878)
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
2020-04-07 18:56:19 +08:00
Cai Yudong
bf6d22e2f5
#1873 fix index file serialize to incorrect path (#1874)
* #1873 fix index file serialize to incorrect path

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* not create sq8h index when gpu disabled

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-04-07 16:25:46 +08:00
Cai Yudong
dc707bb8a3
Caiyd 1858 disable s3 build (#1860)
* #1858 disable s3 build

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* disable s3 in unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-04-03 16:35:48 +08:00
op-hunter
54337e9361
refactor knowhere inner unittest for annoy and hnsw, which a… (#1855)
* skip ci, refactor knowhere inner unittest for annoy and hnsw, which aimed to remove logs of std output

Signed-off-by: cmli <chengming.li@zilliz.com>

* skip ci, fix lint error

Signed-off-by: cmli <chengming.li@zilliz.com>

* fix annoy load bugs and add serialize test 4 annoy and hnsw

Signed-off-by: cmli <chengming.li@zilliz.com>

* update changelog

Signed-off-by: cmli <chengming.li@zilliz.com>

* update lower bound of annoy search_k param

Signed-off-by: cmli <chengming.li@zilliz.com>

* lint errorgit add src/utils/ValidationUtil.cpp

Signed-off-by: cmli <chengming.li@zilliz.com>

Co-authored-by: cmli <chengming.li@zilliz.com>
2020-04-03 14:14:04 +08:00
Cai Yudong
7ccfa5b9f0
Caiyd 1627 move rw index (#1833)
* #1627 move read/write index into codec

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang-format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code optimize

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update WriteVectorIndex

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang-format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-04-02 22:44:45 +08:00
Xiaohai Xu
f82aa4f493
#1849 Support NSG Deletion (#1839)
* nsg delete

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix request

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* Fix clang

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix typo

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* [skip ci] add changelog

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix test_nsg and changelog

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix test

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix test case

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix clang-format

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* Polish

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
2020-04-02 20:46:58 +08:00
Jin Hai
3e03850169
Merge from 0.7.1 to master (#1840)
* update mishards images to 0.7.1 (fix #1799) (#1802)

* update mishards images to 0.7.1 (fix #1799)

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* [skip ci] Skip CI

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* [skip ci] Update CHANGELOG

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* fix lsn meta bug (#1813)

* fix lsn meta bug

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix mysql lsn meta bug

Signed-off-by: groot <yihua.mo@zilliz.com>

* Update CHANGELOG.md (#1814)

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix compile (#1815)

* Update CHANGELOG.md

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Update

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix compile (#1816)

* Update CHANGELOG.md

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Update

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Update

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix compile (#1817)

* Update CHANGELOG.md

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Update

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Update

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Increase Resources

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Decrease the compiling thread

Signed-off-by: jinhai <hai.jin@zilliz.com>

* #1818 Duplicate data generated after restart milvus server (#1819)

* #1818 Duplicate data generated after restart milvus server

Signed-off-by: groot <yihua.mo@zilliz.com>

* set compile thread number

Signed-off-by: groot <yihua.mo@zilliz.com>

* [skip-ci]Remove auto flush interval from config (#1837)

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix lint

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix compile error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

Co-authored-by: BossZou <40255591+BossZou@users.noreply.github.com>
Co-authored-by: groot <yhmo@zeronedata.com>
2020-04-02 14:20:33 +08:00
op-hunter
6e7fb831bd
add annoy params validate check (#1829)
* fix bug & solve compile warning from annoylib

Signed-off-by: cmli <chengming.li@zilliz.com>

* fix lint error

Signed-off-by: cmli <chengming.li@zilliz.com>

Co-authored-by: cmli <chengming.li@zilliz.com>
2020-04-02 09:47:45 +08:00
yukun
c30a394805
Add annoy index type in C++ sdk (#1826)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2020-04-01 18:46:57 +08:00
Cai Yudong
531fca8869
Caiyd 1655 gpu ivfflat delete (#1809)
* support GPUIVF delete

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* calc bitset offset correctly

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* support GPUSQ8 delete

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* support GPUIVFPQ delete

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update unittest for GPU delete

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* delete FAISS compile option -DFAISS_USE_FLOAT16

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code opt

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-04-01 10:42:48 +08:00
JinHai-CN
4574f9d998 Merge branch '0.7.1' 2020-03-29 18:50:10 +08:00
JinHai-CN
c14adeb5b1 Update CHANGELOG
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-03-29 17:44:54 +08:00
BossZou
4cd5d8dd1b
Add Superstructure/Substructure in http module (#1785)
* add sub/super-structure in http module

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update http test case

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* add Superstructure/Substructure in http module (fix #1784)

Signed-off-by: Yhz <yinghao.zou@zilliz.com>
2020-03-28 14:41:24 +08:00
Cai Yudong
a7932b0365
#1781 fix search hang with SQ8H (#1783)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-03-28 13:08:42 +08:00
Jin Hai
d92bcd2e54
Update CHANGLOG.md (#1771)
* Update CHANGLOG.md

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Update

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Update

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-03-27 14:02:54 +08:00
groot
e4264d8461
Server is not forbidden to create new partition which tag is _default (#1770)
* HasTable with a not exist table name, return error status

Signed-off-by: groot <yihua.mo@zilliz.com>

* Server is not forbidden to create new partition which tag is _default

Signed-off-by: groot <yihua.mo@zilliz.com>
2020-03-27 10:00:51 +08:00
op-hunter
310d5d70bc
Annoy support (#1746)
* add annoy source code

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* add annoy knowhere

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* annoy local gtest passed

Signed-off-by: lichengming <chengming.li@zilliz.com>

* fix lint error and update changelog

Signed-off-by: lichengming <chengming.li@zilliz.com>

* fix compile error

Signed-off-by: cmli <chengming.li@zilliz.com>

* Update connect timeout in test cases

Signed-off-by: zw <zw@milvus.io>

* fix some potential bugs

Signed-off-by: cmli <chengming.li@zilliz.com>

* retry ci

Signed-off-by: cmli <chengming.li@zilliz.com>

* rerun ci!

Signed-off-by: cmli <chengming.li@zilliz.com>

* fix errors tested by c++ sdk

Signed-off-by: cmli <chengming.li@zilliz.com>

* fix lint error

Signed-off-by: cmli <chengming.li@zilliz.com>

Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
Co-authored-by: lichengming <chengming.li@zilliz.com>
Co-authored-by: zw <zw@milvus.io>
2020-03-27 09:52:31 +08:00
JinHai-CN
7763caef09 Update CHANGLOG.md
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-03-26 23:45:00 +08:00
JinHai-CN
65ffaedae3 Update CHANGLOG.md
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-03-26 18:35:49 +08:00
Tinkerrr
243e843aac
fix #1747 (#1757)
* fix

Signed-off-by: Nicky <nicky.xj.lin@gmail.com>

* update.

Signed-off-by: Nicky <nicky.xj.lin@gmail.com>

* lint pass

Signed-off-by: xiaojun.lin <xiaojun.lin@zilliz.com>

* UPDATE CASE: search with tag not exist

Signed-off-by: zw <zw@zilliz.com>

Co-authored-by: zw <zw@zilliz.com>
2020-03-26 18:28:24 +08:00
groot
c4b7be8ed6
#1728 Optimize request handler to combine similar query (#1743)
* modify changelog

Signed-off-by: groot <yihua.mo@zilliz.com>

* improve search qps

Signed-off-by: groot <yihua.mo@zilliz.com>

* changelog

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix hang bug

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix combine request result bug

Signed-off-by: groot <yihua.mo@zilliz.com>

* add unittest for combine request

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix python test failure

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix python test failure

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix unittest failure

Signed-off-by: groot <yihua.mo@zilliz.com>
2020-03-26 14:26:02 +08:00
Cai Yudong
578d2633eb
#1756 fix memory exhausted during searching (#1758)
* #1756 fix memory exhausted during searching

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #1756 update const names

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-03-26 13:55:11 +08:00
BossZou
5a83ba4e49
Optimize behavior to get file ids from metadata in mishards (#1754) (#1755)
* update mishards get file ids

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update behaviour for getting file ids

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* check default partition

Signed-off-by: yhz <413554850@qq.com>

* remove exception when file id not found

Signed-off-by: yhz <413554850@qq.com>

* Optimize behavior to get file ids from metadata in mishards (fix #1754)

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* [skip ci] Update changlog

Signed-off-by: Yhz <yinghao.zou@zilliz.com>
2020-03-26 12:28:20 +08:00