Commit Graph

5356 Commits

Author SHA1 Message Date
Accagain
c7eb9b0fa2
add safty check 4 annoy load index (#2022)
Signed-off-by: cms <chenmaosen0@gmail.com>
2020-04-22 10:00:46 +08:00
nezha2017
00e76219f6
use #include "segment/SegmentWriter.h" instead (#2020)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2020-04-22 09:46:33 +08:00
mileyzjq
72d80aca74 fix unused code and clarify params
Signed-off-by: mileyzjq <mileyzha@gmail.com>
2020-04-21 22:08:50 +08:00
shiyu09
0320259976
rename table_info into collection_info (#2016)
Signed-off-by: shiyu09 <39143280+shiyu09@users.noreply.github.com>
2020-04-21 20:30:16 +08:00
adream307
a3d99de5de
Context-head-file (#2021)
* Context-head-file

Signed-off-by: adream307 <adream307@163.com>

* clang-format

Signed-off-by: adream307 <adream307@163.com>

Co-authored-by: adream307 <adream307@163.com>
2020-04-21 19:04:06 +08:00
del-zhenwu
7b6ff08466
Update timeout strategy in CI (#2000)
Signed-off-by: zw <zw@zilliz.com>
2020-04-21 18:46:21 +08:00
break2017
65a6d8cce7
[skip-ci] remove redundant code in test cases (#2019)
Signed-off-by: break2017 <zhenxiang0708@163.com>
2020-04-21 16:40:02 +08:00
nameczz
5bcd3034c6
remove unused function "normalize_vector" (#2015)
Signed-off-by: nameczz <nameczz@126.com>
2020-04-21 15:53:17 +08:00
bo huang
574ff66c01
add safty check 4 annoy load index (#2009)
Signed-off-by: bo-huang <1013767522@qq.com>
2020-04-21 14:23:42 +08:00
Cai Yudong
f039032c8c
Caiyd 1965 unify metric cal (#2008)
* #1965 add metric algorithm benchmark for FAISS

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #1965 add metric algorithm benchmark for NSG

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #1965 add metric algorithm benchmark for ANNOY

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #1965 add metric algorithm benchmark for HNSW

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code opt

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* calculate average time

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #1965 annoy/nsg/hnsw all use faiss distance algorithm

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #1965 support AVX512/AVX2/SSE42

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #1965 update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #1965 fix hnsw ip calculation error

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-04-21 14:22:33 +08:00
groot
a5eec9d7b4
create index notify (#2003)
* create index notify

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: groot <yihua.mo@zilliz.com>
2020-04-21 14:22:18 +08:00
xudalin0609
9e30c468ff
remove unused var (#2006)
Signed-off-by: xudl <isxudl@163.com>

Co-authored-by: xudl <isxudl@163.com>
2020-04-21 12:13:23 +08:00
Jin Hai
7350337884
Update readme cn (#2011)
* Update README.md

Signed-off-by: Jin Hai <haijin.chn@gmail.com>

* [skip ci] Update README_CN.md

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2020-04-21 11:42:27 +08:00
Jin Hai
23526f0d72
[skip ci] Update README.md (#2010)
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2020-04-21 11:39:58 +08:00
Morry Niu
75ca0d96b6
keep bitset naming consistent (#2004)
Signed-off-by: Xinyao Niu <morryniu123@gmail.com>
2020-04-21 09:40:49 +08:00
del-zhenwu
55cc6be6c9
Set case level (#1968)
* add mishards test stage

Signed-off-by: zw <zw@zilliz.com>

* add clean up

Signed-off-by: zw <zw@zilliz.com>

* Set case level

Signed-off-by: zw <zw@zilliz.com>

* update test timeout

Signed-off-by: zw <zw@zilliz.com>

* remove some test cases

Signed-off-by: zw <zw@zilliz.com>

* udpate

Signed-off-by: zw <zw@zilliz.com>

* remove connect

Signed-off-by: zw <zw@zilliz.com>

* update case

Signed-off-by: zw <zw@zilliz.com>
2020-04-21 01:33:05 +00:00
JUN
a2e7a91964
Modify data type in CommonUtil.cpp (#1996)
Signed-off-by: JUN <gujun720@foxmail.com>
2020-04-20 20:42:16 +08:00
Jin Hai
315dac3338
Automerge1 (#1984)
* [skip ci] remove auto-merge git-action

Signed-off-by: jinhai <hai.jin@zilliz.com>

* [skip ci] update CHANGELOG.md

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Stop strip the binary

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Improve code

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix compiling error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix warning

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-04-20 20:39:22 +08:00
Yaning Zhong
814b8f0c53
move const char* to .cpp file (#1994)
Signed-off-by: ynzhong <zhongyn522@gmail.com>

Co-authored-by: ynzhong <zhongyn522@gmail.com>
2020-04-20 15:41:38 +08:00
Tracy199
c50de3de9a
use TimeRecorder instead of chrono (#1989)
Signed-off-by: Tracy199 <houkx@vip.sina.com>
2020-04-20 01:29:11 +00:00
HuangHua
008f24a7c2
replace metaOptions with meta_options (#1979)
* replace metaOptions with meta_options

Signed-off-by: huanghua <bruce.huang@hotmail.com>

* typo

Signed-off-by: groot <yihua.mo@zilliz.com>

Co-authored-by: groot <yihua.mo@zilliz.com>
2020-04-19 08:00:50 +00:00
smiling849
50ddc49332
fix hnsw code format (#1987)
Signed-off-by: Zhonglei Song <szl_84@sina.com>
2020-04-19 12:00:45 +08:00
bigbraver
e8f759c22b
code optimize (#1986)
Signed-off-by: bigbraver <bigbraver@qq.com>
2020-04-19 03:54:23 +00:00
Hanyang Ke
25b6efc8e8
fix warning (#1988)
Signed-off-by: kehycs <kehycs@foxmail.com>
2020-04-19 11:12:01 +08:00
anchun
29acf6a594
update some variables in validation utils. (#1978)
Signed-off-by: anchun <nkterry@163.com>
2020-04-18 21:59:26 +08:00
Jin Hai
ecafa5ab3f
[skip ci] update mergify (#1983)
Signed-off-by: jinhai <hai.jin@zilliz.com>
2020-04-18 20:38:49 +08:00
zaqwsxqwer111
54fb8ef357
remove pragma once from .cpp files (#1982)
Signed-off-by: zaqwsxqwer111 <2536173042@qq.com>
2020-04-18 20:28:10 +08:00
Jin Hai
930df204f6
Automerge (#1981)
* [skip ci] add mergify config

Signed-off-by: jinhai <hai.jin@zilliz.com>

* [skip ci] update mergify config

Signed-off-by: jinhai <hai.jin@zilliz.com>

* [skip ci] update changelog

Signed-off-by: jinhai <hai.jin@zilliz.com>
2020-04-18 20:23:35 +08:00
mergify[bot]
06335ad80b
Merge pull request #1974 from zhangch5n/master
not to printf the log when record type is None
2020-04-18 11:49:07 +00:00
mergify[bot]
b97682529a
Merge pull request #1963 from JinHai-CN/test1
[skip ci] update change log
2020-04-18 11:31:57 +00:00
Jin Hai
e89c231c84
[skip ci] add mergify config (#1980)
Signed-off-by: jinhai <hai.jin@zilliz.com>
2020-04-18 19:22:56 +08:00
zhangchen
b02a37d79d not to printf the log when record type is None
Signed-off-by: zhangchen <zhangch5n@163.com>
2020-04-18 17:29:12 +08:00
Rayn Wei
375fd709a9
. (#1972)
Signed-off-by: RyanWei <rainmanwy@gmail.com>
2020-04-18 13:22:47 +08:00
groot
85de706d22
#1929 Skip mysql meta schema field width check (#1937)
* #1929 Skip mysql meta schema field width check

Signed-off-by: groot <yihua.mo@zilliz.com>

* typo

Signed-off-by: groot <yihua.mo@zilliz.com>

* merge master

Signed-off-by: groot <yihua.mo@zilliz.com>
2020-04-18 10:26:22 +08:00
Jin Hai
59035d9892
#1454 Improve code quality (#1948)
* Improve code quality

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Improve code quality

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Improve code quality

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix lint

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix compiling error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix compiling error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix compiling error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix issue

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-04-17 21:05:17 +08:00
jinhai
b6a8477629 [skip ci] update change log
Signed-off-by: jinhai <hai.jin@zilliz.com>
2020-04-17 12:20:18 +08:00
jinhai
35ab255f59 [skip ci] update change log
Signed-off-by: jinhai <hai.jin@zilliz.com>
2020-04-17 11:46:28 +08:00
Jin Hai
6b106cbef1
[skip ci] Update CHANGELOG (#1909)
* [skip ci] Update CHANGELOG

Signed-off-by: jinhai <hai.jin@zilliz.com>

* [skip ci] update auto merge

Signed-off-by: jinhai <hai.jin@zilliz.com>
2020-04-17 11:45:22 +08:00
del-zhenwu
f33ba8f8c9
add mishards test stage (#1960)
* add mishards test stage

Signed-off-by: zw <zw@zilliz.com>

* add clean up

Signed-off-by: zw <zw@zilliz.com>
2020-04-16 23:35:48 +08:00
XuPeng-SH
753c2ff906
Mishards k8s fix (#1958)
* [skip ci](shards): update kubernetes related

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* [skip ci](shards): add pending buff in kubernetes provider

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* [skip ci](shards): some bug fix in k8s provider

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* [skip ci](shards): update log

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* [skip ci](shards): update k8s config

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* [skip ci](shards): update Makefile

Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-04-16 22:04:47 +08:00
yukun
3e1b2ab4a0
Filtering by numeric scalar fields prototype (#1919)
* Add hybrid request handler

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add C++ sdk for createcollection and insertentities

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add RequestHandler

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add test case for hybrid insert

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix sqlite bug for createcollection

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add HybridQuery Handler DBImpl and ExecBinaryQuery

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add HybridSearch sdk

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add HybridSearch test case

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix HybridSearch bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix HybridSearch crash bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Change void* to vector<uint8_t> in Attr codec

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add context and new search task

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add merge for Hybrid

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add AST validation

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add unittest for hybrid

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix hybrid search nq bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix bugs after merge master

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix clang format

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix unittest bugs

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix Codacy

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix compact unittest bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Remove grpc request in hybridsearchcontext

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix some codacy quality issue

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix HYBRID_DB_TEST bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Annotate new search task

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add Hybrid RPC handler unittest

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix logs

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add HybridSearch unittest

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2020-04-16 14:54:12 +08:00
quicksilver
6a36eaa982
Implement a retry option for helm deploy failed stages in Jenkins Pipeline (#1907) 2020-04-16 09:37:29 +08:00
groot
93244a2a79
#1705 Limit the insert data batch size (#1936)
* #1705 Limit the insert data batch size

Signed-off-by: groot <yihua.mo@zilliz.com>

* typo

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix unittest

Signed-off-by: groot <yihua.mo@zilliz.com>
2020-04-16 00:18:23 +08:00
Jin Hai
0323aa1aad
Merge 080 (#1940)
* #1910 C++ SDK GetIDsInSegment could not work for large dataset (#1911)

Signed-off-by: groot <yihua.mo@zilliz.com>

* #1903 Fix invalid annoy result (#1912)

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* #1914: Partition max size should be 4096 (#1915)

Signed-off-by: jinhai <hai.jin@zilliz.com>

* add log (#1913)

* add log

Signed-off-by: groot <yihua.mo@zilliz.com>

* add log

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix ut

Signed-off-by: groot <yihua.mo@zilliz.com>

* partition limit 4096

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix py test

Signed-off-by: groot <yihua.mo@zilliz.com>

* update server version (#1916)

Signed-off-by: zw <zw@zilliz.com>

* Update to 0.8.0 (#1918)

* Create new branch 0.8.0 and change preload_table to preload_collection

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix format

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Update CHANGELOG

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Update CHANGELOG

Signed-off-by: jinhai <hai.jin@zilliz.com>

* update helm version

Signed-off-by: zw <zw@zilliz.com>

* Update CHANGELOG

Signed-off-by: jinhai <hai.jin@zilliz.com>

Co-authored-by: zw <zw@zilliz.com>

* fix issue 1901 (#1920)

* fix issue 1901

Signed-off-by: cmli <chengming.li@zilliz.com>

* update change log

Signed-off-by: cmli <chengming.li@zilliz.com>

Co-authored-by: cmli <chengming.li@zilliz.com>

* #1900 (#1923)

* add log

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix #1900

Signed-off-by: groot <yihua.mo@zilliz.com>

* Upgrade mishards to 0.8.0 (#1933)

* update grpc server of milvus & rename table name to collection

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update changlog

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* [skip ci] Skip CI

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* [skip ci] Update changlog

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Caiyd 1883 fix rw (#1926)

* #1883 use DiskIO

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix logic error

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* retry CI

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* Update CHANGELOG

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Co-authored-by: JinHai-CN <hai.jin@zilliz.com>

* #1928 Too many data and uid copies when loading files (#1931)

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

Co-authored-by: Jin Hai <hai.jin@zilliz.com>

* Update mishards configure files (#1938)

* Update web readme

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* [skip ci] update configure files

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* [skip ci] rename table to collection

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Update test.groovy

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Update test.groovy

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix lint

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix compiling error

Signed-off-by: jinhai <hai.jin@zilliz.com>

Co-authored-by: groot <yhmo@zeronedata.com>
Co-authored-by: shengjun.li <49774184+shengjun1985@users.noreply.github.com>
Co-authored-by: del-zhenwu <56623710+del-zhenwu@users.noreply.github.com>
Co-authored-by: zw <zw@zilliz.com>
Co-authored-by: op-hunter <ophunter52@gmail.com>
Co-authored-by: cmli <chengming.li@zilliz.com>
Co-authored-by: BossZou <40255591+BossZou@users.noreply.github.com>
Co-authored-by: Cai Yudong <yudong.cai@zilliz.com>
2020-04-15 21:32:20 +08:00
Wang XiangYu
890fe08e7c
Refactor LOG macro and improve log format (#1927)
* update log format

Signed-off-by: wxyu <xy.wang@zilliz.com>

* add new log macro

Signed-off-by: wxyu <xy.wang@zilliz.com>

* use new log macro instead

Signed-off-by: Xiangyu Wang <xy.wang@zilliz.com>

* add SetThreadName function

Signed-off-by: wxyu <xy.wang@zilliz.com>

* clang-format

Signed-off-by: wxyu <xy.wang@zilliz.com>

* set thread name

Signed-off-by: Xiangyu Wang <xy.wang@zilliz.com>

* add changelog

Signed-off-by: wxyu <xy.wang@zilliz.com>

* add git ignore .swp file

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update log level in LogUtil.cpp

Signed-off-by: wxyu <xy.wang@zilliz.com>
2020-04-15 14:44:00 +08:00
Snyk bot
407cedd74b
fix: upgrade org.apache.commons:commons-lang3 from 3.9 to 3.10 (#1934)
Snyk has created this PR to upgrade org.apache.commons:commons-lang3 from 3.9 to 3.10.

See this package in NPM:
https://www.npmjs.com/package/org.apache.commons:commons-lang3

See this project in Snyk:
https://app.snyk.io/org/cxie/project/2f701ce6-b421-4bbf-aaad-9d2bdb8569e4?utm_source=github&utm_medium=upgrade-pr
2020-04-15 10:27:58 +08:00
Cai Yudong
a63d832772
#1921 use TimeRecorder instead of chrono (#1922)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-04-13 18:42:31 +08:00
Jin Hai
f8a28699b4
Change to 0.8.0 and rename from preload_table to preload_collection (#1904)
* Create new branch 0.8.0 and change preload_table to preload_collection

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix format

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Update CHANGELOG

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Update CHANGELOG

Signed-off-by: jinhai <hai.jin@zilliz.com>

* update helm version

Signed-off-by: zw <zw@zilliz.com>

Co-authored-by: zw <zw@zilliz.com>
2020-04-11 19:01:29 +08:00
Jin Hai
d0a01c4e59
[skip ci] Add auto-merge action (#1908)
* [skip ci] Add auto-merge action

Signed-off-by: jinhai <hai.jin@zilliz.com>

* [skip ci] Update auto-merge action

Signed-off-by: jinhai <hai.jin@zilliz.com>

* [skip ci] Update auto-merge action

Signed-off-by: jinhai <hai.jin@zilliz.com>
2020-04-11 17:00:22 +08:00
groot
ad4645fce0
check task table limit (#1902)
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-04-11 13:13:32 +08:00