Commit Graph

5042 Commits

Author SHA1 Message Date
Jin Hai
929a976b11
Merge pull request #1559 from yhmo/ppp
#1556 Index file not create after table and index create
2020-03-08 14:01:08 +08:00
JinHai-CN
a96a837186 Update CHANGELOG.md
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-03-08 12:39:00 +08:00
groot
9f7bf032da typo
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-03-08 12:14:05 +08:00
groot
fcb5656819 refine c++sdk
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-03-08 12:13:06 +08:00
groot
1829624a43 #1556 Index file not create after table and index create
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-03-08 11:59:25 +08:00
Jin Hai
ab8339ced4
Merge pull request #1555 from yhmo/master
fix sqlite bug
2020-03-08 11:58:21 +08:00
yudong.cai
7cd2138e3c Merge remote-tracking branch 'main/master' into caiyd_1546_move_config 2020-03-08 11:11:09 +08:00
groot
f8fa76ac96 remove search by id interface from c++sdk
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-03-08 10:44:19 +08:00
groot
a145791060 fix sqlite bug
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-03-08 10:27:36 +08:00
Jin Hai
748e1b461f
Merge pull request #1542 from milvus-io/snyk-fix-aaa0973a886c271d6f1ca77febcd9a36
[Snyk] Security upgrade urllib3 from 1.25.3 to 1.25.8
2020-03-08 00:30:59 +08:00
Jin Hai
ba701808a3
#1532 Search with ivf_flat failed with open-dataset: sift-256-hamming (#1553)
* #1532 Search with ivf_flat failed with open-dataset: sift-256-hamming

Signed-off-by: groot <yihua.mo@zilliz.com>

* for meta migration

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix typo

Signed-off-by: groot <yihua.mo@zilliz.com>
2020-03-08 00:29:59 +08:00
groot
b78521021b fix typo
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-03-07 22:48:02 +08:00
groot
9de0dea134 fix typo
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-03-07 22:34:07 +08:00
groot
b729d04236 Merge remote-tracking branch 'source/master' 2020-03-07 22:26:18 +08:00
yudong.cai
c29877901f Merge remote-tracking branch 'main/master' into caiyd_1546_move_config 2020-03-07 21:09:49 +08:00
BossZou
0fcdc31c7c
Update resources name in HTTP module (#1545)
* set preload_table correctly (fix #1524)

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update res name in http module (fix #1544)

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* add bool convertor function

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* fix test fail

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* optimize update config method & add boolean conversion comments

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* remove comments

Signed-off-by: Yhz <yinghao.zou@zilliz.com>
2020-03-07 20:56:01 +08:00
Cai Yudong
0fd9bf8e70
#1529 fix server crash when cache_insert_data enabled (#1541)
* #1529 fix server crash when cache_insert_data enabled

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #1529 fix bug

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #1529 code opt

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #1529 set uids for CopyGpuToCpu()

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-03-07 20:54:26 +08:00
groot
e0886f84d6 for meta migration
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-03-07 20:49:27 +08:00
groot
c2b5b00592 #1532 Search with ivf_flat failed with open-dataset: sift-256-hamming
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-03-07 19:50:57 +08:00
yudong.cai
b277b48308 #1546 add changelog
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-03-07 17:04:07 +08:00
yudong.cai
6b81dd55df #1536 move Config.cpp to config directory
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-03-07 17:02:21 +08:00
Xiaohai Xu
8e0dccdde0 update config test
Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au>
2020-03-07 17:00:02 +08:00
Xiaohai Xu
7a23cf23a4 update config test
Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au>
2020-03-07 16:35:51 +08:00
groot
a08b51c2b6
merge json to master to get docker image (#1500)
* General proto api for NNS libraries

Signed-off-by: groot <yihua.mo@zilliz.com>

* refactor confadapter

Signed-off-by: Nicky <nicky.xj.lin@gmail.com>

* fix unittest failures

Signed-off-by: groot <yihua.mo@zilliz.com>

* update test_add

Signed-off-by: zhenwu <zw@zilliz.com>

* update knowhere

Signed-off-by: Nicky <nicky.xj.lin@gmail.com>

* update test cases

Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au>

* Update cases

* C++ sdk for json parameters

Signed-off-by: groot <yihua.mo@zilliz.com>

* update unittest

Signed-off-by: Nicky <nicky.xj.lin@gmail.com>

* fix unittest failures

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix case

Signed-off-by: del-zhenwu <zw@zilliz.com>

* modify test_index.py

Signed-off-by: shengjh <jianghong.sheng@zilliz.com>

* update

Signed-off-by: Nicky <nicky.xj.lin@gmail.com>

* update sptag

Signed-off-by: Nicky <nicky.xj.lin@gmail.com>

* update...

Signed-off-by: Nicky <nicky.xj.lin@gmail.com>

* Build Pass

Signed-off-by: xiaojun.lin <xiaojun.lin@zilliz.com>

* knowhere/wrapper ut pass

Signed-off-by: xiaojun.lin <xiaojun.lin@zilliz.com>

* update util

Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au>

* fix wal case

Signed-off-by: del-zhenwu <zw@zilliz.com>

* modify test_search_vectors

Signed-off-by: shengjh <jianghong.sheng@zilliz.com>

* update ci

Signed-off-by: del-zhenwu <zw@zilliz.com>

* update util

Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au>

* modify test_search_vectoes

Signed-off-by: shengjh <jianghong.sheng@zilliz.com>

* add hnsw in http module & modify index apis

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* modify search in http module

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* fix build error

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix typo in test_index and test_search

Signed-off-by: shengjh <jianghong.sheng@zilliz.com>

* update...

Signed-off-by: Nicky <nicky.xj.lin@gmail.com>

* index apis in http module done

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* fix build index bug

Signed-off-by: groot <yihua.mo@zilliz.com>

* search apis unittest pass

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* web test pass

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update confadapter

Signed-off-by: Nicky <nicky.xj.lin@gmail.com>

* update util

Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au>

* code format

Signed-off-by: groot <yihua.mo@zilliz.com>

* code format

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix vectors results bug (fix #1476)

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* clang format

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update test

Signed-off-by: shengjh <jianghong.sheng@zilliz.com>

* fix unittest

Signed-off-by: groot <yihua.mo@zilliz.com>

* add test_config

Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au>

* add log

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix a build error

Signed-off-by: groot <yihua.mo@zilliz.com>

* add invalid param search test

Signed-off-by: shengjh <jianghong.sheng@zilliz.com>

* fix range check

Signed-off-by: Nicky <nicky.xj.lin@gmail.com>

* cpmpact/flush case passed

Signed-off-by: del-zhenwu <zhenxiang.li@zilliz.com>

* fix unittest failures

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix unittest failures

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix unittest failures

Signed-off-by: groot <yihua.mo@zilliz.com>

* validate json parameters in request

Signed-off-by: groot <yihua.mo@zilliz.com>

* add unittest cases

Signed-off-by: groot <yihua.mo@zilliz.com>

* update test index/search

Signed-off-by: shengjh <jianghong.sheng@zilliz.com>

* update test_config

Signed-off-by: sahuang <xiaohaix@student.unimelb.edu.au>

* fix

Signed-off-by: shengjh <jianghong.sheng@zilliz.com>

* support nsg and ivf-nlist

Signed-off-by: Nicky <nicky.xj.lin@gmail.com>

* update

Signed-off-by: xiaojun.lin <xiaojun.lin@zilliz.com>

* fix validation bug

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix python test bug

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix python test bug

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix python test bug

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix python test bug

Signed-off-by: groot <yihua.mo@zilliz.com>

* code format

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix python test failure

Signed-off-by: groot <yihua.mo@zilliz.com>

* remove rnsg cases

Signed-off-by: zhenwu <zw@zilliz.com>

* fix python test failure

Signed-off-by: groot <yihua.mo@zilliz.com>

* Update changelog

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix typo

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* add pq to test_index && multithread test

Signed-off-by: shengjh <jianghong.sheng@zilliz.com>

* add pq to test_search

Signed-off-by: shengjh <jianghong.sheng@zilliz.com>

* Fix format

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Update CHANGELOG

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix compiling error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix compiling error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* fix config bug

Signed-off-by: groot <yihua.mo@zilliz.com>

* code format

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix config test

Signed-off-by: xiaojun.lin <xiaojun.lin@zilliz.com>

* Update CHANGELOG.md

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Update CHANGELOG.md

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* disable config test case

Signed-off-by: zhenwu <zw@zilliz.com>

Co-authored-by: Nicky <nicky.xj.lin@gmail.com>
Co-authored-by: zhenwu <zw@zilliz.com>
Co-authored-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au>
Co-authored-by: shengjh <jianghong.sheng@zilliz.com>
Co-authored-by: xiaojun.lin <xiaojun.lin@zilliz.com>
Co-authored-by: Yhz <yinghao.zou@zilliz.com>
Co-authored-by: del-zhenwu <zhenxiang.li@zilliz.com>
Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
2020-03-07 15:23:34 +08:00
Cai Yudong
692986760b
Merge branch 'master' into caiyd_codec_opt 2020-03-07 14:38:57 +08:00
Cai Yudong
2a101eaab1
#1524 update config preload_table description (#1543)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-03-07 13:18:14 +08:00
snyk-bot
bdb87db380 fix: shards/requirements.txt to reduce vulnerabilities
The following vulnerabilities are fixed by pinning transitive dependencies:
- https://snyk.io/vuln/SNYK-PYTHON-URLLIB3-559452
2020-03-07 02:46:33 +00:00
BossZou
1963f3b9e9
Make cache_insert_data take effect in-service(# 1521) (#1540)
* make cache_insert_data take effect in-service (fix #1521)

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* complete omitted config setter

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* cache_insert_data take effect for MemTable

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* add config test

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* rename config handler

Signed-off-by: Yhz <yinghao.zou@zilliz.com>
2020-03-07 09:34:35 +08:00
Tinkerrr
8cd1ec9819
fix 1507 (#1509)
* fix

Signed-off-by: Nicky <nicky.xj.lin@gmail.com>

* update..

Signed-off-by: Nicky <nicky.xj.lin@gmail.com>

* update...

Signed-off-by: Nicky <nicky.xj.lin@gmail.com>

* update...

Signed-off-by: Nicky <nicky.xj.lin@gmail.com>

* update...

Signed-off-by: Nicky <nicky.xj.lin@gmail.com>

* comment ut

Signed-off-by: xiaojun.lin <xiaojun.lin@zilliz.com>

* Fix lint error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* update....

Signed-off-by: xiaojun.lin <xiaojun.lin@zilliz.com>

* update....

Signed-off-by: xiaojun.lin <xiaojun.lin@zilliz.com>

* Fix typo

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix typo

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

Co-authored-by: Jin Hai <hai.jin@zilliz.com>
2020-03-06 22:14:58 +08:00
Tinkerrr
626c9bc593
set size after load (#1539)
Signed-off-by: xiaojun.lin <xiaojun.lin@zilliz.com>
2020-03-06 21:10:42 +08:00
yudong.cai
5c34c48d87 #1537 fix clang format
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-03-06 19:19:39 +08:00
yudong.cai
560f35a9d3 #1537 update changelog
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-03-06 19:19:39 +08:00
yudong.cai
417ebb7cb9 add read_vectors_internal() and read_uids_internal()
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-03-06 19:19:39 +08:00
Zhiru Zhu
5bb1d001e2
Fix#1530 (#1533)
* update

Signed-off-by: youny626 <zzhu@fandm.edu>

* update

Signed-off-by: Zhiru Zhu <zzhu@fandm.edu>

* update

Signed-off-by: Zhiru Zhu <zzhu@fandm.edu>

Co-authored-by: Jin Hai <hai.jin@zilliz.com>
2020-03-06 15:21:39 +08:00
Cai Yudong
d28d1db907
#1531 remove S3 related config (#1534)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-03-06 13:33:35 +08:00
BossZou
df47fad78c
Add setter API for config preload_table (#1526)
* update code

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* optimize http module code

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* move gpu res handler to config holder

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Add setter API for config preload_table (fix #1525)

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* add else branch at end of if condition

Signed-off-by: Yhz <yinghao.zou@zilliz.com>
2020-03-06 13:26:01 +08:00
Zhiru Zhu
b43e0fbada
Update compact (#1522)
* update

Signed-off-by: youny626 <zzhu@fandm.edu>

* update

Signed-off-by: youny626 <zzhu@fandm.edu>

* update

Signed-off-by: Zhiru Zhu <zzhu@fandm.edu>

* update

Signed-off-by: Zhiru Zhu <zzhu@fandm.edu>

* [skip ci] update CHANGELOG

Signed-off-by: Zhiru Zhu <zzhu@fandm.edu>

* update

Signed-off-by: Zhiru Zhu <zzhu@fandm.edu>

* update

Signed-off-by: Zhiru Zhu <zzhu@fandm.edu>

* update

Signed-off-by: Zhiru Zhu <zzhu@fandm.edu>

* update

Signed-off-by: Zhiru Zhu <zzhu@fandm.edu>
2020-03-06 09:40:29 +08:00
Jin Hai
19b8565748
Fix vectors results bug(#1476) & change vector id type to string(#1511) (#1514)
* add sign-off

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* add index HNSW in http module

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

Co-authored-by: Jin Hai <hai.jin@zilliz.com>
2020-03-05 16:30:30 +08:00
Jin Hai
570ecf4266
Add cfg fun (#1501)
* save work

Signed-off-by: lichengming <chengming.li@zilliz.com>

* wal config&gtest

Signed-off-by: lichengming <chengming.li@zilliz.com>

* fix dco and lint error

Signed-off-by: lichengming <chengming.li@zilliz.com>

* try to fix lint error

Signed-off-by: lichengming <chengming.li@zilliz.com>

* try to fix clang-format issue

Signed-off-by: lichengming <chengming.li@zilliz.com>

* update change log

Signed-off-by: lichengming <chengming.li@zilliz.com>

* ajdjust the code by yudong's advice

Signed-off-by: lichengming <chengming.li@zilliz.com>

* fix error

Signed-off-by: lichengming <chengming.li@zilliz.com>

* recommit pr 1501,local gtest is ok

Signed-off-by: lichengming <chengming.li@zilliz.com>
2020-03-05 14:08:07 +08:00
Jin Hai
8c7ef47f20
Fix duplicate vector ID issue (#1508)
* #1499 Fix duplicated ID number issue

Signed-off-by: jinhai <hai.jin@zilliz.com>

* #1499 Fix duplicated ID number issue - part 1

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* #1499 Fix duplicated ID number issue - part 2

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* #1499 Fix duplicated ID number issue - part 3

Signed-off-by: jinhai <hai.jin@zilliz.com>

* #1499 Fix duplicated ID number issue - part 4

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix format issue

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Update CHANGELOG

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Add return value check

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-03-05 12:38:22 +08:00
Jin Hai
9899cfefb7
#1475 add debug timing for map uids (#1513)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-03-05 11:23:36 +08:00
Zhiru Zhu
fc8a6e0779
Fix #1491 and #1504 (#1506)
* update

Signed-off-by: Zhiru Zhu <zzhu@fandm.edu>

* update

Signed-off-by: Zhiru Zhu <zzhu@fandm.edu>

* update

Signed-off-by: Zhiru Zhu <zzhu@fandm.edu>

* fix centos compile error

Signed-off-by: Zhiru Zhu <zzhu@fandm.edu>
2020-03-04 21:10:11 +08:00
yudong.cai
f6b4ed3335 #1463 enable config version check
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-03-04 11:01:05 +08:00
Cai Yudong
600dac7500
#1463 update config version to 0.2 (#1466)
* #1463 update config version to 0.2

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #1463 disable config version check

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-03-04 00:02:48 +08:00
del-zhenwu
962a00c366
Test mysql-meta (#1489)
* fix cases: test_search_id/test_wal

Signed-off-by: del-zhenwu <zw@zilliz.com>

* Run test with mysql-meta

Signed-off-by: del-zhenwu <zw@zilliz.com>
2020-03-03 19:20:15 +08:00
Zhiru Zhu
01b6ad3627
Fix#1484 (#1490)
* update

Signed-off-by: Zhiru Zhu <zzhu@fandm.edu>

* #1426 Support to configure whether to enabled autoflush (#1468)

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
Signed-off-by: Zhiru Zhu <zzhu@fandm.edu>

* fix cases: test_search_id/test_wal (#1486)

Signed-off-by: del-zhenwu <zw@zilliz.com>
Signed-off-by: Zhiru Zhu <zzhu@fandm.edu>

* #1480 add return code for AVX512 selection (#1482)

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
Signed-off-by: Zhiru Zhu <zzhu@fandm.edu>

* update

Signed-off-by: Zhiru Zhu <zzhu@fandm.edu>

* update

Signed-off-by: Zhiru Zhu <zzhu@fandm.edu>

Co-authored-by: shengjun.li <49774184+shengjun1985@users.noreply.github.com>
Co-authored-by: del-zhenwu <56623710+del-zhenwu@users.noreply.github.com>
Co-authored-by: Cai Yudong <yudong.cai@zilliz.com>
2020-03-03 18:31:49 +08:00
Cai Yudong
d460f48ce9
#1480 add return code for AVX512 selection (#1482)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-03-03 13:55:26 +08:00
del-zhenwu
5eca1fce90
fix cases: test_search_id/test_wal (#1486)
Signed-off-by: del-zhenwu <zw@zilliz.com>
2020-03-03 12:43:06 +08:00
shengjun.li
6e3991d167
#1426 Support to configure whether to enabled autoflush (#1468)
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-03-03 11:52:50 +08:00
Jin Hai
23e0816baa
Fix Markdown format (#1485)
* Fix #839 and improve the code quality

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix compiling error and building threads number

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix Markdown quality

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix Markdown quality

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix Markdown quality

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix Markdown format

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix Markdown format

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix Markdown format

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-03-03 00:25:22 +08:00