test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-05 05:18:52 +08:00
Code Issues Actions 14 Packages Projects Releases Wiki Activity
11,066 Commits 96 Branches 105 Tags 186 MiB
5ec895fedc
Commit Graph

151 Commits

Author SHA1 Message Date
groot
314eed2172
Update storage unittest (#7637) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2021-09-09 17:12:55 +08:00
groot
a3666c843f
Add unittest for chunk manager (#7628) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2021-09-09 17:12:15 +08:00
groot
dbd2a02462
Add unittest for VectorChunkManager (#7610) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2021-09-09 14:34:00 +08:00
XuanYang-cn
4785e452be
Fix Arrow unable to find Boost error (#7552) 
Resolve: #7549

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-09-08 10:59:59 +08:00
Cai Yudong
7edd96989f
Improve codecov for paramtable and typeutil (#7451) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-09-02 22:18:10 +08:00
Xiangyu Wang
c62f23e201
[skip ci]Remove neza2017 from reviewers (#7324) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-08-27 00:27:36 +08:00
godchen
8cc24e09c9
Make users get vectors without using local storage (#7134) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-08-18 16:30:11 +08:00
Cai Yudong
e771bda92f
optimize retrieve output vector code structure (#7102) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-08-14 11:18:10 +08:00
Xiangyu Wang
5e78fd6a97
[skip ci]Update approvers (#7063) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-08-12 19:36:09 +08:00
bigsheeper
7307332fab
Add nil check (#6832) 
* add nil check

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* use grpc functions

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* use grpc functions

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* fix indexCoord crash

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-07-29 20:11:23 +08:00
godchen
61786f769c
Add file exist check (#6840) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-07-28 16:31:22 +08:00
godchen
db94d7771f
Read vector from disk (#6707) 
* Read vector from disk

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* go fmt

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix git action error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix test error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix action error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix caculate error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* change var name

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* remove unused method

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* remove unused method

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix len error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* remove unused code

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* change bytes to float method

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* change float to bytes method

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix action error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-07-24 09:25:22 +08:00
godchen
b9f5949680
Add vector chunk manager (#6613) 
* add file manager

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* add file manager

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* add vector file manager

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* rename

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* go fmt

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix ut error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix action error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix memory error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-07-20 15:06:09 +08:00
Cai Yudong
a992dcf6a8
Support query return vector output field (#6570) 
* improve code readibility

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add offset in RetrieveResults

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add VectorFieldInfo into Segment struct

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add new interface for query vector

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update load vector field logic

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update load vector field logic

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fill in field name in query result

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add FieldId into FieldData

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add fillVectorOutputFieldsIfNeeded

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update data_codec_test.go

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add DeserializeFieldData

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* realize query return vector output field

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix static-check

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* disable query vector case

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-07-16 17:19:55 +08:00
congqixia
bc15b74244
Handle datasorter edge cases (#6465) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-07-13 14:37:54 +08:00
quicksilver
7d920e9c9c
Fix compile error on CentOS (#6359) 
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
 2021-07-08 15:43:45 +08:00
Cai Yudong
3387b07dfd
Optimize code under storage (#6335) 
* rename AddOneStringToPayload/GetOneStringFromPayload to AddStringToPayload/GetStringFromPayload

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code optimize

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename print_binglog_test to print_binlog_test

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update chap08_binlog.md

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* use SetEventTimestamp() to replace SetStartTimestamp() and SetEndTimestamp()

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code optimize

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename AddStringToPayload/GetStringFromPayload to AddOneStringToPayload/GetOneStringFromPayload

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-07-07 19:10:07 +08:00
Cai Yudong
5a1f7e4feb
rename masterservice to rootcoord (#5875) 
* rename masterservice to rootcoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix build error

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename service to coord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename package

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename masterservice to rootcoord under distributed

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update scripts

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* enable more unittests

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-06-18 21:30:08 +08:00
godchen
1c6786f85c
Add blob info (#5792) 
* Add blob info

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-06-16 12:03:57 +08:00
Xiangyu Wang
ca0993c274 Update OWNERS 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-05-30 18:58:47 +08:00
Xiangyu Wang
23c4de0eb8
Flush statistics for all int64 fields (#5318) 
Resolves: #5262

Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-05-20 10:38:45 +00:00
yukun
4e6e502df1
Add data sorter in storage (#5200) 
GetEntityByID needs the flushed segment to be sorted by RowID field, 
then do binary search to get the target id and entities.

See also: #5177 

Signed-off-by: fishpenguin kun.yu@zilliz.com
 2021-05-14 02:59:49 +00:00
Xiangyu Wang
82ccd4cec0
Rename module (#4988) 
* Rename module

Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-04-22 14:45:57 +08:00
godchen
0dfcb90881 Add storage copyright 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-04-19 11:32:24 +08:00
godchen
a5ad70a5ab Add unittest for storage 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-04-19 10:36:19 +08:00
FluorineDog
ce7a5ea699 Support segcoreinit 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2021-04-16 14:02:49 +08:00
godchen
8204546bcd Add unit test for binlog module 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-04-12 14:07:46 +08:00
godchen
f3649f0419 Refactor interface and proto 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-03-12 14:22:09 +08:00
XuanYang-cn
7dd531d159 Fix release collection bug 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-03-11 18:42:25 +08:00
xige-16
09ae985daa Fix wrong error code in master_service_test.go 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-03-10 22:06:22 +08:00
quicksilver
455cc59dbb Update publish-test-images.yaml 
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
 2021-03-09 13:55:35 +08:00
sunby
ddddd65d10 Delete internal/errors package 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-03-05 10:15:27 +08:00
bigsheeper
01e9dc8e3f Remove collection name 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-03 11:52:19 +08:00
bigsheeper
5e781b9370 Remove field name in query node and segCore 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-03 10:10:07 +08:00
sunby
45b99c0cf3 Save indexName and indexID in IndexCodec 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-02 19:56:11 +08:00
neza2017
854accf95b Remove master and writenode 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-01-30 16:41:47 +08:00
bigsheeper
73d2b6a101 Get index param from minio and filter by vector fields 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-01-29 15:22:24 +08:00
neza2017
3a866dab3d Update proto 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-01-28 17:25:43 +08:00
neza2017
2ebeade25e Fix get component states 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-01-28 17:13:00 +08:00
sunby
2be8cc1c4b Add index params serde to IndexCodec 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-01-28 16:41:24 +08:00
xige-16
d22c2cd203 Add proto for query service loading collection 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-01-18 19:32:08 +08:00
neza2017
4015d7245d Merge operation 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-01-06 14:45:50 +08:00
cai.zhang
1cfc2ff0a5 Add make clean in Makefile 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-01-06 13:50:48 +08:00
zhenshan.cao
b06e01f523 Initmate querynode collection manager 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-01-06 11:31:36 +08:00
godchen
1c87faf8b3 Add insert request opentracing 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-01-06 11:17:35 +08:00
FluorineDog
e16a7478da Add verifier 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2021-01-05 20:13:54 +08:00
XuanYang-cn
05f1350e19 Add writenode to ci 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-01-05 15:14:59 +08:00
quicksilver
ca89ece576 Fix go format bug 
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
 2021-01-05 14:20:45 +08:00
XuanYang-cn
fa77783ad7 Fix print ddl binlog bug 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-01-05 12:20:30 +08:00
sunby
95b162ccfd Refactor flush scheduler 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-01-05 10:57:59 +08:00
XuanYang-cn
7ce0f27ebc Add buffer to minIO for binlogs 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2020-12-23 18:06:04 +08:00
sunby
e7ebfcb05a Save index meta to meta table 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-12-23 15:13:45 +08:00
godchen
e52b130ba6 Refactor data codec code 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2020-12-23 15:46:16 +08:00
sunby
0b9cffaf87 Remove index_name in IndexRequest and remove indexType in FieldIndexMeta 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-12-23 11:34:35 +08:00
dragondriver
adfb5187cf Dock build index task with storage & indexbuilder 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2020-12-22 00:14:36 +00:00
XuanYang-cn
87521adfbd Add logic of deserilize rawData into InsertData 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2020-12-21 16:27:03 +08:00
FluorineDog
e65cfe1e3d Enable complex dsl parser 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2020-12-19 10:36:49 +08:00
zhenshan.cao
0d75840ed6 Fix missing file 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-12-18 18:35:03 +08:00
neza2017
0110ba6bd2 List events of binlog 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-12-12 16:52:09 +08:00
dragondriver
da9ea7d2b9 Add cgo wrapper to index builder 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2020-12-18 15:44:27 +08:00
godchen
475d36a2c9 Add data codec 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2020-12-18 15:21:25 +08:00
cai.zhang
54f2b79f1b Add indexbuilder client 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-12-12 17:10:42 +08:00
neza2017
07d68de2a3 Fix data codec 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-12-11 18:14:19 +08:00
bigsheeper
585d3f9831 Refactor param table, and add ddNode 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2020-12-11 17:20:14 +08:00
neza2017
8df05714a8 Add binlog unittest 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-12-11 12:01:38 +08:00
XuanYang-cn
d64aa37db3 Add KFlush msg type 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2020-12-11 12:01:20 +08:00
cai.zhang
62e9002348 Add git-hooks for check code 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-12-11 11:29:07 +08:00
neza2017
9585819154 Add binlog unittest 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-12-10 15:50:09 +08:00
neza2017
9cbebc0221 Add binlog unittest 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-12-10 14:52:42 +08:00
quicksilver
21721d2e03 Cache go mod workflow for jenkins pipeline 
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
 2020-12-10 09:57:14 +08:00
neza2017
5512ce8d8a Add binlog unittest 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-12-10 09:31:29 +08:00
XuanYang-cn
881be7f3e3 Impl minimum runable writenode and writenode flowgraph 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2020-12-09 20:07:27 +08:00
neza2017
de12fa5a10 Add binlog event 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-12-09 11:18:13 +08:00
FluorineDog
968ddc6016 Fix thrift.a not found 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2020-12-09 14:57:07 +08:00
sunby
ed54d3e26c Fix binlog reader bug 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-12-09 09:55:56 +08:00
quicksilver
d09ad77fb2 Add archiving file workflow for cache thirdparty 
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
 2020-12-08 19:53:22 +08:00
sunby
8ace1c3837 Add binlog serialization and deserialization 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-12-08 19:13:39 +08:00
XuanYang-cn
e6f726e73a Add cache for thirdparty files cache 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2020-12-08 18:51:07 +08:00
XuanYang-cn
32977e270c Add impl cgo of parquet 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2020-12-08 14:41:04 +08:00
zhenshan.cao
7bbbc14637 Fix bug: address already used 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-12-07 15:22:20 +08:00
xige-16
37d7526d31 Fix search test failure 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2020-12-07 14:37:42 +08:00
neza2017
0d8273c7cc Add string type payload cgo wrapper 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-12-06 15:15:11 +08:00
neza2017
aeca8f85a5 Run cwrapper unittest in ci 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-12-05 17:39:58 +08:00
cai.zhang
806a97505c Add query node for CI progress 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-12-05 16:12:45 +08:00
neza2017
70710dee47 Add parquet payload 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-12-05 16:11:03 +08:00
FluorineDog
63c8f60c6e Enable term parser and executor 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2020-12-05 09:51:27 +08:00
FluorineDog
6412ebc0d4 Add support of metric type in schema, enable binary vector, fix segfault 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2020-12-05 06:46:01 +08:00
zhenshan.cao
0b6fe0a8cf Refactor paramtable 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-12-04 14:18:17 +08:00
neza2017
cec903da19 Add parquet payload 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-12-02 18:31:56 +08:00
cai.zhang
5bcabffdaf Replace conf struct with ParamsTable 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-11-16 21:10:43 +08:00
dragondriver
c2382adf04 Initialize the task scheduler of Proxy 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2020-11-14 11:24:49 +08:00
cai.zhang
1cd2fd9954 Add the standard of golangci-lint 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-11-12 12:04:12 +08:00
zhenshan.cao
e9122921b9 Use type alias 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-11-04 17:58:43 +08:00
neza2017
d18c17e786 Add Meta table implementation 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-11-04 16:28:14 +08:00
godchen
bade037fb2 Add meta_table implementation 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2020-11-04 16:01:28 +08:00
zhenshan.cao
da18bc4952 Update proto 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-11-03 16:38:16 +08:00
zhenshan.cao
e962b57fbb Add timestamp allocator 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-11-03 14:53:36 +08:00
zhenshan.cao
251bc2a19e Fix conf.LoadConfig failed to load yaml file 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-10-19 18:31:00 +08:00
GuoRentong
974496a2b2 Update issue templates 
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
 2020-10-19 17:45:56 +08:00
zhenshan.cao
6a136c1a76 Change import path 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-10-19 17:39:51 +08:00