Commit Graph

1366 Commits

Author SHA1 Message Date
dragondriver
d2767f920c
Fix index parameters check of RHNSWPQ (#6950)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-08-03 10:25:25 +08:00
XuanYang-cn
47df4a2e57
Fix DataNode BackGroudGC problem (#6551)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-08-03 09:59:24 +08:00
xige-16
87c5a49d9a
Refactor showCollection/showPartition to get load state in memory (#6935)
* refactor showCollection/showPartition to get load state in memory

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* update proto

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* update proto

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* update milvus.proto

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* add show result log to debug

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* fix wrong err message

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* update pymilvus-orm version

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* fix repeated load error

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* update proto

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* fix status nil error

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-08-02 22:39:25 +08:00
congqixia
4d5eeb2890
Fix minio object leaking goroutine (#6944)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-08-02 15:25:24 +08:00
Cai Yudong
fe50f97726
Use id to tell search result validation in proxy reduce stage (#6905)
* use id to tell search result validation

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* enable test_search_binary_hamming_flat_index

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code optimize

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix merge retrieve result issue

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-08-02 10:25:49 +08:00
bigsheeper
e8ea8b51d5
improve query node log (#6897)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-07-31 10:47:22 +08:00
bigsheeper
7307332fab
Add nil check (#6832)
* add nil check

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* use grpc functions

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* use grpc functions

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* fix indexCoord crash

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-07-29 20:11:23 +08:00
bigsheeper
eee0f1e077
Add mutex for segment of segCore in querynode (#6870)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-07-29 16:03:22 +08:00
cai.zhang
d51fec6678
Enhance log when create index failed (#6871)
* Enhance log when create index failed

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Fix bug for repeatedly drop index

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>
2021-07-29 14:47:22 +08:00
godchen
61786f769c
Add file exist check (#6840)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-07-28 16:31:22 +08:00
sunby
23bffe399d
Enhance log print in data coordinator (#6681)
issue: #6680
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-07-28 11:43:22 +08:00
zhenshan.cao
8a1a841011
FixBug: Incorrect handling of time synchronization (#6814)
* FixBug: proxy send wrong sync timetick and DataCoord did not filter channel

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-07-27 14:34:48 +08:00
dragondriver
3cc8ee298e
Filter NaN when reducing search results (#6806)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-07-27 10:19:21 +08:00
xaxys
e43b43e7d8
Refactor RangeExpr & CompareExpr (#6786)
* Changed `RangeExpr` proto to `UnaryRangeExpr` & `BinaryRangeExpr`
Several unit test unpassed.

Signed-off-by: xaxys <tpnnghd@163.com>

* Fix bugs to pass unit test. Fix format.

Signed-off-by: xaxys <tpnnghd@163.com>

* Remove debug information.

Signed-off-by: xaxys <tpnnghd@163.com>

* Fix format.
Remove debug information.
Unify variable name.
Add error information.
Remove `CompareExpr` test in `test_c_api.cpp`.

Signed-off-by: xaxys <tpnnghd@163.com>

* Fix code format.

Signed-off-by: xaxys <tpnnghd@163.com>

* Update `Plan.cpp`.

Signed-off-by: xaxys <tpnnghd@163.com>
2021-07-25 10:23:19 +08:00
zhenshan.cao
2962d31a5f
Fixbug: segment allocate but return value incorrect (#6788)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-07-24 20:49:20 +08:00
godchen
db94d7771f
Read vector from disk (#6707)
* Read vector from disk

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* go fmt

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix git action error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix test error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix action error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix caculate error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* change var name

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* remove unused method

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* remove unused method

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix len error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* remove unused code

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* change bytes to float method

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* change float to bytes method

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix action error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-07-24 09:25:22 +08:00
sunby
aa20d88f6f
Fix datacoord stop bug (#6642)
issue: #6641
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-07-24 09:25:14 +08:00
sunby
fc74a0f578
Auto create new segments when allocating rows more than (#6665)
max number of rows per segment

If user insert too much rows in a request. Now we will return a failed
response. Maybe auto creating new segments to hold that much rows is a
better way.

issue: #6664
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-07-23 21:58:33 +08:00
sunby
63387c83ef
Remove collection check when saving binlog (#6658)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-07-23 21:23:22 +08:00
dragondriver
2821c08560
Add unittests for metric_type: tanimoto (#6773)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-07-23 20:59:33 +08:00
dragondriver
cf8e9d52b1
Reduce the log of proxy (#6766)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-07-23 20:33:34 +08:00
bigsheeper
21bc5810c0
remove redundant log of query node (#6768)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-07-23 20:09:33 +08:00
bigsheeper
2f44fc40a3
fix historical unittest (#6705)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-07-23 19:55:07 +08:00
FluorineDog
aba21baf82
refactor expr executor to use single bitset (#6667)
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-07-23 15:30:12 +08:00
neza2017
5adb69161f
Set CollectionInfo.CreateTime to the timestamp when save the metadata into etcd, (#6763)
not the timestamp when RootCoord receive the CreateCollection Request

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-07-23 14:36:12 +08:00
cai.zhang
e1b52b0583
Add unittest for IndexCoord and IndexNode (#6698)
* Add unittest for IndexCoord and IndexNode

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Fix verifiers

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>
2021-07-23 10:44:12 +08:00
cai.zhang
9aa56cd7b8
Add log for IndexCoord and IndexNode (#6597)
* Add log for IndexCoord and IndexNode

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Make verifier

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Delete unused script

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>
2021-07-22 11:40:11 +08:00
cai.zhang
a3c662c7bf
Add watchWithVersion interface for etcdKV (#6663)
* Add watchWithVersion interface for etcdKV

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Add unittest

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Add LoadWithVersion interface

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>
2021-07-21 18:20:11 +08:00
dragondriver
67c1fdd2c0
Show/Describe collections/partitions with created timestamp (#6700)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-07-21 18:00:14 +08:00
Cai Yudong
cbb4052fdb
Support search and query output fields using wildcard (#6671)
* update wildcard polocy for search and query

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* mark xfail cases

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* mark xfail cases

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize debug log

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix static-check

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-07-21 14:52:12 +08:00
dragondriver
26f890b5f5
Remove too frequent timetick log (#6682)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-07-21 11:26:11 +08:00
congqixia
9b1708f6e5
Fix proxy crashes when retrieve task is called (#6656)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-07-20 22:33:09 +08:00
Cai Yudong
744f2c7f51
optimize test_c_api.cpp (#6637)
* optimize test_c_api.cpp

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* clean redundant file

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* change DIM to const

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize test_c_api.cpp

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update reduce testcase

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-07-20 17:13:21 +08:00
FluorineDog
234954931f
remove deprecated (#6623)
* remove deprecated

Signed-off-by: fluorinedog <fluorinedog@gmail.com>

* fix timeout

Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-07-20 15:18:08 +08:00
godchen
b9f5949680
Add vector chunk manager (#6613)
* add file manager

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* add file manager

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* add vector file manager

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* rename

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* go fmt

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix ut error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix action error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix memory error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-07-20 15:06:09 +08:00
dragondriver
99249a0224
Fix bug: override the compare function of SearchResultPair (#6628)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-07-20 14:38:11 +08:00
sunby
ab5a7cbf44
Remove timesync package (#6619)
issue: #6618
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-07-19 18:53:09 +08:00
sunby
b98b30b96f
Fix datacoord_test.go failed (#6558)
Create a mock rootcoord when testing datacoord to hide the network layer
issue: #6557
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-07-19 10:41:15 +08:00
dragondriver
0e2329c7ad
Refactor snapshot of timestamp statistics (#6600)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-07-17 16:31:29 +08:00
Cai Yudong
a7b27db63e
update fillRetrieveResults parameter (#6598)
* update fillRetrieveResults parameter

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize fillVectorFieldData process

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-07-17 15:17:30 +08:00
dragondriver
2d9361e0ba
Fix bug: remove duplicated primary field id when retrieve entities (#6577)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-07-17 10:53:30 +08:00
Cai Yudong
a992dcf6a8
Support query return vector output field (#6570)
* improve code readibility

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add offset in RetrieveResults

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add VectorFieldInfo into Segment struct

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add new interface for query vector

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update load vector field logic

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update load vector field logic

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fill in field name in query result

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add FieldId into FieldData

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add fillVectorOutputFieldsIfNeeded

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update data_codec_test.go

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add DeserializeFieldData

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* realize query return vector output field

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix static-check

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* disable query vector case

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-07-16 17:19:55 +08:00
紫晴
4c7b57ffc5
Upload codecov report in GitHub Actions (#6565)
* Upload codecov report in GitHub Actions

Signed-off-by: wangting0128 <ting.wang@zilliz.com>
2021-07-16 14:52:21 +08:00
xige-16
2a42244ff6
fix watchQueryChannelTask's info can't be deleted from etcd (#6568)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-07-16 10:21:55 +08:00
dragondriver
84b9791221
Fix snapshot of timestmap statistics (#6543)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-07-15 17:47:55 +08:00
sunby
ee38d14f5d
Fix channel lost after data node crash (#6545)
If we start up 2 data nodes and one of them crashes. We expect that all
channels of crashed node will be allcoated to the alive node. But now we
discover that these channels are lost after data node crash. The reason
is we pass a NodeInfo with empty channel info. We fix it and improve log
print.

issue: #6501
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-07-15 16:38:31 +08:00
bigsheeper
7b9f0d398f
update flowgraph unittest (#6537)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-07-15 14:51:55 +08:00
groot
6b001fd76a
fix debug error in IDE (#6539)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-07-15 14:27:54 +08:00
XuanYang-cn
dca6c6afde
Add unittest for BackGroundGC in DataNode (#6522)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-07-15 10:05:55 +08:00
dragondriver
5016e509be
Support wildcard match on search/query output fields (#6510)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-07-14 18:51:54 +08:00