Commit Graph

50 Commits

Author SHA1 Message Date
congqixia
c490b30db7
Fix segment loader skip wait if all segments are loading (#25643)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-07-17 19:51:18 +08:00
congqixia
a7e6f08183
Fix DeleteCollection not protected (#25599)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-07-14 17:04:31 +08:00
yah01
d216f9abda
Clear collection meta after all channels/segments released (#25486)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-07-14 10:28:30 +08:00
yah01
205a7c430a
Fix QueryNode panic while inserting to released segment (#25495)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-07-12 19:48:29 +08:00
cai.zhang
81278f4aea
Fix bug of missing JSON type when sorting retrieve results (#25412) (#25455)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-07-10 20:02:28 +08:00
yah01
dd5f896dc8
Load batch by batch (#25212)
This will significantly reduce the memory usage while loading
- 1x memory usage and MBs overhead for buffer (memory mode)
- only MBs overhead for buffer (mmap mode)

Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-07-06 13:58:27 +08:00
congqixia
efdd71c640
Make cgo pool size larger than worker pool size (#25318)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-07-05 16:56:26 +08:00
Enwei Jiao
816158e4af
Remove outdated searchplan (#25282)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-07-04 18:30:25 +08:00
yihao.dai
c56a26a556
Add SetMetricType and set metric only for shard leader (#25285)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-07-04 10:20:25 +08:00
yah01
7b5013664d
Support graceful stop (#25226)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-06-30 17:28:23 +08:00
Jiquan Long
8d193a3e56
Skip unnecessary reduce phase during search (#25166)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-06-27 19:22:45 +08:00
yihao.dai
29b9b26de8
Improve get vector error msg (#24838)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-06-27 16:34:45 +08:00
xige-16
33c2012675
Add more metrics (#25081)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-06-26 17:52:44 +08:00
jaime
18df2ba6fd
[Cherry-Pick] Support Database (#24769)
Support Database(#23742)
Fix db nonexists error for FlushAll (#24222)
Fix check collection limits fails (#24235)
backward compatibility with empty DB name (#24317)
Fix GetFlushAllState with DB (#24347)
Remove db from global meta cache after drop database (#24474)
Fix db name is empty for describe collection response (#24603)
Add RBAC for Database API (#24653)
Fix miss load the same name collection during recover stage (#24941)

RBAC supports Database validation (#23609)
Fix to list grant with db return empty (#23922)
Optimize PrivilegeAll permission check (#23972)
Add the default db value for the rbac request (#24307)

Signed-off-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
2023-06-25 17:20:43 +08:00
xige-16
04082b3de2
Migrate the ability to upload and download binlog to cpp (#22984)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-06-25 14:38:44 +08:00
yah01
a413842e38
Fix deleted data is still visible (#24849)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-06-16 17:16:41 +08:00
wei liu
e40d95e093
add search/query request cost metrics on qn (#24413)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-06-15 14:24:38 +08:00
congqixia
41af0a98fa
Use go-api/v2 for milvus-proto (#24770)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-09 01:28:37 +08:00
Enwei Jiao
d3af451d92
Upgrade golangci-lint (#24707)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-06-07 19:34:36 +08:00
yah01
fb66924c90
Set metric type while init the collection (#24665)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-06-06 13:24:36 +08:00
yihao.dai
b0ce4cff83
Assign metric type in search request when user has not explicitly assigned (#24591)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-06-01 18:10:32 +08:00
congqixia
31880ab427
Use conc.PoolOption instead of ants.Option (#24585)
- Add conc.PoolOption to setup conc.Pool
- Change panic default behavior
- Make future has error when job panicks

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-01 15:52:31 +08:00
congqixia
48dff300db
Fix Get segment returns ambiguous result (#24505)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-05-30 12:25:28 +08:00
aoiasd
c84bdcea49
merge stats log when segment flushing or compacting (#23570)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-05-29 10:21:28 +08:00
congqixia
165734de4e
Add unit test for invoking method after segment released (#24414)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-05-26 10:03:27 +08:00
yihao.dai
014387fd94
Forbid to get quantized vector from ChunkManager (#24334)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-24 23:03:27 +08:00
Bingyi Sun
0c8045d66c
Fix crash caused by segment function concurrency (#24351)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2023-05-24 19:23:27 +08:00
yihao.dai
45aa9779e0
Retrieve segments concurrently (#24245)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-19 18:19:24 +08:00
yihao.dai
1a3dca9b5e
Fix dynamic partitions loading (#24112)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-18 09:17:23 +08:00
MrPresent-Han
b74770b636
support try-best query reducer methods(#24075) (#24103)
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-05-17 16:23:23 +08:00
congqixia
73a181d226
Fix get vector it timeout and improve some string const usage (#24141)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-05-16 17:41:22 +08:00
yah01
400364483d
Improve predict the resource usage (#23968)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-05-11 15:33:24 +08:00
yihao.dai
7e0d1492c7
Load delete from channel checkpoint (#23961)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-09 19:10:41 +08:00
congqixia
884f7904a1
Re-calculate segment entry num when binlog field is zero (#23883)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-05-06 14:24:39 +08:00
yihao.dai
02734d9eb0
Fix ignoring error when loading segment (#23830)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-04 15:10:39 +08:00
Gao
e467e0d451
Seperate reading build and load params for diskann (#23758)
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2023-04-27 20:30:35 +08:00
foxspy
6f4ed517de
add growing segment index (#23615)
Signed-off-by: xianliang <xianliang.li@zilliz.com>
2023-04-26 10:14:41 +08:00
yihao.dai
092d743917
Add support for getting vectors by ids (#23450)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-04-23 09:00:32 +08:00
yah01
546080dcdd
Support to retrieve json (#23563)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-04-21 11:46:32 +08:00
Enwei Jiao
967a97b9bd
Support json & array types (#23408)
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
2023-04-20 11:32:31 +08:00
congqixia
4fe363c4b2
Add KNN cgo pool (#23526)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-04-19 23:56:31 +08:00
yah01
dc0eb433ae
Add IoPoolSize config (#23360)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-04-13 14:50:28 +08:00
Jiquan Long
ad7f3d4c3e
Support Count(*) on querynodev2 (#23321)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-04-11 15:40:31 +08:00
congqixia
0527d2706c
Fix offline set always nil (#23340)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-04-11 14:42:30 +08:00
jaime
c9d0c157ec
Move some modules from internal to public package (#22572)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-04-06 19:14:32 +08:00
congqixia
424570b2b8
Remove annoy related unit tests (#23229)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-04-05 08:14:28 +08:00
yihao.dai
9cecdc4f4e
Fix querynode panic (#23181)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-04-03 14:18:24 +08:00
Enwei Jiao
50dab2d394
Add tracing for retrieve (#23033)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-03-29 18:10:02 +08:00
yah01
826c1fb850
Support mmap on QueryNodeV2 (#23073)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-03-28 21:30:05 +08:00
yah01
081572d31c
Refactor QueryNode (#21625)
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: aoiasd <zhicheng.yue@zilliz.com>
2023-03-27 00:42:00 +08:00