Commit Graph

814 Commits

Author SHA1 Message Date
SimFG
c267ad612d
Support Role-Based Access Control II (#18528)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-08-05 16:28:35 +08:00
SimFG
ff0200210a
Support Role-Based Access Control (#18425)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-08-04 11:04:34 +08:00
Cai Yudong
7cd37fc6dd
Merge FloatSearch() and BinarySearch() into SearchOnGrowing() (#18498)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-08-03 11:56:36 +08:00
Enwei Jiao
283f5731d2
config from etcd (#18421)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-08-01 10:04:33 +08:00
zhenshan.cao
a287a2b3fd
Return empty result in advance if all data filtered out (#18329) (#18438)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-07-28 12:36:30 +08:00
bigsheeper
cef8b1e7cc
Enable jemalloc (#18349)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-07-20 22:22:31 +08:00
Ten Thousand Leaves
0ebe407221
Make DataNode not panic if stale segment is not found during flush (#18322)
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-07-20 15:06:30 +08:00
Jeng.Gwan
db71c74cc7
Fix leak of memory in ParseMallocInfo while assert false (#18311)
Signed-off-by: xaxys <zheng.guan@zilliz.com>
2022-07-19 14:26:33 +08:00
Jeng.Gwan
638f6c36e9
Support to get real row count of segment (#18115)
Signed-off-by: xaxys <zheng.guan@zilliz.com>
2022-07-18 09:58:28 +08:00
bigsheeper
73aef14820
Improve check of purge memory and change purge place (#18271)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-07-15 10:22:27 +08:00
xige-16
54d17bc5da
Fix query too slow when insert multi repeated pk data (#18231)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-07-13 10:22:26 +08:00
Jeng.Gwan
a926a9eb66
Fix redundant string copy of PlaceholderGroup (#18236)
Signed-off-by: xaxys <zheng.guan@zilliz.com>
2022-07-12 15:48:25 +08:00
Cai Yudong
015a2f0866
Let growing segment call knowhere brute search API (#18227)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-07-12 11:58:25 +08:00
bigsheeper
22508f36d3
Get search field id from search plan and log if loaded index when search segments (#18183)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-07-08 20:18:22 +08:00
zhagnlu
257da153ce
Fix core dump when nq has no topk result (#17923) (#18051)
Signed-off-by: zhagnlu <lu.zhang@zilliz.com>

Co-authored-by: zhagnlu <lu.zhang@zilliz.com>
2022-07-05 19:48:20 +08:00
Cai Yudong
5edeeb5fcd
Define new SubSearchResult constructor to avoid memory copy for growing (#18000)
segment search

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-07-05 17:14:20 +08:00
xige-16
0702993106
Fix query not empty after delete string (#17953)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-07-04 16:54:20 +08:00
Jiquan Long
6954a5ba3e
Fix search successfully with invalid metric type (#17977)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-07-01 22:28:23 +08:00
bigsheeper
641fb96958
Return malloc_trim result and log if malloc_trim successfully (#17959)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-07-01 10:44:18 +08:00
Cai Yudong
a001412e12
Replace faiss::MetricType with knowhere::MetricType (#17891)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-06-29 14:20:19 +08:00
Letian Jiang
4ae1ca2cac
Fix arrow builder nullptr check in FinishPayloadWriter (#17873)
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-06-28 20:04:17 +08:00
Enwei Jiao
ea901393ef
fix windows ldflags (#17807)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-06-28 19:58:25 +08:00
bigsheeper
b657e58370
Remove temporary variable to prevent memory fragmentation (#17728)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-06-28 16:30:23 +08:00
Enwei Jiao
16c3aedc15
refine complie configuration (#17502)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-06-24 21:12:15 +08:00
Cai Yudong
7385770014
Upgrade to knowhere-v1.1.12 (#17692)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-06-24 10:34:18 +08:00
Jiquan Long
90ca61d948
Fix double-freed global variable in easylogging lib (#17730)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-06-23 19:12:14 +08:00
Cai Yudong
e78269f450
Optimize search related interface in segcore (#17568)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-06-23 18:16:13 +08:00
bigsheeper
1722431699
Checking for existence in map in segcore (#17700)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-06-22 18:16:14 +08:00
Jiquan Long
fdccfb4273
Fix ut heap overflow (#17706)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-06-22 16:56:13 +08:00
bigsheeper
92d06b2e30
Purge memory by the memory state and try to purge after each search (#17565)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-06-17 17:46:10 +08:00
Ten Thousand Leaves
65ab847815
Add segment reference lock for bulk load (#17457)
So we can support DataQueryable and DataIndexed states

issue: #16607
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-06-15 12:20:10 +08:00
Jiquan Long
e751663fdc
Fix err msg when the dimension of query vectors mismatch field (#17553)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-06-15 10:54:09 +08:00
bigsheeper
cdcdfa1ea5
Disable jemalloc and use malloc_trim instead (#17538)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-06-14 11:12:09 +08:00
xige-16
36ad989590
Fix segOffset grater than insert barrier when mark delete (#17444)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-06-10 20:02:08 +08:00
bigsheeper
2d9a52206d
Use jemalloc in QueryNode, DataNode and IndexNode (#17470)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-06-10 11:24:08 +08:00
xige-16
6d078ebe7a
Print more information when loading fields with different lines (#17448)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-06-08 20:16:06 +08:00
bigsheeper
af108b5bda
Add malloc_trim after reduce search result (#17423)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-06-08 00:12:05 +08:00
xige-16
b5c11a216d
Alter varChar type params's name to max_length (#17409)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-06-07 15:58:06 +08:00
bigsheeper
f38637c227
Pass PlaceholderGroup pointer to prevent memory copy in SegCore (#17389)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-06-06 21:34:05 +08:00
bigsheeper
9f786dd752
Remove unnecessary bitset flip in Search and Query (#17387)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-06-06 17:38:05 +08:00
Jiquan Long
e88ffb8a57
Enable marisa trie ut on MacOS (#17316)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-06-02 10:48:03 +08:00
Xiaofan
9579a645c6
Support compile marisa on Macos (#17261)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-05-31 13:28:02 +08:00
SimFG
731870211a
Support role-based access control (only pb) (#16987)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-05-28 00:04:01 +08:00
Letian Jiang
f2a27e0e64
Retry GetShardLeaders until service available or timeout (#17183)
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-05-26 20:28:02 +08:00
xige-16
56778787be
Reverse data from scalar index (#17145)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-05-26 14:58:01 +08:00
xige-16
7d810ac159
Fix query empty when insert same pk after deletion (#17222)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-05-25 22:06:00 +08:00
Ten Thousand Leaves
5e1e7a6896
Fix an issue that causes flush hanging (#17169)
issue: #16628
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-05-25 14:34:00 +08:00
xige-16
a8829554eb
Merge multiple sets of expr's optypes in diff dirs (#17192)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-05-24 21:56:00 +08:00
zhenshan.cao
d05c24a0ff
Delete unused code (#17182)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-05-24 14:49:59 +08:00
zhenshan.cao
08a4dff41d
Add logic for search merging and a simple task scheduler for read tasks (#17022)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Co-authored-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: bigsheeper <yihao.dai@zilliz.com>

Co-authored-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: bigsheeper <yihao.dai@zilliz.com>
2022-05-23 16:41:58 +08:00