nico
f756f01445
test: update test cases ( #35640 )
...
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-08-26 12:00:58 +08:00
sre-ci-robot
6ddfd02f01
[automated] Update Knowhere Commit ( #35688 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-08-26 01:04:57 +08:00
Zhen Ye
4b7357ee80
fix: fix typo in milvus spec ( #35684 )
...
issue: #35611
introduced by #35610
Signed-off-by: chyezh <chyezh@outlook.com>
2024-08-25 16:28:57 +08:00
Patrick Weizhi Xu
4236c3bafb
enhance: enable MV on binary vectors ( #35668 )
...
Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
2024-08-25 16:27:09 +08:00
Ashwin Krishna Kumar
cfa434086a
doc: improve VSCode setup in dev docs ( #35677 )
...
The developer documentation contains instructions for setting up a dev
environment in VSCode. This PR improves on the below two issues:
1. With the current settings, you cannot run and debug Go unit tests
through the VSCode UI. The `tags` and `ldflags` parameters need to be
modified slightly to make this work.
<img width="436" alt="Screenshot 2024-08-23 at 4 37 37 PM"
src="https://github.com/user-attachments/assets/78bae0b9-5abe-4e48-90cf-e844ad9ef292 ">
2. Users need to replace all occurrences of
`/Users/zilliz/workspace/milvus` with the actual folder path where
they've cloned Milvus. This is not necessary when using the
`${workspaceFolder}` variable, which VSCode will substitute
automatically.
Signed-off-by: Ashwin Krishna Kumar <ashwin.kumar6@ibm.com>
2024-08-25 16:25:04 +08:00
smellthemoon
098c10922b
enhance: add some check when create collection ( #35629 )
...
not support nullable==true in vector type. check it when create
collection.
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-08-25 16:23:05 +08:00
Xiaofan
0dc5e89007
enhance: reduce the log level of frequent log ( #35652 )
...
fix #35651
Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
2024-08-25 16:20:57 +08:00
cai.zhang
615a653988
fix: Fix offset out of range for creating Trie index ( #35553 )
...
issue: #35550
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-08-25 15:50:57 +08:00
yihao.dai
f2b83d316b
enhance: Support memory mode chunk cache ( #35347 )
...
Chunk cache supports loading raw vectors into memory.
issue: https://github.com/milvus-io/milvus/issues/35273
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-08-25 15:42:58 +08:00
Xiaofan
50fcfe8ef1
enhance: add nan and inf check ( #35683 )
...
fix #35594
add float check on files
Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
2024-08-25 15:22:57 +08:00
zhagnlu
42f7800b5b
enhance: add bitmap offset cache to speed up retrieve raw data ( #35498 )
...
#35458
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-08-24 01:40:58 +08:00
Zhen Ye
75da36d1aa
enhance: enable asan for milvus ( #35627 )
...
issue: #35626
Signed-off-by: chyezh <chyezh@outlook.com>
2024-08-23 21:06:58 +08:00
SimFG
9dc1311f38
enhance: add the param to control whether to include the current msg ( #35656 )
...
/kind improvement
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-08-23 15:56:57 +08:00
congqixia
59387f0b3a
fix: Use correct release func when datanode unwatch a channel ( #35655 )
...
See also #35654
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-23 14:09:03 +08:00
Gao
e8e3544a11
enhance: add hit segment num metrics for queryHook ( #35577 )
...
issue: #35576
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2024-08-23 12:49:02 +08:00
Zhen Ye
a773836b89
enhance: optimize milvus core building ( #35610 )
...
issue: #35549,#35611,#35633
- remove milvus_segcore milvus_indexbuilder..., add libmilvus_core
- core building only link once
- move opendal compilation into cmake
- fix odr
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-08-23 12:35:02 +08:00
congqixia
ab532ae199
enhance: Add back BF lazy load logic for datanode watch channel ( #35646 )
...
Add back lazy loading statslog when watch dml channel on datanode.
Related to #22994 #27675
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-22 19:42:57 +08:00
wei liu
5c245d51c4
enhance: Refresh proxy cache after restore rbac meta ( #35635 )
...
issue: #35443
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-22 19:09:01 +08:00
congqixia
9fff07936a
fix: Re-read value after once initialization ( #35642 )
...
Related to #35641
See also #35271
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-22 17:24:58 +08:00
congqixia
582d2eec79
enhance: Move datanode/indexnode manager to session pkg ( #35634 )
...
Related to #28861
Move session manager, worker manager to session package. Also renaming
each manager to corresponding node name(datanode, indexnode).
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-22 16:02:56 +08:00
zhagnlu
3107701fe8
enhance: optimize retrieve on dynamic field ( #35580 )
...
#35514
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
Co-authored-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2024-08-22 14:24:56 +08:00
ThreadDao
570a8879e7
test: update gosdk case for valid empty sparse vector ( #35621 )
...
/kind improvement
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2024-08-22 10:54:56 +08:00
congqixia
c992a61a23
enhance: Separate allocator pkg in datacoord ( #35622 )
...
Related to #28861
Move allocator interface and implementation into separate package. Also
update some unittest logic.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-22 10:06:56 +08:00
presburger
024eccbde0
enhance: add pkg-config for knowhere ( #35433 )
...
Signed-off-by: yusheng.ma <yusheng.ma@zilliz.com>
2024-08-22 09:56:56 +08:00
sre-ci-robot
6fa6e2f978
Update all contributors
...
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
2024-08-21 12:00:50 +00:00
yellow-shine
2b6e6f4040
feat: [skip e2e] effective way to use cache ( #35598 )
...
issue: https://github.com/milvus-io/milvus/issues/34876
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-08-21 18:46:55 +08:00
OxalisCu
ed4eaffc9d
enhance: add csv support for bulkinsert ( #34938 )
...
See this issue for details: #34937
---------
Signed-off-by: OxalisCu <2127298698@qq.com>
2024-08-21 17:47:01 +08:00
smellthemoon
ba6db117e3
enhance: add some integration test in null ( #35599 )
...
#31728
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-08-21 17:44:56 +08:00
wei liu
cf242f9e09
fix: fix dynamic update config doesn't works for some param ( #35572 )
...
issue: #35570
milvus support config cache to spped up config access, but only evict
param's cache when param has been updated. but milvus's param may rely
on other param's value, let's say ParamsA relys on paramsB, when paramsB
updated, it will evict paramB's cache, but the paramA's cache still keep
the old value.
This PR evict all config cache to solve the above issue, cause dynamic
update config won't be much frequetly.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-21 11:02:56 +08:00
yanliang567
7ac339ac64
test: Update init collection method ( #35596 )
...
Related issue: #32653
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-08-21 09:22:56 +08:00
SimFG
731d45abbe
enhance: provide more general configuration to control mmap behavior ( #35359 )
...
- issue: #35273
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-08-21 00:22:54 +08:00
congqixia
99f8c64c31
enhance: [GoSDK] support Bitmap Index ( #35585 )
...
See also #32900
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-20 19:44:55 +08:00
Ted Xu
41646c8439
feat: integrate new deltalog format ( #35522 )
...
See #34123
---------
Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-08-20 19:06:56 +08:00
aoiasd
bc2bbdc82f
enhance: Forbid access log print error message with line break ( #34491 )
...
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-08-20 17:36:55 +08:00
smellthemoon
80a7c78f28
enhance: import supports null in parquet and json formats ( #35558 )
...
#31728
---------
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-08-20 16:50:55 +08:00
congqixia
2fbc628994
feat: Support field partial load collection ( #35416 )
...
Related to #35415
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-20 16:49:02 +08:00
Chun Han
031ee6f155
enhance: support httpv1/v2 throttle and add it for httpV2( #35350 ) ( #35470 )
...
related: #35350
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-08-20 16:16:55 +08:00
congqixia
3491608256
fix: Match int8_t and int16_t in Array::get_data ( #35579 )
...
Related to #35578
Previously int16/int8 bitmap index may read int32 array as int16, which
may cause build index with half of the data(if array is full) and half
zeros. This causes BITMAP index lost information.
This PR matches int8_t & int16_t while `get_data` when building index.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-20 16:10:56 +08:00
yanliang567
249dc4d9eb
test: Add tests for upsert with auto id ( #35556 )
...
Related issue: #34668
---------
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-08-20 14:20:56 +08:00
wei liu
e09dc3be58
enhance: Mark query node as read only after suspend ( #35492 )
...
issue: #34985 #35493
after querynode has been suspended, it's not allow to load
segment/channel on it, which means the node is read only. to be
compatible with resource group design, after query node has been
suspend, we remove it from it's original resource group, make it a read
only query node in replica. then two things will happens:
1. it's original resource group will be lacking of query nodes, query
coord will assign new node to it.
2. querycoord will try to move out all segments/channels after querynode
has been suspended
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-20 14:02:54 +08:00
nish112022
3948bd4e79
fix: Added check for validating varchar,array max length ( #35499 )
...
issue : https://github.com/milvus-io/milvus/issues/34150
This is for numpy,parquet,json readers.
---------
Signed-off-by: Nischay Yadav <nischay.yadav@ibm.com>
2024-08-20 11:42:55 +08:00
wei liu
22ced010cd
enhance: make configure load param feature be compatible with old sdk ( #35520 )
...
issue: #31570 #35521
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-20 10:30:55 +08:00
yellow-shine
6e29d713b3
enhance: [skip e2e]dockerfile for milvus-sdk-go test ( #35426 )
...
https://github.com/milvus-io/milvus/issues/35567
---------
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-08-20 09:48:55 +08:00
Zhen Ye
4d69898cb2
enhance: support single pchannel level transaction ( #35289 )
...
issue: #33285
- support transaction on single wal.
- last confirmed message id can still be used when enable transaction.
- add fence operation for segment allocation interceptor.
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-08-19 21:22:56 +08:00
wei liu
853d2f33e0
enhance: Trim space during parsing string list from string ( #35546 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-19 20:48:55 +08:00
yihao.dai
227ecd3901
enhance: Remove the check for channel cp lag when generating compaction plan ( #35383 )
...
issue: https://github.com/milvus-io/milvus/issues/35382
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-08-19 19:40:55 +08:00
CharlesFeng
795339ecbc
enhance: using timer instead of ticker in client/index.go ( #35552 )
...
https://github.com/milvus-io/milvus/issues/35551
Signed-off-by: fengjun2016 <jornfeng@gmail.com>
2024-08-19 18:56:54 +08:00
congqixia
b4743b4ca8
enhance: [GoSDK] Sync latest milvus proto and mockery ( #35511 )
...
Related to #35443
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-19 16:22:55 +08:00
CharlesFeng
b96db798b9
enhance: using timer instead of tickers for need to be executed successfully once ( #35534 )
...
https://github.com/milvus-io/milvus/issues/35533
Signed-off-by: fengjun2016 <jornfeng@gmail.com>
2024-08-19 16:16:55 +08:00
Chun Han
337e065902
fix: querynode hang when failing to allocate disk space for mmap( #35184 ) ( #35187 )
...
related: #35184
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-08-19 15:30:55 +08:00
