congqixia
cedb33ceec
enhance: Improve datacoord segment filtering with collection ( #32831 )
...
See also #32165
This PR modify the `SelectSegments` interface to utilizing collection id
information when selecting segment with provided collection
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-08 21:37:29 +08:00
Jiquan Long
035a508722
fix: make sure inverted index has only one segment ( #32858 )
...
issue: #32717
---------
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-05-08 21:25:30 +08:00
XuanYang-cn
0b0d279515
fix: [skip-e2e] Fix AvgAssignByCount unstable ( #32850 )
...
See also: #32849
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-05-08 19:09:35 +08:00
Bingyi Sun
17a79f4ca9
enhance: The LRU cache evicts items and retries loading if the disk limit is reached. ( #32819 )
...
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-05-08 14:45:30 +08:00
yihao.dai
a8db16ad4b
enhance: Change ddl log to info ( #32838 )
...
/kind improvement
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-05-08 14:17:50 +08:00
yiwangdr
d6e537c91c
fix: allow datanode's server id to be updated ( #31597 )
...
issue: #31516
background: the server id field in data node is redundant. session id
already provides the source of truth.
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
2024-05-08 14:03:29 +08:00
Chun Han
01c2684355
enhance: [skip e2e] disable unstable ut temporarily ( #32836 )
...
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2024-05-08 12:17:29 +08:00
SimFG
8963615b33
enhance: add the cost response for the rest api ( #32620 )
...
issue: https://github.com/milvus-io/milvus/issues/30436
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-08 11:53:29 +08:00
wei liu
fad8f0afa5
enhance: enable stopping balance after balance has been suspended ( #32812 )
...
issue: #32811
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-05-08 10:15:29 +08:00
wei liu
ba02d54a30
enhance: update shard leader cache when leader location changed ( #32470 )
...
issue: #32466
this PR enhance that when shard location changed, update proxy's shard
leader cache. in case of query node failover case, proxy can find
replica recover
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-05-08 10:05:29 +08:00
wei liu
5038036ece
enhance: Reuse hash locations during access bloom fitler ( #32642 )
...
issue: #32530
when try to match segment bloom filter with pk, we can reuse the hash
locations. This PR maintain the max hash Func, and compute hash location
once for all segment, reuse hash location can speed up bf access
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-05-07 06:13:47 -07:00
Jiquan Long
9837ad6a8d
enhance: remove deprecated api ( #32808 )
...
issue: #32728
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-05-07 20:03:30 +08:00
congqixia
40728ce83d
enhance: Add metautil.Channel to convert string compare to int ( #32749 )
...
See also #32748
This PR:
- Add `metautil.Channel` utiltiy which convert virtual name to physical
channel name, collectionID and shard idx
- Add channel mapper interface & implementation to convert limited
physical channel name into int index
- Apply `metautil.Channel` filter in querynode segment manager logic
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-07 19:13:35 +08:00
XuanYang-cn
6843d6d376
fix: Compaction trigger choose 2 same segments ( #32800 )
...
DataNode would stuck at compactor try to lock the
same segmentID
See also: #32765
---------
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-05-07 19:01:31 +08:00
Cai Yudong
bcdbd1966e
feat: Support sparse float vector bulk insert for binlog/json/parquet ( #32649 )
...
Issue: #22837
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
2024-05-07 18:43:30 +08:00
yihao.dai
53874ce245
fix: Fix cannot specify partition name in binlog import ( #32730 )
...
issue: https://github.com/milvus-io/milvus/issues/32807
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-05-07 17:19:30 +08:00
yihao.dai
9db3aa18bc
enhance: Remove deprecated EnableIndex ( #32704 )
...
/kind improvement
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-05-07 17:11:30 +08:00
chyezh
641f702f64
fix: add request resource timeout for lazy load, refactor context usage in cache ( #32709 )
...
issue: #32663
- Use new param to control request resource timeout for lazy load.
- Remove the timeout parameter of `Do`, remove `DoWait`. use `context`
to control the timeout.
- Use `VersionedNotifier` to avoid notify event lost and broadcast,
remove the redundant goroutine in cache.
related dev pr: #32684
Signed-off-by: chyezh <chyezh@outlook.com>
2024-05-07 16:33:30 +08:00
yiwangdr
b1eacb2ae8
feat: datacoord/node watch based on rpc ( #32036 )
...
issue: https://github.com/milvus-io/milvus/issues/25309
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
2024-05-07 15:49:30 +08:00
congqixia
efa0c73c62
fix: Unify querynode metrics cleanup in collection release ( #32805 )
...
Related to #32803
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-07 15:41:29 +08:00
aoiasd
31dca3249e
enhance: add type info for payload writer error message and add log when querynode find new collection ( #32522 )
...
relate: https://github.com/milvus-io/milvus/issues/32668
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-05-07 14:45:29 +08:00
Jiquan Long
1f58cda957
enhance: add more trace for search & query ( #32734 )
...
issue: https://github.com/milvus-io/milvus/issues/32728
---------
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-05-07 13:03:29 +08:00
yihao.dai
4de063ae14
fix: Make the dynamic column optional in parquet import ( #32738 )
...
issue: https://github.com/milvus-io/milvus/issues/32729
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-05-07 11:21:29 +08:00
SimFG
0ea08b008a
enhance: add the config to control the way when fail to init plugin ( #32680 )
...
issue: #32679
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-07 11:01:31 +08:00
SimFG
7da1ca9efb
enhance: check the partition num when creating collection with partition key ( #32670 )
...
issue: #30577
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-07 10:43:29 +08:00
yihao.dai
cf4db3ff4e
enhance: Fix compilation error ( #32797 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-05-06 19:31:49 -07:00
SimFG
48fb8310ba
fix: skip to check auto id when inserting data ( #32775 )
...
/kind improvement
issue: https://github.com/milvus-io/milvus/issues/32591
related pr: #32592
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-07 10:17:30 +08:00
congqixia
02bf47f295
fix: Validate num of rows for insert field data with schema ( #32770 )
...
See also #32769
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-07 10:07:29 +08:00
congqixia
7102403a6b
fix: Add Wrapper and Keepalive for CTraceContext ids ( #32746 )
...
See also #32742
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-07 10:05:35 +08:00
congqixia
53b5f1be17
enhance: Remove legacy L0 segment if watch failed ( #32725 )
...
Like growing segments, legacy l0 segments shall be removed if watch dml
channel execution fails as well.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-07 10:03:42 +08:00
Jiquan Long
600db9d99e
enhance: check field data type of input ( #32777 )
...
issue: #32769
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-05-06 21:15:29 +08:00
Xiaofan
1e47d7afc4
improve: change some frequent log to debug ( #32779 )
...
remove the frequent log "filter insert messages"
Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
2024-05-06 20:51:29 +08:00
Bingyi Sun
fecd9c21ba
feat: LRU cache implementation ( #32567 )
...
issue: https://github.com/milvus-io/milvus/issues/32783
This pr is the implementation of lru cache on branch lru-dev.
Signed-off-by: sunby <sunbingyi1992@gmail.com>
Co-authored-by: chyezh <chyezh@outlook.com>
Co-authored-by: MrPresent-Han <chun.han@zilliz.com>
Co-authored-by: Ted Xu <ted.xu@zilliz.com>
Co-authored-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: wayblink <anyang.wang@zilliz.com>
2024-05-06 20:29:30 +08:00
congqixia
6af5275066
fix: Refine index param check for Create/Alter Index ( #32712 )
...
See also #32711
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-06 15:43:29 +08:00
chyezh
b904c8d377
enhance: resource group unittest refactory ( #32739 )
...
issue: #30647
Signed-off-by: chyezh <chyezh@outlook.com>
2024-05-06 10:17:34 +08:00
chyezh
77f4f0c5a8
fix: gc only do once ( #32722 )
...
issue: #29655
produced by pr: #31740
Signed-off-by: chyezh <chyezh@outlook.com>
2024-04-30 14:19:27 +08:00
wayblink
42d0412e93
enhance: Add channelCPs in FlushResponse ( #32044 )
...
#32609
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-04-30 09:45:27 +08:00
Chun Han
ac82cef04d
enhance: disable reload partstats by config ( #32702 )
...
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2024-04-29 19:11:26 +08:00
SimFG
0359139299
enhance: return the auth collection list when using the show collection api ( #32576 )
...
issue: #32550
/kind improvement
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-04-29 19:09:34 +08:00
liliu-z
a43baa0323
enhance: Upgrade knowhere to 2.3.1 ( #32699 )
...
/kind improvement
Signed-off-by: Li Liu <li.liu@zilliz.com>
2024-04-29 17:21:26 +08:00
PowderLi
6289f3a9eb
fix: build milvus in rockylinux8 ( #32619 )
...
issue: #32299
1. xz utils recovers
2. forget to install ninja
Signed-off-by: PowderLi <min.li@zilliz.com>
2024-04-29 14:53:26 +08:00
Buqian Zheng
858599d831
enhance: sparse float vector to support brute force iterator and range search ( #32635 )
...
issue: #29419
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-04-29 14:35:26 +08:00
wei liu
d900e68440
fix: fix GetShardLeaders return empty node list ( #32685 )
...
issue: #32449
to avoid GetShardLeaders return empty node list, this PR add node list
check in both client side and server side.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-04-29 14:19:26 +08:00
chyezh
ef4c875d4c
fix: resource group ut may failure ( #32688 )
...
issue: https://github.com/milvus-io/milvus/issues/30647
Signed-off-by: chyezh <chyezh@outlook.com>
2024-04-29 14:17:26 +08:00
congqixia
2c1e8f4774
enhance: Use struct{} for sync task future result ( #32673 )
...
Related to #27675
Use `struct{}` instead `error` for sync task future result type to
reduce result size and preventing logci error.
Also change some unused parameter to `_` to suppress lint warning
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-04-29 10:59:26 +08:00
SimFG
09cd56d44f
enhance: add the skip auto id and partition key check config ( #32592 )
...
/kind improvement
issue: #32591
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-04-29 10:29:26 +08:00
wei liu
c0555d4b45
fix: Remove read only node from replica immedaitely after node down ( #32666 )
...
issue: #32665
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-04-28 20:25:25 +08:00
presburger
cb1dbf20c7
enhance:remove openblas from conan ( #32002 )
...
issue: #30683
Signed-off-by: yusheng.ma <yusheng.ma@zilliz.com>
2024-04-28 20:13:25 +08:00
yihao.dai
1594122c0a
enhance: Make the dynamic field file optional during numpy import ( #32596 )
...
1. Make the dynamic field file optional during numpy import
2. Add integration importing test with dynamic
3. Disallow file of pk when autoID=true during numpy import
issue: https://github.com/milvus-io/milvus/issues/32542
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-04-28 19:39:25 +08:00
wei liu
07720f1a95
enhance: expose alter database api in proxy ( #32639 )
...
issue: #30040
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-04-28 18:15:31 +08:00
