wei liu
04a8ec69f6
fix: Segment on stopping query node can't be release successfully ( #32929 )
...
issue: #32901
Cause release segment request need be send to delegator, but it need
replica to info find segment's delegator. but the stopping query node
will be marked as read only in replica, then `replica.Contains()` just
return true for rwNode in replica. then it can't get replica info by
stopping query node and release segment will be blocked.
This PR make `replica.Contains()` return true for both roNode and
rwNode.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-05-10 14:33:30 +08:00
aoiasd
cbbfb5b6d6
enhance: Add update milvus yaml command to makefile ( #32857 )
...
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-05-10 14:31:37 +08:00
Jiquan Long
0783582e2e
fix: temporarily disable two-phase retrieval when lru is enabled ( #32927 )
...
issue: #31822
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-05-10 14:19:45 +08:00
zhuwenxing
e776c807c4
test: add more json datatype in restful test cases ( #32930 )
...
add more json datatype in restful test cases
related to https://github.com/milvus-io/milvus/issues/31051
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-05-10 14:11:30 +08:00
Bingyi Sun
5cbf081111
fix: fix index resource estimation ( #32842 )
...
issue: #32820
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-05-10 11:53:30 +08:00
Bingyi Sun
b7ef8da360
fix: set channel checkpoint to delta position ( #32878 )
...
issue: https://github.com/milvus-io/milvus/issues/32853
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-05-10 11:51:30 +08:00
zhuwenxing
9a269f1489
test: add import checker to chaos test ( #32908 )
...
add import checker to chaos test
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-05-10 11:43:30 +08:00
Buqian Zheng
bb7765cbd6
fix: fix Indexing.Iterator ut: build index with all data at once ( #32844 )
...
issue: #32843
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-05-10 11:31:30 +08:00
Bingyi Sun
1e1fba0588
enhance: add lazy load retry configurations ( #32848 )
...
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-05-10 10:55:30 +08:00
wei liu
c35797c399
enhance: expose DescribeDatabase api in proxy ( #32732 )
...
issue: #32707
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-05-09 22:51:30 +08:00
congqixia
efa58ae423
enhance: Utilize coll2replica mapping when getting rg by collection ( #32892 )
...
See also #32165
In old `GetResourceGroupByCollection` implementation, it iterates all
replicas to match collection id, which is slow and CPU time consuming.
This PR make it utilize the coll2Replicas mapping by calling
`GetByCollection` and mapping replicas into resource group.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-09 19:37:30 +08:00
congqixia
77fa615772
fix: Make SyncManager callback func ignore nil error ( #32891 )
...
introduced by #32865
sync manager callback handler panicked when error is nil
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-09 18:03:31 +08:00
aoiasd
54a51b1236
enhance: Support dynamic config for opentelemetry trace ( #32169 )
...
relate: https://github.com/milvus-io/milvus/issues/31940
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-05-09 17:43:30 +08:00
shaoting-huang
ca0cf9b3b1
doc: fix typos in design docs ( #32885 )
...
Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-05-09 15:43:30 +08:00
congqixia
acb0417a9f
enhance: Avoid iteration over channel results when update leaderview ( #32887 )
...
See also #32165
Cache channel name to channel info to avoid iteration over channel
results when updating leader view version.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-09 15:41:30 +08:00
XuanYang-cn
bc015df962
fix: [skip-e2e] Fix unstable policy_test ( #32883 )
...
See also: #32849
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-05-09 15:39:40 +08:00
Cai Yudong
8bb58d0460
enhance: optimize vector offsets handling for parquet ( #32822 )
...
Issue: #22837
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
2024-05-09 14:43:30 +08:00
congqixia
a06f601c6e
fix: Make syncmgr lock key before returning future ( #32865 )
...
See also #32860
SyncMgr did not ensure task key is locked before `SyncData` returning
which may cause concurrent problem during sync wich multiple policies.
This PR change sync mgr implementation to make sure the key is locked
before returning task result `*conc.Future`
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-09 10:09:30 +08:00
sre-ci-robot
3d78b90fe7
[automated] Update Knowhere Commit ( #32872 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-05-09 01:57:30 +08:00
congqixia
cedb33ceec
enhance: Improve datacoord segment filtering with collection ( #32831 )
...
See also #32165
This PR modify the `SelectSegments` interface to utilizing collection id
information when selecting segment with provided collection
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-08 21:37:29 +08:00
Jiquan Long
035a508722
fix: make sure inverted index has only one segment ( #32858 )
...
issue: #32717
---------
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-05-08 21:25:30 +08:00
sre-ci-robot
5037497929
Update all contributors
...
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
2024-05-08 12:00:53 +00:00
XuanYang-cn
0b0d279515
fix: [skip-e2e] Fix AvgAssignByCount unstable ( #32850 )
...
See also: #32849
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-05-08 19:09:35 +08:00
Bingyi Sun
17a79f4ca9
enhance: The LRU cache evicts items and retries loading if the disk limit is reached. ( #32819 )
...
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-05-08 14:45:30 +08:00
yihao.dai
a8db16ad4b
enhance: Change ddl log to info ( #32838 )
...
/kind improvement
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-05-08 14:17:50 +08:00
yiwangdr
d6e537c91c
fix: allow datanode's server id to be updated ( #31597 )
...
issue: #31516
background: the server id field in data node is redundant. session id
already provides the source of truth.
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
2024-05-08 14:03:29 +08:00
Chun Han
01c2684355
enhance: [skip e2e] disable unstable ut temporarily ( #32836 )
...
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2024-05-08 12:17:29 +08:00
SimFG
8963615b33
enhance: add the cost response for the rest api ( #32620 )
...
issue: https://github.com/milvus-io/milvus/issues/30436
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-08 11:53:29 +08:00
wei liu
fad8f0afa5
enhance: enable stopping balance after balance has been suspended ( #32812 )
...
issue: #32811
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-05-08 10:15:29 +08:00
wei liu
ba02d54a30
enhance: update shard leader cache when leader location changed ( #32470 )
...
issue: #32466
this PR enhance that when shard location changed, update proxy's shard
leader cache. in case of query node failover case, proxy can find
replica recover
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-05-08 10:05:29 +08:00
wei liu
5038036ece
enhance: Reuse hash locations during access bloom fitler ( #32642 )
...
issue: #32530
when try to match segment bloom filter with pk, we can reuse the hash
locations. This PR maintain the max hash Func, and compute hash location
once for all segment, reuse hash location can speed up bf access
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-05-07 06:13:47 -07:00
Jiquan Long
9837ad6a8d
enhance: remove deprecated api ( #32808 )
...
issue: #32728
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-05-07 20:03:30 +08:00
congqixia
40728ce83d
enhance: Add metautil.Channel to convert string compare to int ( #32749 )
...
See also #32748
This PR:
- Add `metautil.Channel` utiltiy which convert virtual name to physical
channel name, collectionID and shard idx
- Add channel mapper interface & implementation to convert limited
physical channel name into int index
- Apply `metautil.Channel` filter in querynode segment manager logic
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-07 19:13:35 +08:00
XuanYang-cn
6843d6d376
fix: Compaction trigger choose 2 same segments ( #32800 )
...
DataNode would stuck at compactor try to lock the
same segmentID
See also: #32765
---------
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-05-07 19:01:31 +08:00
Cai Yudong
bcdbd1966e
feat: Support sparse float vector bulk insert for binlog/json/parquet ( #32649 )
...
Issue: #22837
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
2024-05-07 18:43:30 +08:00
yihao.dai
53874ce245
fix: Fix cannot specify partition name in binlog import ( #32730 )
...
issue: https://github.com/milvus-io/milvus/issues/32807
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-05-07 17:19:30 +08:00
yihao.dai
9db3aa18bc
enhance: Remove deprecated EnableIndex ( #32704 )
...
/kind improvement
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-05-07 17:11:30 +08:00
chyezh
641f702f64
fix: add request resource timeout for lazy load, refactor context usage in cache ( #32709 )
...
issue: #32663
- Use new param to control request resource timeout for lazy load.
- Remove the timeout parameter of `Do`, remove `DoWait`. use `context`
to control the timeout.
- Use `VersionedNotifier` to avoid notify event lost and broadcast,
remove the redundant goroutine in cache.
related dev pr: #32684
Signed-off-by: chyezh <chyezh@outlook.com>
2024-05-07 16:33:30 +08:00
yiwangdr
b1eacb2ae8
feat: datacoord/node watch based on rpc ( #32036 )
...
issue: https://github.com/milvus-io/milvus/issues/25309
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
2024-05-07 15:49:30 +08:00
congqixia
efa0c73c62
fix: Unify querynode metrics cleanup in collection release ( #32805 )
...
Related to #32803
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-07 15:41:29 +08:00
aoiasd
31dca3249e
enhance: add type info for payload writer error message and add log when querynode find new collection ( #32522 )
...
relate: https://github.com/milvus-io/milvus/issues/32668
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-05-07 14:45:29 +08:00
zhuwenxing
29f3cb692b
test:fix retry for network error ( #32804 )
...
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-05-07 14:01:29 +08:00
zhuwenxing
e1eb03ad0d
test: add testcase for import without meta when enabled dynamic ( #32706 )
...
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-05-07 13:59:36 +08:00
Jiquan Long
1f58cda957
enhance: add more trace for search & query ( #32734 )
...
issue: https://github.com/milvus-io/milvus/issues/32728
---------
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-05-07 13:03:29 +08:00
yihao.dai
4de063ae14
fix: Make the dynamic column optional in parquet import ( #32738 )
...
issue: https://github.com/milvus-io/milvus/issues/32729
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-05-07 11:21:29 +08:00
SimFG
0ea08b008a
enhance: add the config to control the way when fail to init plugin ( #32680 )
...
issue: #32679
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-07 11:01:31 +08:00
SimFG
7da1ca9efb
enhance: check the partition num when creating collection with partition key ( #32670 )
...
issue: #30577
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-07 10:43:29 +08:00
yihao.dai
cf4db3ff4e
enhance: Fix compilation error ( #32797 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-05-06 19:31:49 -07:00
SimFG
48fb8310ba
fix: skip to check auto id when inserting data ( #32775 )
...
/kind improvement
issue: https://github.com/milvus-io/milvus/issues/32591
related pr: #32592
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-07 10:17:30 +08:00
cai.zhang
a5d1135512
fix: Fix scalar auto index config incorrect version ( #32795 )
...
issue: #29309
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-05-07 10:13:00 +08:00
