Use an individual buffer size parameter for imports and set buffer size
to 64MB.
issue: https://github.com/milvus-io/milvus/issues/28521
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
See #31944
There is deadlock in concurrent invocation on `cache.DoWait()`. Suppose
2 callers are calling `DoWait` concurrently, the notification for cache
wait queue may be limited to just 1 caller's occupation instead of them
combined.
To fix this issue, this patch is trying to notify all waiters in queue.
Signed-off-by: Ted Xu <ted.xu@zilliz.com>
Should not return `err, nil` when using conc.Future, as the error will
be lost/ignored when using `AwaitAll` to wait for the future.
issue: https://github.com/milvus-io/milvus/issues/31788
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
Issue: #31752
This PR improves the performance for bitset utilities (introduced in PR
#30454), including varchar filtering
Signed-off-by: Alexandr Guzhva <alexanderguzhva@gmail.com>
the purpose of this pr is to build arm-based and x86-based
milvusdb/milvus-env for milvus gpu. milvusdb/milvus-env would be based
on ubuntu22.04
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
issue: #30647
- ReplicaManager manage read only node now, and always do persistent of
node distribution of replica.
- All segment/channel checker using ReplicaManager to get read-only node
or read-write node, but not ResourceManager.
- ReplicaManager promise that only apply unique querynode to one replica
in same collection now (replicas in same collection never hold same
querynode at same time).
- ReplicaManager promise that fairly node count assignment policy if
multi replicas of collection is assigned to one resource group.
- Move some parameters check into ReplicaManager to avoid data race.
- Allow transfer replica to resource group that already load replica of
same collection
- Allow transfer node between resource groups that load replica of same
collection
---------
Signed-off-by: chyezh <chyezh@outlook.com>
See also #30552
FlushTS policy was orignally designed to flushed/L0 segments only, but
in some edge case, new growing segment buffer would by-pass flush
request and hold a buffer before flush ts, which caused flush timeout
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
issue: #31091
This PR add GetByFilter interface in channel dist manager, instead of
all kind of get func
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Related to #31791
This segment meta is implemented in COW pattern. All modification on
segment info shall happen on the copied version of it.
This PR clones the child segment info for `GetSegmentInfo` in case data
race problem.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Feature Introduced:
1. Ensure ImportV2 waits for the index to be built
Enhancements Introduced:
1. Utilization of local time for timeout ts instead of allocating ts
from rootcoord.
3. Enhanced input file length check for binlog import.
4. Removal of duplicated manager in datanode.
5. Renaming of executor to scheduler in datanode.
6. Utilization of a thread pool in the scheduler in datanode.
issue: https://github.com/milvus-io/milvus/issues/28521
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
