test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-11-30 10:59:32 +08:00
Code Issues Actions 11 Packages Projects Releases Wiki Activity
18,647 Commits 94 Branches 105 Tags 184 MiB
bcd6865b29
Commit Graph

428 Commits

Author SHA1 Message Date
PowderLi
bcd6865b29
enhance: add 3 builtin roles (#28961) 
issue: #28960 [milvus-proto
#212](https://github.com/milvus-io/milvus-proto/issues/212)

add new configuration: builtinRoles
user can define roles in config file: `milvus.yaml`

there is an example:
1. db_ro, only have read privileges, include load
2. db_rw, read and write privileges, include create/drop/rename
collection
3. db_admin, not only read and write privileges, but also user
administration

Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-12-18 14:28:41 +08:00
zhagnlu
a602171d06
enhance: Refactor runtime and expr framework (#28166) 
#28165

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2023-12-18 12:04:42 +08:00
wei liu
820ee692fc
enhance: Add config for querycoord auto balance channel (#29231) 
issue: #23726
This PR add control config to querycoord's background auto balance
channel operation

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-12-18 10:00:40 +08:00
congqixia
88b4b8b77c
enhance: make segments SQPool & LoadPool resizable (#29239) 
See also #29223

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-15 20:22:39 +08:00
congqixia
4731c1b0d5
enhance: make SyncManager pool size refreshable (#29224) 
See also #29223

This PR make `conc.Pool` resizable by adding `Resize` method for it. 
Also make newly added datanode `MaxParallelSyncMgrTasks` config
refreshable

---------

Signed-off-by: Congqi.Xia <congqi.xia@zilliz.com>
 2023-12-15 09:58:43 +08:00
congqixia
25a4525297
enhance: Change sync manager parallel config item (#29216) 
Since the sync manager is global in datanode now, the old
`maxParallelSyncTaskNum` does not fit into current implementation
anymore.

This PR add a new param item for sync mgr parallel control and enlarge
default value

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-14 20:46:41 +08:00
congqixia
9407be6eb2
enhance: Add a config item for partition name as regexp feature and disable it by default (#29154) 
See also #29177
Add a config item for partition name as regexp feature and disable it by
default

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-14 10:04:38 +08:00
aoiasd
b5ee563914
fix: accesslog can not print search expression (#28899) 
relate: https://github.com/milvus-io/milvus/issues/28893

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-12-13 18:50:42 +08:00
Jiquan Long
520c07ec9d
fix: autoindex panic with flat index type (#29071) 
issue: https://github.com/milvus-io/milvus/issues/29048

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-12-12 22:08:46 +08:00
Jiquan Long
e5f155612a
fix: txn control not applied to snapshot.Remove (#29124) 
issue: #29123

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-12-12 14:48:38 +08:00
wayblink
51f870da7e
feat: Introduce channelCheckpointUpdater to reduce goroutine use in ttNode (#28570) 
/kind enhancement

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-12-12 13:48:42 +08:00
wei liu
69a71e11d1
enhance: Add config for auto upgrade segment index (#29112) 
This PR add config for auto upgrade segment index to index engine's
version.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-12-12 11:56:37 +08:00
Enwei Jiao
0e65e90338
enhance: Support otlp with insecure (#29115) 
issue: https://github.com/milvus-io/milvus/issues/28914

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-12-12 11:14:37 +08:00
yah01
0a87724f18
enhance: remove merger for load segments (#29062) 
remove merger as now QueryNode could load segments concurrently
fix #29063

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-12-12 10:48:37 +08:00
cai.zhang
2b05460ef9
enhance: Make import-related error message clearer (#28978) 
issue: #28976

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2023-12-08 10:12:38 +08:00
yihao.dai
43abe9ce31
enhance: Delete datanode metric after flowgraph released (#29003) 
`DataNodeFlowGraphBufferDataSize` monitoring metric is leaked. This PR
resolve it.

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-12-07 17:00:35 +08:00
yah01
9b3e06ae86
enhance: add more metrics for level zero segments (#29029) 
- Add SegmentNum metric for level zero segments
- Add level zero segments size metirc

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-12-07 14:34:35 +08:00
wayblink
6736f65345
feat: skip some empty ttMsg in Datanode flowgraph (#28756) 
/kind feature

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-12-07 01:00:37 +08:00
shaoyue
4a067a4c8c
enhance: Add proxy.ginLogSkipPaths (#28945) 
Fix #28944

/cc @xiaofan-luan

Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2023-12-06 10:26:35 +08:00
groot
ad1daebc8e
fix: Change ranges of HNSW parameters according to Knowhere (#28875) 
fix: https://github.com/milvus-io/milvus/issues/28860

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2023-12-05 16:20:42 +08:00
yah01
bf633bb5d7
enhance: refine the retry error (#28573) 
return the last error but not combining all errors, to improve
readability and erorr handling

resolve: #28572

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-11-30 18:34:32 +08:00
shaoyue
8b2b0d412c
enhance: storeageType default value change to remote (#28792) 
/kind enhancement
/cc @PowderLi

Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2023-11-30 11:34:27 +08:00
congqixia
f9bb8e9648
enhance: Change const magic number in querycoord to param (#28819) 
See also #28817

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-11-30 09:06:28 +08:00
cqy123456
3b1b14dd78
fix: update binlog index memory uasge before loading segments (#28528) 
issue: #27678 
when interimIndex = true, memory predict should be update with the
memory usage of binlog index build process.

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2023-11-29 16:42:27 +08:00
Gao
5bd73d5503
enhance: add efficient distance computations in Go (#28657) 
Related #28656
Add more efficient calc_distance at Go side.

---------

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2023-11-28 18:20:26 +08:00
XuanYang-cn
606ec77b66
enhance: Unify levelzero segment config in DN (#28720) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-11-28 18:04:26 +08:00
aoiasd
89d8ce2f73
enhance: refine access log to support format access log by yaml and print name info. (#28319) 
relate: https://github.com/milvus-io/milvus/issues/28086

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-11-28 15:32:31 +08:00
cai.zhang
1b7a503f89
enhance: Revert import support csv format (#28760) 
Revert import support csv format.
issue: #28778

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2023-11-28 14:32:27 +08:00
aoiasd
ce2436127c
fix: log can not reduce level (#28750) 
relate: https://github.com/milvus-io/milvus/issues/28749

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-11-28 10:02:27 +08:00
congqixia
dd682658da
enhance: Add channel checkpoint ts lag metric (#28767) 
See also #28765

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-11-28 07:00:28 +08:00
SimFG
9c46788d87
enhance: Support to trace restful request and request error (#28685) 
issue: #28348

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-11-27 20:14:26 +08:00
cai.zhang
c29b60e18e
enhance: Support Array DataType for bulk_insert (#28341) 
issue: #28272 
Support array DataType for bulk_insert with json, binlog files.

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2023-11-27 13:50:27 +08:00
aoiasd
8a4cfb7d6a
enhance: add l0 metric and fix datacoord no need drop l0 segment when flush (#28373) 
relate: https://github.com/milvus-io/milvus/issues/27675

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-11-24 15:58:24 +08:00
Bingyi Sun
2843664a20
fix: channel name can not contains '-partition-' (#28689) 
issue: https://github.com/milvus-io/milvus/issues/28675

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2023-11-24 10:14:24 +08:00
wei liu
4b8d3eaa41
enhance: Increase heartbeat timeout (#28648) 
it's easy to trigger heartbeat timeout after 100ms when standalone cpu
usage reach 100%.
This PR increase the heartbeat timeout param to 2000ms

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-11-23 19:50:24 +08:00
congqixia
a2fe9dad49
enhance: Make etcd kv request timeout configurable (#28661) 
See also #28660
This pr add request timeout config item for etcd kv request timeout
 Sync the default timeout value to same value for etcdKV & tikv config

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-11-23 19:34:23 +08:00
zhagnlu
bc43c5cf34
fix: Panic if minio port is empty (#28461) 
#28460

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2023-11-23 19:24:25 +08:00
Bingyi Sun
4fedff6d47
feat: integrate storage v2 into the write path (#28440) 
#28378

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2023-11-23 17:26:24 +08:00
SimFG
de13865769
enhance: Add load/release partitions to replicate msg stream (#28399) 
/kind improvement
issue: #25655

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-11-23 15:38:24 +08:00
yah01
c96d07682e
enhance: enhance the accuracy of memory usage (#28554) 
before this, Milvus use container/system's memory info to get the memory
usage, which could be inaccurate.

we allocates the memory by private anon mmap,
then `rss - shared` would be the accurate memory usage

resolve #28553

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-11-23 15:12:23 +08:00
Enwei Jiao
3c3a57cb92
fix: Fix kafka config type error (#28642) 
issue https://github.com/milvus-io/milvus/issues/28588

---------

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-11-23 14:36:22 +08:00
yah01
bfccfcd0ca
enhance: refine error messages (#28424) 
- Split the simple reason and full detail
- Refine existing error messages
related: #28422

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-11-21 17:02:24 +08:00
XuanYang-cn
e88bbaac24
feat: Add universal levelzero segment switch (#28483) 
See also: #27349

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-11-20 19:04:22 +08:00
PowderLi
7029797452
Improve HTTP server include metrics and RESTful API (#28045) 
/kind improvement
issue: #27653

Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-11-19 17:14:20 +08:00
congqixia
f1fc19e8a9
enhance: Add unittest for config.EventDispatcher (#28552) 
These is no enough unittest case for EventDispatcher, see also #28540. 
This PR add unit test case for all methods for EventDispatcher
Related to #28538

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-11-18 19:12:20 +08:00
congqixia
c948a437a9
fix: protect EventDispatcher map with mutex (#28540) 
Add mutex protection for `EventDispatcher.registry` map 
Fix #28538

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-11-17 20:32:20 +08:00
yah01
e9ff7ed13d
enhance: remove legacy error related code (#28385) 
/kind improvement

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-11-14 15:02:18 +08:00
congqixia
371875f650
Fix MQ Factory timeout unit test (#28395) 
This PR change timeout case to a longer duration in case of slow runner
Also change timeout case behavior of using timeout deadline instead of waiting
See also #28394

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-11-13 21:44:18 +08:00
SimFG
cfb6edea61
Support to trace the grpc request (#28349) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-11-13 20:16:18 +08:00
XuanYang-cn
f8aa46419a
Add LevelZeroCompaction configs (#28190) 
See also: #27606

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-11-13 11:18:19 +08:00