Commit Graph

21311 Commits

Author SHA1 Message Date
SimFG
2208b7c2ef
fix: the too long default root password does not take effect (#37983)
- issue: #36987

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-11-26 17:24:35 +08:00
yellow-shine
858eaa668d
enhance: select node (#37998)
Some checks are pending
Code Checker / Code Checker AMD64 Ubuntu 22.04 (push) Waiting to run
Code Checker / Code Checker Amazonlinux 2023 (push) Waiting to run
Code Checker / Code Checker rockylinux8 (push) Waiting to run
Mac Code Checker / Code Checker MacOS 13 (push) Waiting to run
Build and test / Build and test AMD64 Ubuntu 22.04 (push) Waiting to run
Build and test / UT for Cpp (push) Blocked by required conditions
Build and test / UT for Go (push) Blocked by required conditions
Build and test / Integration Test (push) Blocked by required conditions
Build and test / Upload Code Coverage (push) Blocked by required conditions
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-11-26 17:02:41 +08:00
Zhen Ye
2b4f211d84
enhance: add switch for local rpc enabled (#37985)
issue: #33285

- Add switch for local rpc

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2024-11-26 17:00:54 +08:00
aoiasd
db3c002788
fix:wrong warning log when idf oracle find L0 segment lacks (#37873)
relate: https://github.com/milvus-io/milvus/issues/35853

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-11-26 16:26:41 +08:00
ThreadDao
c5327ff150
test: add cases for gosdk v2 partial load (#37924)
issue: #33419

Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2024-11-26 15:08:34 +08:00
congqixia
cb6542339e
enhance: Mark cgo thread with tag name (#38000)
Related to #37999

This PR add `SetThreadName` API for marking cgo thread and utilize it
when initializing cgo worker.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-26 11:22:35 +08:00
congqixia
6b23e668b0
test: Update go_client version & update latest changes (#37793)
Related to #37768

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-26 10:26:40 +08:00
zhuwenxing
27c22d1166
test: remove sparse embedding in output fields for fts cases (#37980)
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-11-25 19:14:33 +08:00
qixuan
4cd79323bf
test: Add cases for restful about default and none (#37943)
Some checks failed
Code Checker / Code Checker AMD64 Ubuntu 22.04 (push) Waiting to run
Code Checker / Code Checker Amazonlinux 2023 (push) Waiting to run
Code Checker / Code Checker rockylinux8 (push) Waiting to run
Mac Code Checker / Code Checker MacOS 13 (push) Waiting to run
Build and test / Build and test AMD64 Ubuntu 22.04 (push) Waiting to run
Build and test / UT for Cpp (push) Blocked by required conditions
Build and test / UT for Go (push) Blocked by required conditions
Build and test / Integration Test (push) Blocked by required conditions
Build and test / Upload Code Coverage (push) Blocked by required conditions
Weekly Release / Run Weekly Release (push) Has been cancelled
issue: #36129

Signed-off-by: qixuan <673771573@qq.com>
2024-11-25 14:06:33 +08:00
tinswzy
e76802f910
enhance: refine querycoord meta/catalog related interfaces to ensure that each method includes a ctx parameter (#37916)
issue: #35917 
This PR refine the querycoord meta related interfaces to ensure that
each method includes a ctx parameter.

Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2024-11-25 11:14:34 +08:00
zhuwenxing
0b9edb62a9
test: add testcases contain growing segments (#37262)
Some checks failed
Code Checker / Code Checker AMD64 Ubuntu 22.04 (push) Waiting to run
Code Checker / Code Checker Amazonlinux 2023 (push) Waiting to run
Code Checker / Code Checker rockylinux8 (push) Waiting to run
Mac Code Checker / Code Checker MacOS 13 (push) Waiting to run
Build and test / Build and test AMD64 Ubuntu 22.04 (push) Waiting to run
Build and test / UT for Cpp (push) Blocked by required conditions
Build and test / UT for Go (push) Blocked by required conditions
Build and test / Integration Test (push) Blocked by required conditions
Build and test / Upload Code Coverage (push) Blocked by required conditions
all-contributors / contributor (push) Has been cancelled
Update Knowhere Commit / update-knowhere-commit (push) Has been cancelled
/kind improvement

---------

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-11-25 10:40:34 +08:00
Zhen Ye
fbb68ca370
enhance: make all index operation async scheduled by tokio (#37946)
issue: #37851
related pr: https://github.com/milvus-io/tantivy/pull/3

Signed-off-by: chyezh <chyezh@outlook.com>
2024-11-25 10:12:34 +08:00
sre-ci-robot
ed73dfca3f
[automated] Update Knowhere Commit (#37965)
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-11-25 01:28:32 +08:00
Yinzuo Jiang
5a06faca39
feat: fp32 vector to fp16/bf16 vector conversion for RESTful API (#37556)
RESTful API. The influenced API are as follows:

- Handler. insert
- HandlerV1. insert/upsert
- HandlerV2. insert/upsert/search

We do not modify search API in Handler/HandlerV1 because they do not
support fp16/bf16 vectors.

module github.com/milvus-io/milvus/pkg:

Add `Float32ArrayToBFloat16Bytes()`, `Float32ArrayToFloat16Bytes()` and
`Float32ArrayToBytes()`. These method will be used in GoSDK in the
future.

issue: #37448

Signed-off-by: Yinzuo Jiang <yinzuo.jiang@zilliz.com>
Signed-off-by: Yinzuo Jiang <jiangyinzuo@foxmail.com>
2024-11-24 17:46:33 +08:00
zhagnlu
62af24c1a1
fix: change search latency metric from us unit to ms unit (#37806)
#37805

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-11-24 17:26:33 +08:00
congqixia
1e76d2b9f1
fix: Use correct policy merging growing&l0 and add unit tests (#37950)
Related to #37574

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-24 17:10:33 +08:00
congqixia
8c9dab5cf7
enhance: [GoSDK] Add Slice method for Vector Columns (#37951)
Related to #37768

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-24 14:48:33 +08:00
sre-ci-robot
930112e55e
[automated] Bump milvus version to v2.5.0-beta (#37961)
Bump milvus version to v2.5.0-beta
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-11-23 21:48:32 +08:00
Zhen Ye
b9a10a2f68
enhance: remove the rpc layer of coordinator when enabling standalone or mixcoord (#37815)
issue: #37764

- add a local client to call local server directly for
querycoord/rootcoord/datacoord.
- enable local client if milvus is running mixcoord or standalone mode.

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2024-11-23 21:36:33 +08:00
sre-ci-robot
74752f2ec7
[automated] Bump milvus version to v2.5.0-beta (#37954)
Bump milvus version to v2.5.0-beta
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-11-22 23:16:41 +08:00
sthuang
bd0811434b
fix: fix grant v2 service (#37945)
issue: https://github.com/milvus-io/milvus/issues/37031

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-11-22 19:42:32 +08:00
smellthemoon
df08d91c0f
enhance: error msg (#37901)
#37900

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-11-22 18:40:32 +08:00
sre-ci-robot
efce709152
[automated] Bump milvus version to v2.4.17 (#37937)
Bump milvus version to v2.4.17
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-11-22 17:56:32 +08:00
jaime
8187fb7d19
enhance: set the maximum database configuration to be refreshable (#37931)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-11-22 17:46:31 +08:00
jaime
7bbfe86bcd
enhance: add list index and segment index retrieval API for WebUI (#37861)
issue: #36621

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-11-22 16:58:34 +08:00
SimFG
7c5a8012cf
enhance: remove the collectionBasicInfo class in the proxy metacache (#37874)
/kind improvement
- issue: #37928

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-11-22 16:10:33 +08:00
yellow-shine
b48f531e63
enhance: [pipeline] limit compute resource (#37888)
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-11-22 14:14:38 +08:00
Bingyi Sun
700a448a54
fix: Escape prefix before search in inverted index (#37925)
issue: https://github.com/milvus-io/milvus/issues/37912

Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-11-22 14:10:33 +08:00
Bingyi Sun
06d73cf2e2
enhance: Remove raw tokenizer register. (#37886)
tantivy already register raw tokenizer by default

Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-11-22 12:02:32 +08:00
congqixia
b34bfb98a0
enhance: Refine Replica manager colle2Replicas secondary index (#37906)
Related to #37630

This PR add a new util coll2Replicas secondary index to reduce map
access & iteration while get replicas by collection

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-22 11:54:32 +08:00
congqixia
1ed686783f
enhance: Use PrimaryKeys to replace interface slice for segment delete (#37880)
Related to #35303

Reduce temporary memory usage for PK interface for segment delete.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-22 11:52:33 +08:00
SimFG
923a661dfe
enhance: filter the fields instead of create a new response obj (#37845)
/kind improvement

Here you only need to filter out the system fields, and you don’t need
to recreate a response, because recreating the response will cause this
part to be easily missed when adding fields later.

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-11-22 11:42:32 +08:00
SimFG
cef55eb093
enhance: add retry termination log (#37894)
/kind improvement

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-11-22 11:40:33 +08:00
ThreadDao
bc131a911c
test: add cases for gosdk v2 load release (#37831)
issue: #33419

Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2024-11-22 10:54:32 +08:00
congqixia
cfa1f1f141
enhance: Add thread watcher to provide actual thread num (#37905)
Related to #37904

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-22 10:20:33 +08:00
congqixia
92e6ee6285
enhance: Use load pool for CreateTextIndex (#37898)
Related to #37895

Only resolves the starving issue which caused goroutine leakage

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-22 10:06:33 +08:00
congqixia
83df725146
enhance: Revert default l0 forward policy to FilterByBF (#37867)
Related to #37767

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-22 09:00:33 +08:00
sre-ci-robot
84259040f7 Update all contributors
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
2024-11-22 00:01:33 +00:00
SimFG
54aaeda63f
fix: add the request ctx for stream pipeline interface (#37835)
- issue: #37834

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-11-22 03:10:34 +08:00
sthuang
19572f5b06
enhance: RBAC new grant/revoke privilege (#37785)
issue: https://github.com/milvus-io/milvus/issues/37031
also fix issues: https://github.com/milvus-io/milvus/issues/37843,
https://github.com/milvus-io/milvus/issues/37842,
https://github.com/milvus-io/milvus/issues/37887

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-11-21 22:20:34 +08:00
wei liu
5f3601a6a5
fix: unstable integration test caused by paramtable.GetNodeID (#37909)
issue: #37908
cause paramtable is global single instance, which cause
paramtable.GetNodeID may return wrong server id in integration test.

This PR use node.GetNodeID to replace paramtable.GetNodeID

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-11-21 22:16:33 +08:00
Zhen Ye
1dc1a97e65
fix: use different thread pool for scheduler and merger (#37911)
issue: #37895
related pr: https://github.com/milvus-io/tantivy/pull/2

Signed-off-by: chyezh <chyezh@outlook.com>
2024-11-21 21:34:33 +08:00
MimoN
3cb9511c57
fix: update webui filters (#37885)
issue: #36621
- fix the empty segment source
- remove http request timeout limit
- support more filters in seg and channel table

Signed-off-by: mimo.oyn <ning.ouyang@zilliz.com>
2024-11-21 20:04:33 +08:00
tinswzy
e247ff9ee7
enhance: refine rootcoord/metatable interfaces to ensure that each method includes a ctx parameter (#37846)
issue: #35917
Before enhancing log trace information, it's necessary to pass the
context to the method entry point.
This PR first refine the rootcoord/metatable interfaces to ensure that
each method includes a ctx parameter.

Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2024-11-21 19:30:32 +08:00
wei liu
965bda6e60
enhance: Add channel name to shard leader log in meta cache (#37856)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-11-21 19:24:31 +08:00
wei liu
97a44b62fd
fix: Data race in datacoord channel manager (#37866)
issue: #37865

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-11-21 19:00:32 +08:00
XuanYang-cn
d7dcc752f1
enhance: Increase task capacity and clean illegal task (#37896)
1. taskQueueCapacity 256 is too small for production when we want to
re-write the entire collection

2. tasks should be cleaned when unable to recover, or the meta will
remain in etcd forever later.

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-11-21 18:48:32 +08:00
wei liu
0a440e0d38
fix: Prevent simultaneous balance of segments and channels (#37850)
issue: #33550
balance segment and balance segment execute at same time, which will
cause bounch of corner case.

This PR disable simultaneous balance of segments and channels

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-11-21 17:56:55 +08:00
wei liu
2159526e80
fix: Deadlock in compaction handler (#37868)
issue: #37803

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-11-21 17:26:46 +08:00
XuanYang-cn
70e6a00ba1
fix: Replace outer lock with concurrent map (#37817)
See also: #37493

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-11-21 16:08:33 +08:00