Commit Graph

373 Commits

Author SHA1 Message Date
Bingyi Sun
4724779b3b
enhance: remove fallback keys for config generator (#32946)
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-05-13 13:33:31 +08:00
aoiasd
54a51b1236
enhance: Support dynamic config for opentelemetry trace (#32169)
relate: https://github.com/milvus-io/milvus/issues/31940

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-05-09 17:43:30 +08:00
Ted Xu
78d32bd8b2
enhance: update milvus.yaml (#31832)
See #32168

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-04-16 16:17:19 +08:00
shaoyue
be53e028a8
doc: [skip e2e] add config-docs-generator (#22134)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>

Part of issue: #21856

Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2024-04-03 14:13:13 +08:00
Xiaofan
b6fefee0cf
fix: etcd not connectable when auth enabled (#31633)
Fix etcd config source didn't respect auth enabled
Also removed pulsar recoverable error when pulsar return ConsumerBusy.
It could happen that pulsar didn't find the original consumer is dead
and recover takes some time.
fix #31631

Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
2024-04-01 15:23:19 +08:00
Bingyi Sun
0ac9bb4a9c
enhance: add mmap migration tool (#30909)
issue: #30908

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-03-25 15:51:09 +08:00
chyezh
0c7474d7e8
enhance: add graceful stop timeout to avoid node stop hang under extreme cases (#30317)
1. add coordinator graceful stop timeout to 5s
2. change the order of datacoord component while stop
3. change querynode grace stop timeout to 900s, and we should
potentially change this to 600s when graceful stop is smooth

issue: #30310
also see pr: #30306

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2024-02-29 17:01:50 +08:00
smellthemoon
9815cf50c9
fix: close rocksmq and natsmq when shutdown (#30706)
related: #30703 #30355

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-02-26 11:38:55 +08:00
chyezh
211143c5e6
enhance: add basic information of milvus into metrics (#29665)
add basic build information and runtime component dependency into
metrics.

issue: #29664

Signed-off-by: chyezh <ye.zhen@zilliz.com>
2024-01-29 15:47:02 +08:00
congqixia
20a3569c14
enhance: Set correct role for non-standalone deployment (#30303)
See also #30211

After fix initialization problem, distributed components do no have
their role set. This will cause logger & tracing miss component service
info when recording information.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-01-26 14:05:01 +08:00
PowderLi
08ca0a2ca5
feat: support etcd authentication (#30226)
issue: #28895
add 3 configuration for ETCD config

Signed-off-by: PowderLi <min.li@zilliz.com>
2024-01-24 11:35:00 +08:00
congqixia
c9860e3d2a
fix: Init paramtable after milvus role setup (#30211)
See also #30176

Move paramtable.Init after env setup in roles.Run. Also introduced a
flag for mixture run to set role correctly for mixture mode.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-01-23 19:10:56 +08:00
Bingyi Sun
dad27cbb40
fix: fix wrong log file name when running as mixture type (#29979)
See also: #25323, #29969
many users reported log file name is incorrect when starting in mixture
type.

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-01-19 11:48:54 +08:00
Jiquan Long
a3bae80b59
enhance: print total memory when milvus starts (#29351)
fix: #29349

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-01-07 20:09:49 +08:00
SimFG
dd9c61831d
enhance: Support to get the param value in the runtime (#29297)
/kind improvement
issue: #29299

Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-12-22 18:36:44 +08:00
Bingyi Sun
e8c5e75786
enhance: ignore proto generated files (#28537)
ignore proto generated files
issue: https://github.com/milvus-io/milvus/issues/28566

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
2023-11-22 10:20:22 +08:00
jaime
ec0d9d8fd4
Fix delete session key with prefix (#28260)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-11-08 15:22:21 +08:00
wei liu
0c69f48ba4
refine stop order (#28016)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-11-01 11:40:15 +08:00
yah01
9658367a3c
Refine chunk manager errors (#27590)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-31 12:18:15 +08:00
Filip Haltmayer
6b1a106a31
Moving etcd client into session (#27069)
Signed-off-by: Filip Haltmayer <filip.haltmayer@zilliz.com>
2023-10-27 07:36:12 +08:00
wayblink
aa1fba79b1
Register metrics before component start (#27744)
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-10-19 17:14:10 +08:00
jaime
ac2d1bb5c2
Support receive signals from parent process (#27756)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-10-18 20:20:11 +08:00
congqixia
2f201c25e2
Remove deprecated io/ioutil usage (#27747)
`io/ioutil` package is deprecated, use `io`,`os` package replacement
also added golangci-lint rule to block future reference

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: guoguangwu <guoguangwu@magic-shield.com>
2023-10-17 20:32:09 +08:00
jaime
ec1fe3549e
Add a stop hook to clean session (#27564)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-10-16 10:24:10 +08:00
SimFG
26f06dd732
Format the code (#27275)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-21 09:45:27 +08:00
yihao.dai
bb6711f28c
Add ChunkCache: support get vector from storage (#26142)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-09-15 10:21:20 +08:00
congqixia
f7b2ad6650
Fix component pointer to component interface never be nil caused panic (#27072)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-14 13:37:18 +08:00
cai.zhang
4fa878d03c
Remove unused log (#27032)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-09-13 16:07:18 +08:00
Enwei Jiao
fb0705df1b
Decouple basetable and componentparam (#26725)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-09-05 10:31:48 +08:00
yah01
3349db4aa7
Refine errors to remove changes breaking design (#26521)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-04 09:57:09 +08:00
congqixia
4bf26dd4ed
Refine standalone components stop order (#26742)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-08-31 16:51:02 +08:00
congqixia
073ac8350b
Refine signal handler for whole milvus role lifetime (#26642)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-08-29 19:28:28 +08:00
jaime
c603f1c244
Remove mysql metastore (#26633)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-08-29 14:36:26 +08:00
Enwei Jiao
533f0ddf6d
Add amazonlinux image, prepare to remove Centos7 image, remove openblas image (#26368)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-08-18 18:32:19 +08:00
SimFG
8b54914287
Add go.mod file for the milvus/pkg package (#26359)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-08-16 14:49:32 +08:00
Jiquan Long
fb5d809599
Fix components' setup (#26182)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-08-09 14:33:15 +08:00
xige-16
33c2012675
Add more metrics (#25081)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-06-26 17:52:44 +08:00
jaime
18df2ba6fd
[Cherry-Pick] Support Database (#24769)
Support Database(#23742)
Fix db nonexists error for FlushAll (#24222)
Fix check collection limits fails (#24235)
backward compatibility with empty DB name (#24317)
Fix GetFlushAllState with DB (#24347)
Remove db from global meta cache after drop database (#24474)
Fix db name is empty for describe collection response (#24603)
Add RBAC for Database API (#24653)
Fix miss load the same name collection during recover stage (#24941)

RBAC supports Database validation (#23609)
Fix to list grant with db return empty (#23922)
Optimize PrivilegeAll permission check (#23972)
Add the default db value for the rbac request (#24307)

Signed-off-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
2023-06-25 17:20:43 +08:00
SimFG
31c08a8135
Improve the panic code about the milvus component #24783 (#24919)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-06-17 10:32:39 +08:00
congqixia
561021ec5a
Refine sigpipe handling (#24872)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-14 10:10:38 +08:00
yiwangdr
4387f36897
make etcdKV private (#24778)
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
2023-06-13 10:52:38 +08:00
congqixia
41af0a98fa
Use go-api/v2 for milvus-proto (#24770)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-09 01:28:37 +08:00
chyezh
f97127ae55
add nats mq wrappers (#24445)
bug fixup, configurable natsmq, add unittest, pass e2e.



move natsmq to pkg project

Signed-off-by: chyezh <ye.zhen@zilliz.com>
Co-authored-by: yiwangdr <yiwangdr@gmail.com>
2023-06-07 10:00:37 +08:00
xige-16
fe18109aab
Clean tmp disk data when start milvus (#24404)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-05-26 15:07:26 +08:00
cai.zhang
43a9e175a3
Exit component process when session key is deleted (#21658) (#22164)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-04-12 20:12:28 +08:00
Cai Yudong
55202ce7d9
Support show knowhere prometheus metrics (#23102)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2023-04-07 15:34:29 +08:00
jaime
c9d0c157ec
Move some modules from internal to public package (#22572)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-04-06 19:14:32 +08:00
aoiasd
1a485044a9
fix NumConsumers metric not work (#23167)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-04-05 08:18:28 +08:00
yah01
7da870f512
Remove useCustomConfig and simpilify the session type (#23166)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-04-03 20:10:24 +08:00
congqixia
8f847884c2
Fix misc copylocks and remove exclude rule (#22962)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-03-24 11:17:58 +08:00