milvus/INSTALL.md

Install Milvus from Source Code

• Build from source

  • Requirements

  • Compilation

  • Launch Milvus server

• Compile Milvus on Docker

  • Step 1 Pull Milvus Docker images

  • Step 2 Start the Docker container

  • Step 3 Download Milvus source code

  • Step 4 Compile Milvus in the container

• Troubleshooting

  • Error message: protocol https not supported or disabled in libcurl

  • Error message: internal compiler error

  • Error message: error while loading shared libraries: libmysqlpp.so.3

  • CMake version is not supported

Build from source

Requirements

• Operating system

• Ubuntu 18.04 or higher

• CentOS 7

  Note: If your Linux operating system does not meet the requirements, we recommend that you pull a Docker image of Ubuntu 18.04 or CentOS 7 as your compilation environment.

• GCC 7.0 or higher to support C++ 17

• CMake 3.14 or higher

• Git

For GPU-enabled version, you will also need:

• CUDA 10.x (10.0, 10.1, 10.2)

• NVIDIA driver 418 or higher

Compilation

Step 1 Download Milvus source code and specify version

Download Milvus source code, change directory and specify version (for example, 0.10.3):

$ git clone https://github.com/milvus-io/milvus
$ cd ./milvus/core
$ git checkout 0.10.3

Step 2 Install dependencies

Install in Ubuntu

$ ./ubuntu_build_deps.sh

Install in CentOS

$ ./centos7_build_deps.sh

Step 3 Build Milvus source code

If you want to use CPU-only:

run build.sh:

$ ./build.sh -t Release

If you want to use GPU-enabled:

1. Add cuda library path to LD_LIBRARY_PATH:

$ export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH

2. Add cuda binary path to PATH:

$ export PATH=/usr/local/cuda/bin:$PATH

3. Add a -g parameter to run build.sh:

$ ./build.sh -g -t Release

When the build completes, everything that you need to run Milvus is under [Milvus root path]/core/milvus.

Launch Milvus server

$ cd [Milvus root path]/core/milvus

Add lib/ directory to LD_LIBRARY_PATH

$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:[Milvus root path]/core/milvus/lib

Then start Milvus server:

$ cd scripts
$ ./start_server.sh

To stop Milvus server, run:

$ ./stop_server.sh

Compile Milvus on Docker

With the following Docker images, you should be able to compile Milvus on any Linux platform that runs Docker. To build a GPU supported Milvus, you need to install NVIDIA Docker first.

Step 1 Pull Milvus Docker images

Ubuntu 18.04

Pull CPU-only image:

$ docker pull milvusdb/milvus-cpu-build-env:v0.7.0-ubuntu18.04

Pull GPU-enabled image:

$ docker pull milvusdb/milvus-gpu-build-env:v0.7.0-ubuntu18.04

CentOS 7

Pull CPU-only image:

$ docker pull milvusdb/milvus-cpu-build-env:v0.7.0-centos7

Pull GPU-enabled image:

$ docker pull milvusdb/milvus-gpu-build-env:v0.7.0-centos7

Step 2 Start the Docker container

Start a CPU-only container:

$ docker run -it -p 19530:19530 -d <milvus_cpu_docker_image>

Start a GPU container:

• For nvidia docker 2:

$ docker run --runtime=nvidia -it -p 19530:19530 -d <milvus_gpu_docker_image>

• For nvidia container toolkit:

docker run --gpus all -it -p 19530:19530 -d <milvus_gpu_docker_image>

To enter the container:

$ docker exec -it [container_id] bash

Step 3 Download Milvus source code

Download Milvus source code:

$ git clone https://github.com/milvus-io/milvus

To enter its core directory:

$ cd ./milvus/core

Specify version (for example, 0.10.3):

$ git checkout 0.10.3

Step 4 Compile Milvus in the container

If you are using a CPU-only image:

1. run build.sh:

$ ./build.sh -t Release

2. Start Milvus server：

$ ./start_server.sh

If you are using a GPU-enabled image:

1. Add cuda library path to LD_LIBRARY_PATH:

$ export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH

2. Add cuda binary path to PATH:

$ export PATH=/usr/local/cuda/bin:$PATH

3. Add a -g parameter to run build.sh:

$ ./build.sh -g -t Release

4. Start Milvus server：

$ ./start_server.sh

Troubleshooting

Error message: protocol https not supported or disabled in libcurl

Follow the steps below to solve this problem:

1. Make sure you have libcurl4-openssl-dev installed in your system.
2. Try reinstalling the latest CMake from source with --system-curl option:

$ ./bootstrap --system-curl
$ make
$ sudo make install

If the --system-curl command doesn't work, you can also reinstall CMake in Ubuntu Software on your local computer.

Error message: internal compiler error

Try increasing the memory allocated to Docker. If this doesn't work, you can reduce the number of threads in CMake build in [Milvus root path]/core/build.sh.

make -j 8 install || exit 1 # The default number of threads is 8.

Note: You might also need to configure CMake build for faiss in [Milvus root path]/core/src/index/thirdparty/faiss.

Error message: error while loading shared libraries: libmysqlpp.so.3

Follow the steps below to solve this problem:

1. Check whether libmysqlpp.so.3 is correctly installed.
2. If libmysqlpp.so.3 is installed, check whether it is added to LD_LIBRARY_PATH.

CMake version is not supported

Follow the steps below to install a supported version of CMake:

1. Remove the unsupported version of CMake.

2. Get CMake 3.14 or higher. Here we get CMake 3.14.

   $ wget https://cmake.org/files/v3.14/cmake-3.14.7-Linux-x86_64.tar.gz
   

3. Extract the file and install CMake.

   $ tar zxvf cmake-3.14.7-Linux-x86_64.tar.gz
   $ mv cmake-3.14.7-Linux-x86_64 /opt/cmake-3.14.7
   $ ln -sf /opt/cmake-3.14.7/bin/* /usr/bin/