milvus/core
jinhai a4eed2e786 Merge branch 'branch-0.5.1' into 'branch-0.5.0'
MS-605 Fix search crash issue

Closes MS-605

See merge request megasearch/milvus!725

Former-commit-id: 5fbfc96e89f6fcf15e757ff30e51c441c3b38e33
2019-10-15 12:12:57 +08:00
..
build-support re-organize project 2019-10-14 09:51:48 +08:00
cmake update ExternalProject_Create_Cache cmake function 2019-10-15 10:54:10 +08:00
conf re-organize project 2019-10-14 09:51:48 +08:00
scripts re-organize project 2019-10-14 09:51:48 +08:00
src Merge from main branch 2019-10-15 12:11:04 +08:00
thirdparty re-organize project 2019-10-14 09:51:48 +08:00
unittest MS-606 fix result reduce bug 2019-10-14 18:20:15 +08:00
.gitignore re-organize project 2019-10-14 09:51:48 +08:00
build.sh re-organize project 2019-10-14 09:51:48 +08:00
CHANGELOG.md Merge from main branch 2019-10-15 12:11:04 +08:00
CMakeLists.txt fix CUSTOMIZATION options bug 2019-10-15 10:24:14 +08:00
CODE_OF_CONDUCT.md re-organize project 2019-10-14 09:51:48 +08:00
CONTRIBUTING.md re-organize project 2019-10-14 09:51:48 +08:00
coverage.sh modify coverage.sh 2019-10-15 12:09:10 +08:00
NOTICE.md re-organize project 2019-10-14 09:51:48 +08:00
README.md re-organize project 2019-10-14 09:51:48 +08:00
start_server.sh re-organize project 2019-10-14 09:51:48 +08:00
stop_server.sh re-organize project 2019-10-14 09:51:48 +08:00
version.h.macro re-organize project 2019-10-14 09:51:48 +08:00

Welcome to Milvus

Firstly, welcome, and thanks for your interest in Milvus! No matter who you are, what you do, we greatly appreciate your contribution to help us reinvent data science with Milvus.

What is Milvus

Milvus is an open source vector search engine that supports similarity search of large-scale vectors. Built on optimized indexing algorithm, it is compatible with major AI/ML models.

Milvus was developed by ZILLIZ, a tech startup that intends to reinvent data science, with the purpose of providing enterprises with efficient and scalable similarity search and analysis of feature vectors and unstructured data.

Milvus provides stable Python, C++ and Java APIs.

Keep up-to-date with newest releases and latest updates by reading Milvus release notes.

  • GPU-accelerated search engine

    Milvus is designed for the largest scale of vector index. CPU/GPU heterogeneous computing architecture allows you to process data at a speed 1000 times faster.

  • Intelligent index

    With a “Decide Your Own Algorithm” approach, you can embed machine learning and advanced algorithms into Milvus without the headache of complex data engineering or migrating data between disparate systems. Milvus is built on optimized indexing algorithm based on quantization indexing, tree-based and graph indexing methods.

  • Strong scalability

    The data is stored and computed on a distributed architecture. This lets you scale data sizes up and down without redesigning the system.

Architecture

Milvus_arch

Get started

Install and start Milvus server

Use Docker

Use Docker to install Milvus is a breeze. See the Milvus install guide for details.

Use source code

Compilation
Step 1 Install necessary tools
# Install tools
Centos7 : 
$ yum install gfortran qt4 flex bison 
$ yum install mysql-devel mysql
    
Ubuntu 16.04 or 18.04: 
$ sudo apt-get install gfortran qt4-qmake flex bison 
$ sudo apt-get install libmysqlclient-dev mysql-client

Verify the existence of libmysqlclient_r.so:

# Verify existence
$ locate libmysqlclient_r.so

If not, you need to create a symbolic link:

# Locate libmysqlclient.so
$ sudo updatedb
$ locate libmysqlclient.so 

# Create symbolic link
$ sudo ln -s /path/to/libmysqlclient.so /path/to/libmysqlclient_r.so
Step 2 Build
$ cd [Milvus sourcecode path]/cpp
$ ./build.sh -t Debug
or 
$ ./build.sh -t Release

When the build is completed, all the stuff that you need in order to run Milvus will be installed under [Milvus root path]/cpp/milvus.

If you encounter the following error message, protocol https not supported or disabled in libcurl

please reinstall CMake with curl:

  1. Install curl development files:

    CentOS 7:   
    $ yum install curl-devel
    Ubuntu 16.04 or 18.04: 
    $ sudo apt-get install libcurl4-openssl-dev
    
  2. Install CMake 3.14:

    $ ./bootstrap --system-curl 
    $ make 
    $ sudo make install
    
code format and linting

Install clang-format and clang-tidy

CentOS 7:   
$ yum install clang
Ubuntu 16.04: 
$ sudo apt-get install clang-tidy
$ sudo su
$ wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | sudo apt-key add -
$ apt-add-repository "deb http://apt.llvm.org/xenial/ llvm-toolchain-xenial-6.0 main"
$ apt-get update
$ apt-get install clang-format-6.0
Ubuntu 18.04: 
$ sudo apt-get install clang-tidy clang-format

$ rm cmake_build/CMakeCache.txt

Check code style

$ ./build.sh -l

To format the code

$ cd cmake_build
$ make clang-format
Run unit test
$ ./build.sh -u
Run code coverage

Install lcov

CentOS 7:   
$ yum install lcov
Ubuntu 16.04 or 18.04: 
$ sudo apt-get install lcov
$ ./build.sh -u -c
Launch Milvus server
$ cd [Milvus root path]/cpp/milvus

Add lib/ directory to LD_LIBRARY_PATH

$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/milvus/lib

Then start Milvus server:

$ cd scripts
$ ./start_server.sh

To stop Milvus server, run:

$ ./stop_server.sh

To edit Milvus settings in conf/server_config.yaml and conf/log_config.conf, please read Milvus Configuration.

Try your first Milvus program

Run Python example code

Make sure Python 3.4 or higher is already installed and in use.

Install Milvus Python SDK.

# Install Milvus Python SDK
$ pip install pymilvus==0.2.0

Create a new file example.py, and add Python example code to it.

Run the example code.

# Run Milvus Python example
$ python3 example.py

Run C++ example code

 # Run Milvus C++ example
 $ cd [Milvus root path]/cpp/milvus/bin
 $ ./sdk_simple

Contribution guidelines

Contributions are welcomed and greatly appreciated. If you want to contribute to Milvus, please read our contribution guidelines. This project adheres to the [code of conduct](CODE OF CONDUCT.md) of Milvus. By participating, you are expected to uphold this code.

We use GitHub issues to track issues and bugs. For general questions and public discussions, please join our community.

Join the Milvus community

To connect with other users and contributors, welcome to join our slack channel.

Milvus Roadmap

Please read our roadmap to learn about upcoming features.

Resources

Milvus official website

Milvus docs

Milvus blog

Milvus CSDN

Milvus roadmap

License

Apache 2.0 license