test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-11-30 02:48:45 +08:00
Code Issues Actions 11 Packages Projects Releases Wiki Activity
b90fa5f459
milvus/scripts
History
XuanYang-cn b90fa5f459
Refine codes of datanode buffer (#23168) 
This PR refines deltabuffer of datanode:
- Add last sync time for compacted segment, see also: #23210
- Ensure all deltabuffermanager handles all delete related operations
- Change usedMemory to atomic.Int64
- Remove allocator in delete buffer

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-04-10 18:42:30 +08:00
..
sql Replace segment index file path with index file id within segment index meta (#19866) 2022-10-19 16:55:27 +08:00
check_proto_product.sh OTEL integration in cpp (#22175) 2023-03-21 16:31:57 +08:00
collect_arrow_dep.sh Add offline build logic for arrow dependency (#17266) 2022-05-30 14:54:02 +08:00
core_build.sh improvement for macos build script (#23082) 2023-03-29 10:10:01 +08:00
devcontainer.sh Fix dev container make failed (#21301) 2022-12-19 16:07:25 +08:00
docker_image_find_tag.sh [skip ci] Update the comments (#9633) 2021-10-11 19:19:02 +08:00
download_milvus_proto.sh Improve the download_milvus_proto bash (#19894) 2022-10-19 20:45:32 +08:00
gofmt.sh [skip ci]Add LF AI&Data foundation license (#9507) 2021-10-09 18:01:27 +08:00
install_deps_embd.sh Upgrade gcc11 for centos7 (#22190) 2023-02-15 17:16:33 +08:00
install_deps_msys.sh Lock conan version at 1.58.0 (#22503) 2023-03-01 19:59:47 +08:00
install_deps.sh improvement for macos build script (#23082) 2023-03-29 10:10:01 +08:00
OWNERS [skip ci]Update OWNERS files (#11898) 2021-11-16 15:41:11 +08:00
package_windows.sh Support compile under windows (#15786) 2022-03-17 17:17:22 +08:00
proto_gen_go.sh Feat: count entities by expression (#22765) 2023-03-16 19:31:55 +08:00
README.md Clean unnecessary deps to make milvus (#19711) 2022-10-12 14:33:23 +08:00
run_cpp_codecov.sh split go and cpp ut (#23239) 2023-04-06 15:56:28 +08:00
run_cpp_unittest.sh config from etcd (#18421) 2022-08-01 10:04:33 +08:00
run_docker.sh [skip ci]Add LF AI&Data foundation license (#9507) 2021-10-09 18:01:27 +08:00
run_go_codecov.sh [skip e2e] Add time caculate for cpp ut && go ut (#20552) 2022-11-14 14:43:06 +08:00
run_go_unittest.sh Add querynode plugin (#22379) 2023-03-24 15:21:59 +08:00
setenv.sh Fix jemalloc.so not found (#21106) 2022-12-13 11:27:21 +08:00
start_cluster.sh Refine codes of datanode buffer (#23168) 2023-04-10 18:42:30 +08:00
start_standalone.sh Set LD_PRELOAD env for jemalloc (#18612) 2022-08-11 19:50:37 +08:00
stop_graceful.sh Fix queryCoord panic during query node down (#21400) 2022-12-28 10:17:30 +08:00
stop.sh Fix the stop error when no milvus process (#20101) 2022-10-26 21:13:32 +08:00
update-api-version.sh Improve the download_milvus_proto bash (#19894) 2022-10-19 20:45:32 +08:00

README.md

Compile and install milvus cluster

Environment

OS: Ubuntu 18.04
go：1.18
cmake: >=3.18
gcc： 7.5

Install dependencies

Install compile dependencies

$ sudo apt install -y g++ gcc make libssl-dev zlib1g-dev libboost-regex-dev \
    libboost-program-options-dev libboost-system-dev libboost-filesystem-dev \
    libboost-serialization-dev python3-dev libboost-python-dev libcurl4-openssl-dev gfortran libtbb-dev
$ export GO111MODULE=on
$ go get github.com/golang/protobuf/protoc-gen-go@v1.3.2

Install OpenBlas library

$ wget https://github.com/xianyi/OpenBLAS/archive/v0.3.9.tar.gz && \
$ tar zxvf v0.3.9.tar.gz && cd OpenBLAS-0.3.9 && \
$ make TARGET=CORE2 DYNAMIC_ARCH=1 DYNAMIC_OLDER=1 USE_THREAD=0 USE_OPENMP=0 FC=gfortran CC=gcc COMMON_OPT="-O3 -g -fPIC" FCOMMON_OPT="-O3 -g -fPIC -frecursive" NMAX="NUM_THREADS=128" LIBPREFIX="libopenblas" INTERFACE64=0 NO_STATIC=1 && \
$ make PREFIX=/usr install

Compile

Generate the go files from proto file

$ make check-proto-product

Check code specifications

$ make verifiers

Compile milvus

$ make milvus

Install docker-compose

refer: https://docs.docker.com/compose/install/

$ sudo curl -L "https://github.com/docker/compose/releases/download/1.27.4/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
$ sudo chmod +x /usr/local/bin/docker-compose
$ sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
$ docker-compose --version

Start service

Start third-party service:

$ cd [milvus project path]/deployments/docker/cluster
$ docker-compose up -d

Start milvus cluster:

$ cd [milvus project path]
$ ./scripts/start_cluster.sh

Run unittest

Run all unittest including go and cpp cases:

$ make unittest

You also can run go unittest only:

$ make test-go

Run cpp unittest only:

$ make test-cpp

Run code coverage

Run code coverage including go and cpp:

$ make codecov

You also can run go code coverage only:

$ make codecov-go

Run cpp code coverage only:

$ make codecov-cpp