Commit Graph

274 Commits

Author SHA1 Message Date
congqixia
1a8cf5c415
Organize all mockery generation commands in Makefile (#26826)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-04 21:19:48 +08:00
congqixia
d343888f3c
Fix pkg codecov & static-check script (#26720)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-08-31 09:59:01 +08:00
yah01
74fb244bd7
Upgrade mockery (#26461)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-08-18 18:10:18 +08:00
wei liu
b47a72bfcf
fix set dirty segment distribution to leader view (#26180)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-08-11 11:21:32 +08:00
yah01
300fef446b
Enable mmap for vector index (#25877)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-08-10 13:59:15 +08:00
cai.zhang
760a2d9aa7
Support AllocTimestamp api for Milvus (#25784)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-07-25 10:05:00 +08:00
zhagnlu
65cb52d06b
Support dynamic simd framework and using term expr as example (#25260)
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-07-13 16:22:30 +08:00
Enwei Jiao
4aed32ff61
Use librdkafka for all platform (#25538)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-07-13 15:34:33 +08:00
wei liu
71d99d4ec2
fix unexpected metrics value cause by wrong time unit (#25240)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-07-06 08:48:27 +08:00
jaime
18df2ba6fd
[Cherry-Pick] Support Database (#24769)
Support Database(#23742)
Fix db nonexists error for FlushAll (#24222)
Fix check collection limits fails (#24235)
backward compatibility with empty DB name (#24317)
Fix GetFlushAllState with DB (#24347)
Remove db from global meta cache after drop database (#24474)
Fix db name is empty for describe collection response (#24603)
Add RBAC for Database API (#24653)
Fix miss load the same name collection during recover stage (#24941)

RBAC supports Database validation (#23609)
Fix to list grant with db return empty (#23922)
Optimize PrivilegeAll permission check (#23972)
Add the default db value for the rbac request (#24307)

Signed-off-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
2023-06-25 17:20:43 +08:00
wei liu
46f7d903a3
enable look aside balancer on replica selection (#24791)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-06-16 18:38:39 +08:00
wei liu
a3437e0ab5
refactor replica selection and retry policy on channel (#24367)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-06-13 10:20:37 +08:00
congqixia
41af0a98fa
Use go-api/v2 for milvus-proto (#24770)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-09 01:28:37 +08:00
Enwei Jiao
d3af451d92
Upgrade golangci-lint (#24707)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-06-07 19:34:36 +08:00
Enwei Jiao
4048caa8ca
Use only one go.mod (#24669)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-06-07 14:22:36 +08:00
zhagnlu
113f9a0ebc
Support SIMD of several Expr (#23715) (#23717)
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-05-12 14:11:20 +08:00
congqixia
5aa9db0d38
Add collection level auto compaction enabled config (#24013)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-05-10 17:45:20 +08:00
yihao.dai
092d743917
Add support for getting vectors by ids (#23450)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-04-23 09:00:32 +08:00
Enwei Jiao
2ced6660e2
Simplify makefile (#23583)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-04-21 14:16:33 +08:00
wei liu
cbfe7a45ef
fix pull target (#23491)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-04-18 18:30:32 +08:00
congqixia
5276529524
Make pkg code change trigger ci workflow (#23312)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-04-11 10:32:31 +08:00
MrPresent-Han
ba02e70301
fix bug when syncing distribution without schema and remove unused logic(#23085) (#23216)
Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
2023-04-07 19:32:29 +08:00
yah01
081572d31c
Refactor QueryNode (#21625)
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: aoiasd <zhicheng.yue@zilliz.com>
2023-03-27 00:42:00 +08:00
XuanYang-cn
93bc805933
Enhance ID allocator in DataNode (#22905)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-03-23 19:43:57 +08:00
Bennu
67658f112a
Update e2e (#22884)
Signed-off-by: Bennu-Li <yunmei.li@zilliz.com>
2023-03-21 17:55:57 +08:00
Bennu
f547c1f8f4
Add ci for milvus gpu (#22845)
Signed-off-by: Bennu-Li <yunmei.li@zilliz.com>
2023-03-20 19:03:57 +08:00
wei liu
6b5dfa6db2
deprecate shard cache immediately if query failed (#22779)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-03-17 16:25:55 +08:00
Cai Yudong
ab3cbdfc61
Partial change to prepare for GPU index type support (#22591)
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
2023-03-14 23:21:56 +08:00
wei liu
a9a263d5a8
fix assign node to replica in nodeUp (#22323)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-02-23 14:15:45 +08:00
wei liu
13b508103e
refine mock querycoord (#22198)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-02-16 15:38:34 +08:00
presburger
9950cacd10
support knowhere 2.0 (#21857)
Signed-off-by: Yusheng.Ma <Yusheng.Ma@zilliz.com>
2023-02-10 14:24:32 +08:00
wayblink
2265e81d97
Add integration test make cmd (#22009)
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-02-07 19:24:30 +08:00
congqixia
09124f87ae
Install separate golangci-lint for static-check (#21842)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-01-29 10:59:49 +08:00
congqixia
66feb295da
Use AdressSantizer build when cpp files changed (#21809)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-01-19 13:43:44 +08:00
congqixia
5f3d3dc4fc
generate-mockery for querycoordv2, querynode and rootcoort (#21714)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-01-16 13:59:42 +08:00
Jiquan Long
9fd9bed2b9
Remove collection meta after GC finished (#21595)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-01-12 09:55:42 +08:00
Jiquan Long
6d09bbed68
[skip e2e] Fix load meta migration (#21584)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-01-11 19:31:39 +08:00
Jiquan Long
ff2a68e65a
Fix collection not exist when tried to do recovery (#21471)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-01-04 16:37:35 +08:00
Jiquan Long
d7156812c1
Try using ASAN in ci ut (#21089)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-12-29 15:29:31 +08:00
Enwei Jiao
9b61170bf0
Fix compile proto failed (#20820)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-11-28 09:47:14 +08:00
Enwei Jiao
958e94f6f0
Use Conan as c++ package manager (#19920)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-11-23 10:39:11 +08:00
Jenny Li
fdefcff84a
Refactor Makefile to reduce duplicate steps for ci-ut (#20288)
Signed-off-by: Jenny Li <jing.li@zilliz.com>

Signed-off-by: Jenny Li <jing.li@zilliz.com>
2022-11-03 14:19:35 +08:00
Jenny Li
fc7e7256f8
Add install support to mac (#20261)
Signed-off-by: Jenny Li <jing.li@zilliz.com>

Signed-off-by: Jenny Li <jing.li@zilliz.com>
2022-11-02 15:39:35 +08:00
SimFG
e10d21ef90
Improve the download_milvus_proto bash (#19894)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-10-19 20:45:32 +08:00
Jenny Li
6f5ff61ac3
[skip e2e] Add publish migration image (#19869)
Signed-off-by: Jenny Li <jing.li@zilliz.com>

Signed-off-by: Jenny Li <jing.li@zilliz.com>
2022-10-18 18:49:26 +08:00
SimFG
7336d0580a
Fix the wrong file name (#19843)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-10-17 19:29:40 +08:00
SimFG
2b512a3371
Improve the making milvus process (#19833)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-10-17 19:19:25 +08:00
Jiquan Long
b0a12116c8
Fix meta migration tool (#19814)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-10-17 15:07:25 +08:00
SimFG
a55f739608
Separate public proto files (#19782)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-10-16 20:49:27 +08:00
Enwei Jiao
5e12a64f20
Fix make binlog (#19766)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-10-13 20:57:23 +08:00
Jenny Li
9aa307d851
Clean unnecessary deps to make milvus (#19711)
Signed-off-by: Jenny Li <jing.li@zilliz.com>

Signed-off-by: Jenny Li <jing.li@zilliz.com>
2022-10-12 14:33:23 +08:00
Jiquan Long
a8a074162f
Add meta migration tool (#19709)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-10-12 11:37:23 +08:00
SimFG
a85e44d169
Support the RELEASE param to update the milvus/api version dynamically (#19656)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-10-09 14:28:59 +08:00
zhenshan.cao
0ac4bc32a5
Add logic of autoindex (#19570)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-10-08 15:38:58 +08:00
xige-16
6466291953
Add make option to compile for disk index (#19530)
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-09-29 11:08:54 +08:00
XuanYang-cn
253fb514bd
Add Revert in DataCoord meta catalog (#19313)
See also: #19072

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-09-23 10:50:51 +08:00
zhenshan.cao
43d367b7d8
Add submodule unit test command entry (#19108)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-09-08 17:36:35 +08:00
Jenny Li
fc977cb90f
Fix git safe issue (#19101)
Signed-off-by: Jenny Li <jing.li@zilliz.com>

Signed-off-by: Jenny Li <jing.li@zilliz.com>
2022-09-08 17:14:36 +08:00
Enwei Jiao
aab060aeff
upgrade Go 1.18 in go.mod (#18165)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-08-01 14:28:33 +08:00
jaime
3ec147acfe
Upgrade confluent kafka go version to 1.9.1 (#18293)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-07-22 13:28:28 +08:00
Enwei Jiao
a97962b6da
fix make binlog (#18355)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-07-21 19:48:27 +08:00
Enwei Jiao
ea901393ef
fix windows ldflags (#17807)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-06-28 19:58:25 +08:00
Enwei Jiao
16c3aedc15
refine complie configuration (#17502)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-06-24 21:12:15 +08:00
Ten Thousand Leaves
65ab847815
Add segment reference lock for bulk load (#17457)
So we can support DataQueryable and DataIndexed states

issue: #16607
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-06-15 12:20:10 +08:00
Enwei Jiao
70324f937a
upgrade ruleguard and integrate into golang-ci (#17476)
Signed-off-by: Enwei Jiao <jiaoew2011@gmail.com>
2022-06-10 14:58:07 +08:00
jaime
6110e3a06d
Fix make clean (#17164)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-05-26 22:58:03 +08:00
zhenshan.cao
08a4dff41d
Add logic for search merging and a simple task scheduler for read tasks (#17022)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Co-authored-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: bigsheeper <yihao.dai@zilliz.com>

Co-authored-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: bigsheeper <yihao.dai@zilliz.com>
2022-05-23 16:41:58 +08:00
Ten Thousand Leaves
4855ba6fa3
Optimize Mac M1 support some tweaks to embedded Milvus (#16650)
issue: #16500
issue: #16621

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-26 14:21:46 +08:00
Ten Thousand Leaves
f93862fa2b
Fix build and tests for Mac M1, also enabled more tests for OSX systems (#16552)
issue: #16500

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-20 23:03:41 +08:00
Ten Thousand Leaves
6f75d02c65
Disable knowhere logging for embedded Milvus and some other tweaks (#16496)
/kind enhancement

issue: #15711
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-20 17:23:46 +08:00
Ten Thousand Leaves
e76a8c5ec2
Update compile options and config for embedded Milvus (#16472)
/kind enhancement

issue: #15711
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-14 19:57:34 +08:00
zhenshan.cao
7a3384734a
Add test coverage target in Makefile (#16439)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-04-11 09:49:34 +08:00
congqixia
4cb7c28bfa
Fix ReplicaInfo package name (#16400)
Fix #16399
PR 16373 breaks the build after ReplicaInfo moved from querypb to milvuspb
Also fix  problem in Makefile
Limit action/setup-go version to ~1.17.1(>=1.17.1 <1.18.0)

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-04-06 19:47:31 +08:00
Jiquan Long
fd589baca7
Integrates marisa trie index (#16192)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-04-01 15:31:29 +08:00
Ji Bin
3cd28420f1
Support compile under windows (#15786)
This patch makes compile milvus under windows(MSYS), including:
- some cpp adaptation for compile under msys/gcc-10.3
- install toolchain scripts for setup from MinGW/MSYS `scripts/install_deps_msys.sh`
- adaptation for POSIX API use in golang
  * using gofrs/flock instead of syscall.Flock
  * using x/exp/mmap instead of syscall.Mmap
- introducing github actions for build milvus.exe under windows/MSYS
- rocksdb's patch for MSYS
- adaptation for compile knowhere under windows
- a windows package script for pack zip file, `scripts/package_windows.sh`

issue #7706

Signed-off-by: Ji Bin <matrixji@live.com>
2022-03-17 17:17:22 +08:00
Ten Thousand Leaves
a3a29fed06
Add entry point for embedded Milvus. (#16008)
/kind feature

issue: #15711
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-03-16 16:51:22 +08:00
zhenshan.cao
f3eeecf146
Reduce the number of metrics (#16030)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-03-15 21:51:21 +08:00
zhenshan.cao
dff08dbf47
Refactor querynode unittest (#15929)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-03-08 17:39:58 +08:00
Nemo
dc65b1aeea
Switch to knowhere v1.0.1. Also switch to forked gorocksdb v0.0.1 (#15603)
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-02-17 19:05:49 +08:00
jaime
61d2063d5a
Fix make clean can't remove build directories (#15507)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-02-10 17:03:48 +08:00
jaime
307a8ce535
Support compile and run on Mac (#15491)
Co-authored-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: Cai Yudong <yudong.cai@zilliz.com>
Co-authored-by: Jenny Li <jing.li@zilliz.com>
Co-authored-by: Nemo <yuchen.gao@zilliz.com>
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: Cai Yudong <yudong.cai@zilliz.com>
Co-authored-by: Jenny Li <jing.li@zilliz.com>
Co-authored-by: Nemo <yuchen.gao@zilliz.com>
2022-02-09 14:27:46 +08:00
zhenshan.cao
6bc2efe429
Fixbug: IndexNode should panic when save meta failed to MetaKV (#15347)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-01-24 17:18:46 +08:00
Jiquan Long
30abe860fe
[skip e2e] Support to build Milvus in debug mode (#15342)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-01-21 19:51:38 +08:00
Bingyi Sun
9a564a7a5e
Upgrade pulsar client (#15175)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-01-14 17:13:35 +08:00
zhenshan.cao
b10c361822
Fix bug: should use Lock instead of RLock (#14797)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-01-04 23:33:24 +08:00
shaoyue
1c6b1f81e7
[skip e2e] Fix typo in Makefile rpm related sections (#14602)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2021-12-30 16:59:20 +08:00
shaoyue
d5b8ae7a00
[skip e2e] Add RPM build spec (#14153)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2021-12-28 09:52:10 +08:00
dragondriver
9051d64f47
Print build env infomation (#13300)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-15 10:21:10 +08:00
congqixia
5c19eef8ce
Add rocksdb compile step for ut (#13156)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-10 14:01:07 +08:00
Cai Yudong
255c45fb8e
Update test-cpp and test-go to run unittest only (#12903)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-12-08 10:55:04 +08:00
Xiangyu Wang
ad4c808207
Upgrade the golangci-lint version in Dockerfile (#12878)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-12-07 17:59:02 +08:00
Xiangyu Wang
c641e66f98
Upgrade golangci-lint to support more linters (#12784)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-12-06 18:26:08 +08:00
Xiangyu Wang
a37a1062e1
Remove unused dependency (#12534)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-12-02 10:53:32 +08:00
bigsheeper
921ef0f2ea
Fix seek query channel error (#10723)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-10-27 23:00:24 +08:00
zhenshan.cao
40750f03c7
Enhance querycoord log (#10523)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-10-24 15:16:00 +08:00
groot
c53f3b56fb
Gen cpp codecov info (#10072)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-10-18 12:32:35 +08:00
Xiangyu Wang
2eabcd8ebe
Update the target docker in Makefile (#9469)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-10-08 19:30:58 +08:00
zhenshan.cao
d1d5f9e409
Replace proto.MarshalTextString with proto.Marshal (#8542)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-09-29 20:26:00 +08:00
edward.zeng
7b955548a2
[skip ci] Refine makefile comments (#8841)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-09-29 14:41:58 +08:00
groot
77e88144fc
Update code coverage script (#8280)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-09-22 16:01:53 +08:00