Commit Graph

394 Commits

Author SHA1 Message Date
congqixia
e68f5cf08e
Fix integration tests logic (#24063)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-05-15 14:45:21 +08:00
wei liu
4c956fab73
enable config collection level rate limit (#24012)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-05-12 18:13:26 +08:00
yah01
7d0c47dd65
Check the length of JSON (#24035)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-05-12 11:33:20 +08:00
yah01
400364483d
Improve predict the resource usage (#23968)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-05-11 15:33:24 +08:00
congqixia
5aa9db0d38
Add collection level auto compaction enabled config (#24013)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-05-10 17:45:20 +08:00
cai.zhang
9715a850fa
Support expr with json field (#23804)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-05-10 10:19:19 +08:00
congqixia
de4b4dafef
Refresh etcd source config with Searializable option (#23954)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-05-09 11:52:40 +08:00
yihao.dai
fc010ee86b
Improve the metric of stored binlog size to the level of collection and segment (#23859)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-07 20:52:38 +08:00
Enwei Jiao
086f3bd748
Add it for refresh config (#23773)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-05-06 17:34:39 +08:00
congqixia
2cacda0d95
Fix Int8~Int32 empty result set type (#23852)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-05-06 14:16:39 +08:00
Jiquan Long
7be7e6f360
Refactor check logic of index parameters (#23856)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-05-06 10:40:39 +08:00
MrPresent-Han
b517bc9e6a
refine balance mechanism including:(#23454) (#23763) (#23791)
1. balance granuity to replica to avoid influence unrelated replicas
2. avoid balance back and forth

Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
2023-05-04 12:22:40 +08:00
yihao.dai
8c060215e9
Add collection number quota per DB (#23656)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-04-28 11:02:35 +08:00
smellthemoon
45fbe1d1a7
Change proxy max shard num (#23777)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-04-27 22:20:35 +08:00
Gao
e467e0d451
Seperate reading build and load params for diskann (#23758)
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2023-04-27 20:30:35 +08:00
yah01
eab94489ba
Refine the merge algorithm (#23767)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-04-27 18:26:35 +08:00
smellthemoon
912cf4ef0f
Change some configurations, include change the defaultChannelNum to 16 (#23617)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-04-27 14:26:35 +08:00
Xiaofan
016311ad48
Fix Has Collection Failed on datacoord (#23710)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-04-27 09:50:34 +08:00
yihao.dai
ed8836cd15
Add disk quota at the collection level (#23704)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-04-26 21:52:36 +08:00
SimFG
5cd21893c8
Fix superusers' password verification problem (#23733)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-04-26 21:16:34 +08:00
smellthemoon
f0ababb420
Use merr in querynodev2 (#23652)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-04-26 17:06:34 +08:00
foxspy
6f4ed517de
add growing segment index (#23615)
Signed-off-by: xianliang <xianliang.li@zilliz.com>
2023-04-26 10:14:41 +08:00
wei liu
4fb8919a97
support collection level rate limit (#22767)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-04-25 15:54:35 +08:00
yihao.dai
092d743917
Add support for getting vectors by ids (#23450)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-04-23 09:00:32 +08:00
zhenshan.cao
03ec804e68
Change default shard number to 1 (#23586)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2023-04-21 07:08:32 +08:00
congqixia
55d76f0cee
Fix rootcoord histgram metrics uses default buckets (#23556)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-04-20 14:08:30 +08:00
Enwei Jiao
967a97b9bd
Support json & array types (#23408)
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
2023-04-20 11:32:31 +08:00
Cai Yudong
2725d38b9e
Add COSINE metric type (#23350)
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
2023-04-20 10:20:31 +08:00
wei liu
4336ed8609
fix node up (#23415)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-04-20 09:52:31 +08:00
congqixia
4fe363c4b2
Add KNN cgo pool (#23526)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-04-19 23:56:31 +08:00
congqixia
eb690ef033
Fix querynodev2 stop logic (#23487)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-04-18 17:56:30 +08:00
aoiasd
695cc769df
Fix watch event timeout when IO Pool busy (#23293)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-04-17 11:02:30 +08:00
cqy123456
7ff6a3a246
Set query node thread pool size in milvus.yaml (#23286)
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2023-04-17 10:44:29 +08:00
congqixia
ba84f52119
Fix watcher loop quit and channel shouldDrop logic (#23402)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-04-14 09:54:28 +08:00
yah01
dc0eb433ae
Add IoPoolSize config (#23360)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-04-13 14:50:28 +08:00
Cai Yudong
178559a875
Ignore gather c metrics fail (#23366)
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
2023-04-13 10:26:27 +08:00
Cai Yudong
49eb4b8af4
Clean ANNOY params (#23353)
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
2023-04-11 17:54:35 +08:00
congqixia
5276529524
Make pkg code change trigger ci workflow (#23312)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-04-11 10:32:31 +08:00
Xiaofan
680ad482b7
Check balance checker chore to 10s (#23304)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-04-09 16:14:32 +08:00
dependabot[bot]
dbdd879de9
Bump golang.org/x/net from 0.6.0 to 0.7.0 in /pkg (#23256)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.6.0 to 0.7.0.
- [Release notes](https://github.com/golang/net/releases)
- [Commits](https://github.com/golang/net/compare/v0.6.0...v0.7.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-07 17:58:29 +08:00
Cai Yudong
55202ce7d9
Support show knowhere prometheus metrics (#23102)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2023-04-07 15:34:29 +08:00
jaime
c9d0c157ec
Move some modules from internal to public package (#22572)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-04-06 19:14:32 +08:00
Ten Thousand Leaves
a3a29fed06
Add entry point for embedded Milvus. (#16008)
/kind feature

issue: #15711
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-03-16 16:51:22 +08:00
zhenshan.cao
64295db471 Refact master and proxy and add etcdutil
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-10-15 21:31:50 +08:00
shengjh
3d7181617b Add ci and format code for proxy
Signed-off-by: shengjh <1572099106@qq.com>
2020-10-15 16:32:22 +08:00
zhenshan.cao
9c15fc550e Enable CICD
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-10-14 17:46:16 +08:00
xige-16
d7e6b99394 Add config reader
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-10-13 15:03:46 +08:00
neza2017
bb517eeca1 Add test for proxy-go
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-10-12 13:54:00 +08:00
bigsheeper
d4e485d37a Add pulsar authentication
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-10-10 11:06:07 +08:00
zhenshan.cao
70fc122338 Update proto
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-10-09 16:16:11 +08:00
neza2017
b0189c2450 Update time sync
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-10-09 15:37:24 +08:00
bigsheeper
5e429b2a94 Fix segment insertion error, and improve search strategy
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-09-26 17:37:40 +08:00
bigsheeper
567232b10f Create segments by query node num
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-09-26 15:29:29 +08:00
bigsheeper
649732c1b0 Fix search error about search time
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-09-25 14:53:06 +08:00
shengjh
3bd6c5417c Fix core dump
Signed-off-by: shengjh <1572099106@qq.com>
2020-09-25 11:46:30 +08:00
rain
90cd86de17 Update controller and fix kvbase bug
Signed-off-by: rain <boyan.wang@zilliz.com>
2020-09-25 10:23:48 +08:00
bigsheeper
f4d29c97ed Fix search timestamp error
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-09-24 14:19:29 +08:00
xige-16
d8b1a7cf8d Add config for write node and query node
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-09-22 21:10:51 +08:00
rain
bb03b9bfde Read config from file in master
Signed-off-by: rain <boyan.wang@zilliz.com>
2020-09-22 15:26:38 +08:00
rain
93b9a06b3b Fix the bug of segment
Signed-off-by: rain <boyan.wang@zilliz.com>
2020-09-22 10:56:19 +08:00
zhenshan.cao
a3635b9cdc Update proto
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-09-21 17:53:59 +08:00
rain
a95f6843aa Add create index interface and update the proto of master
Signed-off-by: rain <boyan.wang@zilliz.com>
2020-09-21 15:10:54 +08:00
FluorineDog
c123b57e98 Support Benchmark
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-09-21 10:32:20 +08:00
shengjh
8c260e8112 Fix bug and add meta wrapper
Signed-off-by: shengjh <1572099106@qq.com>
2020-09-19 17:10:23 +08:00
rain
6bc7e6d372 Update collection and segment id type and Refactor channel controller
Signed-off-by: rain <boyan.wang@zilliz.com>
2020-09-18 15:55:33 +08:00
FluorineDog
74c0e3542c Remove tbb
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-09-18 15:44:09 +08:00
FluorineDog
fe9040f326 Fix BruteForce, add restriction for indexing builder
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-09-18 11:03:28 +08:00
zhenshan.cao
3acd55f675 Fix config path error
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-09-18 01:53:18 +08:00
FluorineDog
a177f48577 Support Query Dispatch, enable offset for bitset
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-09-17 18:37:54 +08:00
rain
7f798c5fce Synchronous creation of collection instead of goroutine
Signed-off-by: rain <boyan.wang@zilliz.com>
2020-09-17 10:00:03 +08:00
bigsheeper
2474d06a10 Add index building of segment
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-09-16 18:19:29 +08:00
rain
fa6e228a91 Update the proto of master
Signed-off-by: rain <boyan.wang@zilliz.com>
2020-09-16 11:46:52 +08:00
rain
455b3ace2f Update segment struct and change collection to collection id
Signed-off-by: rain <boyan.wang@zilliz.com>
2020-09-16 10:40:04 +08:00
rain
11125bc6eb Implement the marshal logic when data to kv store
Signed-off-by: rain <boyan.wang@zilliz.com>
2020-09-15 17:55:02 +08:00
bigsheeper
d7056ce046 Fix test when search failed
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-09-15 15:53:10 +08:00
FluorineDog
b80de55ac8 Add Testcase for indexing
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-09-15 10:00:00 +08:00
rain
13ebbdfcca Delete some useless mock code
Signed-off-by: rain <boyan.wang@zilliz.com>
2020-09-14 11:26:35 +08:00
zhenshan.cao
be58ccb65e Delete unused
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-09-12 16:57:37 +08:00
shengjh
4cf6e079bc Add etcd watcher
Signed-off-by: shengjh <1572099106@qq.com>
2020-09-11 18:55:43 +08:00
cai.zhang
34b670f654 Add interface for collect result
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-09-11 10:43:08 +08:00
rain
c10cf9bf8f Update the collection proto and add the go files generated by proto
Signed-off-by: rain <boyan.wang@zilliz.com>
2020-09-10 19:55:03 +08:00
neza2017
be35568af1 Add unittest for timesync when there is only one proxy
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-09-10 19:12:23 +08:00
rain
ef25004205 Update the frequence of getting message of informer
Signed-off-by: rain <boyan.wang@zilliz.com>
2020-09-09 16:52:35 +08:00
bigsheeper
1370da9498 Add message sending and key2segment, use hard code schema
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-09-09 15:24:07 +08:00
shengjh
8f9a8d83e4 Enable timesyc
Signed-off-by: shengjh <1572099106@qq.com>
2020-09-09 14:34:03 +08:00
rain
e704667bc5 Finish all feature and update the README of master
Signed-off-by: rain <boyan.wang@zilliz.com>
2020-09-08 16:57:48 +08:00
cai.zhang
815a6ca1cf Add aggregation for query result
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-09-08 14:16:03 +08:00
rain
c9c7c78340 Fix bug that max time cant be marshaled
Signed-off-by: rain <boyan.wang@zilliz.com>
2020-09-08 10:01:30 +08:00
xige-16
81e70ee6c9 Hash message to partitioned topic
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-09-07 21:07:36 +08:00
zhenshan.cao
a90b725ba8 Add timestamp for key2segment message
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-09-07 19:46:14 +08:00
rain
642554e7bd Refactor directory structure and update the default value of server
Signed-off-by: rain <boyan.wang@zilliz.com>
2020-09-07 19:19:50 +08:00
zhenshan.cao
cae0f36bdd Add config of pulsar
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-09-07 16:56:57 +08:00
zhenshan.cao
8992611898 refactor proto logic
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-09-06 21:13:06 +08:00
bigsheeper
bb9c906ef6 Use go mod instead of GO_PATH and add more cgo interfeces
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-09-01 16:23:39 +08:00