Commit Graph

5007 Commits

Author SHA1 Message Date
xiyichan
ca129d4308
Support configurable msgstream (#16131)
Signed-off-by: xiyichan <2863768433@qq.com>
2022-03-24 10:15:25 +08:00
groot
478890a700
modify import proto (#16155)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-03-23 19:29:25 +08:00
Letian Jiang
321105cc01
Add column-based insert interface in segcore (#16100)
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-03-23 19:05:25 +08:00
groot
1c4b949a1d
modify import rpc interface (#16129)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-03-22 15:11:24 +08:00
groot
2078b24dca
Prepare for import (#16083)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-03-21 15:47:23 +08:00
Jiquan Long
f8d9bc919d
Unify interface of vector index & scalar index. (#15959)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-03-21 14:23:24 +08:00
XuanYang-cn
ad41e2a082
Tidy getQueryChannel for readability (#16096)
- Remove useless codes in query_coord.go
- Tidy slice initialization in LoadCollection task
- Add ut for getQueryChannel

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-03-21 11:25:24 +08:00
shaoyue
97e5d77953
implements restful entities api (#15916)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2022-03-21 11:21:23 +08:00
XuanYang-cn
15a3fe41c5
[skip e2e]Fix comments of exposed functions in QC (#16094)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-03-21 11:17:24 +08:00
Xiaofan
787d4f8bfd
Refine prometheus metrics and etcd log (#16084)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-03-18 15:51:22 +08:00
yah01
31c5b286da
Fix wrong unittest, check whether loading collection will cause OOM (#16068)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-03-18 13:17:22 +08:00
godchen
78557ca6d7
Replace minio kv with minio chunk manager (#15936)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-03-17 18:03:23 +08:00
Ji Bin
3cd28420f1
Support compile under windows (#15786)
This patch makes compile milvus under windows(MSYS), including:
- some cpp adaptation for compile under msys/gcc-10.3
- install toolchain scripts for setup from MinGW/MSYS `scripts/install_deps_msys.sh`
- adaptation for POSIX API use in golang
  * using gofrs/flock instead of syscall.Flock
  * using x/exp/mmap instead of syscall.Mmap
- introducing github actions for build milvus.exe under windows/MSYS
- rocksdb's patch for MSYS
- adaptation for compile knowhere under windows
- a windows package script for pack zip file, `scripts/package_windows.sh`

issue #7706

Signed-off-by: Ji Bin <matrixji@live.com>
2022-03-17 17:17:22 +08:00
Cai Yudong
d65cf5309f
Remove useless LogMgr (#16080)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-03-16 21:19:21 +08:00
edward.zeng
d4ad785b76
Bump go version in documentation (#15979)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2022-03-16 18:51:21 +08:00
Ten Thousand Leaves
a3a29fed06
Add entry point for embedded Milvus. (#16008)
/kind feature

issue: #15711
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-03-16 16:51:22 +08:00
Bingyi Sun
256ccb8f65
Fix rootcoord goroutine leak (#16071)
issue: #15658
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-03-16 16:47:21 +08:00
Jiquan Long
ffdad82b9c
Set load info in reschedule tasks (#16065)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-03-16 11:43:21 +08:00
jaime
cf4c014b2e
Remove collect query result by mq (#15988)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-03-16 10:19:21 +08:00
zhenshan.cao
f3eeecf146
Reduce the number of metrics (#16030)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-03-15 21:51:21 +08:00
Jiquan Long
c984c4ea73
Set load meta info in load balance task (#16060)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-03-15 20:37:21 +08:00
congqixia
b61e8201e7
Refactor querynode historical replica (#15992)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-03-15 19:15:20 +08:00
Xiaofan
c51c8c0a74
Fix get flused segment errror handling (#15155)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-03-15 18:51:21 +08:00
Bingyi Sun
d3d5a41b76
Fix memory leak casued by unclosed channel in msgstream (#16051)
issue: #16045
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-03-15 15:55:21 +08:00
zhenshan.cao
0bd07daae3
Add grpc healthy check (#16050)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-03-15 15:05:21 +08:00
jaime
97b1ed7bca
Remove reader API from Mqstream and replace it with consumer API (#15971)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-03-15 14:45:22 +08:00
XuanYang-cn
d4a0c5eba6
Alter comments in parallel.go for readability (#16032)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-03-15 10:26:01 +08:00
xige-16
09b2c46b61
Support create collection with VarChar field (#15904)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-03-14 23:20:02 +08:00
bigsheeper
caf9cbfcd4
Fix load failure and remove parition release related code (#16038)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-03-14 23:18:02 +08:00
congqixia
fab0263616
Use RmqFactory in DataNode unit test (#16039)
Use Rocksmq instead of Pulsar in DataNode unittest
- Avoid pulsar seek problem caused timeout issue
- Remove third party component dependency in unit test

See also: #15986

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-03-14 19:22:01 +08:00
yah01
d4225117ee
Enable Zstd compression for binlog (#15840)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-03-14 16:00:01 +08:00
congqixia
6a66faa2fb
Explictly specify Load operation meta (#16027)
Fix #15996

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-03-14 09:50:01 +08:00
jaime
29975a7a26
Remove Consume() API from mqstream interface (#15886)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-03-11 20:09:59 +08:00
groot
89356ebcb9
Modify import rpc interfaces (#16006)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-03-11 17:13:59 +08:00
yah01
7d2934e4c2
Add methods to compress/decompress small blocks (#15980)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-03-11 14:55:59 +08:00
Jiquan Long
3121619758
Chunk manager support scalar data (#16010)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-03-11 14:39:59 +08:00
yah01
4f3d5fb6eb
Make load operation parallel improve performance (#15928)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-03-11 14:28:01 +08:00
Letian Jiang
8f52e5b6c7
Add unit tests in segcore (#15960)
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-03-10 16:33:59 +08:00
Ten Thousand Leaves
d5bae6710f
Make all logs configurable for standalone/embedded Milvus. (#15926)
Resolves: #15919, #15708
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-03-10 10:35:59 +08:00
groot
5fdef607d5
Add rpc interfaces for import (#15930)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-03-09 18:33:59 +08:00
Xiaofan
d013134b44
Enhance error message when coordinator not healthy (#15950)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-03-09 15:57:58 +08:00
XuanYang-cn
c2fe12bc97
Handle error in topic creation of rocksmq (#15922)
Reopen for: #14429

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-03-09 15:47:59 +08:00
congqixia
6513be5c44
Extend watch/release channel logic in DataNode (#15925)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-03-09 15:39:58 +08:00
bigsheeper
9ad5c14a4a
Return error when create message stream failed in QueryNode (#15317)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-03-09 15:17:59 +08:00
顾炎
4f8c08f912
fix checkSegmentSize overflows int64 (#15858)
Signed-off-by: guyan <632722781@qq.com>
2022-03-09 13:29:58 +08:00
yah01
565524ed3c
Add zstd compressor in util (#15779)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-03-08 18:17:58 +08:00
zhenshan.cao
dff08dbf47
Refactor querynode unittest (#15929)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-03-08 17:39:58 +08:00
Letian Jiang
74f66dce3b
Fix passing empty binlog content in compaction (#15909) (#15927)
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-03-08 16:34:01 +08:00
bigsheeper
6fec7ed6fb
Add search benchmark in QueryNode (#15854)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-03-08 10:15:59 +08:00
yah01
7124e91d53
Refactor assigning loading segments task in QueryCoord (#15895)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-03-07 19:23:58 +08:00