Commit Graph

5042 Commits

Author SHA1 Message Date
Jiquan Long
cd0f3cbcd3
Refine log (#16269)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-03-29 22:43:28 +08:00
cai.zhang
e2e345079f
Add indexMeta version for IndexCoord log (#16266)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-03-29 21:03:27 +08:00
jaime
d5d6e115bd
Refine delete segment data log (#16241)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-03-29 18:41:27 +08:00
Jiquan Long
3219b4e392
Skip dimension check for scalar index (#16257)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-03-29 15:35:27 +08:00
Cai Yudong
1160693e4c
Fix centos build error (#16251)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-03-29 15:13:30 +08:00
aakejiang
de70dfe0c3
refine log (#16256)
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
2022-03-29 15:05:27 +08:00
congqixia
91d5436807
Improve log level and param function in QueryCollection (#16254)
Use debug level for all debug information
Use specific parameter function for appliable ones

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-03-29 12:31:27 +08:00
zhenshan.cao
11cea5aad0
Refine proxy log (#16246)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-03-29 02:29:27 +08:00
Cai Yudong
2ac53c89e9
Revert #16242 (#16247)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-03-29 00:53:27 +08:00
XuanYang-cn
23b3a9ed8a
Log state in datanode handleWatchInfo (#16231)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-03-29 00:15:28 +08:00
Cai Yudong
9a6b2b8806
Fix centos build error (#16242)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-03-28 23:23:27 +08:00
bigsheeper
9d673006e0
Improve primary keys print (#16232)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-03-28 23:15:27 +08:00
xige-16
d2424fc50a
Print msgID instead of loadInfo when load segment (#16238)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-03-28 23:13:27 +08:00
congqixia
2c03287e79
Improve log level in proxy QueryTask (#16234)
Use warn instead of debug in error case

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-03-28 23:11:26 +08:00
XuanYang-cn
a77dd1034c
Adopt the extended state in DataCoord (#16200)
ChannelWatchInfo with ToWatch, ToRelease will trigger a timer.
ChannelManager now reacts to different ChannelWatch states.

- WatchSuccess > log this info

- WatchFailure/WatchTimeout > ToRelease

- ReleaseSuccess > Delete, reassign if not from DropCollection

- ReleaseFailure/ReleaseTimeout > Cleanup subscription and Delete,
  reassgin if not from DropCollection.

Some Notes:
1. Reassignment will add this channel to buffer if there's only one node.

See also: #15846

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-03-28 22:33:27 +08:00
Jiquan Long
973ea406ca
Improve log of show collections (#16233)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-03-28 21:47:27 +08:00
cai.zhang
c8688a6275
Organize log for GetIndexFilePaths (#16230)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-03-28 21:27:27 +08:00
Ji Bin
4f08f6d8c6
Workaround for knowhere under windows (#16214)
issue #16205

Signed-off-by: Ji Bin <matrixji@live.com>
2022-03-28 17:41:27 +08:00
yah01
178eb8cce1
Modify proto to support replicas (#16218)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-03-28 16:43:27 +08:00
Ten Thousand Leaves
40c703dacd
Support check index state in root coord, for bulk load feature. (#16198)
issue: #15604

/kind feature

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-03-28 16:41:28 +08:00
Bingyi Sun
e0ffe54547
Add interface to get replications by collection ID (#16213)
issue: #16158
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-03-28 14:35:27 +08:00
Cai Yudong
3e3756d3dc
Upgrade to knowhere v1.1.1 to support Mac M1 build (#16210)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-03-28 14:33:26 +08:00
godchen
478f6ca11e
Add cache for vector chunk (#15912)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-03-26 22:05:26 +08:00
xige-16
205c92e54b
Support insert string data (#15993)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-03-25 14:27:25 +08:00
yah01
227889b0d0
Add replica interfaces for Meta and Cluster (#16190)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-03-25 14:03:25 +08:00
Cai Yudong
f4ebd3a9ce
Upgrade to knowhere v1.1.0 (#16186)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-03-25 13:49:25 +08:00
XuanYang-cn
a00968fdfe
Fix unregister in datacoord cluster (#16182)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-03-25 12:09:25 +08:00
yah01
0d16ecf0dd
Add protobuf messages to support multi memory replica (#16153)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-03-25 11:13:25 +08:00
Ten Thousand Leaves
a23ca745bb
Add Etcd support for Import Tasks. (#16164)
issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-03-25 11:03:25 +08:00
bigsheeper
2fcf7b9760
Fix GetMemoryCount, get min as total memory (#15931)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-03-24 20:41:25 +08:00
yah01
9deb245d77
Fix data race in unittest (#16165)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-03-24 20:19:25 +08:00
bigsheeper
7fee4743f5
Improve log print in OOM error info (#16170)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-03-24 20:17:25 +08:00
Jiquan Long
48706f416f
Migrate scalar index from knowhere (#16174)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-03-24 14:57:26 +08:00
congqixia
661469445c
Copy index file content to c memory (#16162)
Might be the root cause of #15896
The index content bytes were passed via unsafe.Pointer,
which shall be copy to cpp managed memory inmediately

This solution might cause a lot of memory consumption.
Maybe we need to store the index bytes in cpp memory in the first place.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-03-24 14:55:25 +08:00
yah01
63a71e247a
Loading segments concurrency level fallbacks to the value that won't cause OOM (#16172)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-03-24 13:39:25 +08:00
xiyichan
ca129d4308
Support configurable msgstream (#16131)
Signed-off-by: xiyichan <2863768433@qq.com>
2022-03-24 10:15:25 +08:00
groot
478890a700
modify import proto (#16155)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-03-23 19:29:25 +08:00
Letian Jiang
321105cc01
Add column-based insert interface in segcore (#16100)
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-03-23 19:05:25 +08:00
groot
1c4b949a1d
modify import rpc interface (#16129)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-03-22 15:11:24 +08:00
groot
2078b24dca
Prepare for import (#16083)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-03-21 15:47:23 +08:00
Jiquan Long
f8d9bc919d
Unify interface of vector index & scalar index. (#15959)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-03-21 14:23:24 +08:00
XuanYang-cn
ad41e2a082
Tidy getQueryChannel for readability (#16096)
- Remove useless codes in query_coord.go
- Tidy slice initialization in LoadCollection task
- Add ut for getQueryChannel

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-03-21 11:25:24 +08:00
shaoyue
97e5d77953
implements restful entities api (#15916)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2022-03-21 11:21:23 +08:00
XuanYang-cn
15a3fe41c5
[skip e2e]Fix comments of exposed functions in QC (#16094)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-03-21 11:17:24 +08:00
Xiaofan
787d4f8bfd
Refine prometheus metrics and etcd log (#16084)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-03-18 15:51:22 +08:00
yah01
31c5b286da
Fix wrong unittest, check whether loading collection will cause OOM (#16068)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-03-18 13:17:22 +08:00
godchen
78557ca6d7
Replace minio kv with minio chunk manager (#15936)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-03-17 18:03:23 +08:00
Ji Bin
3cd28420f1
Support compile under windows (#15786)
This patch makes compile milvus under windows(MSYS), including:
- some cpp adaptation for compile under msys/gcc-10.3
- install toolchain scripts for setup from MinGW/MSYS `scripts/install_deps_msys.sh`
- adaptation for POSIX API use in golang
  * using gofrs/flock instead of syscall.Flock
  * using x/exp/mmap instead of syscall.Mmap
- introducing github actions for build milvus.exe under windows/MSYS
- rocksdb's patch for MSYS
- adaptation for compile knowhere under windows
- a windows package script for pack zip file, `scripts/package_windows.sh`

issue #7706

Signed-off-by: Ji Bin <matrixji@live.com>
2022-03-17 17:17:22 +08:00
Cai Yudong
d65cf5309f
Remove useless LogMgr (#16080)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-03-16 21:19:21 +08:00
edward.zeng
d4ad785b76
Bump go version in documentation (#15979)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2022-03-16 18:51:21 +08:00