Commit Graph

940 Commits

Author SHA1 Message Date
xige-16
205c92e54b
Support insert string data (#15993)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-03-25 14:27:25 +08:00
Cai Yudong
f4ebd3a9ce
Upgrade to knowhere v1.1.0 (#16186)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-03-25 13:49:25 +08:00
Ten Thousand Leaves
a23ca745bb
Add Etcd support for Import Tasks. (#16164)
issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-03-25 11:03:25 +08:00
bigsheeper
2fcf7b9760
Fix GetMemoryCount, get min as total memory (#15931)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-03-24 20:41:25 +08:00
xiyichan
ca129d4308
Support configurable msgstream (#16131)
Signed-off-by: xiyichan <2863768433@qq.com>
2022-03-24 10:15:25 +08:00
groot
1c4b949a1d
modify import rpc interface (#16129)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-03-22 15:11:24 +08:00
Jiquan Long
f8d9bc919d
Unify interface of vector index & scalar index. (#15959)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-03-21 14:23:24 +08:00
Xiaofan
787d4f8bfd
Refine prometheus metrics and etcd log (#16084)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-03-18 15:51:22 +08:00
Ji Bin
3cd28420f1
Support compile under windows (#15786)
This patch makes compile milvus under windows(MSYS), including:
- some cpp adaptation for compile under msys/gcc-10.3
- install toolchain scripts for setup from MinGW/MSYS `scripts/install_deps_msys.sh`
- adaptation for POSIX API use in golang
  * using gofrs/flock instead of syscall.Flock
  * using x/exp/mmap instead of syscall.Mmap
- introducing github actions for build milvus.exe under windows/MSYS
- rocksdb's patch for MSYS
- adaptation for compile knowhere under windows
- a windows package script for pack zip file, `scripts/package_windows.sh`

issue #7706

Signed-off-by: Ji Bin <matrixji@live.com>
2022-03-17 17:17:22 +08:00
edward.zeng
d4ad785b76
Bump go version in documentation (#15979)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2022-03-16 18:51:21 +08:00
zhenshan.cao
f3eeecf146
Reduce the number of metrics (#16030)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-03-15 21:51:21 +08:00
zhenshan.cao
0bd07daae3
Add grpc healthy check (#16050)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-03-15 15:05:21 +08:00
XuanYang-cn
d4a0c5eba6
Alter comments in parallel.go for readability (#16032)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-03-15 10:26:01 +08:00
xige-16
09b2c46b61
Support create collection with VarChar field (#15904)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-03-14 23:20:02 +08:00
jaime
29975a7a26
Remove Consume() API from mqstream interface (#15886)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-03-11 20:09:59 +08:00
groot
89356ebcb9
Modify import rpc interfaces (#16006)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-03-11 17:13:59 +08:00
yah01
7d2934e4c2
Add methods to compress/decompress small blocks (#15980)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-03-11 14:55:59 +08:00
Ten Thousand Leaves
d5bae6710f
Make all logs configurable for standalone/embedded Milvus. (#15926)
Resolves: #15919, #15708
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-03-10 10:35:59 +08:00
groot
5fdef607d5
Add rpc interfaces for import (#15930)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-03-09 18:33:59 +08:00
yah01
565524ed3c
Add zstd compressor in util (#15779)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-03-08 18:17:58 +08:00
Jiquan Long
f71651e294
Support column-based insert data in message stream (#15802)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-03-04 15:09:56 +08:00
Cai Yudong
3e862ecdfb
Merge msgChannelConfig and knowhereConfig into commonConfig (#15843)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-03-04 11:17:56 +08:00
jaime
7a3a721380
Reconstruct mqstream module (#15784)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-03-03 21:57:56 +08:00
Cai Yudong
92c8e32ebd
Let MemoryKV.Load return error when key not exist (#15814)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-03-02 18:51:55 +08:00
XuanYang-cn
e394ba8c88
Refactor rocksmq comments and logs for readability (#15801)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-03-01 18:55:55 +08:00
Jiquan Long
001567640e
Fix shallow copy of typeutil.AppendFieldData (#15790)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-03-01 15:13:54 +08:00
Bingyi Sun
d6857f4b24
Remove globalZapWrapper in logutil.go (#15726)
issue: #15720

Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-02-28 17:23:54 +08:00
yah01
867cf620ee
Add Extract() method to take out C pointer from BytesConverter without releasing it (#15614)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2022-02-27 09:47:53 +08:00
Nemo
cdcb3627b4
Make Etcd log level configurable. (#15737)
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-02-25 15:03:53 +08:00
godchen
fcf0887d92
Add and implement chunk manager interface. (#15541)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-02-24 23:53:53 +08:00
shaoyue
83fdc1da7c
Add web server in proxy with sample handler (#15647)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2022-02-23 14:37:52 +08:00
congqixia
c7f55c2e34
Make SessionWatch keep watch even no Rewatch func when find ErrCompacted (#15497)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-02-15 15:07:48 +08:00
yah01
684110bc9a
Avoid to copy during converting C bytes to Go bytes (#15551)
* Avoid to copy during converting C bytes to Go bytes

Signed-off-by: yah01 <yah2er0ne@outlook.com>

* Add BytesConverter to achieve no-copy converting C bytes to Go bytes

Signed-off-by: yah01 <yah2er0ne@outlook.com>
2022-02-15 14:17:48 +08:00
Xiaofan
be42627ab2
Move Logutil to Util dir (#15447)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-02-09 18:55:46 +08:00
Cai Yudong
8255b713ac
Move ProxySubName/QueryNodeSubName/DataNodeSubName to msgChannelConfig (#15501)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-02-09 14:41:45 +08:00
jaime
307a8ce535
Support compile and run on Mac (#15491)
Co-authored-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: Cai Yudong <yudong.cai@zilliz.com>
Co-authored-by: Jenny Li <jing.li@zilliz.com>
Co-authored-by: Nemo <yuchen.gao@zilliz.com>
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: Cai Yudong <yudong.cai@zilliz.com>
Co-authored-by: Jenny Li <jing.li@zilliz.com>
Co-authored-by: Nemo <yuchen.gao@zilliz.com>
2022-02-09 14:27:46 +08:00
xige-16
55cdc5aa35
Get indexInfo and segment size by queryCoord (#14207)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-02-08 21:57:46 +08:00
Cai Yudong
1f337891e1
Rename base_param to service_param, rename global_param to component_param (#15463)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-02-08 20:57:47 +08:00
JunSung Park
f6873d3dc1
Implement automatic item expiration on compaction (#14418)
* Implement automatic item expiration on compaction

Signed-off-by: JunSung Park <writtic@gmail.com>

* Change entityExpiration value comparison logic

Signed-off-by: JunSung Park <writtic@gmail.com>

* Add logging for the number of expired entity

Signed-off-by: JunSung Park <writtic@gmail.com>

* Set default value of CompactionEntityExpiration to off

Signed-off-by: JunSung Park <writtic@gmail.com>

* Fix errors for code checker

Signed-off-by: JunSung Park <writtic@gmail.com>

* Set default entityExpiration to max value

Signed-off-by: JunSung Park <writtic@gmail.com>
2022-02-07 22:45:46 +08:00
Cai Yudong
f1ea9613ae
Add EtcdConfig and move PulsarConfig/RocksdbConfig/MinioConfig into BaseParamTable (#15434)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-02-07 10:09:45 +08:00
Cai Yudong
4bccc85534
Use MsgChannelConfig in GlobalParams for all components (#15163)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-02-02 00:35:43 +08:00
Bingyi Sun
515fe962d5
Fix subscription path (#15373)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-01-25 13:27:44 +08:00
Bingyi Sun
3987574448
Unsubscribe topic in DataCoord (#15353)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-01-24 19:47:38 +08:00
xige-16
1cd2363bd8
Delete usless params SkipQueryChannelRecovery (#15289)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-01-21 21:43:40 +08:00
Bingyi Sun
c877f33084
Downgrade pulsar client (#15325)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-01-20 22:25:38 +08:00
congqixia
1e61112b5a
Add retry logic in pulsar consumer unsubscribe (#15284)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-01-20 01:23:37 +08:00
bigsheeper
7f775096c5
Unsubscribe when close pulsar consumer (#15221)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-01-17 18:53:36 +08:00
xige-16
dfc6670f0f
Unsubscribe channel after query node down (#15230)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-01-17 17:37:37 +08:00
Jiquan Long
675e6d352b
QueryNodes send search/query results by rpc (#15223)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-01-17 14:41:35 +08:00
zhenshan.cao
1601a61b78
Move logic of checking available port to service (#15222)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-01-14 21:17:34 +08:00