Commit Graph

248 Commits

Author SHA1 Message Date
Xiaofan
a9b1d71a8f
change the etcd compareAndSwap interface (#18068)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-07-06 13:54:21 +08:00
cai.zhang
ef0fc37ad7
Fixed the bug that IndexCoord lost some meta information (#17818)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-27 21:52:17 +08:00
Enwei Jiao
16c3aedc15
refine complie configuration (#17502)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-06-24 21:12:15 +08:00
Xiaofan
327bd6abbe
Support Txn interface in rockskv (#17027)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-05-16 19:17:55 +08:00
Eng Zer Jun
5375161777
test: use T.TempDir to create temporary test directory (#16758)
This commit replaces `ioutil.TempDir` with `t.TempDir` in tests. The
directory created by `t.TempDir` is automatically removed when the test
and all its subtests complete.

Prior to this commit, temporary directory created using `ioutil.TempDir`
needs to be removed manually by calling `os.RemoveAll`, which is omitted
in some tests. The error handling boilerplate e.g.
	defer func() {
		if err := os.RemoveAll(dir); err != nil {
			t.Fatal(err)
		}
	}
is also tedious, but `t.TempDir` handles this for us nicely.

Fixes: #16759
Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-05-04 11:35:50 +08:00
jaime
68b1b82faf
Remove DataKV interface (#16692)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-04-28 21:03:47 +08:00
jaime
c0a73dbe98
Using a temporary dir instead of a specified dir within embed etcd test (#16694)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-04-28 11:17:47 +08:00
Ten Thousand Leaves
e76a8c5ec2
Update compile options and config for embedded Milvus (#16472)
/kind enhancement

issue: #15711
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-14 19:57:34 +08:00
jaime
ed3c29eae7
Support Kafka (#16149)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-04-12 19:47:33 +08:00
Ten Thousand Leaves
1e6f6bd072
Implement set segment state call from RootCoord to DataCoord. (#16374)
/kind feature

issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-06 15:33:32 +08:00
aakejiang
de70dfe0c3
refine log (#16256)
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
2022-03-29 15:05:27 +08:00
Ten Thousand Leaves
a23ca745bb
Add Etcd support for Import Tasks. (#16164)
issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-03-25 11:03:25 +08:00
godchen
78557ca6d7
Replace minio kv with minio chunk manager (#15936)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-03-17 18:03:23 +08:00
congqixia
a5c3f4eef4
Add CompareFailErr for CAS operation error check (#15893)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-03-07 18:01:58 +08:00
Letian Jiang
08d12e62f3
Add ByteSlice Method for embed etcd (#15844)
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-03-04 15:14:01 +08:00
Cai Yudong
92c8e32ebd
Let MemoryKV.Load return error when key not exist (#15814)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-03-02 18:51:55 +08:00
Letian Jiang
442e791128
Add ByteSlice Method for memkv (#15774)
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-03-02 17:29:56 +08:00
yah01
d67c801511
Add bytes methods for RocksdbKV (#15749)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-02-25 17:19:53 +08:00
yah01
1a9933a7c3
Add methods to load/save bytes for MinIOKV (#15732)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-02-25 16:21:53 +08:00
congqixia
a05df77e96
Remove unused ValueKV (#15748)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-02-25 14:47:52 +08:00
Letian Jiang
b56ec7ea97
Add ByteSlice Method for Etcd-kv (#15738)
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-02-25 11:25:53 +08:00
Nemo
3eb074a9b0
Update some comments for data coord with some minor code refactoring. (#15686)
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-02-22 13:15:51 +08:00
jaime
87dd4e8f92
Fix clean up data dir failed for TestEtcdRestartLoad (#15506)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-02-09 18:37:45 +08:00
jaime
307a8ce535
Support compile and run on Mac (#15491)
Co-authored-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: Cai Yudong <yudong.cai@zilliz.com>
Co-authored-by: Jenny Li <jing.li@zilliz.com>
Co-authored-by: Nemo <yuchen.gao@zilliz.com>
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: Cai Yudong <yudong.cai@zilliz.com>
Co-authored-by: Jenny Li <jing.li@zilliz.com>
Co-authored-by: Nemo <yuchen.gao@zilliz.com>
2022-02-09 14:27:46 +08:00
Cai Yudong
1f337891e1
Rename base_param to service_param, rename global_param to component_param (#15463)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-02-08 20:57:47 +08:00
Cai Yudong
f1ea9613ae
Add EtcdConfig and move PulsarConfig/RocksdbConfig/MinioConfig into BaseParamTable (#15434)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-02-07 10:09:45 +08:00
congqixia
69def6b34e
[skip e2e] Add comment and implementation assertion for EmbedEtcdKV (#14905)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-01-06 13:32:27 +08:00
Xieql
4c043fd86d
[skip e2e] Improve annotation (#14653)
Signed-off-by: Xieql <qianglong.xie@zilliz.com>
2022-01-04 09:07:20 +08:00
Xiaofan
98e4ff33a8
Fix RocksDB Slow (#14614) 2021-12-30 20:08:15 +08:00
congqixia
07cf4f9ffc
Fix zero length declaration in rocksdbkv (#14576)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-30 19:13:28 +08:00
Xiaofan
69087ff8dd
Support embedded etcd (#14333)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-12-29 14:35:21 +08:00
Xiaofan
029b153259
Remove data directory for test (#14363)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-12-29 10:08:47 +08:00
cxytz01
e47a651be4
[skip e2e] Grammar correction (#14057)
Signed-off-by: cxytz01 <xueyou.chen@zilliz.com>
2021-12-23 14:41:52 +08:00
Xiaofan
02d1282949
Fix RocksMQ UT (#13819)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-12-22 11:39:03 +08:00
Xieql
42c6088911
[skip e2e] improve annotation (#13836)
Signed-off-by: Xieql <qianglong.xie@zilliz.com>
2021-12-21 11:53:14 +08:00
jaime
1a030bba59
[skip e2e] Fix grammar mistake in minio_kv (#13677)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: yun.zhang <yun.zhang@zilliz.com>
2021-12-19 12:34:51 +08:00
Xiaofan
b6cca25d1f
Refine rocksdb (#12743)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-12-17 23:44:42 +08:00
jaime
0a99e1aeba
[skip ci] Fix go format error (#13212)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: yun.zhang <yun.zhang@zilliz.com>
2021-12-11 22:23:13 +08:00
yukun
197bcbba08
[skip ci]Format rocksdbkv log (#13191)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-12-10 21:16:04 +08:00
yukun
f6ec741114
[skip ci]Format rocksdbkv log (#13190)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-12-10 21:14:14 +08:00
jaime
ea7d8be913
[skip ci] Refine error message (#11728)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: yun.zhang <yun.zhang@zilliz.com>
2021-12-10 16:50:52 +08:00
yukun
c71b4d32f6
[skip ci]Format rocksdbkv log (#13094)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-12-10 10:08:45 +08:00
yukun
506ca42c5a
[skip ci]Format rocksdbkv log (#13093)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-12-10 10:06:31 +08:00
jaime
7691876fff
[skip ci] Fix golint error in mem_kv (#13020)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: yun.zhang <yun.zhang@zilliz.com>
2021-12-09 10:49:25 +08:00
jaime
efaca29ee4
[skip ci] fix a grammar error of the comment in etcd_kv (#13019)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: yun.zhang <yun.zhang@zilliz.com>
2021-12-09 09:23:08 +08:00
shaoyue
34fadc9ab6
[skip ci] Add comment for EmbedEtcdKV (#12964)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2021-12-08 15:11:33 +08:00
edward.zeng
d3c23ca5e9
[skip ci] Refine etcd kv comment (#12872)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-07 15:55:03 +08:00
edward.zeng
d1c7a1574d
[skip ci] Refine rocksdb kv comment (#12867)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-07 14:47:37 +08:00
jaime
e4746e8c6e
[skip ci] Add a function comment for LoadWithPrefix2 (#12828)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: yun.zhang <yun.zhang@zilliz.com>
2021-12-07 09:42:15 +08:00
edward.zeng
1d2311d4da
[skip ci] Refine minio kv comment (#12724)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-03 19:21:33 +08:00
shaoyue
bbdd1dbef9
[skip ci] Add comment for etcd_kv.go (#12449)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2021-11-30 16:58:14 +08:00
shaoyue
f00ee2a3b1
[skip ci] Add comment for mem_kv.go (#12448)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2021-11-30 16:08:59 +08:00
edward.zeng
c9d31f0b73
[skip ci] Refine etcd kv comment (#12374)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-11-29 19:25:49 +08:00
edward.zeng
772857d9c1
[skip ci] Refine minio kv comment (#12369)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-11-29 17:55:41 +08:00
bigsheeper
18463ebde4
Reduce memory cost when load from minio (#12055)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-11-23 14:49:15 +08:00
edward.zeng
3cfc740a05
Refine minio kv FGetObject function (#12078)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-11-21 22:53:13 +08:00
edward.zeng
8f40450d93
[skip ci] Refine rocksdb kv comment (#12093)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-11-18 19:27:13 +08:00
edward.zeng
a92a6f34b4
[skip ci] Fix typo in etcd kv comment (#12091)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-11-18 19:25:19 +08:00
edward.zeng
77264ae493
[skip ci] Refine etcd kv comment (#12090)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-11-18 19:23:25 +08:00
yukun
3b51232d59
[skip ci]Format rocksdbkv (#12034)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-11-17 22:03:24 +08:00
yukun
18b9f7810c
[skip ci]Format rocksdbkv (#12033)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-11-17 22:01:31 +08:00
congqixia
b7de3bda3b
Fix mem_kv.go slice 0 length declaration (#11996)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-17 20:57:17 +08:00
edward.zeng
2b566eb622
[skip ci] Refine comments in mem kv (#12006)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-11-17 19:27:27 +08:00
edward.zeng
bcb7cce026
[skip ci] Refine comment in minio kv (#12004)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-11-17 19:25:20 +08:00
edward.zeng
776a01b5b9
[skip ci] Add comment to embed etcd kv (#12003)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-11-17 19:23:25 +08:00
shaoyue
6e5334ab11
[skip ci] Add comments for exported funcs in mem_kv.go (#11979)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2021-11-17 16:11:16 +08:00
shaoyue
585d9be439
[skip ci] Add comments for exported funcs in etcd_kv.go (#11975)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2021-11-17 16:09:23 +08:00
shaoyue
6542b092b2
[skip ci] Add comments of exported items in etcd_kv.go (#11884)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2021-11-16 17:41:11 +08:00
Xiangyu Wang
12f50cb22c
[skip ci]Update OWNERS files (#11898)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-11-16 15:41:11 +08:00
shaoyue
53c29088b0
[skip ci] Add comment for exported funcs in mem_key.go (#11877)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2021-11-16 13:23:10 +08:00
yukun
5e19629b3b
[skip ci]Format rocksdbkv (#11854)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-11-16 09:23:45 +08:00
yukun
36966504b1
[skip ci]Format rocksdbkv (#11853)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-11-16 09:21:48 +08:00
yukun
8593d75fd1
[skip ci]Format rocksdbkv (#11852)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-11-16 09:19:55 +08:00
XuanYang-cn
72c0711919
Refine minio kv test (#11255)
This PR refines the minio_kv_test, makes sure the minIO test
is in the scope of configed rootPath

See also: #11237

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-05 23:05:10 +08:00
Xiaofan
2b0f1a14ac
Remove secretkey and accesskey log print for MinIO (#11130)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-11-03 13:04:09 +08:00
dragondriver
232f75a001
[skip ci] Fix golint warning of kv (#11087)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-11-02 21:40:12 +08:00
dragondriver
c208363b0d
[skip ci] Fix golint warning of MinIOKV (#10913)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-10-29 17:51:28 +08:00
XuanYang-cn
41f1378a01
[skip ci]Update license for kv kv (#10896)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-10-29 13:15:04 +08:00
XuanYang-cn
848fa77d13
[skip ci]Update license for kv rocksdb (#10895)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-10-29 13:13:15 +08:00
XuanYang-cn
005d177f68
[skip ci]Update license for kv etcdkv (#10894)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-10-29 13:11:24 +08:00
XuanYang-cn
978229517d
[skip ci]Update license for kv mem (#10893)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-10-29 13:09:33 +08:00
XuanYang-cn
08cd25c14f
[skip ci]Update license for kv minio (#10892)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-10-29 13:07:43 +08:00
XuanYang-cn
8885c510ef
[skip ci]Update license for kv etcd (#10891)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-10-29 12:12:44 +08:00
congqixia
95b8953bac
Add kv Value interface (#10469)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-29 00:16:38 +08:00
zhenshan.cao
5320698e29
[skip ci]Fix golint error in mem_kv (#10862)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-10-28 23:54:55 +08:00
zhenshan.cao
25551ffa2f
[skip ci]Fix golint error in memory_kv (#10861)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-10-28 23:42:39 +08:00
Xiaofan
93f205b49e
Rename param names (#10853)
* Rename param names

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

* Rename configs

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-10-28 23:16:40 +08:00
BIKI DAS
1f2f76789e
Combined function params (#10786)
Signed-off-by: Biki-das <bikid475@gmail.com>
2021-10-28 15:58:29 +08:00
zhenshan.cao
d598c78bbf
[skip ci]Fix golint error in minio_kv (#10702)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-10-26 22:42:25 +08:00
zhenshan.cao
dcac9c71b1
[skip ci]Fix golint error in minio_kv (#10701)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-10-26 22:40:34 +08:00
zhenshan.cao
4ca4c81ce3
[skip ci]Fix golint error in minio_kv (#10700)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-10-26 22:38:44 +08:00
zhenshan.cao
58d202f200
[skip ci]Fix golint error in minio_kv (#10699)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-10-26 22:36:53 +08:00
cai.zhang
5b42a3223c
Increase compatibility for EstimateMemorySize interface (#10603)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-10-26 15:34:21 +08:00
congqixia
ed2b8d67c0
Reduce info saved in SnapshotMeta (#10288)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-21 14:04:36 +08:00
godchen
8daeb0d519
Change bloom filter use pk (#10193)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-19 20:18:47 +08:00
zhenshan.cao
e97b62cf67
Change RocksDBLRUCache size to 0 (#9820)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-10-13 20:28:34 +08:00
Cai Yudong
c7566345eb
Simplify meta_snapshot interface (#9778)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-10-13 15:54:33 +08:00
dragondriver
41bd866ff6
Support LoadPartial interface for DataKV (#9554)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-10-09 19:33:03 +08:00
yukun
a6e97e0d44
Change LRUCache size of rocksdb to 512MB (#9509)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-10-09 11:08:56 +08:00
yukun
d9115fe78d
Set PinL0FilterAndIndexBlocksInCache to true for rocksdb (#8855)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-10-08 10:19:06 +08:00