Commit Graph

98 Commits

Author SHA1 Message Date
congqixia
3333160b8d
enhance: Fix lint issues from recent PRs (#34482)
See also #34483
Some lint issues are introduced due to lack of static check run. This PR
fixes these problems.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-09 10:06:24 +08:00
charleskakumanu
05df70973c
enhance: retry etcd server start with 3 default retries (#34208)
Added retry method and unit test cases for retrying etcd server start.
New to open source and Go language. Please reject if this is not the
requirement/ specify the modifications needed in code.
issue : #17569

Signed-off-by: Charles Kakumanu <charles_kakumanu@apple.com>
Co-authored-by: Charles Kakumanu <charles_kakumanu@apple.com>
2024-07-05 20:24:11 +08:00
jaime
9630974fbb
enhance: move rocksmq from internal to pkg module (#33881)
issue: #33956

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-06-25 21:18:15 +08:00
gcmutator
6edd06083f
chore: remove repetitive words (#31153)
Signed-off-by: gcmutator <329964069@qq.com>
2024-03-20 10:17:07 +08:00
PowderLi
08ca0a2ca5
feat: support etcd authentication (#30226)
issue: #28895
add 3 configuration for ETCD config

Signed-off-by: PowderLi <min.li@zilliz.com>
2024-01-24 11:35:00 +08:00
wei liu
9fc5f1176c
fix: Drop segment meta info with prefix (#29856)
If segment has more than 128 log fils, drop segment will exceed etcd txn
ops limit, which will failed the drop segment request
This PR drop segment meta info with prefix, to avoid drop segment meta
failed

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-01-11 12:22:51 +08:00
congqixia
a2fe9dad49
enhance: Make etcd kv request timeout configurable (#28661)
See also #28660
This pr add request timeout config item for etcd kv request timeout
 Sync the default timeout value to same value for etcdKV & tikv config

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-11-23 19:34:23 +08:00
Xiaofan
38aa53f523
Fix datacoord start timeout (#28026)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-10-31 10:04:12 +08:00
yah01
15746b5b5f
Refine KV errors (#27588)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-13 17:15:34 +08:00
congqixia
a3dd2756cf
Add predicates for TxnKV operations (#27365)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-27 10:21:26 +08:00
SimFG
26f06dd732
Format the code (#27275)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-21 09:45:27 +08:00
yiwangdr
f85af0732c
remove MultiRemoveWithPrefix (#26924)
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
2023-09-12 15:07:17 +08:00
Enwei Jiao
fb0705df1b
Decouple basetable and componentparam (#26725)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-09-05 10:31:48 +08:00
Bingyi Sun
6539396bbe
Add HasPrefix in kv (#25462)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2023-07-11 10:28:28 +08:00
Bingyi Sun
f75201bbfd
Add Has interface in kv (#25439)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2023-07-10 19:40:27 +08:00
xige-16
33c2012675
Add more metrics (#25081)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-06-26 17:52:44 +08:00
yiwangdr
c7b851f870
add interface for non-watch metakv (#25092)
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
2023-06-26 09:20:44 +08:00
yiwangdr
4387f36897
make etcdKV private (#24778)
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
2023-06-13 10:52:38 +08:00
yah01
ebd0279d3f
Check error by Error() and NoError() for better report message (#24736)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-06-08 15:36:36 +08:00
yiwangdr
37c02c9927
clean up meta interface (#24729)
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
2023-06-08 14:18:35 +08:00
jaime
c9d0c157ec
Move some modules from internal to public package (#22572)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-04-06 19:14:32 +08:00
Enwei Jiao
697dedac7e
Use cockroachdb/errors to replace other error pkg (#22390)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-02-26 11:31:49 +08:00
wayblink
88293b8b79
Add log to analysis when etcd txn operation failed (#21558)
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-02-02 15:57:51 +08:00
congqixia
5986106037
Make paramtable init only once (#21782)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-01-19 14:53:44 +08:00
jaime
58b79eb74c
Add based on timetravel GC for snapshot KV (#21417)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-01-04 21:37:35 +08:00
Enwei Jiao
2ecdb4ba4a
Etcd config source support TLS (#20874)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-11-30 18:23:15 +08:00
Enwei Jiao
c05b9ad539
Add event dispatcher for config (#20393)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-11-17 18:59:09 +08:00
Eng Zer Jun
438790744d
test: use T.Setenv to set env vars in tests (#19649)
This commit replaces `os.Setenv` with `t.Setenv` in tests. The
environment variable is automatically restored to its original value
when the test and all its subtests complete.

Reference: https://pkg.go.dev/testing#T.Setenv
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-10-11 18:43:23 +08:00
Bingyi Sun
626854cf0c
Refactor QueryCoord (#18836)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Wei Liu <wei.liu@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Wei Liu <wei.liu@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
2022-09-15 18:48:32 +08:00
jaime
816967c000
Add metrics for etcd (#18962)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-09-02 17:46:59 +08:00
jaime
237d912625
Add log for the large value during etcd saving (#18229)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-09-01 09:47:02 +08:00
SimFG
1366f130a2
Add a specific KeyNotExistError err for the metastore load interface (#18582)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-08-10 17:20:41 +08:00
Ten Thousand Leaves
097f144db1
Improve slow Etcd logs by specifying keys (#18547)
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-08-09 21:14:37 +08:00
Enwei Jiao
283f5731d2
config from etcd (#18421)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-08-01 10:04:33 +08:00
Xiaofan
a9b1d71a8f
change the etcd compareAndSwap interface (#18068)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-07-06 13:54:21 +08:00
cai.zhang
ef0fc37ad7
Fixed the bug that IndexCoord lost some meta information (#17818)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-27 21:52:17 +08:00
Eng Zer Jun
5375161777
test: use T.TempDir to create temporary test directory (#16758)
This commit replaces `ioutil.TempDir` with `t.TempDir` in tests. The
directory created by `t.TempDir` is automatically removed when the test
and all its subtests complete.

Prior to this commit, temporary directory created using `ioutil.TempDir`
needs to be removed manually by calling `os.RemoveAll`, which is omitted
in some tests. The error handling boilerplate e.g.
	defer func() {
		if err := os.RemoveAll(dir); err != nil {
			t.Fatal(err)
		}
	}
is also tedious, but `t.TempDir` handles this for us nicely.

Fixes: #16759
Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-05-04 11:35:50 +08:00
jaime
c0a73dbe98
Using a temporary dir instead of a specified dir within embed etcd test (#16694)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-04-28 11:17:47 +08:00
aakejiang
de70dfe0c3
refine log (#16256)
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
2022-03-29 15:05:27 +08:00
Ten Thousand Leaves
a23ca745bb
Add Etcd support for Import Tasks. (#16164)
issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-03-25 11:03:25 +08:00
congqixia
a5c3f4eef4
Add CompareFailErr for CAS operation error check (#15893)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-03-07 18:01:58 +08:00
Letian Jiang
08d12e62f3
Add ByteSlice Method for embed etcd (#15844)
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-03-04 15:14:01 +08:00
Letian Jiang
b56ec7ea97
Add ByteSlice Method for Etcd-kv (#15738)
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-02-25 11:25:53 +08:00
jaime
87dd4e8f92
Fix clean up data dir failed for TestEtcdRestartLoad (#15506)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-02-09 18:37:45 +08:00
Cai Yudong
1f337891e1
Rename base_param to service_param, rename global_param to component_param (#15463)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-02-08 20:57:47 +08:00
Cai Yudong
f1ea9613ae
Add EtcdConfig and move PulsarConfig/RocksdbConfig/MinioConfig into BaseParamTable (#15434)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-02-07 10:09:45 +08:00
congqixia
69def6b34e
[skip e2e] Add comment and implementation assertion for EmbedEtcdKV (#14905)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-01-06 13:32:27 +08:00
Xiaofan
69087ff8dd
Support embedded etcd (#14333)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-12-29 14:35:21 +08:00
cxytz01
e47a651be4
[skip e2e] Grammar correction (#14057)
Signed-off-by: cxytz01 <xueyou.chen@zilliz.com>
2021-12-23 14:41:52 +08:00
jaime
0a99e1aeba
[skip ci] Fix go format error (#13212)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: yun.zhang <yun.zhang@zilliz.com>
2021-12-11 22:23:13 +08:00