Commit Graph

6440 Commits

Author SHA1 Message Date
wayblink
88293b8b79
Add log to analysis when etcd txn operation failed (#21558)
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-02-02 15:57:51 +08:00
aoiasd
6a6818ac32
Fix rocksmq retention test is not stable (#21925)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-02-02 14:09:51 +08:00
Xiaofan
2213ed7bed
Fix bulkload segment can not be flushed (#21919)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-02-02 13:11:51 +08:00
yah01
bdcff5b921
Avoid counting all bits to improve query performance (#21909)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-02-01 18:49:51 +08:00
yah01
9b491858a5
Fix datanode unit test not stable (#21911)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-02-01 18:19:50 +08:00
Enwei Jiao
53ae40b8c6
Make ratelimiter's config refreshable (#21757)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-02-01 16:03:51 +08:00
Bingyi Sun
b12740e108
Remove a goroutine when creating a proxy client (#21904)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2023-02-01 15:49:50 +08:00
Cai Yudong
e3d1624e97
Handle distance Inf correctly (#21828)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2023-02-01 14:59:51 +08:00
congqixia
ec282b1958
Apply lifetime control for indexnode (#21892)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-01-31 18:37:50 +08:00
congqixia
01e59f8ec2
Fix master semver value (#21873)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-01-31 17:41:50 +08:00
wayblink
45ff568470
Fix probable ut fail caused by session key confict (#21883)
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-01-31 17:39:49 +08:00
congqixia
11a3d0a50b
Use embed ETCD for datacoord unit test (#21890)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-01-31 14:47:49 +08:00
yah01
61c2f29ab3
Fix miss to throw exception when failed to create dir (#21886)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-01-31 13:49:50 +08:00
congqixia
d329cf33a0
Use embed etcd for ConnectManager unit test (#21884)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-01-31 13:35:49 +08:00
Xiaofan
f1daef22a1
Fix failed bulkload segment marked as sealed (#21799)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-01-31 12:41:53 +08:00
cai.zhang
a04238c25f
Keep the configuration of indexCoord (#21879)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-01-31 12:07:50 +08:00
congqixia
8af2aebb66
Fix msg channel prefix config priority for backward compatibility (#21871)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-01-31 10:09:51 +08:00
Jiquan Long
270c5b0502
Refine log when initialize dml channels (#21863)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-01-30 14:05:54 +08:00
jaime
5ed6ae2f27
Remove repeated etcd endpoint configuration (#21860)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-01-30 12:07:48 +08:00
jaime
e14f96a8e4
[skip e2e]Fix missed logger with information (#21859)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-01-30 11:51:49 +08:00
cai.zhang
bcae97b125
Register after start to prevent there are tow coordinator at the same time (#21641) (#21707)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-01-30 11:11:50 +08:00
smellthemoon
bb086f47e4
Swap execution flow order in enqueue (#21845)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-01-30 10:27:50 +08:00
wei liu
73c44d4b29
resource group impl (#21609)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-01-30 10:19:48 +08:00
congqixia
66027790a2
Implement detailed lifetime control for querynode (#21851)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-01-29 17:45:49 +08:00
yah01
43d633cfed
Fix fill the string field twice when string index exists (#21852)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-01-29 17:43:48 +08:00
congqixia
0480719e1a
Fix useasan format not correct (#21829)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-01-28 11:43:47 +08:00
Xiaofan
949d5d078f
Fix memory calculation in dataCodec (#21800)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-01-28 11:09:52 +08:00
jaime
d7bb908148
Fix rename collection data race (#21827)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-01-24 10:01:46 +08:00
Xiaofan
9258f7059d
Fix Bulkload ut print too much result (#21821)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-01-19 21:31:44 +08:00
congqixia
54ec22e879
Allow empty AK/SK for minio cfg (#21815)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-01-19 15:49:44 +08:00
congqixia
5986106037
Make paramtable init only once (#21782)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-01-19 14:53:44 +08:00
jaime
1dcd9eeb79
Support rename collection (#21693)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-01-19 14:13:43 +08:00
Bingyi Sun
c82b5d15b6
Add unsubscribe dmchannel task (#21513)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2023-01-18 17:45:44 +08:00
Bingyi Sun
b0b0929c6c
Change CreateAt to UpdatedAt when checking timeout (#21769)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2023-01-18 17:29:43 +08:00
Ten Thousand Leaves
defb7660c8
Add refresh option for LoadCollection/LoadPartitions interfaces (#21776)
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2023-01-18 16:41:44 +08:00
Enwei Jiao
cecd59d940
Disable time-travel by default (#21775)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-01-18 13:57:43 +08:00
Xiaofan
3e21d25de5
Add slow gc warning log (#21734)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-01-18 11:41:43 +08:00
groot
0af53d1e2e
Fix bulkinsert report bug (#21778)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2023-01-18 10:33:43 +08:00
yah01
78fd1987eb
Fix data race caused by no clone while updating index (#21772)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-01-17 18:11:42 +08:00
smellthemoon
ad6cbc990f
Combine similar function (#21752)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-01-17 17:53:42 +08:00
Bingyi Sun
b91bb5a729
Fix load timeout after next target updates (#21759)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2023-01-17 15:15:46 +08:00
presburger
e64f55cd27
downgrade curl for compiled on aarch64 platform (#21739)
Signed-off-by: Yusheng.Ma <Yusheng.Ma@zilliz.com>
2023-01-17 14:53:42 +08:00
wayblink
de584b508e
Fix active-standby switch fail bug (#21755)
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-01-17 11:43:43 +08:00
yah01
c8f89907b6
Fix current target may be updated to an invalid target (#21742)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-01-17 11:41:51 +08:00
jaime
81326ca0a1
[skip e2e]Fix cpp lint errors (#21750)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-01-17 09:43:42 +08:00
yah01
6e66c10114
Fix check time tick lag with no channel (#21737)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-01-16 18:49:43 +08:00
Xiaofan
e7f947ed0a
Check ScalarIndex deserialization (#21724)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-01-16 17:29:43 +08:00
yah01
836773f1a0
Optimize performance of reducing segments (#21722)
- Improve the performance of reducing from O(knlogn) to O(nlogk)

Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-01-16 15:25:42 +08:00
congqixia
5f3d3dc4fc
generate-mockery for querycoordv2, querynode and rootcoort (#21714)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-01-16 13:59:42 +08:00
yah01
837e3162d7
Fix ready notifiers leak when collection released (#21712)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-01-14 21:55:41 +08:00
yah01
32fb409e57
Fix may update the current target to an unavailable target when node down (#21698)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-01-13 17:11:41 +08:00
yah01
c66ce4aeba
Add warning comments (#21697)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-01-13 16:43:41 +08:00
groot
c0e7a122d4
Fix bulkinsert progress bug (#21680)
Signed-off-by: groot <yihua.mo@zilliz.com>
2023-01-13 16:25:40 +08:00
Enwei Jiao
90d9e165d4
Fix some configs not shown (#21653)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-01-13 15:31:41 +08:00
smellthemoon
4620aebd46
Fix data race (#21706)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-01-13 14:29:41 +08:00
congqixia
782f942b6f
Add MLogger.WithRateGroup for logger (#21703)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-01-13 14:11:40 +08:00
Enwei Jiao
564173dcbc
Upgrade protobuf to 3.21.9 (#21679)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-01-13 14:05:41 +08:00
yah01
f4d80b4211
Fix reading config from etcd may block forever (#21688)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-01-13 11:29:41 +08:00
Xiaofan
f8e1566b24
Support zstd compression in grpc (#21689)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-01-13 10:55:41 +08:00
congqixia
204db183f3
Prevent panicking when index segment info not exists (#21664)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-01-12 21:39:40 +08:00
wayblink
6a722396bd
Integration test framework (#21283)
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-01-12 19:49:40 +08:00
Enwei Jiao
49f53003e9
upgrade libcurl to fix mac build failed (#21684)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-01-12 19:03:40 +08:00
wei liu
c67070ab1f
add resource group interface (#21676)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-01-12 18:31:41 +08:00
Enwei Jiao
fb42466c65
Use opentelemetry (#21509)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-01-12 16:09:39 +08:00
Enwei Jiao
9fccbbb92b
Remove factory in querycoord (#21659)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-01-12 15:59:39 +08:00
SimFG
b307ee8c07
Make the rbac error message clearer (#21645)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-01-12 13:55:42 +08:00
Xiaofan
8f202274a7
Fix lag behind error with 0 servicetime (#21637)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-01-12 10:53:39 +08:00
Jiquan Long
d0b974de07
Make rootcoord client compatible with older version (#21651)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-01-12 10:09:39 +08:00
Jiquan Long
9fd9bed2b9
Remove collection meta after GC finished (#21595)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-01-12 09:55:42 +08:00
Jiquan Long
6d09bbed68
[skip e2e] Fix load meta migration (#21584)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-01-11 19:31:39 +08:00
Ten Thousand Leaves
8c2c04517e
Fix an issue where flushed segments can turn into sealed (#21566)
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2023-01-11 18:57:43 +08:00
groot
9f8095996b
Report bulkinsert progress (#21612) (#21638)
Signed-off-by: groot <yihua.mo@zilliz.com>
2023-01-11 17:37:44 +08:00
cai.zhang
f02ccfb422
Set GetIndexBuildProgress is compatible with SDK version 2.1 (#21622)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-01-11 16:59:42 +08:00
cai.zhang
e127cf7b99
Reset indexpb for upgrade (#21620)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-01-11 14:35:40 +08:00
groot
df0ffd08ce
Update segment id for import task (#21621)
Signed-off-by: groot <yihua.mo@zilliz.com>
2023-01-11 11:55:39 +08:00
liliu-z
cc5ecabac6
Skip search GRPC call for standalon (#21383)
Signed-off-by: Li Liu <li.liu@zilliz.com>
2023-01-10 21:09:38 +08:00
bigsheeper
2146af1fb2
Return insufficient memory error when load failed (#21574)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-01-10 20:35:39 +08:00
congqixia
c550427e81
Use singleflight in localCache.load (#21606)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-01-10 16:21:39 +08:00
yah01
3b41a6931d
Add QueryCoord OWNERS file (#21552)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-01-10 16:15:37 +08:00
groot
aab20a4031
Streaming read numpy (#21579)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2023-01-10 14:53:38 +08:00
cai.zhang
5d660f16d6
Forbid IndexCoord start with version<2.3 (#21597)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-01-10 14:33:38 +08:00
Ten Thousand Leaves
23ceb9c638
Remove index checking phase for bulk insert (#21578)
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2023-01-09 21:15:38 +08:00
congqixia
42ab1b0a48
Add segment gc restraint of dml position before channel cp (#21564)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-01-06 21:33:36 +08:00
Xiaofan
dc2bfb0901
Add stack trace for core dump (#21555)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-01-06 16:53:35 +08:00
Xiaofan
4b4944ecee
Support sync all segments while close (#21421)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-01-06 14:49:36 +08:00
MrPresent-Han
6fb3542f2a
enable auto balance paramter(#21504) (#21507)
Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
2023-01-06 14:45:35 +08:00
jaime
73d8509600
Improve ListSegments performance for datacoord catalog (#21477)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-01-06 14:33:36 +08:00
bigsheeper
79535931ab
Return error code when deny to read/write (#21561)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-01-06 14:31:37 +08:00
cai.zhang
aa203acfb3
Low IndexCoord weight (#21548)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-01-06 14:21:37 +08:00
yah01
e8e2696430
avoid checking search params many times (#21516)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-01-06 13:15:35 +08:00
Ten Thousand Leaves
b3da554705
Increase bulk insert file size limit (#21526)
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2023-01-06 13:09:36 +08:00
jaime
6d53a99171
Improve LoadWithPrefix performance for SuffixSnapshot (#21524)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-01-06 10:11:36 +08:00
jaime
09e1d4fc83
Remove redundant LoadPrefix requests for Catalog ListCollections (#21551)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-01-06 10:09:36 +08:00
Enwei Jiao
24d5faf7aa
Fix show configuration bug (#21544)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-01-06 09:55:36 +08:00
congqixia
3d0f516fd7
Add With method for MLogger (#21530)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-01-05 13:13:34 +08:00
cai.zhang
efefba4da4
Fix checkpoint will not update after delete and compact (#21495) (#21499)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-01-05 11:31:35 +08:00
cai.zhang
9dd2e1dcd8
Use tt msg stream for consume delete msg (#21478) (#21500)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-01-05 11:03:35 +08:00
jaime
58b79eb74c
Add based on timetravel GC for snapshot KV (#21417)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-01-04 21:37:35 +08:00
Jiquan Long
6e521dd4b3
Optimize GetLoadingProgress to save RPC (#21493)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-01-04 20:33:35 +08:00
cai.zhang
e5f408dceb
Merge IndexCoord and DataCoord (#21267)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-01-04 19:37:36 +08:00