test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-03 04:19:18 +08:00
Code Issues Actions 13 Packages Projects Releases Wiki Activity
18,901 Commits 95 Branches 105 Tags 184 MiB
e52ce370b6
Commit Graph

70 Commits

Author SHA1 Message Date
smellthemoon
e52ce370b6
enhance:don't store logPath in meta to reduce memory (#28873) 
don't store logPath in meta to reduce memory, when service get
segmentinfo, generate logpath from logid.
#28885

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-01-18 22:06:31 +08:00
smellthemoon
1c1f2a1371
enhance:change some logs (#29579) 
related #29588

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-01-05 16:12:48 +08:00
wei liu
69a71e11d1
enhance: Add config for auto upgrade segment index (#29112) 
This PR add config for auto upgrade segment index to index engine's
version.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-12-12 11:56:37 +08:00
SimFG
ef18e6a532
enhance: Use a non-blocking method to trigger compaction when saveBinlogPath is executed (#28941) 
/kind improvement
issue: #28924

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-12-12 10:16:39 +08:00
XuanYang-cn
5bac7f7897
fix: Fix L0 compaction in datacoord (#28814) 
See also: #27606

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-12-05 18:44:37 +08:00
smellthemoon
adce6aab4f
fix:handle err when ManualCompaction (#28804) 
handle err when ManualCompaction
#28644

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-11-30 17:06:31 +08:00
XuanYang-cn
9b371067d2
feat: Add Compaction views and triggers (#27906) 
- Add Compaction l0 views
- Add Compaction scheduler
- Add Compaction triggerv2
- Add Compaction view manager

See also: #27606

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-11-23 17:30:25 +08:00
Enwei Jiao
7445d3711c
feat: trigger compaction to handle index version (#28442) 
issue: https://github.com/milvus-io/milvus/issues/28441

---------

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-11-21 09:26:22 +08:00
SimFG
26f06dd732
Format the code (#27275) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-09-21 09:45:27 +08:00
XuanYang-cn
b2e7cbdf4b
Remove TimeTravel in compactor (#26785) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-09-04 17:41:48 +08:00
wayblink
21eeb37ce9
Save binlog timestampFrom, timestampTo meta when compact (#26203) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-08-08 21:17:21 +08:00
xige-16
6f18587f35
Fix small segment compaction (#21327) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2023-07-26 14:49:01 +08:00
cai.zhang
b15e34db21
Add contraint for compaction based indexed segments (#25709) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-07-23 21:31:00 +08:00
Enwei Jiao
66fdc71479
Refactor logs in DataCoord & DataNode (#25574) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-07-14 15:56:31 +08:00
congqixia
41af0a98fa
Use go-api/v2 for milvus-proto (#24770) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-06-09 01:28:37 +08:00
Enwei Jiao
7e645dabf1
Add trace log for datacoord (#24046) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-05-12 09:53:21 +08:00
congqixia
5aa9db0d38
Add collection level auto compaction enabled config (#24013) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-05-10 17:45:20 +08:00
XuanYang-cn
72485c9eeb
Remove index check to trigger compaction (#23657) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-04-24 20:26:34 +08:00
jaime
c9d0c157ec
Move some modules from internal to public package (#22572) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-04-06 19:14:32 +08:00
yah01
319f1773af
Fix segment not found if forward delete to empty segment (#22528) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-03 14:13:49 +08:00
cai.zhang
e5f408dceb
Merge IndexCoord and DataCoord (#21267) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-01-04 19:37:36 +08:00
Jiquan Long
ff2a68e65a
Fix collection not exist when tried to do recovery (#21471) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-01-04 16:37:35 +08:00
Ten Thousand Leaves
ef3fcdf65b
Small candidate compaction should only happen with >1 segments (#21249) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-12-16 10:01:23 +08:00
Ten Thousand Leaves
a81c9cc11c
Recalculate segment max size during compaction (#21077) 
issue: #21077

/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-12-13 15:39:21 +08:00
Xiaofan
e977e014a9
Fix flush didn't respect binaryvector and other schemas (#21120) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-12-12 10:33:26 +08:00
Ten Thousand Leaves
d67db2ed9b
Make compaction target able to exceed segment max size by a bit (#21068) 
issue: #21053
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-12-09 16:03:20 +08:00
Enwei Jiao
89b810a4db
Refactor all params into ParamItem (#20987) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-12-07 18:01:19 +08:00
Ten Thousand Leaves
2f5b03fef7
Split small segment proportion and small segment compactable proportion (#20996) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-12-06 14:37:18 +08:00
XuanYang-cn
d1cb380ee0
Enable to gen compaction plan for stale segments (#20741) 
See also: #20502

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-11-22 17:33:16 +08:00
Ten Thousand Leaves
480a2b8632
Improve compaction logs (#20490) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-11-15 11:13:07 +08:00
congqixia
5f5fdb0789
Compaction bypass stale segments (#20202) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-31 19:13:34 +08:00
SimFG
a55f739608
Separate public proto files (#19782) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-10-16 20:49:27 +08:00
congqixia
2a2d32c4fc
Fix datacoord panics when collection info not found (#19706) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-11 21:39:24 +08:00
jaime
31db01b6ae
Support TTL for collection level (#19383) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-10-10 20:31:22 +08:00
Ten Thousand Leaves
b30c9d4f60
Support Bulk Load in Milvus 2.2 (#18982) 
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-09-26 18:06:54 +08:00
zhagnlu
59f8e96522
Change compactor policy from loading all binlogs to loading serially (#13716) (#19280) 
Signed-off-by: zhagnlu <lu.zhang@zilliz.com>

Signed-off-by: zhagnlu <lu.zhang@zilliz.com>
Co-authored-by: zhagnlu <lu.zhang@zilliz.com>
 2022-09-25 20:12:52 +08:00
yah01
ee9d634a43
Compaction requires the source segments indexed (#19368) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-09-22 18:34:50 +08:00
yah01
71d4a32ce6
Fix deadlock (#19340) 
The IsParentDropped() acquires read lock of meta,
which leads to recursively acquire read lock

Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-09-22 15:48:50 +08:00
SimFG
d7f38a803d
Separate some proto files (#19218) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-09-16 16:56:49 +08:00
yah01
11b352c3ae
Guarantee flushed segments are all indexed (#19062) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-09-16 11:32:48 +08:00
Enwei Jiao
096149e33f
fix GetCompactionState hang (#18829) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-08-26 14:22:55 +08:00
cai.zhang
c924f73105
Refactor for IndexCoord to support cloud (#18643) 
Co-authored-by: Zach41 <zongmei.zhang@zilliz.com>
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: Zach41 <zongmei.zhang@zilliz.com>
 2022-08-25 15:48:54 +08:00
jaime
c9174d55ba
Refine merge operation during compacting phase (#18399) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-07-28 14:52:31 +08:00
yah01
ef6859a4d6
Refactor handoff handler (#17806) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Co-authored-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-06-29 23:26:19 +08:00
Xiaofan
6fdf88f452
Fix compaction selection policy (#17690) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-06-22 19:00:14 +08:00
cai.zhang
ed94ecf847
Segments can't be compacted when they have reference lock (#17649) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-06-20 21:56:12 +08:00
Xiaofan
54aeb077e9
Refine compaction selection poliy (#17486) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-06-15 23:14:10 +08:00
XuanYang-cn
c70af73005
Fix datacoord set wrong state for node registering (#17376) 
Fix datacoord datarace

See also: #17335

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-06-06 15:10:05 +08:00
Xiaofan
bd7d294c6d
Refine log level in data (#15821) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-03-02 15:35:55 +08:00
congqixia
5751759c85
Implement mix compaction logic (#15542) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-02-18 14:51:49 +08:00