mirror of
https://gitee.com/dotnetchina/MiniExcel.git
synced 2024-12-02 11:57:45 +08:00
289 lines
9.2 KiB
Markdown
289 lines
9.2 KiB
Markdown
[](https://www.nuget.org/packages/MiniExcel) [](https://www.nuget.org/packages/MiniExcel) [](https://ci.appveyor.com/project/shps951023/miniexcel/branch/master) [](#) [](#) [](#)
|
||
|
||
---
|
||
|
||
[English](README.md) / [繁體中文](README.zh-tw.md)
|
||
|
||
---
|
||
|
||
### 簡介
|
||
|
||
MiniExcel 簡單、高效避免OOM的.NET處理Excel工具。
|
||
|
||
目前主流框架大多需要將資料全載入到記憶體方便操作，但這會導致記憶體消耗問題，MiniExcel 嘗試以 Stream 角度寫底層算法邏輯，能讓原本1000多MB占用降低到幾MB，避免記憶體不夠情況。
|
||
|
||
### 特點
|
||
- 低記憶體耗用，避免OOM(out of memoery)
|
||
- 支持`即時`操作每行資料
|
||
|
||
- 兼具搭配 LINQ 延遲查詢特性，能辦到低消耗、快速分頁等複雜查詢
|
||
圖片:與主流框架對比的消耗、效率差
|
||
|
||
- 輕量，不依賴任何套件，DLL小於100KB
|
||
- 簡便操作的 Dapper API 風格
|
||
|
||
### 安裝
|
||
|
||
請查看 [from NuGet](https://www.nuget.org/packages/MiniExcel)
|
||
|
||
### 更新日誌
|
||
|
||
請查看 [Release Notes](https://github.com/shps951023/MiniExcel/tree/master/docs)
|
||
|
||
### TODO
|
||
|
||
請查看 [Project · todo](https://github.com/shps951023/MiniExcel/projects/1?fullscreen=true)
|
||
|
||
### 性能測試
|
||
|
||
以 [**Test1,000,000x10.xlsx**](https://github.com/shps951023/MiniExcel/blob/master/samples/xlsx/Test1%2C000%2C000x10/Test1%2C000%2C000x10.xlsx) 做基準做性能測試，總共 1千萬筆 "HelloWorld"，檔案大小 23 MB
|
||
|
||
Benchmarks 邏輯可以在 [MiniExcel.Benchmarks](https://github.com/shps951023/MiniExcel/tree/master/benchmarks/MiniExcel.Benchmarks) 查看或是提交 PR，運行指令
|
||
|
||
```
|
||
dotnet run -p .\benchmarks\MiniExcel.Benchmarks\ -c Release -f netcoreapp3.1 -- -f * --join
|
||
```
|
||
|
||
最後一次運行結果 :
|
||
|
||
```
|
||
BenchmarkDotNet=v0.12.1, OS=Windows 10.0.19042
|
||
Intel Core i7-7700 CPU 3.60GHz (Kaby Lake), 1 CPU, 8 logical and 4 physical cores
|
||
[Host] : .NET Framework 4.8 (4.8.4341.0), X64 RyuJIT
|
||
Job-ZYYABG : .NET Framework 4.8 (4.8.4341.0), X64 RyuJIT
|
||
IterationCount=3 LaunchCount=3 WarmupCount=3
|
||
```
|
||
|
||
| 框架 | 記憶體耗用 | 方法 | 平均 | 誤差 | 標準偏差 | Gen 0 | Gen 1 | Gen 2 |
|
||
| --------------- | -------------: | ---------: | --------------: | --------------: | --------------: | -----------: | ----------: | ---------: |
|
||
| MiniExcel | 299.71 KB | QueryFirst | 564.4 μs | 36.35 μs | 21.63 μs | 72.2656 | 17.5781 | - |
|
||
| ExcelDataReader | 2629975.14 KB | QueryFirst | 12,455,316.6 μs | 266,606.83 μs | 158,653.45 μs | 642000.0000 | 1000.0000 | - |
|
||
| Epplus | 6258769.32 KB | QueryFirst | 23,369,553.1 μs | 2,909,345.17 μs | 1,731,304.64 μs | 1081000.0000 | 273000.0000 | 13000.0000 |
|
||
| ClosedXml | 12650295.38 KB | QueryFirst | 60,567,701.6 μs | 3,905,377.40 μs | 2,324,027.45 μs | 2036000.0000 | 708000.0000 | 10000.0000 |
|
||
|
||
### Query 查詢 Excel 返回`強型別` IEnumerable 資料 [[Try it]](https://dotnetfiddle.net/w5WD1J)
|
||
|
||
推薦使用 Stream.Query 效率會相對較好。
|
||
|
||
```C#
|
||
public class UserAccount
|
||
{
|
||
public Guid ID { get; set; }
|
||
public string Name { get; set; }
|
||
public DateTime BoD { get; set; }
|
||
public int Age { get; set; }
|
||
public bool VIP { get; set; }
|
||
public decimal Points { get; set; }
|
||
}
|
||
|
||
var rows = MiniExcel.Query<UserAccount>(path);
|
||
|
||
// or
|
||
|
||
using (var stream = File.OpenRead(path))
|
||
var rows = stream.Query<UserAccount>();
|
||
```
|
||
|
||
|
||
|
||
|
||
### Query 查詢 Excel 返回`Dynamic` IEnumerable 資料 [[Try it]](https://dotnetfiddle.net/w5WD1J)
|
||
|
||
* Key 系統預設為 `A,B,C,D...Z`
|
||
|
||
| MiniExcel | 1 |
|
||
| -------- | -------- |
|
||
| Github | 2 |
|
||
|
||
```C#
|
||
|
||
var rows = MiniExcel.Query(path).ToList();
|
||
|
||
// or
|
||
using (var stream = File.OpenRead(path))
|
||
{
|
||
var rows = stream.Query().ToList();
|
||
|
||
Assert.Equal("MiniExcel", rows[0].A);
|
||
Assert.Equal(1, rows[0].B);
|
||
Assert.Equal("Github", rows[1].A);
|
||
Assert.Equal(2, rows[1].B);
|
||
}
|
||
```
|
||
|
||
### 查詢資料以第一行數據當Key [[Try it]](https://dotnetfiddle.net/w5WD1J)
|
||
|
||
note : 同名以右邊數據為準
|
||
|
||
Input Excel :
|
||
|
||
| Column1 | Column2 |
|
||
| -------- | -------- |
|
||
| MiniExcel | 1 |
|
||
| Github | 2 |
|
||
|
||
|
||
```C#
|
||
|
||
var rows = MiniExcel.Query(useHeaderRow:true).ToList();
|
||
|
||
// or
|
||
|
||
using (var stream = File.OpenRead(path))
|
||
{
|
||
var rows = stream.Query(useHeaderRow:true).ToList();
|
||
|
||
Assert.Equal("MiniExcel", rows[0].Column1);
|
||
Assert.Equal(1, rows[0].Column2);
|
||
Assert.Equal("Github", rows[1].Column1);
|
||
Assert.Equal(2, rows[1].Column2);
|
||
}
|
||
```
|
||
|
||
### Query 查詢支援延遲加載(Deferred Execution)，能配合LINQ First/Take/Skip辦到低消耗、高效率複雜查詢
|
||
|
||
Query First
|
||
```C#
|
||
var row = MiniExcel.Query(path).First();
|
||
Assert.Equal("HelloWorld", row.A);
|
||
|
||
// or
|
||
|
||
using (var stream = File.OpenRead(path))
|
||
{
|
||
var row = stream.Query().First();
|
||
Assert.Equal("HelloWorld", row.A);
|
||
}
|
||
```
|
||
|
||
### 建立 Excel 檔案 [[Try it]](https://dotnetfiddle.net/w5WD1J)
|
||
|
||
1. 必須是 non-abstract 類別有公開建構式
|
||
2. MiniExcel SaveAs 支援 `IEnumerable參數``延遲查詢`，除非必要請不要使用 ToList 等方法讀取全部資料到記憶體，請看圖片了解差異
|
||
|
||
圖片 : 是否呼叫 ToList 的記憶體差別
|
||
|
||
|
||
Anonymous or strongly type:
|
||
```C#
|
||
var path = Path.Combine(Path.GetTempPath(), $"{Guid.NewGuid()}.xlsx");
|
||
MiniExcel.SaveAs(path, new[] {
|
||
new { Column1 = "MiniExcel", Column2 = 1 },
|
||
new { Column1 = "Github", Column2 = 2}
|
||
});
|
||
```
|
||
|
||
Datatable:
|
||
```C#
|
||
var path = Path.Combine(Path.GetTempPath(), $"{Guid.NewGuid()}.xlsx");
|
||
var table = new DataTable();
|
||
{
|
||
table.Columns.Add("Column1", typeof(string));
|
||
table.Columns.Add("Column2", typeof(decimal));
|
||
table.Rows.Add("MiniExcel", 1);
|
||
table.Rows.Add("Github", 2);
|
||
}
|
||
|
||
MiniExcel.SaveAs(path, table);
|
||
```
|
||
|
||
Dapper:
|
||
```C#
|
||
using (var connection = GetConnection(connectionString))
|
||
{
|
||
var rows = connection.Query(@"select 'MiniExcel' as Column1,1 as Column2 union all select 'Github',2");
|
||
MiniExcel.SaveAs(path, rows);
|
||
}
|
||
```
|
||
|
||
`IEnumerable<IDictionary<string, object>>`
|
||
```C#
|
||
var values = new List<Dictionary<string, object>>()
|
||
{
|
||
new Dictionary<string,object>{{ "Column1", "MiniExcel" }, { "Column2", 1 } },
|
||
new Dictionary<string,object>{{ "Column1", "Github" }, { "Column2", 2 } }
|
||
};
|
||
MiniExcel.SaveAs(path, values);
|
||
```
|
||
|
||
output :
|
||
|
||
| Column1 | Column2 |
|
||
| -------- | -------- |
|
||
| MiniExcel | 1 |
|
||
| Github | 2 |
|
||
|
||
### SaveAs 支援 Stream [[Try it]](https://dotnetfiddle.net/JOen0e)
|
||
|
||
```C#
|
||
using (var stream = File.Create(path))
|
||
{
|
||
stream.SaveAs(values);
|
||
}
|
||
```
|
||
|
||
|
||
### 例子 : SQLite & Dapper 讀取大數據新增到資料庫
|
||
|
||
note : 請不要呼叫 call ToList/ToArray 等方法，這會將所有資料讀到記憶體內
|
||
|
||
```C#
|
||
using (var connection = new SQLiteConnection(connectionString))
|
||
{
|
||
connection.Open();
|
||
using (var transaction = connection.BeginTransaction())
|
||
using (var stream = File.OpenRead(path))
|
||
{
|
||
var rows = stream.Query();
|
||
foreach (var row in rows)
|
||
connection.Execute("insert into T (A,B) values (@A,@B)", new { row.A, row.B }, transaction: transaction);
|
||
transaction.Commit();
|
||
}
|
||
}
|
||
```
|
||
|
||
效能:
|
||
|
||
|
||
|
||
### 例子 : ASP.NET Core 3.1 or MVC 5 下載 Excel Xlsx API Demo
|
||
|
||
```C#
|
||
public class ExcelController : Controller
|
||
{
|
||
public IActionResult Download()
|
||
{
|
||
var values = new[] {
|
||
new { Column1 = "MiniExcel", Column2 = 1 },
|
||
new { Column1 = "Github", Column2 = 2}
|
||
};
|
||
var stream = new MemoryStream();
|
||
stream.SaveAs(values);
|
||
return File(stream,
|
||
"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
|
||
"demo.xlsx");
|
||
}
|
||
}
|
||
```
|
||
|
||
### Excel 類別自動判斷
|
||
|
||
MiniExcel 預設會根據擴展名或是 Stream 類別判斷是 xlsx 還是 csv，但會有失準時候，請自行指定。
|
||
```C#
|
||
stream.SaveAs(excelType:ExcelType.CSV);
|
||
//or
|
||
stream.SaveAs(excelType:ExcelType.XLSX);
|
||
//or
|
||
stream.Query(excelType:ExcelType.CSV);
|
||
//or
|
||
stream.Query(excelType:ExcelType.XLSX);
|
||
```
|
||
|
||
|
||
### 侷限與警告
|
||
- 目前不支援 xls 或是加密檔案。
|
||
|
||
### 參考
|
||
- 讀取邏輯 : [ExcelDataReader](https://github.com/ExcelDataReader/ExcelDataReader)
|
||
- API 設計方式 : [StackExchange/Dapper](https://github.com/StackExchange/Dapper)
|