- [New] Support QueryAsDataTable method [#216](https://github.com/shps951023/MiniExcel/issues/216)
- [New] SaveAs support IDataReader value parameter [#211](https://github.com/shps951023/MiniExcel/issues/211)
- [Bug] Fix numeric format string will be cast to numeric type [#I3OSKV](https://gitee.com/dotnetchina/MiniExcel/issues/I3OSKV)
- [Opt] Optimize SaveAs convert value type logic to improve performance
This commit is contained in:
wei 2021-04-30 17:59:12 +08:00
parent 8a455901ba
commit 44b89cccac
10 changed files with 150 additions and 104 deletions

View File

@ -227,6 +227,22 @@ foreach(IDictionary<string,object> row in MiniExcel.Query(path))
#### 9. Query Query Excel return DataTable
Not recommended, because DataTable will load all data into memory and lose MiniExcel's low memory consumption feature.
```C#
var table = MiniExcel.QueryAsDataTable(path, useHeaderRow: true);
```
![image](https://user-images.githubusercontent.com/12729184/116673475-07917200-a9d6-11eb-947e-a6f68cce58df.png)
### Create Excel <a name="getstart2"></a> ### Create Excel <a name="getstart2"></a>
1. Must be a non-abstract type with a public parameterless constructor . 1. Must be a non-abstract type with a public parameterless constructor .
@ -302,6 +318,12 @@ using (var stream = File.Create(path))
} }
``` ```
#### 6. Support IDataReader value parameter
```csharp
MiniExcel.SaveAs(path, reader);
```
### Fill Data To Excel Template <a name="getstart3"></a> ### Fill Data To Excel Template <a name="getstart3"></a>
@ -760,38 +782,6 @@ public static IEnumerable<T> Page<T>(IEnumerable<T> en, int pageSize, int page)
### FAQ ### FAQ
#### Q: How to convert query results to DataTable
Reminder: Not recommended, because DataTable will load all data into memory and lose MiniExcel's low memory consumption function.
```csharp
public static DataTable QueryAsDataTable(string path)
{
var rows = MiniExcel.Query(path, true);
var dt = new DataTable();
var first = true;
foreach (IDictionary<string, object> row in rows)
{
if (first)
{
foreach (var key in row.Keys)
{
var type = row[key]?.GetType() ?? typeof(string);
dt.Columns.Add(key, type);
}
first = false;
}
dt.Rows.Add(row.Values.ToArray());
}
return dt;
}
```
![image](https://user-images.githubusercontent.com/12729184/115068722-3105c480-9f25-11eb-8f5a-994416754134.png)
#### Q: Excel header title not equal class property name, how to mapping? #### Q: Excel header title not equal class property name, how to mapping?
A. Please use ExcelColumnName attribute A. Please use ExcelColumnName attribute

View File

@ -231,7 +231,15 @@ foreach(IDictionary<string,object> row in MiniExcel.Query(path))
} }
``` ```
#### 9. Query 读 Excel 返回 DataTable
提醒 : 不建议使用因为DataTable会将数据`全载入内存`失去MiniExcel低内存消耗功能。
```C#
var table = MiniExcel.QueryAsDataTable(path, useHeaderRow: true);
```
![image](https://user-images.githubusercontent.com/12729184/116673475-07917200-a9d6-11eb-947e-a6f68cce58df.png)
@ -297,7 +305,7 @@ output :
| MiniExcel | 1 | | MiniExcel | 1 |
| Github | 2 | | Github | 2 |
#### 5. SaveAs 支 Stream [[Try it]](https://dotnetfiddle.net/JOen0e) #### 5. SaveAs 支 Stream [[Try it]](https://dotnetfiddle.net/JOen0e)
```csharp ```csharp
using (var stream = File.Create(path)) using (var stream = File.Create(path))
@ -308,6 +316,14 @@ using (var stream = File.Create(path))
#### 6. 支持 IDataReader 参数
```csharp
MiniExcel.SaveAs(path, reader);
```
### 模板填充 Excel <a name="getstart3"></a> ### 模板填充 Excel <a name="getstart3"></a>
@ -769,36 +785,6 @@ public static IEnumerable<T> Page<T>(IEnumerable<T> en, int pageSize, int page)
### FAQ 常见问题 ### FAQ 常见问题
#### Q: 如何将查询结果转为 DataTable
提醒 : 不建议使用因为DataTable会将数据`全载入内存`失去MiniExcel低内存消耗功能。
```csharp
public static DataTable QueryAsDataTable(string path)
{
var rows = MiniExcel.Query(path, true);
var dt = new DataTable();
var first = true;
foreach (IDictionary<string, object> row in rows)
{
if (first)
{
foreach (var key in row.Keys)
{
var type = row[key]?.GetType() ?? typeof(string);
dt.Columns.Add(key, type);
}
first = false;
}
dt.Rows.Add(row.Values.ToArray());
}
return dt;
}
```
![image](https://user-images.githubusercontent.com/12729184/115068549-fac84500-9f24-11eb-9f12-884b19cf1489.png)
#### Q: Excel 表头标题名称跟 class 属性名称不一致,如何对应? #### Q: Excel 表头标题名称跟 class 属性名称不一致,如何对应?
A. 请使用 ExcelColumnName 作 mapping A. 请使用 ExcelColumnName 作 mapping

View File

@ -229,9 +229,21 @@ foreach(IDictionary<string,object> row in MiniExcel.Query(path))
} }
``` ```
![image](https://user-images.githubusercontent.com/12729184/116673475-07917200-a9d6-11eb-947e-a6f68cce58df.png)
#### 9. Query 讀 Excel 返回 DataTable
提醒 : 不建議使用因為DataTable會將數據`全載入內存`失去MiniExcel低記憶體消耗功能。
```C#
var table = MiniExcel.QueryAsDataTable(path, useHeaderRow: true);
```
![image](https://user-images.githubusercontent.com/12729184/116673475-07917200-a9d6-11eb-947e-a6f68cce58df.png)
### 寫 Excel <a name="getstart2"></a> ### 寫 Excel <a name="getstart2"></a>
@ -304,7 +316,11 @@ using (var stream = File.Create(path))
} }
``` ```
#### 6. 支持 IDataReader 參數
```csharp
MiniExcel.SaveAs(path, reader);
```
@ -768,36 +784,6 @@ public static IEnumerable<T> Page<T>(IEnumerable<T> en, int pageSize, int page)
### FAQ 常見問題 ### FAQ 常見問題
#### Q: 如何將查詢結果轉為 DataTable
提醒 : 不建議使用因為DataTable會將數據`全載入記憶體`失去MiniExcel低記憶體消耗功能。
```csharp
public static DataTable QueryAsDataTable(string path)
{
var rows = MiniExcel.Query(path, true);
var dt = new DataTable();
var first = true;
foreach (IDictionary<string, object> row in rows)
{
if (first)
{
foreach (var key in row.Keys)
{
var type = row[key]?.GetType() ?? typeof(string);
dt.Columns.Add(key, type);
}
first = false;
}
dt.Rows.Add(row.Values.ToArray());
}
return dt;
}
```
![image](https://user-images.githubusercontent.com/12729184/115068722-3105c480-9f25-11eb-8f5a-994416754134.png)
#### Q: Excel 表頭標題名稱跟 class 屬性名稱不一致,如何對應? #### Q: Excel 表頭標題名稱跟 class 屬性名稱不一致,如何對應?
A. 請使用 ExcelColumnName 作 mapping A. 請使用 ExcelColumnName 作 mapping

View File

@ -8,6 +8,7 @@
### 0.13.5 ### 0.13.5
- [New] Support QueryAsDataTable method [#216](https://github.com/shps951023/MiniExcel/issues/216) - [New] Support QueryAsDataTable method [#216](https://github.com/shps951023/MiniExcel/issues/216)
- [New] SaveAs support IDataReader value parameter [#211](https://github.com/shps951023/MiniExcel/issues/211)
- [Bug] Fix numeric format string will be cast to numeric type [#I3OSKV](https://gitee.com/dotnetchina/MiniExcel/issues/I3OSKV) - [Bug] Fix numeric format string will be cast to numeric type [#I3OSKV](https://gitee.com/dotnetchina/MiniExcel/issues/I3OSKV)
- [Opt] Optimize SaveAs convert value type logic to improve performance - [Opt] Optimize SaveAs convert value type logic to improve performance

View File

@ -9,6 +9,7 @@
### 0.13.5 ### 0.13.5
- [New] 新增 QueryAsDataTable 方法 [#216](https://github.com/shps951023/MiniExcel/issues/216) - [New] 新增 QueryAsDataTable 方法 [#216](https://github.com/shps951023/MiniExcel/issues/216)
- [New] SaveAs 支持 IDataReader value 参数 [#211](https://github.com/shps951023/MiniExcel/issues/211)
- [Bug] 修正数字格式的字串会被强制转换为decimal类型 [#I3OSKV](https://gitee.com/dotnetchina/MiniExcel/issues/I3OSKV) - [Bug] 修正数字格式的字串会被强制转换为decimal类型 [#I3OSKV](https://gitee.com/dotnetchina/MiniExcel/issues/I3OSKV)
- [Opt] 优化 SaveAs 类别转换算法,避免效率浪费 - [Opt] 优化 SaveAs 类别转换算法,避免效率浪费

View File

@ -9,6 +9,7 @@
### 0.13.5 ### 0.13.5
- [New] 新增 QueryAsDataTable 方法 [#216](https://github.com/shps951023/MiniExcel/issues/216) - [New] 新增 QueryAsDataTable 方法 [#216](https://github.com/shps951023/MiniExcel/issues/216)
- [New] SaveAs 支持 IDataReader value 參數 [#211](https://github.com/shps951023/MiniExcel/issues/211)
- [Bug] 修正數字格式的字串會被強制轉換為decimal類型 [#I3OSKV](https://gitee.com/dotnetchina/MiniExcel/issues/I3OSKV) - [Bug] 修正數字格式的字串會被強制轉換為decimal類型 [#I3OSKV](https://gitee.com/dotnetchina/MiniExcel/issues/I3OSKV)
- [Opt] 優化 SaveAs 類別轉換算法,避免效率浪費 - [Opt] 優化 SaveAs 類別轉換算法,避免效率浪費

View File

@ -106,7 +106,7 @@
/// <summary> /// <summary>
/// This method is not recommended, because it'll load all data into memory. /// This method is not recommended, because it'll load all data into memory.
/// </summary> /// </summary>
public static DataTable QueryAsDataTable(string path, bool useHeaderRow = false, string sheetName = null, ExcelType excelType = ExcelType.UNKNOWN, IConfiguration configuration = null) public static DataTable QueryAsDataTable(string path, bool useHeaderRow = true, string sheetName = null, ExcelType excelType = ExcelType.UNKNOWN, IConfiguration configuration = null)
{ {
using (var stream = Helpers.OpenSharedRead(path)) using (var stream = Helpers.OpenSharedRead(path))
return QueryAsDataTable(stream, useHeaderRow, sheetName, GetExcelType(path, excelType), configuration); return QueryAsDataTable(stream, useHeaderRow, sheetName, GetExcelType(path, excelType), configuration);
@ -115,7 +115,7 @@
/// <summary> /// <summary>
/// This method is not recommended, because it'll load all data into memory. /// This method is not recommended, because it'll load all data into memory.
/// </summary> /// </summary>
public static DataTable QueryAsDataTable(this Stream stream, bool useHeaderRow = false, string sheetName = null, ExcelType excelType = ExcelType.UNKNOWN, IConfiguration configuration = null) public static DataTable QueryAsDataTable(this Stream stream, bool useHeaderRow = true, string sheetName = null, ExcelType excelType = ExcelType.UNKNOWN, IConfiguration configuration = null)
{ {
var dt = new DataTable(); var dt = new DataTable();
dt.TableName = sheetName; dt.TableName = sheetName;

View File

@ -1,7 +1,7 @@
<Project Sdk="Microsoft.NET.Sdk"> <Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup> <PropertyGroup>
<TargetFrameworks>net45;netstandard2.0;net5.0</TargetFrameworks> <TargetFrameworks>net45;netstandard2.0;net5.0</TargetFrameworks>
<Version>0.13.4</Version> <Version>0.13.5</Version>
</PropertyGroup> </PropertyGroup>
<PropertyGroup> <PropertyGroup>
<AssemblyName>MiniExcel</AssemblyName> <AssemblyName>MiniExcel</AssemblyName>

View File

@ -165,6 +165,10 @@ namespace MiniExcelLibs.OpenXml
{ {
GenerateSheetByDataTable(writer, archive, value as DataTable, printHeader); GenerateSheetByDataTable(writer, archive, value as DataTable, printHeader);
} }
else if (value is IDataReader)
{
GenerateSheetByIDataReader(writer, archive, value as IDataReader, printHeader);
}
else else
{ {
throw new NotImplementedException($"Type {type.Name} & genericType {genericType.Name} not Implemented. please issue for me."); throw new NotImplementedException($"Type {type.Name} & genericType {genericType.Name} not Implemented. please issue for me.");
@ -332,6 +336,55 @@ namespace MiniExcelLibs.OpenXml
writer.Write("</x:sheetData></x:worksheet>"); writer.Write("</x:sheetData></x:worksheet>");
} }
private void GenerateSheetByIDataReader(StreamWriter writer, MiniExcelZipArchive archive, IDataReader value, bool printHeader)
{
var xy = ExcelOpenXmlUtils.ConvertCellToXY("A1");
writer.Write($@"<?xml version=""1.0"" encoding=""utf-8""?><x:worksheet xmlns:x=""http://schemas.openxmlformats.org/spreadsheetml/2006/main"">");
{
var yIndex = xy.Item2;
// TODO: dimension
//var maxRowIndex = value.Rows.Count + (printHeader && value.Rows.Count > 0 ? 1 : 0);
//var maxColumnIndex = value.Columns.Count;
//writer.Write($@"<x:dimension ref=""{GetDimensionRef(maxRowIndex, maxColumnIndex)}""/>");
writer.Write("<x:sheetData>");
int fieldCount = value.FieldCount;
if (printHeader)
{
writer.Write($"<x:row r=\"{yIndex.ToString()}\">");
var xIndex = xy.Item1;
for (int i = 0; i < fieldCount; i++)
{
var r = ExcelOpenXmlUtils.ConvertXyToCell(xIndex, yIndex);
writer.Write($"<x:c r=\"{r}\" t=\"str\">");
writer.Write($"<x:v>{value.GetName(i)}");
writer.Write($"</x:v>");
writer.Write($"</x:c>");
xIndex++;
}
writer.Write($"</x:row>");
yIndex++;
}
while (value.Read())
{
writer.Write($"<x:row r=\"{yIndex.ToString()}\">");
var xIndex = xy.Item1;
for(int i = 0; i < fieldCount; i++)
{
var cellValue = value.GetValue(i);
WriteCell(writer, yIndex, xIndex, cellValue);
xIndex++;
}
writer.Write($"</x:row>");
yIndex++;
}
}
writer.Write("</x:sheetData></x:worksheet>");
}
private void GenerateContentTypesXml(MiniExcelZipArchive archive, Dictionary<string, ZipPackageInfo> packages) private void GenerateContentTypesXml(MiniExcelZipArchive archive, Dictionary<string, ZipPackageInfo> packages)
{ {
//[Content_Types].xml //[Content_Types].xml

View File

@ -12,6 +12,8 @@ using Newtonsoft.Json;
using MiniExcelLibs.Attributes; using MiniExcelLibs.Attributes;
using MiniExcelLibs.Tests.Utils; using MiniExcelLibs.Tests.Utils;
using System.Data; using System.Data;
using System.Data.SQLite;
using Dapper;
namespace MiniExcelLibs.Tests namespace MiniExcelLibs.Tests
{ {
@ -23,6 +25,32 @@ namespace MiniExcelLibs.Tests
this.output = output; this.output = output;
} }
/// <summary>
/// [Can SaveAs support iDataReader export to avoid the dataTable consuming too much memory · Issue #211 · shps951023/MiniExcel]
/// (https://github.com/shps951023/MiniExcel/issues/211)
/// </summary>
[Fact]
public void Issue211()
{
var path = PathHelper.GetNewTemplateFilePath();
var tempSqlitePath = Path.Combine(Path.GetTempPath(), $"{Guid.NewGuid()}.db");
var connectionString = $"Data Source={tempSqlitePath};Version=3;";
using (var connection = new SQLiteConnection(connectionString))
{
var reader = connection.ExecuteReader(@"select 1 Test1,2 Test2 union all select 3 , 4 union all select 5 ,6");
MiniExcel.SaveAs(path, reader);
var rows = MiniExcel.Query(path,true).ToList();
Assert.Equal((double)1, rows[0].Test1);
Assert.Equal((double)2, rows[0].Test2);
Assert.Equal((double)3, rows[1].Test1);
Assert.Equal((double)4, rows[1].Test2);
}
}
/// <summary> /// <summary>
/// [When reading Excel, can return IDataReader and DataTable to facilitate the import of database. Like ExcelDataReader provide reader.AsDataSet() · Issue #216 · shps951023/MiniExcel](https://github.com/shps951023/MiniExcel/issues/216) /// [When reading Excel, can return IDataReader and DataTable to facilitate the import of database. Like ExcelDataReader provide reader.AsDataSet() · Issue #216 · shps951023/MiniExcel](https://github.com/shps951023/MiniExcel/issues/216)
/// </summary> /// </summary>
@ -34,14 +62,14 @@ namespace MiniExcelLibs.Tests
MiniExcel.SaveAs(path, value); MiniExcel.SaveAs(path, value);
{ {
var dt = MiniExcel.QueryAsDataTable(path, true); var table = MiniExcel.QueryAsDataTable(path);
var columns = dt.Columns; var columns = table.Columns;
Assert.Equal("Test1", dt.Columns[0].ColumnName); Assert.Equal("Test1", table.Columns[0].ColumnName);
Assert.Equal("Test2", dt.Columns[1].ColumnName); Assert.Equal("Test2", table.Columns[1].ColumnName);
Assert.Equal("1", dt.Rows[0]["Test1"]); Assert.Equal("1", table.Rows[0]["Test1"]);
Assert.Equal((double)2, dt.Rows[0]["Test2"]); Assert.Equal((double)2, table.Rows[0]["Test2"]);
Assert.Equal("3", dt.Rows[1]["Test1"]); Assert.Equal("3", table.Rows[1]["Test1"]);
Assert.Equal((double)4, dt.Rows[1]["Test2"]); Assert.Equal((double)4, table.Rows[1]["Test2"]);
} }
{ {