My problem is that I need to export 90.000+ rows / 143 cols from a DataGridView (populated from MySQL database) to Excel. Whatever I do I always end up with 'System.Out.Of.Memory' exception after 45k-60k rows depending of the solution. I know that there could be questions like 'Why do you need so much rows' and I would answer that 'Unfortunately this is needed.' I have searched forums about my problem but haven't found any working solution. I tried StreamWriter to CSV, processing data in chunks (the solution below), also using multiple Excel or CSV files, but nothing helped. Every time during execution RAM usage is growing and not released after a successful export, when I try with less amount of rows. I don't know when and if after a successful execution RAM is released.
Test machines have 8 GB of RAM and are using Windows 10. Unfortunately I am not able to use the resources of MySQL server for processing of Excel export there and then output file to be shared with user, so I need to use the client machines.
Below is my latest not-working solution, where data is read from DGV and wrote to Excel in chunks. Changing the size of chunks is not reducing memory consumption and if I make it smaller (like 500 to 2000) the only effect is that exporting is getting slower.
Imports Excel = Microsoft.Office.Interop.Excel
Private Sub Button2_Click(sender As Object, e As EventArgs) Handles Button2.Click
If DataGridView1.Rows.Count > 0 Then
Dim filename As String = ""
Dim SV As SaveFileDialog = New SaveFileDialog()
SV.FileName = "Worst_cells"
SV.Filter = "xlsx files (*.xlsx)|*.xlsx|All files (*.*)|*.*"
SV.FilterIndex = 1
SV.RestoreDirectory = True
Dim result As DialogResult = SV.ShowDialog()
If result = DialogResult.OK Then
filename = SV.FileName
Dim XCELAPP As Microsoft.Office.Interop.Excel.Application = Nothing
Dim XWORKBOOK As Microsoft.Office.Interop.Excel.Workbook = Nothing
Dim XSHEET As Microsoft.Office.Interop.Excel.Worksheet = Nothing
Dim misValue As Object = System.Reflection.Missing.Value
XCELAPP = New Excel.Application()
XWORKBOOK = XCELAPP.Workbooks.Add(misValue)
XCELAPP.DisplayAlerts = False
XCELAPP.Visible = False
XSHEET = XWORKBOOK.ActiveSheet
XSHEET.Range("B1").ColumnWidth = 11
For Each column As DataGridViewColumn In DataGridView1.Columns
XSHEET.Cells(1, column.Index + 1) = column.HeaderText
Next
Dim rowCnt As Integer = DataGridView1.Rows.Count
Dim colCnt As Integer = DataGridView1.Columns.Count
Dim batchSize As Integer = 10000
Dim currentRow As Integer = 0
Dim valueObjArray As Object(,) = New Object(batchSize - 1, colCnt - 1) {}
While currentRow < rowCnt
Dim rowIndex As Integer = 0
While rowIndex < batchSize AndAlso currentRow + rowIndex < rowCnt
For colIndex As Integer = 0 To colCnt - 1
valueObjArray(rowIndex, colIndex) = DataGridView1(colIndex, currentRow + rowIndex).Value
Next
rowIndex += 1
End While
Dim colName As String = ColumnLetter(colCnt)
If (currentRow + batchSize + 1) < rowCnt Then
XSHEET.Range("A" + (currentRow + 2).ToString(), colName + (currentRow + batchSize + 1).ToString()).Value2 = valueObjArray
Else
XSHEET.Range("A" + (currentRow + 2).ToString(), colName + (rowCnt + 1).ToString()).Value2 = valueObjArray
End If
XWORKBOOK.SaveAs(filename)
currentRow += batchSize
End While
XCELAPP.DisplayAlerts = True
XWORKBOOK.Close(False)
XCELAPP.Quit()
Try
System.Runtime.InteropServices.Marshal.ReleaseComObject(XSHEET)
System.Runtime.InteropServices.Marshal.ReleaseComObject(XWORKBOOK)
System.Runtime.InteropServices.Marshal.ReleaseComObject(XCELAPP)
Catch
End Try
GC.Collect()
GC.WaitForPendingFinalizers()
GC.Collect()
GC.WaitForPendingFinalizers()
End If
End If
End Sub
Rangeobjects. Typically it would be something likeRange r = ...; r.Value2 = ...; Marshal.ReleaseComObject(r);Double-dotting in your code should be avoided.DataRow.ItemArrayinstead of through the DGV Cell.Value to avoid making copies of the data.For Eachloop over the rows collection.