Stack Overflow Exception in VB.net

Question

I was troubleshooting an application today at work that runs periodically to handle some data processing needs, it often runs just fine but sometimes the application will crash and someone will have to manually restart it.

After looking at the source and some debugging it seems the core of the program relies on a recursive function that is removing rows from a DataTable and then stopping once the DataTable no longer has any rows remaining.

Here is the code:

Public Sub ProcessData(ByRef dtTable As DataTable)
  Dim DataView1 As New DataView()
  Dim UID As String = ""
  Dim dtUID As New DataTable

  dtUID = dtTable.Clone()
  UID = dtTable.Rows(0)("UID")    

  DataView1 = dtTable.DefaultView

  Dim expression As String = "UID ='" & UID & "'"
  DataView1.RowFilter = expression

  For n As Integer = 0 To DataView1.Count - 1
      dtUID.ImportRow(DataView1.Item(n).Row)
  Next

  Dim foundRows() As DataRow
  foundRows = dtTable.Select(expression)

  For n As Integer = 0 To foundRows.GetUpperBound(0)
      dtTable.Rows.Remove(foundRows(n))
  Next

  For n As Integer = 0 To dtUID.Rows.Count - 1
      ProcessRecord(dtUID, n)
  Next

  If dtTable.Rows.Count > 0 Then
      ProcessData(dtTable)
  End If
End Sub

ProcessRecord() is a function that applies all the business logic to the record and also does a write to a database. The unhandled stack overFlow exception happens at:

For n As Integer = 0 To DataView1.Count - 1

In the real world this function would be fed a DataTable with around 100,000 records in it. In testing it, it will run through about 40,000-65,000 pretty consistently after many trial runs before it has the stack overflow. The code that creates the DataTable and ProcessRecord() flag records that have already been processed, so if you're working through a 100,000 record large data set then subsequent calls to this method after a crash will be passed a smaller DataTable (35,000-60,000 records.) Interestingly the stack overflow will happen more and more often with smaller DataTables, I can consistently get through 40k-65k of a 100k DataTable but cannot consistently get through 15k of a 15k DataTable without a crash (eventually it will complete all records, with multiple crashes over the last few thousand.)

Once I saw what the problem was I basically scrapped most of this code and used an implementation I've used for similar tasks in the past. But I really wanted to understand exactly why this code was causing this problem.

eventually it will complete all records, with multiple crashes over the last few thousand. That's not possible. Stack trace required. — Hans Passant
– Hans Passant, Commented Apr 5, 2011 at 23:21

Tim Sylvester · Accepted Answer · 2011-04-05 22:34:00Z

2

The problem is simply that it is a recursive function.

Each function invocation uses stack space to store its parameters and local variables, and stack space is limited.

There's no reason that this particular operation should be recursive. In fact, when the maximum number of recursions cannot be estimated or limited easily, it's a bad idea to use a recursive solution.

answered Apr 5, 2011 at 22:34

Tim Sylvester

23.2k2 gold badges82 silver badges99 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Brandon Over a year ago

Alright, that makes sense. In the past I have typically just used a loop to iterate through rows in a DataTable in scenarios such as this. Since I'm relatively new when I saw this code I was working under the assumption the developer who originally wrote it had some reason for his approach. The MSDN documentation I had seen mentioned looking out for infinite recursion but that was obviously not applicable here.

Collectives™ on Stack Overflow

Stack Overflow Exception in VB.net

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related