In order to see if a cell contains #NA you need to trap this type of error, which is a 2-Steps If.
Also, allways loop backwards when deleting Rows, use For r = lastRow2 To 3 Step -1.
Try the code below, explanations inside the code's comments:
Option Explicit
Sub DelNARows()
Dim lastRow2 As Long
Dim r As Long
Dim CellVal As Variant
With Worksheets("csvFile") ' use with statement to qualify all Range, Rows and Cells object nested inside
lastRow2 = .Cells(.Rows.Count, 1).End(xlUp).Row
For r = lastRow2 To 3 Step -1 ' always loop backward when deleting rows
' trap #NA error section
On Error Resume Next
CellVal = .Cells(r, 1).Value
On Error GoTo 0
If IsError(CellVal) Then
If CellVal = CVErr(xlErrNA) Then ' check if current error if xlErrNA (2042)
.Rows(r).Delete
End If
End If
Next r
End With
End Sub
Edit 1: a faster way to delete multiple rows, is to delete all the rows at one shot, not one-by-one. You can achieve that by merging all the rows to be deleted in a DelRng (it's a Range object which is combined from all the rows that needs to be deleted).
Code
For r = lastRow2 To 3 Step -1
' trap #NA error section
On Error Resume Next
CellVal = .Cells(r, 1).Value
On Error GoTo 0
Dim DelRng As Range ' define a new range to save all the rows to delete
If IsError(CellVal) Then
If CellVal = CVErr(xlErrNA) Then ' check if current error if xlErrNA (2042)
If Not DelRng Is Nothing Then
Set DelRng = Application.Union(DelRng, .Rows(i))
Else
Set DelRng = .Rows(i)
End If
End If
End If
Next r
' delete the entire rows at once
If Not DelRng Is Nothing Then DelRng.Delete
Cells(r, 1).valueIf Worksheets("csvFile").Cells(r, 1) == "#N/A" Then