Ok, here's a function I wrote a while ago in VB 6 which was painfully slow:

Code:

Public Function lcsArr_byte(ByRef a1() As Byte, ByRef a2() As Byte) As Byte()
On Error Resume Next
' Builds the LCS table for two arrays of Bytes
Dim res() As Byte
Dim i As Long
Dim j As Long
Dim lSizeA As Long
Dim lSizeB As Long
lSizeA = UBound(a1)
lSizeB = UBound(a2)
ReDim res(0 To lSizeA, 0 To lSizeB)
For i = 0 To lSizeA
For j = 0 To lSizeB
If a1(i) = a2(j) Then
res(i + 1, j + 1) = res(i, j) + 1
Else
res(i + 1, j + 1) = MaxB(res(i, j + 1), res(i + 1, j))
End If
Next ' j
Next ' i
lcsArr_byte = res
End Function
Public Function MaxB(ByVal a As Byte, ByVal b As Byte) As Byte
' Returns the MAX() of a pair of Bytes
If a >= b Then
MaxB = a
Else
MaxB = b
End If
End Function

So if you read 1 MB (1048576 bytes) into each buffer, just to build the table means 1,099,511,627,776 iterations because the inner loop (j) is processed for every outer loop (i). Now imagine doing this for every 1 MB in a 3.5 GB file.

That's just for computing the LCS table, nevermind actually determining the DIFF.