How to: Query for Duplicate Files in a Directory Tree (LINQ) (Visual Basic)
Sometimes files that have the same name may be located in more than one folder. For example, under the Visual Studio installation folder, several folders have a readme.htm file. This example shows how to query for such duplicate file names under a specified root folder. The second example shows how to query for files whose size and creation times also match.
Example
Module QueryDuplicateFileNames
Public Sub Main()
Dim path As String = "C:\Program Files\Microsoft Visual Studio 9.0\Common7"
QueryDuplicates1(path)
' Uncomment to run this query instead
' QueryDuplicates2(path)
End Sub
Sub QueryDuplicates1(ByVal root As String)
Dim dir As New System.IO.DirectoryInfo(root)
Dim duplicates = From aFile In dir.GetFiles("*.*", System.IO.SearchOption.AllDirectories) _
Order By aFile.Name _
Group aFile By aFile.Name Into newGroup = Group _
Where newGroup.Count() >= 2 _
Select newGroup
' Page the display so that the results can be read.
Dim trimLength = root.Length
PageOutput(duplicates, trimLength)
End Sub
Sub QueryDuplicates2(ByVal root As String)
' This time a composite key is used. This sub finds all files
' that have been copied into multiple subfolders.
Dim dir As New System.IO.DirectoryInfo(root)
Dim duplicates = From aFile In Dir.GetFiles("*.*", System.IO.SearchOption.AllDirectories) _
Order By aFile.Name _
Group aFile By aFile.Name, aFile.CreationTime, aFile.Length Into newGroup = Group _
Where newGroup.Count() >= 2 _
Select newGroup
' Page the display so that the results can be read.
Dim trimLength = root.Length
PageOutput(duplicates, trimLength)
End Sub
' Pages console display for large query results. No more than one group per page.
' This sub specifically works with group queries of FileInfo objects
' but can be modified for any type.
Sub PageOutput(ByVal groupQuery, ByVal charsToSkip)
' "3" = 1 line for extension key + 1 for "Press any key" + 1 for input cursor.
Dim numLines As Integer = Console.WindowHeight - 3
' Flag to indicate whether there are more results to display
Dim goAgain As Boolean = True
For Each fg As IEnumerable(Of System.IO.FileInfo) In groupQuery
' Start a new extension at the top of a page.
Dim currentLine As Integer = 0
Do While (currentLine < fg.Count())
Console.Clear()
' Get the next page of results
' No more than one filename per page
Dim resultPage = From file In fg _
Skip currentLine Take numLines
' Execute the query. Trim the paths in the output.
For Each line In resultPage
Console.WriteLine(vbTab & line.FullName.Substring(charsToSkip))
Next
' Advance the current position
currentLine = numLines + currentLine
' Give the user a chance to break out of the loop
Console.WriteLine("Press any key for next page or the 'End' key to exit.")
Dim key As ConsoleKey = Console.ReadKey().Key
If key = ConsoleKey.End Then
goAgain = False
Exit For
End If
Loop
Next
End Sub
End Module
The first query uses a simple key to determine a match; this finds files that have the same name but whose contents might be different. The second query uses a compound key to match against three properties of the FileInfo object. This query is much more likely to find files that have the same name and similar or identical content.
Compile the code
Create a Visual Basic console application project, with an Imports statement for the System.Linq namespace.
