How to Find Duplicate Files on a Mac
Two files with the same name aren't necessarily duplicates, and two files with different names might be byte-for-byte identical. Finder's search only sees the first case.
Why Finder search isn't a duplicate finder
Searching for a filename in Finder returns files that share that name, or that share text somewhere in the file. It has no concept of content matching. A photo saved twice as IMG_4021.jpg and vacation_photo.jpg is invisible to it, and a real duplicate. Two entirely different spreadsheets that both happen to be named budget.xlsx will show up as a false match.
The Terminal way: hashing
A cryptographic hash turns any file into a short fixed string, and identical content always produces an identical hash, regardless of filename or location. Two files with the same hash are duplicates, full stop.
To hash every file in a folder and list ones that match:
find ~/Downloads -type f -exec md5 -r {} \; | sort | uniq -w 32 -D The -r flag puts the hash first on each line, ahead of the filename, which is what makes the next part work. sort groups matching hashes together, and uniq -w 32 -D prints every line where the first 32 characters (an MD5 hash is always 32) repeat. What's left is pairs of files with identical content, regardless of what they're named or where they sit inside the folder.
What a real duplicate scan needs to get right
- True content comparison, not filename matching, so renamed copies are caught.
- Coverage across every folder that accumulates duplicates: Documents, Downloads, Desktop, Movies, Pictures and Music, not just one directory at a time.
- A safe review step before deleting, so nothing gets removed on a hash collision or a moment of carelessness. MD5 collisions are astronomically rare for ordinary files, but the review step is what makes the difference between a tool and a loaded gun.
Full MD5 comparison across every folder that matters, with review before delete
CleanMachine's Duplicate Finder hashes files across Documents, Downloads, Desktop, Movies, Pictures and Music at once, and shows you every match before anything is removed. Free to scan.
↓ Download Free & ScanPhotos specifically are their own case, near-duplicates like burst shots and slight crops don't hash the same even when they're visually redundant. That needs a different approach, covered in finding duplicate photos.