Duplicate Files

I’m using XP.

If I run a program that detects duplicate files (by hashing not size) on the FS would the results be reliable?

Thanx 4 looking.

Hashing algorithms don’t guarantee uniqueness, they are designed such that files that share the same hash have to be substantially different.

The chances of you getting a collision where there isn’t one is really, REALLY small.

1 in 340,282,366,920,938,463,463,374,607,431,768,211,456 for MD5 for example and MD5 is considered “broken” in some applications.

I personally would trust MD5 to find duplicates but if you want additional assurance, diff any collisions before deleting copies.

Thanks for replying noidd.

My concern was regarding using software designed for NTFS on the non-NTFS Drobo FS. Sorry for not being more specific.

Well, something that does file checksums shouldn’t do anything filesystem specific unless it does things like attempt to checksum the metatdata too, but that would be dumb.


Thanx for the encouragement.

I’ve run over 3TB of file copies with CRC-checks (using ViceVersa Pro) on my Drobo, both direct-connected and on DroboShare with no trouble.