March 01, 2004 (research)
Linux password files, MD5 Algorithms
Apart from reading more of Schneier's "Secrets & Lies" did some googling on
- Linux Passwords
- Linux Shadow
- Linux Password Algorithm
- Linux MD5
- Linux Password MD5 Algorithm
And some other searchings using different iterations of the above mentioned terms.
What are the benefits of the Shadow File
Because usernames and uid/gid's are stored within the passwd file it is required to be read by the system. This allows anyone to easily gain access to the hashed passwords stored in the /etc/passwd file. To remedy this the /etc/shadow file is used, which is set to read-only for a few authorised users (usually only root by default).
This means the username's and uid/gid's are still stored in /etc/passwd but the hashed password is stored in a stronger-permissioned file. ie /etc/shadow.
There are more benefits, such as specifying when a password has to be updated and the specified maximum age of a password too.
Benefits of MD5 hashed passwords compared to DES hashed passwords
DES's plaintext has a maximum of 8 characters, MD5 can accept much larger lengths. (In fact, isn't MD5 not limited by length at all, md5sum being able to return a 128-bit hash on any file length input?)
"Foiling the Cracker: A Survey Of, and Improvements to, Password Security" by Daniel V. Klein (1990)
This text was too old to be of any use. The author discusses the potential security benefits of using shadow'd password files instead of shared password files, but then follows on saying that most systems won't upgrade their software.. when in fact most modern unix/linux OS's enable Shadow'd password files by default.
RFC1321 on MD5 Algorithm
Very interesting, the Memo sent out about the new MD5 algorithm, this text document also includes the c code down the bottom of the page, and on initial inspection seems very similar to the MD5 code used by John. (although some changes appear to have been made, are these optimisations for JtR? or for other systems? .. )
RSA Lab's Bulletin "On Recent Results for MD2, MD4 and MD5" by M.J.B. Robshaw (1996)
This mentions that the weakness of the MD5 hashing algorithm isn't so much that collisions are discovered easily, it's more that pseudo-collisions occur. Where these are collisions which happen because of the compression part of the MD5 algorithm as opposed to the actual hashing itself. Dobbertin's article (2 pager) describes the situation used to cause this pseudo-collision.
"Cryptanalysis of MD5 Compress" by Hans Dobbertin (1996)
As mentioned, this outlines the conditions used to force pseudo-collisions by the MD5 hashing algorithm.