Password strength testers are becoming more common in the account sign-up process. Their purpose is to indicate whether the passwords chosen by users are weak, good, strong, or very strong – the implication being that good, strong, and very strong passwords will help protect the account from brute force attacks. But how accurate are password strength testers?
To find out, we ran a test pitching five variations of commonly-used passwords against five password strength testers. We also ran the same test with a randomly-generated password and a randomly-generated passphrase to determine if accuracy issues existed with more complex passwords and NIST-recommended passphrases.
How We Set Up Our Passwords
We picked five commonly-used passwords from the top 100 passwords identified in data breaches by the Open Web Application Security Project (OWASP). Because all password managers will identify these as weak passwords, we amended the passwords to add complexity – either by adding numbers, letters, or substituting numbers and letters for special characters. Consequently:
password became password2022. By adding the year to the second most commonly used password, we increased the entropy count (randomness) from 37.60 to 62.04; which – according to a 1Password blog – is just about “sufficient for any purpose”. As you will see, it isn’t.
qwerty became qwertybank. Like adding the year to “password” (above) or adding your date of birth to your name, adding the nature of an account to a weak password is a common practice. Unfortunately, this does not turn a weak password into a strong password.
letmein became letmeintowalmart. This change to one of the most commonly-breached passwords resulted in a 16-character password – longer than the minimum password length recommended by Microsoft (14 characters) and many other leading security experts.
trustno1 became trustñ01. The password trustno1 implies the user of this password is security-conscious, but this password is the 37th most-breached password. We strengthened it by exchanging the “n” for an “ñ” and the letter “o” for the number “0”.
abc123 became abc123ABC!@#. Many password security blogs recommend using a combination of upper- and lower-case letters, numbers, and special characters. In this case, we used the special characters by holding down the shift key when typing 123 on a U.S. keyboard.
How We Chose Our Password Strength Testers
Although there are hundreds of password strength testers, the majority only tell you if a password is weak, good, strong, or very strong. We wanted to access the accuracy of password strength testers by using a more sophisticated measurement, so we picked five testers that indicate how long it would take a brute force algorithm to crack the password.
The five password strength testers chosen represent a cross-section of those freely available on the Internet:
- All Things Secured is a multi-purpose online security site acting as an affiliate for many different security software vendors.
- Bitwarden is a leading open source password manager with a good range of plans for individuals, families, and businesses.
- NordPass is a password manager developed by the people behind NordVPN that has a less-than-transparent subscription structure.
- Psono is a feature-limited open source password manager which can represent value in some use cases but with a high management overhead.
- Security.org is another multi-purpose online security site full of reviews pointing you towards the best affiliate deals (for them – not you).
Once we had chosen our five testing tools, we ran our five amended passwords through each one to ascertain how long might survive a brute force attack.
The results of our test show a massive inconsistency between password strength testers. The All Things Secured tester produced the results we largely expected, and – with the exception of the “Walmart” password – so did Bitwarden. However, businesses checking the strength of user passwords against the NordPass, Psono, or Security.org testers could be left with a false sense of security.
How about Random Complex Passwords and Passphrases?
It is well chronicled that humans are useless are creating random complex passwords and passphrases. Therefore, to conduct the second half of our test (and to ensure impartiality), we used Dashlane’s password tool to generate “T66sEmj2Gi@X” and randompassphrasegenerator.com to generate the passphrase “scheme stir grass” (with spaces to add entropy). The results were:
T66sEmj2Gi@X
- All Things Secured – centuries to crack
- Bitwarden – 3 years to crack
- NordPass – 3 years to crack
- Psono – 748 years to crack
- Security.org – 34,000 years to crack
scheme stir grass
- All Things Secured – centuries to crack
- Bitwarden – 92 years to crack
- NordPass – centuries to crack
- Psono – 747,212,696 years to crack
- Security.org – 10 billion years to crack
These tests further highlight the inconsistency of password strength testers. It also makes you wonder how good Dashlane’s password tool is at generating complex passwords considering the results we got from Bitwarden’s and NordPass’ testers. In terms of accuracy, although we like the precision of Psono’s tester, we feel that Bitwarden’s testing tool is probably the most accurate.
Conclusion – Don’t Rely on One Password Strength Tester
In all things related to online security, it is important to verify any information provided to you. Therefore, we recommend you repeat our tests with your own passwords and test any new or replacement passwords across multiple password strength testers to ensure they are sufficiently strong to resist brute force attacks.
We also recommend you add an extra layer of security to online accounts by applying two-factor authentication wherever possible. Most online websites support two-factor authentication, while most password managers provide free tools to mitigate any perceived inconvenience of introducing an extra step into your online security.