Typically sites store their users' passwords "encrypted" or "hashed" (which applies a function or algorithm to the user's password to produce a different value) - though this is not always the case. In order to "crack passwords" (that have been "hashed", and presumably retrieved when a website is hacked) we need to test input values, combined with the same function or algorithm (and sometimes a "salt"), so see which produces the same hash value - allowing us to discover the original password (this is a bit like working out what numbers multiplied together resulted in a known total, eg: "20 = 5 x ?").
A popular tool to do this is "hashcat" which has a whole bunch of options and functionality. This is blog post is a dump of a noob's thoughts and findings related to hashcat, rulesets and wordlists.
1. Common password prefixes and suffixes
Passwords come in all shapes and sizes, sometimes due to human choice and "randomness" and sometimes due to company/site password rules, but as it turns out humans tend to behave fairly similarly. A common way to "secure" a password is to add letters, numbers, or words to the beginning or end of a password. I wanted to get an idea of what the most common prefixes and suffixes are, so put together a script to lookup known passwords starting or ending with common dictionary words:
#!/bin/bash
WORDLIST=$1
# 1. make our "lookup" dictionary (lower cased unique words without punctuation from system dictionary file)
rg "[a-zA-Z]{3,99}$" /usr/share/dict/american-english | tr '[:upper:]' '[:lower:]' | sort | uniq > /tmp/dict-lower
echo "Created /tmp/dict-lower"
# 2. find all wordlist lines ending with a word in our "lookup" dictionary
rm all_prefixes.txt
cat /tmp/dict-lower | while read line; do
echo "prefix $line"
rg -io ".*$line\$" $WORDLIST | tr '[:upper:]' '[:lower:]' | sed "s/$line//g" >> all_prefixes.txt
done;
echo "Created all_prefixes.txt"
# 3. find all wordlist lines starting with a word in our "lookup" dictionary
rm all_suffixes.txt
cat /tmp/dict-lower | while read line; do
echo "suffix: $line"
rg -io "^$line.*" $WORDLIST | tr '[:upper:]' '[:lower:]' | sed "s/$line//g" >> all_suffixes.txt
done;
echo "Created all_suffixes.txt"
echo "Sorting and counting..."
sort all_prefixes.txt | sed '/^$/d' | uniq -c | sort -nr > prefixes.txt
sort all_suffixes.txt | sed '/^$/d' | uniq -c | sort -nr > suffixes.txt
echo "Created prefixes.txt and suffixes.txt"
(note this is slow, even using RipGrep, it should probably make use of the "-f" flag to read rules in from a file rather than invoking ripgrep for every line... oh well)
I ran this against the "Xato top 10 million passwords", RockYou password list, Dropbox leaked passwords, and Matrix.org passwords and collected what seemed like the most popular and common prefixes.
There are numerous flaws with this approach (eg: many single-letter "prefixes" for dictionary words actually just make other dictionary words), and I had to take some liberties rather than just taking the "most common", but...
Common prefixes:
the
my
ilove
a
123
1
red
love
blue
black
big
super
Common suffixes:
1
s
123
2
7
99
69
1234
er
man
ing
ed
We can use these with hashcat's "combinator" attack to add prefixes/suffixes to an existing "wordlist". I've put the prefix+suffix collections up on GitHub: https://github.com/hypn/common_password_prefixes_and_suffixes
2. Hashcat "brain" for faster cracking
Version 5 of Hashcat came with a new "brain" feature, for which seems to be fairly little documentation online. What it does, in short, is provide a "brain server" against which "clients" (hashcat) can check if an algorithm+password combination has already been tried to avoid re-calculating a costly algorithm when duplicates would appear (either in a single wordlist, or when one or more rules would result in words already in the wordlist or generated by other rules). In short it should reduce cracking time.
D:\Programs\hashcat-5.1.0>hashcat64.exe --brain-server --brain-password password123
1560084695.828542 | 0.00s | 0 | Brain server started
If the "--brain-password" parameter (and value) can be left out the password will be generated randomly, but you'll need to watch for it to use with your hashing clients. Hashcat can then be run like normal with extra parameters telling it to use the brain server:
hashcat64.exe --brain-client --brain-password password123 -m 2500 -a 0 d:\temp\some-wifi.hccapx rockyou.txt
...
Session..........: hashcat (Brain Session/Attack:0x704c3423/0x4df7161b)
Status...........: Running
Hash.Type........: WPA-EAPOL-PBKDF2
Hash.Target......: d:\temp\some-wifi.hccapx
Guess.Base.......: File (rockyou.txt)
...
Brain.Link.#1....: RX: 54.2 kB (0.00 Mbps), TX: 122.0 kB (0.00 Mbps), idle
Note the "Brain.Link" line and session name - you'll likely also see a higher "Rejected" rate than normal. More info here and here.
3. Hashcat Rules Using "rules" hashcat's able to modify an input wordlist of possible hashes to create new variants of the word - such as adding or removing letters or numbers, changing the word to upper or lower case, removing spaces, etc. Using a rule is easy, you simply include a "-r " parameter (it can either be the full path, or "rules.rule" if you copied it in to hahcat's "rules" directory.
The rule syntax looks a bit intimidating at first, but we can fairly easily write some trivial rules (note "#" is used for comments), eg:
# add a "1" or "s" suffix
$1
$s
$1$2$3
# add or "the" and "my" prefix
^e^h^t
^y^m
I've you've worked with regular expressions, the "^" indicating a start of line and "$" for end of line might be familiar. The first two rules are easy enough, at the end of each word add something, but the 3rd rule effectively has 3 steps: add "1", then add "2", then add "3" resulting in "123" being added. In the same way adding to the front of a word is done one character at a time, each time changing the word, requiring the characters to be added in reverse order. This seems a little confusing but "^t^h^e" on "password" would result in "tpassword", then "htpassword", then "ehtpassword"... which is not what we want.
We can test our rule set by saving it to file (eg: "d:\temp\myrules.txt"), creating a basic wordlist (eg: "echo password> d:\temp\wordlist.txt") and having hashcat output the result of the two:
hashcat64.exe d:\temp\wordlist.txt -r d:\temp\myrules.txt --stdout
password1
passwords
password123
thepassword
mypassword
Note that "password" (our original word) is not shown, to have our ruleset also try our base rules we need to include a rule of ":" in our rule file.
A great, but huge, ruleset useful for cracking hashes is the "OneRuleToRuleThemAll.rule" by stealthsploit which you can read about here. We can give it a try with our trivial wordlist to see the variations it generates:
hashcat64.exe d:\temp\wordlist.txt -r rules\OneRuleToRuleThemAll.rule --stdout
...
gunspassword
snspassword
unicornspassword
...
buttonspassword
simpsonspassword
demonspassword
lionspassword
dragonspassword
falconspassword
twinspassword
kevinspassword
bruinspassword
penguinspassword
martinspassword
redskinspassword
dolphinspassword
muffinspassword
trainspassword
...
Wait, what?
Now we get back to point 1 above... prefixes and suffixes. These just don't look like great prefixes to me, how many passwords really start with "simpsons" (not even "thesimpsons")? And do we really want a rule file applying prefixes instead of using the combinator attack mentioned above? I'm not convinced. There are over 50k rules in this rulefile, with 10k of them being prefixes... that's a potential 20% we could shrink our cracking time by if we removed these rules and have a good wordlist. The reduced ruleset, for better or worse, is at https://github.com/hypn/Optimised-hashcat-Rule
It's also worth noting that we can relatively easily "debug" rules by outputting the results of them to a file, and looking up the line-number of the offending generated password in the rule file (so long as you remove all comments and blank lines from the rule file).
4. All the passwords in the world (and hashes.org) The fastest way to crack a password hash, in my opinion, is to put it in a text file and upload it to hashes.org. Initially I thought this site was a community drive effort to crack individual files, and wasn't sure how to help out or get people to crack my hashes, but as it turns out they appear to just lookup the hashes and return plaintext value if they've seen it before (basically a "rainbow table" attack). While this is convenient, it's not really ethical to do with a client's corporate password hashes if you've been hired to pentest their systems.
The alternative is cracking the hashes ourselves with hashcat hopefully using a good wordlist and ruleset. The popular "rockyou.txt" wordlist has over 14 million passwords, the "OneRuleToRuleThemAll" (before my changes) would make that 14mil x 50k = 700,000,000,000 password combinations (likely with many duplicates). On the other hand hashes.org also provides a "found lists" where you can download literally all the passwords they've ever come across. These "found" files, when added together with duplicates removed, come out to around 640,000,000 passwords (if I did things correctly)... less than 1/1000th the size of rockyou + OneRuleToRuleThemAll... not bad!