Cracking Story – How I Cracked Over 122 Million SHA1 and MD5 Hashed Passwords
by m3g9tr0n on Aug.28, 2012, under Crack1ng, Hack1ng. 11,051 views
This is the story about how I cracked 122 million* password hashes with John the Ripper and oclHashcat-plus.
It was several months ago, when I saw a tweet from KoreLogic about a torrent file containing various password hash lists for a total of 146 million passwords. This very big amount of password hashes at first discouraged me, as I only own a classic computer configuration with an AMD Phenom II 4 cores at 3,2 Mhz in addition to an ATI/AMD 5770 graphics card. But at least, I really wanted to give them a try because the field of password cracking fascinates me.
The password cracking tools I used during this long trip were John the Ripper and oclHashcat-plus. This article is about cracking the provided MD5 hashes of KoreLogic only, but the same strategy was also applied to SHA1 hashes.
Updates:
- 08/29/2012 – New example in John the Ripper section: “Crack double MD5 hashes with the help of dict2hash.pl script”
- 08/29/2012 – New download! All in one sorted and cleaned version.
Dealing with hashes…
First of all the KoreLogic torrent file file must be decompressed, it contains a folder named “hashes”. Let’s see the content of this folder…
root@m3g9tr0n:~/hashes$ ls longer_salts raw-md5.hashes.txt salted_with_md5 SHA1 vBulletin-v3.8.4
We will concentrate here on the raw-md5.hashes.txt list. This file is 4.3 GB and includes 139444502 lines according to wc utility.
root@m3g9tr0n:~/hashes$ wc -l raw-md5.hashes.txt 139444502 raw-md5.hashes.txt
As you consider, both John the Ripper and oclHashcat-plus are not able to load this file because it is too big. For that reason, we need to split this file. Under Linux we have a nice utility called split that does this job very well.
root@m3g9tr0n:~$ split --help Usage: split [OPTION]... [INPUT [PREFIX]] Output fixed-size pieces of INPUT to PREFIXaa, PREFIXab, ...; default size is 1000 lines, and default PREFIX is `x'. With no INPUT, or when INPUT is -, read standard input. Mandatory arguments to long options are mandatory for short options too. -a, --suffix-length=N use suffixes of length N (default 2) -b, --bytes=SIZE put SIZE bytes per output file -C, --line-bytes=SIZE put at most SIZE bytes of lines per output file -d, --numeric-suffixes use numeric suffixes instead of alphabetic -l, --lines=NUMBER put NUMBER lines per output file --verbose print a diagnostic just before each output file is opened --help display this help and exit --version output version information and exit SIZE may be (or may be an integer optionally followed by) one of following: KB 1000, K 1024, MB 1000*1000, M 1024*1024, and so on for G, T, P, E, Z, Y.
We can use the –lines=NUMBER parameter to split our raw-md5.hashes.txt file.
root@m3g9tr0n:~/hashes$ split -l 3000000 raw-md5.hashes.txt part
Note that we can also split the file based on the amount of MBs by taking into consideration that each MD5 hash is 32 bytes long.
Cracking Passwords with oclHashcat-plus
I started with oclHashcat-plus because it contains the –remove option, which enable remove of hash from hashfile once it is cracked and is really convenient. The only limitation oclHashcat-plus has, is the constraint on password length. In other words, it is only able to crack passwords up to 15 characters. The rules that I used for oclHashcat-plus are base64.rule, passwordspro.rule, T0XlC.rule and in some cases d3ad0ne.rule. There rules can be found directly from the oclHashcat-plus suite.
Bruteforce techniques were not my first choice. I used wordlists which I download from g0tm1lk’s blogspot. You will find on g0tmi1k’s article other external links for more wordlists. The biggest part of cracking process was done by using these wordlists with the rules mentioned above. Let’s see some examples…
Using a single rule:
./oclHashcat-plus64.bin -m 0 ~/hashes/md5_1 ~/Wordlists/d3ad0ne.dic -r rules/best64.rule -o Ultimate_Crack/eNtr0pY_1 --remove
Using Rules’ combination:
./oclHashcat-plus64.bin -m 0 ~/hashes/md5_1 ~/Wordlists/d3ad0ne.dic -r rules/best64.rule r rules/passwordspro.rule -o Ultimate_Crack/eNtr0pY_1 --remove
Bruteforce attack with mask (you can specify whatever charset you want):
./oclHashcat-plus64.bin -a 3 -1 ?l?d?u?s -m 0 ~/hashes/md5_1 ?1?1?1?1?1?1?1?1 -o Ultimate_Crack/eNtr0pY_1 --remove
Combination attack:
./oclHashcat-plus64.bin -a 1 -m 0 ~/hashes/md5_1 ~/Wordlists/d3ad0ne.dic ~/Wordlists/list -o Ultimate_Crack/eNtr0pY_1 --remove
Combination attack with rules:
./oclHashcat-plus64.bin -a 1 -m 0 ~/hashes/md5_1 ~/Wordlists/d3ad0ne.dic ~/Wordlists/list -r rules/passwordspro.rule -o Ultimate_Crack/eNtr0pY_1 --remove
Permutation attack:
./oclHashcat-plus64.bin -a 4 -m 0 ~/hashes/md5_1 ~/Wordlists/d3ad0ne.dic -o Ultimate_Crack/eNtr0pY_1 --remove
Permutation attack with rules:
./oclHashcat-plus64.bin -a 4 -m 0 ~/hashes/md5_1 ~/Wordlists/d3ad0ne.dic -r rules/best64.rule -o Ultimate_Crack/eNtr0pY_1 --remove
In some cases, I used the hybrid + mask attack technique:
./oclHashcat-plus64.bin -a 6 -1 ?l?d -m 0 ~/hashes/md5_1 ~/Wordlists/d3ad0ne.dic ?1?1 -o Ultimate_Crack/eNtr0pY_1 --remove
Hybrid + mask attack with rules:
./oclHashcat-plus64.bin -a 6 -1 ?l?d -m 0 ~/hashes/md5_1 ~/Wordlists/d3ad0ne.dic ?1?1 -r rules/best64.rule -o Ultimate_Crack/eNtr0pY_1 --remove
At this point, I did not use these last two methods as they are very time consuming. I rather found a better one using KoreLogic’s Rules for John the Ripper by piping the output of John the Ripper to oclHashcat-plus. As I mentioned, oclHashcat-plus is able to crack passwords up to 15 characters. For that reason, I had to define everytime, via the –stdout option, the length of the produced word. If you own a very fast GPU you do not have to use the following example.
./john --wordlist=~/Wordlists/all.lst -rules:KoreLogicRulesPrependYears --stdout=10 | ./oclHashcat-plus64.bin -m 0 ~/hashes/md5_1 -o Ultimate_Crack/eNtr0pY_1 --remove
Of course you can use other prepend rules created from Korelogic, like KoreLogicRulesPrependNumNum, or even better create your own rules!
It was time to produce a wordlist from the cracked passwords and use it to crack the remaining hashes. From eNtr0pY_1, I removed the MD5 hashes with the following command.
cut -b34- eNtr0pY_1 > eNtr0pY_1.dic
By using the above produced wordlist, a big amount of MD5 hashes were cracked with the fingerprint attack. You can read more about this attack from Martin Bos @purehate and I guarantee that this technique is very successful!
Of course you can also use the binaries included into hashcat-utils and pipe the output of each util to oclHashcat-plus.
root@m3g9tr0n:~/oclHashcat-plus-0.08/hashcat-utils$ ls combinator.bin expander.bin gate.bin len.bin mp32.bin permute.bin prepare.bin req.bin splitlen.bin
Cracking Passwords with John the Ripper
After testing all my wordlist collection and after several days, it was time to move to John the Ripper for cracking the rest of password hashes…
I used magnum-ripper compiled with OpenCL for ATI/AMD graphics card because I wanted to use the –format=raw-md5-opencl parameter. Compared to –format=raw-md5, it is way faster as it uses your CPU and GPU!
The Rules that were used with John the Ripper are wordlist, Single, NT, Extra, KoreLogicRulesAppendNumbers_and_Specials_Simple, KoreLogicRulesAppend6Num, KoreLogicRulesPrependAndAppendSpecial, KoreLogicRulesAppendNumNum_AddSpecialEverywhere, KoreLogicRulesAppendNumNumNum_AddSpecialEverywhere and KoreLogicRulesL33t.
Furthermore you can download these rules and add them to your john.conf file.
Let’s see now some examples with John the Ripper…
Using –rules=Single
./john --format=raw-md5-opencl --wordlist=../../Wordlists/all.lst --rules:Single ~/hashes/md5_1
The results of cracked hashes are stored in the john.pot file by default. You can examine its contents with cat, more, head and tail.
root@m3g9tr0n:~/Tools/Password_Cracking/magnum-jumbo-OpenCL/run$ tail -n 9 john.pot $MD5$0fad81e7a61b47d387dde893fcf8e88a:anacarolinagu $MD5$0f82fc9a81f5db07eb9289767390fd2b:fabulousfoodsu $MD5$0e22933267b2e7df062703c4e5842029:fabuloustravelu $MD5$0d40086a54fefe993c9816d1441672ac:modularhomeu $MD5$0ed8181fc4d18e260dd8e36633124bfd:greenshoppingu $MD5$0d6e8da4017ec5c384ac5536087da44d:lawofattractionu $MD5$0eb916d3c6a66a32cedd4acc6edb1dbb:hotreportu $MD5$0e241f99b5c13d56686ec618ab54d5fa:flightsandholidaysu $MD5$0f3c99478362aae389d2cbf716394269:stthomasmoresu
To produce a wordlist from the john.pot file, you can use the following command.
cut -d: -f2 john.pot | sort -u > cracked.dic
The created wordlist can be used to crack more hashes when combined with the abovementioned rules.
When I was cracking MD5 hashes with oclHashcat-plus, I observed that some produced passwords were rejected. This is because oclHashcat-plus has a limitation about characters’ length. For that reason, I piped hashcat’s output to John the Ripper with the additional advantage of using hashcat rules with John the Ripper.
./hashcat-cli64.bin --stdout ~/Wordlists/d3ad0ne.dic -r rules/best64.rule | ./john --format=raw-md5-opencl --stdin ~/hashes/md5_1
After trying all the wordlists combined with the rules mentioned above, it was time to move to bruteforce attacks with John the Ripper. Unfortunately, John the Ripper does not use the mask attacks to produce passwords when implementing bruteforce attacks. We have to create our own charset based on cracked passwords contained in john.pot.
./john --make-charset=eNtr0pY.chr Loaded 7948325 plaintexts Generating charsets... 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 DONE Generating cracking order... DONE Successfully written charset file: eNtr0pY.chr (95 characters)
Many of you will wonder about “31 DONE”… This is just because I compiled John the Ripper with 31 characters’ length. By default, John the Ripper is compliled with 8 characters’ length, so it is best to change it by modifying the following lines of the header file params.h located in the scr folder of John the Ripper.
#define CHARSET_MIN ' ' #define CHARSET_MAX 0x7E #define CHARSET_SIZE (CHARSET_MAX - CHARSET_MIN + 1) #define CHARSET_LENGTH 8 //Change that to 31 or whatever you wish
At last you have to include your created charset to john.conf as given in this example:
# Incremental modes [Incremental:eNtr0pY] File = $JOHN/eNtr0pY.chr MinLen = 0 MaxLen = 31 CharCount = 95
Now it is time to use bruteforce attacks with our own charstet!
./john --format=raw-md5-opencl --incremental=eNtr0pY ~/hashes/md5_1
If you look into john.conf you will see some bruteforce attack modes characterized as extrernals. These are Double, Strip, Keyboard (which uses neighbor combinations produced from keyboard characters), KnownForce, DateTime, Repeats, Sequence, Subsets and DumbForce for crazy password formats.
./john --format=raw-md5-opencl --external=DumbForce ~/hashes/md5_1
We would also like to crack double MD5 hashes with the help of dict2hash.pl script provided here.
perl dict2hash.pl < rockyou.txt | ./john --format=raw-md5-opencl --stdin ~/md5_1
Here you can see some samples of cracked md5s with John the Ripper:
Personally, I believe a password like “$MD5$0b26a0faf1344d6e772bf55628e10e29:n34=mn { .clipboard $me }“ is impossible to crack with bruteforce attacks.
Note: All the abovementioned techniques can be used with oclHashcat-plus by defining -m 100 and with John the Ripper by defining –format=raw-sha1-opencl for SHA1 cracking with OpenCL!
Password Analysis
Finally, it worths to see an analysis using pipal (a password analyser) of a collected sample generated from cracking results.
root@m3g9tr0n:~/pipal$ ruby1.9.1 pipal.rb \ -o eNtr0pY_1 ~/Wordlists/Ultimate/Part1/eNtr0pY_5.dic Total entries = 759103 Total unique entries = 758299 Top 10 passwords niezgadniesz123 = 3 (0.0%) ubqu = 3 (0.0%) amonys = 3 (0.0%) centralitie = 3 (0.0%) bobydu = 3 (0.0%) hanghuynh = 3 (0.0%) hmadyousi = 3 (0.0%) matthewperman = 3 (0.0%) shadowninja2 = 3 (0.0%) lhz4 = 3 (0.0%) Top 10 base words august = 219 (0.03%) july = 205 (0.03%) april = 199 (0.03%) june = 195 (0.03%) march = 165 (0.02%) alex = 161 (0.02%) love = 132 (0.02%) chris = 130 (0.02%) daniel = 128 (0.02%) dragon = 122 (0.02%) Password length (length ordered) 1 = 13 (0.0%) 2 = 103 (0.01%) 3 = 1332 (0.18%) 4 = 16781 (2.21%) 5 = 19831 (2.61%) 6 = 95800 (12.62%) 7 = 202414 (26.66%) 8 = 158562 (20.89%) 9 = 103855 (13.68%) 10 = 75652 (9.97%) 11 = 46023 (6.06%) 12 = 24997 (3.29%) 13 = 8423 (1.11%) 14 = 3772 (0.5%) 15 = 1560 (0.21%) Password length (count ordered) 7 = 202414 (26.66%) 8 = 158562 (20.89%) 9 = 103855 (13.68%) 6 = 95800 (12.62%) 10 = 75652 (9.97%) 11 = 46023 (6.06%) 12 = 24997 (3.29%) 5 = 19831 (2.61%) 4 = 16781 (2.21%) 13 = 8423 (1.11%) 14 = 3772 (0.5%) 15 = 1560 (0.21%) 3 = 1332 (0.18%) 2 = 103 (0.01%) 1 = 13 (0.0%) | | | || || || || ||| |||| |||| ||||| ||||| |||||| |||||| ||||||||| ||||||||||||||||| 00000000001111111 01234567890123456 One to six characters = 133854 (17.63%) One to eight characters = 494828 (65.19%) More than eight characters = 264275 (34.81%) Only lowercase alpha = 154996 (20.42%) Only uppercase alpha = 14072 (1.85%) Only alpha = 169068 (22.27%) Only numeric = 119581 (15.75%) First capital last symbol = 6088 (0.8%) First capital last number = 73611 (9.7%) Months january = 109 (0.01%) february = 45 (0.01%) march = 247 (0.03%) april = 251 (0.03%) may = 850 (0.11%) june = 246 (0.03%) july = 223 (0.03%) august = 300 (0.04%) september = 80 (0.01%) october = 134 (0.02%) november = 113 (0.01%) december = 115 (0.02%) Days monday = 59 (0.01%) tuesday = 20 (0.0%) wednesday = 7 (0.0%) thursday = 38 (0.01%) friday = 46 (0.01%) saturday = 7 (0.0%) sunday = 70 (0.01%) Months (Abreviated) jan = 1482 (0.2%) feb = 249 (0.03%) mar = 8397 (1.11%) apr = 692 (0.09%) may = 850 (0.11%) jun = 889 (0.12%) jul = 1051 (0.14%) aug = 785 (0.1%) sept = 215 (0.03%) oct = 512 (0.07%) nov = 821 (0.11%) dec = 874 (0.12%) Days (Abreviated) mon = 4319 (0.57%) tues = 28 (0.0%) wed = 217 (0.03%) thurs = 44 (0.01%) fri = 758 (0.1%) sat = 769 (0.1%) sun = 1018 (0.13%) Includes years 1975 = 411 (0.05%) 1976 = 388 (0.05%) 1977 = 446 (0.06%) 1978 = 432 (0.06%) 1979 = 441 (0.06%) 1980 = 541 (0.07%) 1981 = 453 (0.06%) 1982 = 519 (0.07%) 1983 = 533 (0.07%) 1984 = 603 (0.08%) 1985 = 585 (0.08%) 1986 = 616 (0.08%) 1987 = 710 (0.09%) 1988 = 641 (0.08%) 1989 = 941 (0.12%) 1990 = 931 (0.12%) 1991 = 995 (0.13%) 1992 = 935 (0.12%) 1993 = 905 (0.12%) 1994 = 907 (0.12%) 1995 = 4021 (0.53%) 1996 = 858 (0.11%) 1997 = 486 (0.06%) 1998 = 443 (0.06%) 1999 = 416 (0.05%) 2000 = 1024 (0.13%) 2001 = 643 (0.08%) 2002 = 586 (0.08%) 2003 = 1132 (0.15%) 2004 = 1254 (0.17%) 2005 = 796 (0.1%) 2006 = 818 (0.11%) 2007 = 1442 (0.19%) 2008 = 1019 (0.13%) 2009 = 742 (0.1%) 2010 = 767 (0.1%) 2011 = 516 (0.07%) 2012 = 925 (0.12%) 2013 = 165 (0.02%) 2014 = 142 (0.02%) 2015 = 146 (0.02%) 2016 = 118 (0.02%) 2017 = 139 (0.02%) 2018 = 131 (0.02%) 2019 = 172 (0.02%) 2020 = 179 (0.02%) |
Years (Top 10) 1995 = 4021 (0.53%) 2007 = 1442 (0.19%) 2004 = 1254 (0.17%) 2003 = 1132 (0.15%) 2000 = 1024 (0.13%) 2008 = 1019 (0.13%) 1991 = 995 (0.13%) 1989 = 941 (0.12%) 1992 = 935 (0.12%) 1990 = 931 (0.12%) Colours black = 485 (0.06%) blue = 549 (0.07%) brown = 184 (0.02%) gray = 89 (0.01%) green = 348 (0.05%) orange = 125 (0.02%) pink = 262 (0.03%) purple = 73 (0.01%) red = 2974 (0.39%) white = 179 (0.02%) yellow = 85 (0.01%) violet = 63 (0.01%) indigo = 22 (0.0%) Single digit on the end = 92080 (12.13%) Two digits on the end = 87587 (11.54%) Three digits on the end = 103715 (13.66%) Last number 0 = 45407 (5.98%) 1 = 64764 (8.53%) 2 = 52570 (6.93%) 3 = 52890 (6.97%) 4 = 43719 (5.76%) 5 = 55185 (7.27%) 6 = 42826 (5.64%) 7 = 46169 (6.08%) 8 = 42475 (5.6%) 9 = 44930 (5.92%) | | | | | ||| | |||| | | | |||||||||| |||||||||| |||||||||| |||||||||| |||||||||| |||||||||| |||||||||| |||||||||| |||||||||| |||||||||| |||||||||| 0123456789 Last digit 1 = 64764 (8.53%) 5 = 55185 (7.27%) 3 = 52890 (6.97%) 2 = 52570 (6.93%) 7 = 46169 (6.08%) 0 = 45407 (5.98%) 9 = 44930 (5.92%) 4 = 43719 (5.76%) 6 = 42826 (5.64%) 8 = 42475 (5.6%) Last 2 digits (Top 10) 95 = 14675 (1.93%) 23 = 12192 (1.61%) 12 = 9230 (1.22%) 11 = 8214 (1.08%) 01 = 7606 (1.0%) 00 = 7131 (0.94%) 07 = 6295 (0.83%) 10 = 6182 (0.81%) 21 = 5881 (0.77%) 99 = 5868 (0.77%) Last 3 digits (Top 10) 123 = 6857 (0.9%) 995 = 4122 (0.54%) 971 = 2916 (0.38%) 972 = 2850 (0.38%) 007 = 2514 (0.33%) 000 = 1868 (0.25%) 234 = 1725 (0.23%) 666 = 1465 (0.19%) 777 = 1389 (0.18%) 004 = 1347 (0.18%) Last 4 digits (Top 10) 1995 = 3886 (0.51%) 1234 = 1379 (0.18%) 2007 = 1325 (0.17%) 2004 = 1121 (0.15%) 2003 = 1016 (0.13%) 2008 = 869 (0.11%) 2000 = 846 (0.11%) 1991 = 819 (0.11%) 2012 = 809 (0.11%) 1990 = 789 (0.1%) Last 5 digits (Top 10) 12345 = 743 (0.1%) 23456 = 652 (0.09%) 54321 = 189 (0.02%) 23123 = 140 (0.02%) 56789 = 127 (0.02%) 34567 = 102 (0.01%) 11111 = 99 (0.01%) 45678 = 75 (0.01%) 00000 = 73 (0.01%) 88888 = 68 (0.01%) US Area Codes 971 = Oregon: Metropolitan Portland, Salem/Keizer area, incl Cricket Wireless (OR) 972 = Texas: Dallas Metro (TX) 234 = NE Ohio: Canton, Akron (OH) Character sets loweralphanum: 330937 (43.6%) loweralpha: 154996 (20.42%) numeric: 119581 (15.75%) mixedalphanum: 41121 (5.42%) upperalphanum: 41078 (5.41%) mixedalpha: 28464 (3.75%) upperalpha: 14072 (1.85%) loweralphaspecial: 10222 (1.35%) loweralphaspecialnum: 5735 (0.76%) mixedalphaspecial: 4724 (0.62%) upperalphaspecial: 2939 (0.39%) mixedalphaspecialnum: 2247 (0.3%) specialnum: 648 (0.09%) upperalphaspecialnum: 374 (0.05%) special: 47 (0.01%) Character set ordering stringdigit: 349534 (46.05%) allstring: 197532 (26.02%) alldigit: 119581 (15.75%) digitstring: 28873 (3.8%) othermask: 18649 (2.46%) stringdigitstring: 14577 (1.92%) stringspecial: 10441 (1.38%) digitstringdigit: 9981 (1.31%) stringspecialstring: 5469 (0.72%) stringspecialdigit: 3075 (0.41%) specialstring: 834 (0.11%) specialstringspecial: 510 (0.07%) allspecial: 47 (0.01%) Hashcat masks (Top 10) ?d?d?d?d?d?d?d: 85053 (11.2%) ?l?l?l?l?l?l: 38400 (5.06%) ?l?l?l?l?l?l?l?l: 36217 (4.77%) ?l?l?l?l?l?l?l: 35468 (4.67%) ?l?l?l?l?l?l?d?d?d: 24051 (3.17%) ?l?l?l?l?l?l?d?d: 18591 (2.45%) ?l?l?l?l?l?d?d?d: 18047 (2.38%) ?d?d?d?d?d?d: 16048 (2.11%) ?l?l?l?l?l?l?l?l?l: 14236 (1.88%) ?l?l?l?l?d?d?d: 13802 (1.82%) |
Conclusion
This was a very time consuming and hard job because I do not own the fastest card. The whole cracking process took about 5 months to accomplish because I had to finish my studies about CCNP certification. The lesson learned from this is that with a good and smart dictionary combined with handy rules either for hashcat or John the Ripper even the most impossible password can be cracked. Based on the upon statement, admins should use a stronger hash algorithm (with salt) to store your passwords or even better from your side just change your passwords in a regular basis.
Thanks for reading.
You can find me on twitter, @m3g9tr0n.
Downloads
You can download the results of cracked hashes:
The provided KoreLogic torrent file contains various but unique password hashes. For that reason you may find duplicated passwords in these wordlists, as a single password can be hashed using various algorithmes! Meaning that 122 million unique hashes (MD5, SHA1, double MD5, etc.) were cracked and result in 83,6 million unique passwords.
You can download the “all in one” version, cleaned and sorted:
The command used to generate this “all in one” CLEANED wordlist was:
export LC_ALL='C' && cat * | sort | uniq > eNtr0pY_ALL_sort_uniq.dic
References
- hashcat Wiki [ http://hashcat.net/wiki/ ]
- John the Ripper Wiki [ http://openwall.info/wiki/john ]
Incoming search terms:
14 Comments for this entry
1 Trackback or Pingback for this entry
-
Episode 749 – Firefox 15, Privitize VPN, Oracle ignores Zero Day Warnings, 122 Million Cracked Passwords, and 2nd Lulzsec member arrested | InfoSec Daily
August 30th, 2012 on 2:49 am[...] Source: http://blog.thireus.com/cracking-story-how-i-cracked-over-122-million-sha1-and-md5-hashed-passwords [...]
August 28th, 2012 on 8:23 pm
Nice writeup, and excellent job!
One thing I will comment on though, for “Combination attack with rules” and “Permutation attack with rules” with oclhc-plus, those attack modes actually do not support using rules. It will not give an error if you try to use rules, and it will even tell you that it is using rules, but it doesn’t actually use the rules
So if you did want to do a combinator attack + rules, you would have to use combinator.bin from hashcat-utils and oclhc-plus in stdin mode with rules. Using your command line as an example:
~/hashcat-utils-0.9/combinator.bin ~/Wordlists/d3ad0ne.dic ~/Wordlists/list | ./oclHashcat-plus64.bin ~/hashes/md5_1 -r rules/passwordspro.rule -o Ultimate_Crack/eNtr0pY_1 –remove
You can use cpu hashcat in stdout mode piped into oclhc-plus to do permutation attack or table attack with rules in a similar fashion. In that way the permutations would happen on the cpu, and the gpu would do all of the rule calculations. Slower, but very potent.
Some other suggestions… I noticed you said you only used d3ad0ne.rule in only a few cases. I would encourage you to use it more, because it is the most effective ruleset that ships with hashcat. Also, I’d like to introduce you to rule chaining: you can specify multiple rule files with oclhc-plus. One of my favorite rule chains is -r rules/leetspeak.rule -r rules/toggles5.rule, which essentially emulates a table attack on the gpu.
Keep up the good work!
August 28th, 2012 on 11:17 pm
Hi,
Excelent paper, thank you for sharing !
I have an question about your JTR compilation, i see you take the “MAgnum” Jumbo for take advantage of your GPU, can you please tell me what you modify exactly on the”Makefile”.
Otherwize, the Parameter.h file, you just add “31″ & that’s all ? & rebuild ?
Thanks again,
Fred
August 29th, 2012 on 12:01 am
very good post here, thx for sharing with us, remind us to use salt + pepper lol
August 29th, 2012 on 5:51 am
Hehe nice reference IKONspirasi
Thanks goes to m3g9tr0n.
August 29th, 2012 on 9:23 am
That’s very impressive for a 3.2 MHz processor.
August 29th, 2012 on 9:33 am
@epixoip
Thank you very much for your kind words and of course for your details.
As far as the d3ad0ne’s rule is concerned, with no doubt it is the best
rule but when applying to big wordlists my not so fast GPU, ATI 5770,
needed too much time to accomplish the cracking process. Also the
temperature was going to high levels.
@Fred
Thank you for your kind words. So let’s get them from the beggining.
Step 1
Download Magnum Ripper via the following command:
git clone https://github.com/magnumripper/magnum-jumbo.git
Initialized empty Git repository in /home/m3g9tr0n/Desktop/magnum-jumbo/.git/
remote: Counting objects: 7262, done.
remote: Compressing objects: 100% (2325/2325), done.
remote: Total 7262 (delta 5114), reused 7072 (delta 4932)
Receiving objects: 100% (7262/7262), 3.27 MiB | 832 KiB/s, done.
Resolving deltas: 100% (5114/5114), done.
Step 2
Navigate to src folder and type “make”. You will see something like
the following output:
make
To build John the Ripper, type:
make clean SYSTEM
where SYSTEM can be one of the following:
([i] is an optional letter for pre-built intrinsics, eg. -sse2i vs -sse2):
linux-x86-64-gpu Linux, x86-64 with SSE2, CUDA and OpenCL (experimental)
linux-x86-64-opencl Linux, x86-64 with SSE2 and OpenCL (experimental)
linux-x86-64-cuda Linux, x86-64 with SSE2 and CUDA (experimental)
linux-x86-64-avx Linux, x86-64 with AVX (2011+ Intel CPUs)
linux-x86-64-xop Linux, x86-64 with AVX and XOP (2011+ AMD CPUs)
linux-x86-64[i] Linux, x86-64 with SSE2 (most common)
linux-x86-64-icc Linux, x86-64 compiled with icc
linux-x86-64-clang Linux, x86-64 compiled with clang
linux-x86-gpu Linux, x86 32-bit with SSE2, CUDA and OpenCL (experimental)
linux-x86-opencl Linux, x86 32-bit with SSE2 and OpenCL (experimental)
linux-x86-cuda Linux, x86 32-bit with SSE2 and CUDA (experimental)
linux-x86-sse2[i] Linux, x86 32-bit with SSE2 (most common, 32-bit)
linux-x86-mmx Linux, x86 32-bit with MMX (for old computers)
linux-x86-any Linux, x86 32-bit (for truly ancient computers)
linux-alpha Linux, Alpha
linux-sparc Linux, SPARC 32-bit
linux-ppc32-altivec Linux, PowerPC w/AltiVec (best)
linux-ppc32 Linux, PowerPC 32-bit
linux-ppc64 Linux, PowerPC 64-bit
linux-ia64 Linux, IA-64
freebsd-x86-64[i] FreeBSD, x86-64 with SSE2 (best)
freebsd-x86-sse2[i] FreeBSD, x86 with SSE2 (best if 32-bit)
freebsd-x86-mmx FreeBSD, x86 with MMX
freebsd-x86-any FreeBSD, x86
freebsd-alpha FreeBSD, Alpha
openbsd-x86-64[i] OpenBSD, x86-64 with SSE2 (best)
openbsd-x86-sse2[i] OpenBSD, x86 with SSE2 (best if 32-bit)
openbsd-x86-mmx OpenBSD, x86 with MMX
openbsd-x86-any OpenBSD, x86
openbsd-alpha OpenBSD, Alpha
openbsd-sparc64 OpenBSD, SPARC 64-bit (best)
openbsd-sparc OpenBSD, SPARC 32-bit
openbsd-ppc32 OpenBSD, PowerPC 32-bit
openbsd-ppc64 OpenBSD, PowerPC 64-bit
openbsd-pa-risc OpenBSD, PA-RISC
openbsd-vax OpenBSD, VAX
netbsd-sparc64 NetBSD, SPARC 64-bit
netbsd-vax NetBSD, VAX
solaris-sparc64-cc Solaris, SPARC V9 64-bit, cc (best)
solaris-sparc64-gcc Solaris, SPARC V9 64-bit, gcc
solaris-sparcv9-cc Solaris, SPARC V9 32-bit, cc
solaris-sparcv8-cc Solaris, SPARC V8 32-bit, cc
solaris-sparc-gcc Solaris, SPARC 32-bit, gcc
solaris-x86-64-cc Solaris, x86-64 with SSE2, cc
solaris-x86-64[i]-gcc Solaris, x86-64 with SSE2, gcc
solaris-x86-sse2-cc Solaris 9 4/04+, x86 with SSE2, cc
solaris-x86-sse2[i]-gcc Solaris 9 4/04+, x86 with SSE2, gcc
solaris-x86-mmx-cc Solaris, x86 with MMX, cc
solaris-x86-mmx-gcc Solaris, x86 with MMX, gcc
solaris-x86-any-cc Solaris, x86, cc
solaris-x86-any-gcc Solaris, x86, gcc
sco-x86-any-gcc SCO, x86, gcc
sco-x86-any-cc SCO, x86, cc
tru64-alpha Tru64 (Digital UNIX, OSF/1), Alpha
aix-ppc32 AIX, PowerPC 32-bit
macosx-x86-64 Mac OS X 10.5+, Xcode 3.0+, x86-64 with SSE2 (best)
macosx-x86-64-opencl Mac OS X 10.5+, Xcode 3.0+, x86-64 with SSE2, OpenCL support
macosx-x86-sse2 Mac OS X, x86 with SSE2
macosx-ppc32-altivec Mac OS X, PowerPC w/AltiVec (best)
macosx-ppc32 Mac OS X, PowerPC 32-bit
macosx-ppc64 Mac OS X 10.4+, PowerPC 64-bit
macosx-universal Mac OS X, Universal Binary (x86 + x86-64 + PPC)
hpux-pa-risc-gcc HP-UX, PA-RISC, gcc
hpux-pa-risc-cc HP-UX, PA-RISC, ANSI cc
irix-mips64-r10k IRIX, MIPS 64-bit (R10K) (best)
irix-mips64 IRIX, MIPS 64-bit
irix-mips32 IRIX, MIPS 32-bit
dos-djgpp-x86-mmx DOS, DJGPP, x86 with MMX
dos-djgpp-x86-any DOS, DJGPP, x86
win32-cygwin-x86-sse2[i] Win32, Cygwin, x86 with SSE2 (best)
win32-cygwin-x86-mmx Win32, Cygwin, x86 with MMX
win32-cygwin-x86-any Win32, Cygwin, x86
win32-mingw-x86-sse2[i] Win32, MinGW, x86 with SSE2 (best)
win32-mingw-x86-mmx Win32, MinGW, x86 with MMX
win32-mingw-x86-any Win32, MinGW, x86
beos-x86-sse2 BeOS, x86 with SSE2 (best)
beos-x86-mmx BeOS, x86 with MMX
beos-x86-any BeOS, x86
generic Any other Unix-like system with gcc
Depending on your GPU architecture try the appropriate command. For example, if
you own an ATI GPU type “make linux-x86-opencl” for 32 bits architecture.
Furthermore, do not forget that ATI owners should install ATI-STREAM-SDK
in order to be able to compile JtR for GPU. You can download ATI SDK from
that link http://developer.amd.com/sdks/AMDAPPSDK/downloads/Pages/default.aspx.
Step 3
The file is called “params.h” and is located to “src” folder. Yes, if you follow
the instructions with success you will compile JtR for 31 characters password
cracking.
If you have any other question please do not hesitate to ask.
@IKONspirasi
Thank you my friend!Thireus also has done a great job about building and
using JtR with MPI and OpenMP.
August 29th, 2012 on 12:13 pm
Thanks for the writeup!
Just a couple of minor comments:
I am surprised you found raw-md5-opencl significantly faster than raw-md5. At this time, it is not supposed to be; this is early development code only (it works, but its speed is CPU-like).
Since you don’t mind trying out unreleased code, you could want to try the bleeding-jumbo branch of JtR (“git clone git://github.com/magnumripper/magnum-jumbo -b bleeding-jumbo bleeding-jumbo”). It uses a little bit less RAM for raw MD5 hashes (I’ve just measured 90 MB for 1 million hashes, vs. magnum-jumbo’s 95 MB), and it uses bitmaps in addition to hash tables for better performance at fast and saltless hashes like this when you load a lot of them at once, which you do. Overall, it’d probably let you load more hashes at once and it’d crack them faster.
BTW, a 64-bit build of John the Ripper should in fact be able to load all of the 146 million hashes at once on a computer with enough RAM (tens of gigabytes). I understand that in practice (with smaller amounts of RAM) you have to split, though.
August 29th, 2012 on 1:51 pm
A minor comment on the cut command. When you use:
cut -d: -f2 john.pot | sort -u > cracked.dic
if a password contains a colon, only the characters occuring before the colon will be written to your dictionary file.
cut -d: -f2- john.pot | sort -u > cracked.dic
This will write the entire password.
August 29th, 2012 on 2:51 pm
I’ve just confirmed that it is possible to load 146 million raw MD5 hashes at once. I took bleeding-jumbo, changed PASSWORD_HASH_SIZE_FOR_LDR in params.h from 4 to 6 to speedup loading of the huge file (I’d have to wait tens of minutes otherwise), and ran John on a file with exactly 146 million of raw MD5 hashes (32 hex chars per line, 4818000000 bytes file size, hashes are of strings “0″ through “145999999″), using the –external=DumbForce mode (could as well be incremental or wordlist, but not “single crack” or batch mode as these need login names or the like). The machine I ran this on is a 2xE5420 with 16 GB RAM, but only one CPU core was used and only 8 GB RAM ended up being used by the “john” process (meaning this would work on an 8 GB RAM machine with a little bit of swap as well, although performance could suffer then). The loading completed in under 3 minutes.
August 29th, 2012 on 3:44 pm
@Solar Designer
Hello Alexander. It is really an honor for us to have a comment of you in our blog.
When I was implementing the cracking process of these hashes, about 5 month ago,
I had a machine with ubuntu 10.10 x86 installed under 4GB of RAM.
Like I said, I started with oclHashcat because it has the –remove option.
This was very convenient because was cleared many passwords up to 15 characters.
The remaining hashes cracked via John the Ripper. In that point I would like to ask
a favor from you about including something similar to JtR like –remove option. That
would be very convenient. I will try to implement your instructions about Bleeding Jumbo
under my new laptop which has 8GB of RAM and Intel i7 8 cpu cores processors.
Again thanks a lot Solar Designer and hope to meet you some day and discuss
about Password Cracking techniques
@Wilbur W.
Thanks a lot my friend!!That was very clever and handy!!I appreciate your comment!
August 29th, 2012 on 5:51 pm
@m3g9tr0n – being unfamiliar with hashcat, I asked about its -remove option on #openwall (irc.freenode.net), and the consensus appears to be that there’s no point in adding that to JtR since it reads and excludes hashes found in john.pot on every load anyway. However, you may find –show=left useful for occasions when you need e.g. to run another tool on the uncracked hashes. I hope this helps.
Oh, and let me invite you to the john-users mailing list. I think you should have announced your blog post in there. I’ll probably do that for you.
August 29th, 2012 on 10:38 pm
@m3g9tr0n – regarding your added double MD5 example and the dict2hash.pl script, this task is better approached by using JtR’s dynamic mode, and in particular the predefined –format=dynamic_2 for the double MD5 case.
August 29th, 2012 on 10:53 pm
@Solar Designer
Thank you for quick feedback I really appreciate that!I will try that way!
Greetings
August 30th, 2012 on 5:24 am
Wow, i am impress & really appreciated your verry detailed explain, understand perfectly, thank’s a lot !