[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Wed, 15 May 2013 20:05:17 +0200
From: magnum <john.magnum@...hmail.com>
To: john-dev@...ts.openwall.com
Subject: Re: Incremental mode in 1.7.9.14
On 14 May, 2013, at 11:09 , Solar Designer <solar@...nwall.com> wrote:
>> Despite 1.7.9 (unstable) running 25% slower, it does crack more hashes here. This is with exact same training as bleeding.
>
> This is unexpected and troubling - we don't want to be making things
> worse than what we had before.
Unfortunately further tests seem to show the same. Something else than this threshold need to be tweaked. Lots of data but no silver bullet.
>> For giving more weight to short words, my gut feeling is this is too steep. A simple and less steep function would be "1 / (1 << length)":
>>
>> $ perl -e 'foreach $i (0..9) { printf("%2d%15f\n", $i + 1, 1/(1<<$i))}'
>> 1 1.000000
>> 2 0.500000
>> 3 0.250000
>> 4 0.125000
>> 5 0.062500
>> ...
>> 10 0.001953
>
> Yes, I think this is better - or maybe "1.0 / (2 << length)" (so start
> at 0.5 for length = 0, which actually means length 1).
Actually any formula or constant I have tried (including crazy things like "length + 1" and "10.0") made very little difference. I enclose an odf spread sheet with lots of test data. The old code wins over anything. Contrary to what I thought, larger values (>1.0) seem to benefit longer lengths.
> Perhaps you could use AutoStatus external mode to have status printed
> after e.g. 1000, 1M, 1G, 10G, 100G candidates? At ~20M raw c/s, it'd
> take you 1.5 hours to get to 100G, but that's the speed on one CPU core,
> so you'll run several such tests in parallel (perhaps for all setting
> you want to test/compare).
AutoStatus eats 25% of the performance. I did the same thing but in crk_process_key() instead. That was just a 2.5% hit.
These tests are similar the the ones in the spread sheet but here we get some information over time (slower machine used here btw, old AMD):
Here's the original 0.001:
17g, 100p 0:00:00:00 850.0g/s 4800p/s 4800c/s 358921KC/s alick..12488
47g, 1Kp 0:00:00:00 522.2g/s 11066p/s 11066c/s 827255KC/s arv24..saris
210g, 10Kp 0:00:00:00 840.0g/s 39984p/s 39984c/s 2984MC/s ducie..ducya
864g, 100Kp 0:00:00:00 1464g/s 169484p/s 169484c/s 7421MC/s baspo..badge
3360g, 1Mp 0:00:00:01 2507g/s 746265p/s 746265c/s 72606MC/s cynco..cynon
6684g, 10Mp 0:00:00:03 2101g/s 3144Kp/s 3144Kc/s 232312MC/s 14l520..14l52b
11591g, 100Mp 0:00:00:12 898.5g/s 7751Kp/s 7751Kc/s 544245MC/s 0m0yma..0m0ymb
19082g, 1Gp 0:00:01:45 180.8g/s 9477Kp/s 9477Kc/s 561370MC/s asybase..asybama
27312g, 10Gp 0:00:19:10 23.74g/s 8695Kp/s 8695Kc/s 445852MC/s iagp2859..iagp2868
33634g, 100Gp 0:03:24:37 2.739g/s 8145Kp/s 8145Kc/s 358826MC/s n6zogp1..n6zogpl
...and length distribution of the guesses from the above run:
0 1
1 7
2 7
3 278
4 664
5 1529
6 13302
7 9243
8 7599
9 1001
10 3
0.01:
17g, 100p 0:00:00:00 566.6g/s 3200p/s 3200c/s 239281KC/s alick..12488
47g, 1Kp 0:00:00:00 427.2g/s 9054p/s 9054c/s 676845KC/s arv24..saris
210g, 10Kp 0:00:00:00 724.1g/s 34468p/s 34468c/s 2572MC/s ducie..ducya
864g, 100Kp 0:00:00:00 1329g/s 153840p/s 153840c/s 7421MC/s baspo..badge
3360g, 1Mp 0:00:00:01 2400g/s 714282p/s 714282c/s 72606MC/s cynco..cynon
6684g, 10Mp 0:00:00:03 1926g/s 2881Kp/s 2881Kc/s 232312MC/s 14l520..14l52b
11591g, 100Mp 0:00:00:13 874.7g/s 7547Kp/s 7547Kc/s 502380MC/s 0m0yma..0m0ymb
19084g, 1Gp 0:00:02:00 157.8g/s 8269Kp/s 8269Kc/s 491179MC/s asybabo..asybab7
27312g, 10Gp 0:00:21:12 21.47g/s 7861Kp/s 7861Kc/s 403086MC/s iagp29ty..iagp29te
33634g, 100Gp 0:03:27:22 2.703g/s 8037Kp/s 8037Kc/s 354067MC/s n6yp718..n6yp71g
0 1
1 7
2 7
3 278
4 664
5 1529
6 13302
7 9243
8 7599
9 1001
10 3
0.1:
17g, 100p 0:00:00:00 566.6g/s 3200p/s 3200c/s 239281KC/s alick..12488
47g, 1Kp 0:00:00:00 427.2g/s 9054p/s 9054c/s 676845KC/s arv24..saris
210g, 10Kp 0:00:00:00 724.1g/s 34468p/s 34468c/s 2572MC/s ducie..ducya
864g, 100Kp 0:00:00:00 1329g/s 153840p/s 153840c/s 7421MC/s baspo..badge
3360g, 1Mp 0:00:00:01 2349g/s 699297p/s 699297c/s 72606MC/s cynco..cynon
6688g, 10Mp 0:00:00:03 1832g/s 2739Kp/s 2739Kc/s 232307MC/s 14m9kh..14m9nn
11594g, 100Mp 0:00:00:14 790.3g/s 6816Kp/s 6816Kc/s 466472MC/s 0m0y0r..0m0ych
19086g, 1Gp 0:00:02:00 158.4g/s 8302Kp/s 8302Kc/s 491165MC/s asygduy..asygdrm
27313g, 10Gp 0:00:20:34 22.13g/s 8103Kp/s 8103Kc/s 415496MC/s iagpeon2..iagpeo11
33640g, 100Gp 0:03:05:36 3.020g/s 8979Kp/s 8979Kc/s 395573MC/s n6d7vsg..n6d7vv7
0 1
1 7
2 7
3 278
4 664
5 1529
6 13302
7 9243
8 7599
9 1001
10 6
11 3
0.5:
17g, 100p 0:00:00:00 566.6g/s 3200p/s 3200c/s 239281KC/s alick..12488
47g, 1Kp 0:00:00:00 470.0g/s 9960p/s 9960c/s 744530KC/s arv24..saris
210g, 10Kp 0:00:00:00 777.7g/s 37022p/s 37022c/s 2763MC/s ducie..ducya
864g, 100Kp 0:00:00:00 1252g/s 144921p/s 144921c/s 7421MC/s baspo..badge
3369g, 1Mp 0:00:00:01 1851g/s 549448p/s 549448c/s 72602MC/s cyrkk..cyr1a
6688g, 10Mp 0:00:00:04 1482g/s 2217Kp/s 2217Kc/s 174223MC/s 14m94c..14m9cd
11596g, 100Mp 0:00:00:15 753.4g/s 6497Kp/s 6497Kc/s 435364MC/s 0m1z0m..0m1z6w
19086g, 1Gp 0:00:01:47 177.5g/s 9302Kp/s 9302Kc/s 550832MC/s asy04as..asy04au
27314g, 10Gp 0:00:17:27 26.06g/s 9551Kp/s 9551Kc/s 489696MC/s iagbbosy..iagbboly
33639g, 100Gp 0:02:56:38 3.173g/s 9435Kp/s 9435Kc/s 415626MC/s n9/m53d..n9/m53z
0 1
1 7
2 7
3 278
4 664
5 1529
6 13302
7 9243
8 7599
9 1001
10 5
11 3
0.9:
17g, 100p 0:00:00:00 566.6g/s 3200p/s 3200c/s 239281KC/s alick..12488
47g, 1Kp 0:00:00:00 470.0g/s 9960p/s 9960c/s 744530KC/s arv24..saris
210g, 10Kp 0:00:00:00 840.0g/s 39984p/s 39984c/s 2984MC/s ducie..ducya
870g, 100Kp 0:00:00:00 1000g/s 114937p/s 114937c/s 7421MC/s bee15..beias
3369g, 1Mp 0:00:00:02 1684g/s 499998p/s 499998c/s 36300MC/s cyrky..cyr13
6688g, 10Mp 0:00:00:04 1429g/s 2136Kp/s 2136Kc/s 174223MC/s 14m972..14m963
11596g, 100Mp 0:00:00:16 711.4g/s 6134Kp/s 6134Kc/s 408146MC/s 0m1z0m..0m1z6w
19086g, 1Gp 0:00:01:55 165.0g/s 8645Kp/s 8645Kc/s 512513MC/s asy097j..asy0964
27318g, 10Gp 0:00:19:17 23.59g/s 8643Kp/s 8643Kc/s 443127MC/s iagdie0!..iagdiely
33641g, 100Gp 0:03:24:11 2.745g/s 8162Kp/s 8162Kc/s 359537MC/s n9lbdl7..n9lbdmb
0 1
1 7
2 7
3 278
4 664
5 1529
6 13302
7 9243
8 7599
9 1001
10 7
11 3
1.0:
17g, 100p 0:00:00:00 566.6g/s 3200p/s 3200c/s 239281KC/s alick..12488
47g, 1Kp 0:00:00:00 391.6g/s 8300p/s 8300c/s 620441KC/s arv24..saris
210g, 10Kp 0:00:00:00 656.2g/s 31237p/s 31237c/s 2331MC/s ducie..ducya
870g, 100Kp 0:00:00:00 878.7g/s 101006p/s 101006c/s 7421MC/s bee15..beias
3369g, 1Mp 0:00:00:02 1635g/s 485434p/s 485434c/s 36300MC/s cyrky..cyr13
6688g, 10Mp 0:00:00:04 1506g/s 2252Kp/s 2252Kc/s 174223MC/s 14m972..14m963
11596g, 100Mp 0:00:00:15 729.3g/s 6289Kp/s 6289Kc/s 435355MC/s 0mghei..0mghis
19087g, 1Gp 0:00:01:49 173.5g/s 9092Kp/s 9092Kc/s 540723MC/s asy2njd..asy2n1a
27318g, 10Gp 0:00:17:56 25.36g/s 9293Kp/s 9293Kc/s 476482MC/s iagrabll..iagraby2
33642g, 100Gp 0:03:12:01 2.920g/s 8679Kp/s 8679Kc/s 382316MC/s n9cx842..n9cx84k
0 1
1 7
2 7
3 278
4 664
5 1529
6 13302
7 9243
8 7599
9 1001
10 7
11 4
10.0 (!):
17g, 100p 0:00:00:00 425.0g/s 2400p/s 2400c/s 179460KC/s alick..12488
54g, 1Kp 0:00:00:00 145.9g/s 2691p/s 2691c/s 201210KC/s morgi..melar
215g, 10Kp 0:00:00:00 255.9g/s 11900p/s 11900c/s 888149KC/s dodys..doops
872g, 100Kp 0:00:00:01 593.1g/s 68024p/s 68024c/s 7420MC/s jhj20..jhhel
3373g, 1Mp 0:00:00:02 1317g/s 390623p/s 390623c/s 36299MC/s bbtlv..bbtot
6692g, 10Mp 0:00:00:04 1360g/s 2032Kp/s 2032Kc/s 174214MC/s 10cwe1..10cw04
11606g, 100Mp 0:00:00:17 656.0g/s 5652Kp/s 5652Kc/s 384097MC/s 0ny9fg..0nnpao
19111g, 1Gp 0:00:02:09 147.9g/s 7743Kp/s 7743Kc/s 456735MC/s alghtu1..alght87
27324g, 10Gp 0:00:18:34 24.51g/s 8976Kp/s 8976Kc/s 460098MC/s iadebuz7..iadebu04
33677g, 100Gp 0:03:24:56 2.738g/s 8132Kp/s 8132Kc/s 358043MC/s h8yxnsn..h8yxnwr
0 1
1 7
2 7
3 278
4 665
5 1529
6 13302
7 9243
8 7599
9 1001
10 39
11 6
1 / (1 << length):
17g, 100p 0:00:00:00 566.6g/s 3200p/s 3200c/s 239281KC/s alick..12488
47g, 1Kp 0:00:00:00 391.6g/s 8300p/s 8300c/s 620441KC/s arv24..saris
210g, 10Kp 0:00:00:00 700.0g/s 33320p/s 33320c/s 2487MC/s ducie..ducya
866g, 100Kp 0:00:00:00 1255g/s 144921p/s 144921c/s 7421MC/s beiah..bezya
3362g, 1Mp 0:00:00:01 1989g/s 591713p/s 591713c/s 72605MC/s cyndz..cynea
6686g, 10Mp 0:00:00:04 1595g/s 2386Kp/s 2386Kc/s 174229MC/s 14m9nm..14m9nl
11593g, 100Mp 0:00:00:15 727.2g/s 6273Kp/s 6273Kc/s 435383MC/s 0m0yfb..0m0yin
19084g, 1Gp 0:00:01:55 165.6g/s 8679Kp/s 8679Kc/s 512538MC/s asy0yaa..asy0yea
27312g, 10Gp 0:00:17:39 25.77g/s 9442Kp/s 9442Kc/s 484161MC/s iagp19ds..iagp19ej
33634g, 100Gp 0:02:48:37 3.324g/s 9884Kp/s 9884Kc/s 435436MC/s n65/BM4..n65/B34
0 1
1 7
2 7
3 278
4 664
5 1529
6 13302
7 9243
8 7599
9 1001
10 3
1 / (2 << length):
17g, 100p 0:00:00:00 425.0g/s 2400p/s 2400c/s 179460KC/s alick..12488
47g, 1Kp 0:00:00:00 427.2g/s 9054p/s 9054c/s 676845KC/s arv24..saris
210g, 10Kp 0:00:00:00 750.0g/s 35700p/s 35700c/s 2664MC/s ducie..ducya
864g, 100Kp 0:00:00:00 1107g/s 128200p/s 128200c/s 7421MC/s baspo..badge
3362g, 1Mp 0:00:00:02 1578g/s 469481p/s 469481c/s 36302MC/s cyndz..cynea
6686g, 10Mp 0:00:00:04 1482g/s 2217Kp/s 2217Kc/s 174229MC/s 14m9nm..14m9nl
11593g, 100Mp 0:00:00:17 668.1g/s 5763Kp/s 5763Kc/s 384161MC/s 0m0yfb..0m0yin
19083g, 1Gp 0:00:01:55 165.4g/s 8671Kp/s 8671Kc/s 512539MC/s asy0yaz..asy0yet
27312g, 10Gp 0:00:17:22 26.20g/s 9596Kp/s 9596Kc/s 492060MC/s iagp19ea..iagp1233
33634g, 100Gp 0:03:05:45 3.017g/s 8972Kp/s 8972Kc/s 395271MC/s n67jw35..n67jw3K
0 1
1 7
2 7
3 278
4 664
5 1529
6 13302
7 9243
8 7599
9 1001
10 3
Old incremental code compiled with 0x01 0xff and 24 and trained the same as above. NOTE I accidentally ran this without bumping -max-len to 12:
guesses: 18, 100p time: 0:00:00:00 0.00% c/s: 15951K trying: mishine - marliam
guesses: 62, 1Kp time: 0:00:00:01 0.00% c/s: 67073K trying: mina12 - millia
guesses: 251, 10Kp time: 0:00:00:02 0.00% c/s: 350173K trying: shorgie1 - shorgel1
guesses: 1082, 100Kp time: 0:00:00:04 0.00% c/s: 1852M trying: mariston - mariseda
guesses: 2940, 1Mp time: 0:00:00:06 0.00% c/s: 12107M trying: semsex - semsie
guesses: 7323, 10Mp time: 0:00:00:13 0.00% c/s: 53306M trying: 1apraa - 1apriz
guesses: 12998, 100Mp time: 0:00:00:31 0.00% c/s: 207484M trying: jav1ngi - jav1nix
guesses: 20125, 1Gp time: 0:00:02:14 0.00% c/s: 430986M trying: ts062mis - ts062mcm
guesses: 27037, 10Gp time: 0:00:17:21 0.00% c/s: 487169M trying: grnetr39 - grnetr24
guesses: 32579, 100Gp time: 0:02:40:51 0.00% c/s: 461849M trying: 0nhel0n5 - 0nhel0us
0 1
1 7
2 7
3 278
4 664
5 1507
6 13233
7 8864
8 8018
Here's an incomplete run at max-length=12:
guesses: 18/100p time: 0:00:00:00 c/s: 8065K trying: mishae1 - marissy
guesses: 46/1Kp time: 0:00:00:01 c/s: 41828K trying: sepping13 - seppiel12
guesses: 241/10Kp time: 0:00:00:03 c/s: 218798K trying: 11193 - 14787
guesses: 1004/100Kp time: 0:00:00:06 c/s: 1236M trying: sunshars - sunshero
guesses: 2752/1Mp time: 0:00:00:10 c/s: 7280M trying: brazygody - brazygon1
guesses: 6654/10Mp time: 0:00:00:19 c/s: 36675M trying: byalty - byalda
guesses: 12607/100Mp time: 0:00:00:39 c/s: 166328M trying: lokan1! - lokaid7
guesses: 19839/1Gp time: 0:00:02:16 c/s: 428027M trying: pff6l1 - pff3E4
guesses: 27338/10Gp time: 0:00:15:40 c/s: 539557M trying: cutsest24 - cutsestud
At 10Gp, we already have more than 1500 guesses longer than 8:
0 1
1 7
2 7
3 275
4 654
5 1408
6 12135
7 6747
8 4534
9 1201
10 299
11 37
12 4
I'm not sure we are even tweaking the right thing at all. The original thought I had was that after hundreds of millions of guesses, we should ideally know that shorter lengths are exhausted, probable or not. I'm not sure if this could be handled specially some way without affecting performance.
magnum
Download attachment "results.ods" of type "application/vnd.oasis.opendocument.spreadsheet" (28863 bytes)
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
