```Date: Mon, 19 Nov 2012 09:57:15 -0600
From: Richard Miles <richard.k.miles@...glemail.com>
To: john-users@...ts.openwall.com
Subject: Re: How does incremental mode works?

Hi Simon,

it. To say the truth the math part was very complex to understand, but the
example helped me a bit to have an idea.

I still have one question and one suggestion if you don't mind.

On Sat, Nov 17, 2012 at 10:37 AM, Simon Marechal <simon@...quise.net> wrote:

> On 11/17/2012 02:14 AM, Richard Miles wrote:
> > Thanks for your answer. Nice to know I'm not the only one that is unable
> to
> > understand how it works and the difference in a high level between
> > incremental and markov. :)
> >
> > Maybe Solar or Simon may help us?
>
> I will answer about Markov mode. The statistics file that it uses contains
> :
> * the probability that character c is the first character of a password
> * the probability that character c_n follows c_(n-1) (the previous
> character)
>
> It doesn't actually store the raw probability, but something like:
>
>     P' = - N log(P)
>
> That way, something very likely (P ~ 1) will have P' ~ 0, and something
> highly unlikely (P ~ 0) will have a very high P'.
>
> You compute the "markov strength" of a password by adding all those P'.
> You can check this with the mkvcalcproba program. For example:
>
> p4ssw0rd!       28+58+47+23+46+56+56+30+76   =  420
>
> Notice how the first letter being identical, the first P' is identical
> between passwords, and how unlikely transitions cost more.
>

Very interesting, but how have you created it? I mean, what was your
original dictionary file? Because I created a short one just to test what
you wrote and the cost results are different, see please:

\$ cat test
123456
abc123
qazwsx
test

\$ ./calc_stat test stats
zero -10*log proba2[49*256+50] (2) / 2, converted to 1 to prevent infinite
length candidates
zero -10*log proba2[50*256+51] (2) / 2, converted to 1 to prevent infinite
length candidates
zero -10*log proba2[51*256+52] (1) / 1, converted to 1 to prevent infinite
length candidates
zero -10*log proba2[52*256+53] (1) / 1, converted to 1 to prevent infinite
length candidates
zero -10*log proba2[53*256+54] (1) / 1, converted to 1 to prevent infinite
length candidates
zero -10*log proba2[98*256+99] (1) / 1, converted to 1 to prevent infinite
length candidates
zero -10*log proba2[99*256+49] (1) / 1, converted to 1 to prevent infinite
length candidates
zero -10*log proba2[100*256+109] (1) / 1, converted to 1 to prevent
infinite length candidates
zero -10*log proba2[101*256+115] (1) / 1, converted to 1 to prevent
infinite length candidates
zero -10*log proba2[105*256+110] (1) / 1, converted to 1 to prevent
infinite length candidates
zero -10*log proba2[109*256+105] (1) / 1, converted to 1 to prevent
infinite length candidates
zero -10*log proba2[111*256+114] (2) / 2, converted to 1 to prevent
infinite length candidates
zero -10*log proba2[112*256+97] (2) / 2, converted to 1 to prevent infinite
length candidates
zero -10*log proba2[113*256+97] (1) / 1, converted to 1 to prevent infinite
length candidates
zero -10*log proba2[114*256+100] (2) / 2, converted to 1 to prevent
infinite length candidates
zero -10*log proba2[116*256+101] (1) / 1, converted to 1 to prevent
infinite length candidates
zero -10*log proba2[122*256+119] (1) / 1, converted to 1 to prevent
infinite length candidates

.//mkvcalcproba stats test
32 lines parsed [0x136d010]
password    12+1+9+10+10+4+1+1    48    8    55426111306    281
admin    12+16+1+1+1    31    5    1783065    257
password    12+1+9+10+10+4+1+1    48    8    55426111306    281
123456    19+1+1+1+1+1    24    6    305859    258
abc123    12+16+1+1+1+1    32    6    40710807    258
qazwsx    19+1+16+1+10+17    64    6    111228825    283
test    19+1+1+17    38    4    230340    272
freeing stuff ...
charsetsize = 23

>
> The markov incremental mode with JtR, given a maximum strength, will
> crack all passwords with a strength that is lower than or identical with
> the given maximum. This means that -markov:200 will crack none of the
> previous passwords, and -markov:250 will crack the easiest.
>

Interesting. One thing that called my attention in your example is the
following:

- "password" is really easier and consequently with a smaller cost (217).
- however if we are targeting real companies and not public leaks (that in
general do not enforce password policy or enforce very poor ones) I think
that "p4ssw0rd!" with a higher cost (420) will be more likely because of
Domain Controllers) will prevent for example the use of "password" but may
accept for example "p4ssw0rd!" or "P4ssw0rd!".
- So, while I agree that Markov computes it in a very smart way I guess it
may not be the best for real target. Do you think that is possible to adapt
Markov method or create and variation to target password hashes created
with an average or strong password policy?

> with the max strength parameter. You can use the genmkvpwd program to
> count them.
>
> I will give a hopefully better description of all of this at Passwords^12.
>

Very nice, I wish you good luck.

Thanks .

Best regard.

```