john-users - Re: tutorials on the wiki

Openwall GNU/*/Linux 3.0 - a small security-enhanced Linux distro for servers
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Fri, 13 Nov 2009 02:55:07 +0300
From: Solar Designer <solar@...nwall.com>
To: john-users@...ts.openwall.com
Subject: Re: tutorials on the wiki

Jim,

On Thu, Nov 05, 2009 at 07:30:52PM -0600, JFoug wrote:
> I will get an advanced howto tutorial put together on how to build a 'new' 
> generic md5 formats, into the john.conf.    Would you want this as native 
> wiki pages, or as downloadable data?

On the wiki, this falls under "development", not "tutorials".  You may
make it a sub-page under "development" and/or include it as a
documentation file added with your patch (or you may refer to the wiki
page from there).

> I figure for this, I will need to:
> 
> 1.  Explain the data structures within the generic environment. Data such 
> as input buffers, crypt 'output' buffers, salt buffers (yes, I do have 
> salt2 working), constants buffers (I have added ability to have up to 8 
> 'constants), count of passwords, the passwords themselves, etc.
> 
> 2.  Explain the layout of the script within john.conf
> 
> 3.  Explain the 'flags' which are the characterstics of the format.
> 
> 4.  Describe the functions.   Also describe how the function script is run, 
> just a simple walk down a list of functions.
> 
> 5.  Explain the 'flags' which are special optimization flags.  Explain the 
> side effects, and show scripts written with and without them.  Most of the 
> optimization flags allow us to reduce the number of crypts, to eliminate 
> certain steps at startup, etc.

All of the above sounds specific to your code, so it is up to you where
to document it best.

> Explain a little of the guts of how john 
> works (enough to explain how and why certain things can be sped up, with 
> precomputations).  Information of this type, is knowledeg that john sets 
> all of the passwords (128 for generic md5), and then simply tries salts one 
> a time, then sets each salt and calls crypt_all, then testing.  Also, that 
> how loading of the hashes, and salts is done, and what information about 
> the salts is used within john, and how it can be manipulated for speed 
> gain. This knowledge of the underbelly of john is required for most of the 
> optimizations (as it is with doing most any format).

Now this is far more generic, applicable to the official JtR as well -
so I and others might want to participate in editing documentation of
this sort.  Please place the above on the wiki, and perhaps it deserves
a separate page (not inter-mixed with specifics of your patches).

> 6.  Go through several examples.  From simple ones, to some more complex, 
> to some complex where they can be done in multiple ways, such as with some 
> of the optimization functions.

Sounds good.  Similarly to the above, maybe you (or someone else) can
place an example of adding a new "format" to the official JtR (not
relying on interfaces added by any patches yet) on a separate wiki page?
Then you can also have a page with examples for further changes to
JtR-with-JimF-patch.

The current "development" page is here:

http://openwall.info/wiki/john/development

It has some content by Pete, which needs to be moved to sub-pages.
Please don't be discouraged from adding content in there just because
the page is already "full" - instead, feel free to move its content and
replace it with an index page.

Thanks,

Alexander