[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Date: Tue, 9 Aug 2011 20:12:26 +0400
From: Aleksey Cherepanov <aleksey.4erepanov@...il.com>
To: john-users@...ts.openwall.com
Subject: 'search patterns' workflow, collaboration, helper scripts
Frank, all,
I would like to have some scripts to support contest workflow.
Probably this workflow is not specific for the contest. I wrote dirty
prototype. Explanations are below.
During the contest I figured out following workflow around patterns:
crack some passwords,
analyze them for patterns,
try first pattern,
filter out passwords of found pattern to simplify next patterns search,
do the same with next pattern,
when all patterns found crack using general attacks again.
Search of patterns was already considered as important at the previous
contest. I think such workflow could be automated in many aspects.
Important thing here is to filter out known patterns. It simplifies
search of new patterns and improves possible charsets because charsets
would not have known patterns.
However there were big gap between moment when I found pattern and even
wrote filter for it and moment when I wrote rule. Rules writing was
hard for me. Rules writing includes different steps and I would like
to simplify them all: learning rules (help to try small examples),
quality assurance (help to collect statistics about rule
effectiveness), using this rules in attacks (help to not forget to use
some rules against some types of passwords, help to stop ineffective
attacks).
Also it would be good to have support for collaboration with ability
to have repository of rules and manage information of attacks.
So I wrote prototype of part of tool to support such workflow. It is
dirty and not finished yet. This prototype could be useful for
someone as is. However I expect some feedback to be sure that I chose
right way and to have more ideas. I would like to hear team members
that had problems with rules and especially Frank Dittrich which in
my opinion has a lot of ideas about rules but from different point of
view. Please!
I will explain what is already coded and how I see other part.
Scripts are written in bash, python and perl. Also they needs w3m.
Scripts are intended to be started in a separate directory. They
create many files during its works in this directory. Also they expect
John's 'run' directory to be in this directory. Example configuration
file is enclosed. It contains configuration for the contest
environment with uncracked password in 'uncracked' directory.
I tried to solve formats problem: you describe format of files once
and then if you run John using scripts they manage formats itself. So
there are 'typed_files.list' file. Each line contains format and path
to with hashes. It has some limitations: there should only one line
for one format. Currently it could work with many lines for one file
but with different formats. However later I will probably broke it.
So assume that we are in directory with unpacked scripts and we have
run directory here and there are uncracked hashes in files liste in
'typed_files.list' file.
First script to be used is init.sh. It will initialize directory.
There is a script to fast try rule:
./tryrule.sh word rule1 rule2 ...
(All scripts have short description information in first lines of code)
It creates wordlist with word and config file with listed rules then
it calls John to show generated words (uses -stdout). So it did not
try rules against hashes it only provide an ability to easily look how
rule works.
Example:
$ ./tryrule.sh mi : c
words: 2 time: 0:00:00:00 DONE (Tue Aug 9 18:33:59 2011) w/s: 100 current: Mi
mi
Mi
There is a script to create/overwrite pattern definition. Pattern
definition includes name of wordlist file and list of rules just like
tryrule.sh takes. That script saves pattern definition, tries pattern
against already cracked passwords and show statistics including count
of matches with current cracked passwords set.
I would like to add such statistics as size of overlap with other
patterns and size of complement in other patterns.
Example:
$ echo mi > mi.lst
$ ./pattern.sh mi_pattern mi.lst : c
words: 2 time: 0:00:00:00 DONE (Tue Aug 9 18:47:19 2011) w/s: 200 current: Mi
Matches: 0
Percent of matches: 0
To start attack against pattern defined use ./attack.sh. It will start
one John per hash type (per line in typed_files.list). Each attack has
its own number. It is like name of attack. So you will be able to
address attack by number later.
Example:
$ ./attack.sh mi
Your attack was numbered as 1
For custom attacks there is a launcher ./runalltypes.sh. This script
numbers attack and dispatches one John per hash type.
Example:
$ ./runalltypes.sh -ru:single -w:mi.lst
Your attack was numbered as 2
To stop attack there is ./killattack.sh. It kills all Johns that work
within some attack. You could provide number of attack you want to
kill or you could omit this value then script will be applied to the
last attack you did.
Example:
$ ./killattack.sh 1
However there are no ability to resume attack (yet).
To get statistics about certain attack use ./astatus.sh. It will show
table for attack with following columns: type (hash type), status,
c/s, guesses, time, time since last crack, tries since last crack (time
* c/s). This scripts uses 'john -status' and reads logs. So there are
two values in some columns: just value is obtained from John, value in
braces is obtained from log. It is needed because -status does not
work for finished sessions. In that case primary value would be 0
while value in braces would be actual.
Example:
$ ./astatus.sh 1
Type Status ...
md5_gen(22) finished ...
des finished ...
...
If you do not like tables in text you could view it with your
preferred browser using ./astatus.html file.
There is also ./results.sh. This script is intended to extract
results from pot. However I did not implement it fully. 'cracked' file
is a result of script's work. I would like to add ability to split cracked
passwords by hash type of source. Also I expect that script to prepare
filtered files, files that do not contain passwords that could be
caught with any rule we defined.
Other scripts are not intended for direct use.
And now some words about collaboration. I think there should be an
ability to publicize your attack (automatically in ./attack.sh), to
report status of that attack (something on cron), to suspend and
resume attack on different machines (someone suspends attack and his
session files are sent to server, when someone want to resume that
attack these files are downloaded from server and session resumes, of
course all with command like ones above), to start equal attack (if
someone thinks that has better hardware while first attacker do not
want to suspend his attack or session files on the server is
corrupted). So if one could publicize his great rule he starts attack
with this rule, look results for fast hashes, suspends attack so
others will see results, analyze them and make decision about this
attack effectiveness.
As improvement I could imagine cluster management. I do not think
about it as about one big cluster. I imagine it like: each user has
its own cluster and manages it itself however there are support for
dispatching attacks on remote computers with load balancing, status
collection and so on. However it seems to be ineffective in case of
dumb force need (someone could prepare wordlist, split it into
small packes, register them as different attacks but this could be
done in more comfortable way with specific tools).
Also there was attempt to write script that finds similar passwords to
help to search patterns. However it seems to be separate thing that is
too big for this mail.
What do you think? Suggestions? Criticisms?
Regards,
Aleksey Cherepanov
Download attachment "john_helper_scripts.tar.xz" of type "application/octet-stream" (4328 bytes)
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
