Openwall GNU/*/Linux - a small security-enhanced Linux distro for servers
[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Date: Sat, 9 Jun 2012 21:18:32 +0200
From: Frank Dittrich <frank_dittrich@...mail.com>
To: john-dev@...ts.openwall.com
Subject: Patch: allow --markov=SECTION in addition to --markov[=[MIN_LEVEL-]LEVEL[:START[:END[:[MIN_LENGHT-]LENGTH]]]]

Hi all,

this (or something similar) has been discussed in the past, see
http://www.openwall.com/lists/john-dev/2011/03/16/5
and the three follow-up messages (6-8 on the same day).

As suggested the last time --markov=section has been discussed, I made
sure that the old behavior still works (and is the default when --markov
is used without specifying any values).

--markov=SECTION is the only way to use several Markov sessions at the
same time which use different stats files - unless you use separate
config files.


May be it is easier to discuss an existing implementation, so I prepared
a patch.

That's why my implementation requires these variables to be specified
in the [Markov:SECTION] that corresponds to the --markov=SECTION command
line option:
Statsfile
MkvLvl
MkvMaxLen
These variables are optional (0 being the default if not specified):
MkvStart
MkvEnd
MkvMinLvl
MkvMinLen


To be able to distinguish --markov=LEVEL and --markov=SECTION, the
current implementation tries to use --markov=LEVEL unless the first
character of the value is a letter.
(This could be relaxed a bit, treating any value that contains at least
one letter and no colon as a section name.)


I also modified doc/MARKOV, run/john.bash_completion and john.conf.


I verified that the old logic still works, and that I can interrupt and
restore a session using --markov=section

I did not see any negative side effects, but please feel free to test
the patch and/or suggest changes.
(Most important for me are your thoughts regarding changes in the user
interface, because I'd hate to add changes that are not backwards
compatible to a previously released version.


Frank

>From 9d8cbdc20eb039d9286fb2e7c73482fd6ac7328b Mon Sep 17 00:00:00 2001
From: Frank Dittrich <frank_dittrich@...mail.com>
Date: Sat, 9 Jun 2012 21:08:59 +0200
Subject: [PATCH] Implement --markov=SECTION in addition to
 --markov[=[MIN_LEVEL-]LEVEL[:START[:END[:[MIN_LENGHT-]LENGTH]]]]

---
 doc/MARKOV               |   75 ++++++++++++++++++++++++++++++++++++++++++++-
 run/john.bash_completion |   24 ++++++++++++++-
 run/john.conf            |   36 +++++++++++++++++++++-
 src/john.c               |   43 +++++++++++++++++++++++++-
 src/mkv.c                |    8 ++---
 src/mkv.h                |    2 +-
 src/options.c            |    6 ++-
 src/options.h            |    1 +
 8 files changed, 182 insertions(+), 13 deletions(-)

diff --git a/doc/MARKOV b/doc/MARKOV
index accbbc0..e112eec 100644
--- a/doc/MARKOV
+++ b/doc/MARKOV
@@ -1,11 +1,17 @@
 BASIC USAGE
 The Markov mode is based from [1], tested and applied to "classical" password
 cracking in [2]. This mode similar to the "wordlist" mode because it will only
-crack a fixed quantity of passwords. Its parameters are:
+crack a fixed quantity of passwords.
 
+There are two alternatives to specify the Markov mode parameters.
+
+The first one is:
 --markov:LEVEL:START:END:LENGTH
+or
+--markov=LEVEL:START:END:LENGTH
+
+with these parameters:
 
-Where:
 * LEVEL is the "Markov level". This value is the maximum strength of passwords
 that are going to be cracked. When LEVEL increases, the quantity of passwords
 that are going to be tested increases exponentially.
@@ -18,6 +24,71 @@ set to 0, it will represent the last possible password.
 using --markov:100:0:0:12 will let john check every password whose length is 12
 or less and whose "Markov strength" is 100 or less.
 
+If no Markov parameters are specified (--markov instead of --markov=LEVEL...),
+the default values specified in the section [Options:Markov] of the config file
+will be used.
+
+The second alternative is:
+--markov:SECTION
+or
+--markov=SECTION
+
+In this case, SECTION is a name of a section [Markov:SECTION] defined in
+the config file (john.conf or the one specified with --config=...)
+
+Instead of parameters specified on the command line, Markov mode will use
+the values specified in this section.
+
+To distinguish the two alternatives, the SECTION name must begin with a letter
+([A-Za-z]), and must not contain a colon (:) or an equal sign(=).
+
+A Markov section corresponding to the default values (hard coded or
+corresponding to the values listed in the [Options:Markov] section) would be:
+
+[Markov:Default]
+# These 3 variables have to be specified, there is no fallback to the
+# variables specified in [Options:Markov]. This means, the parameters of
+# an interrupted Markov section remain unchanged even if the values
+# in the [Options:Markov] section get changed.
+#
+Statsfile = $JOHN/stats
+# LEVEL
+MkvLvl = 200
+# LENGTH
+MkvMaxLen = 12
+# These 4 variables are optional, the following lines represent the default
+# values which would be used when a variable is not specified here.
+# START
+MkvStart = 0
+# END
+MkvEnd = 0
+# MIN_LEVEL
+MkvMinLvl = 0
+# MIN_LENGTH
+MkvMinLen = 0
+
+A Markov section that corresponds to the --markov:100:0:0:12 command line
+parameter would be:
+
+[Markov:lvl100len12]
+Statsfile = $JOHN/stats
+MkvLvl = 100
+MkvMaxLen
+# The following 2 variables are optional, since 0 is their default value
+MkvStart = 0
+MkvEnd = 0
+
+A Markov section corresponding to the --markov=201-280:0:0:8-12 command line
+parameter would be:
+
+[Markov:Example]
+Statsfile = $JOHN/stats
+MkvMinLvl = 201
+MkvLvl = 220
+MkvMinLen = 8
+MkvMaxLen = 12
+
+
 
 SELECTING THE PARAMETERS
 The "LEVEL" parameter should be selected based on the desired maximum running
diff --git a/run/john.bash_completion b/run/john.bash_completion
index 363a5e3..65f758e 100644
--- a/run/john.bash_completion
+++ b/run/john.bash_completion
@@ -359,10 +359,32 @@ _john()
 			;;
 		--markov)
 			if [[ "${valopts}" == *${cur}* ]] ; then
-				COMPREPLY=( $(compgen -W "--markov --markov=LEVEL[:START[:END[:LENGTH]]]" -- ${cur}) )
+				if [[ "_${__john_completion}" == "_2" ]] ; then
+					COMPREPLY=( $(compgen -W "${cur}=" -- ${cur}) )
+					compopt -o nospace
+				else
+					# FIXME: I think not all Markov implementations support an interval for level and/or length - how to find out?
+					COMPREPLY=( $(compgen -W "--markov --markov=SECTION --markov=[MIN_LEVEL-]LEVEL[:START[:END[:[MIN_LENGHT-]LENGTH]]]" -- ${cur}) )
+					# FIXME: Should I check the markov section names (require those names to begin with a letter)
+					#        to allow completion for -?(-)mar?(k|ko|kov)+(=|:)[a-zA-Z]*)
+				fi
+			fi
+			return 0
+			;;
+		-?(-)mar?(k|ko|kov)+(:|=)*)
+			if [[ "${valopts}" == *--markov* ]] ; then
+				# ignore the  --markov=[MINLVL-]LEVEL[:START[:END[:[MINLEN-]LENGTH]]] alternative,
+				# just try completion for --markov=SECTION
+				if [[ "${hidden}" == *--list=* || "${valopts}" == *--list=* ]] ; then
+					cur=`echo ${cur#*[=:]}|LC_ALL=C tr A-Z a-z`
+					# Don't include section names that don't begin with a letter
+					list=`${first} --list=Markov 2>/dev/null | LC_ALL=C sed 's#^[^a-zA-Z].*$##'`
+					COMPREPLY=( $(compgen -W "${list}" -- ${cur}) )
+				fi
 			fi
 			return 0
 			;;
+			#
 		--test)
 			if [[ "${valopts}" == *${cur}* ]] ; then
 				COMPREPLY=( $(compgen -W "--test --test=SECONDS" -- ${cur}) )
diff --git a/run/john.conf b/run/john.conf
index 6c01ace..5c2d777 100644
--- a/run/john.conf
+++ b/run/john.conf
@@ -62,7 +62,9 @@ LogCrackedPasswords = N
 
 
 [Options:Markov]
-# Default Markov mode settings
+# Default Markov mode settings, used for parameters that are not
+# specified on the command line when using this --markov variant:
+# --markov[=[MIN_LEVEL-]LEVEL[:START[:END[:[MIN_LENGHT-]LENGTH]]]]
 Statsfile = $JOHN/stats
 MkvLvl = 200
 MkvMaxLen = 12
@@ -99,6 +101,38 @@ Device = 0
 #cryptsha512_GWS = 8192
 
 
+# Markov sections allow interrupting and restoring a markov session
+# without caring about config changes in the "global" [Options:Markov]
+# section
+# These sections are read when --markov=SECTION instead of
+# --markov[=[MINLVL-]LEVEL[:START[:END[:[MINLEN-]LENGTH]]]]
+# is used on the command line.
+# Markov section names must begin with a letter [A-Za-z]
+#
+# A section that corresponds to the default settings defined
+# in [Options:Markov]
+[Markov:Default]
+# These 3 variables have to be specified, there is no fallback to the
+# variables specified in [Options:Markov], to allow the parameters of
+# an interrupted Markov section to remain unchanged even if the values
+# in the [Options:Markov] section get changed.
+Statsfile = $JOHN/stats
+# LEVEL
+MkvLvl = 200
+# LENGTH
+MkvMaxLen = 12
+# These 4 variables are optional, the following lines represent the default
+# values which would be used when a variable is not specified here.
+# START
+MkvStart = 0
+# END
+MkvEnd = 0
+# MIN_LEVEL
+MkvMinLvl = 0
+# MIN_LENGTH
+MkvMinLen = 0
+
+
 # A user defined character class is named with a single digit, ie. 0..9. After
 # the equal-sign, just list all characters that this class should match. You
 # can specify ranges within brackets, much like pre-processor ranges in rules.
diff --git a/src/john.c b/src/john.c
index cec024d..a200a85 100644
--- a/src/john.c
+++ b/src/john.c
@@ -750,7 +750,48 @@ static void john_run(void)
 			do_incremental_crack(&database, options.charset);
 		else
 		if (options.flags & FLG_MKV_CHK)
-			do_markov_crack(&database, options.mkv_level, options.mkv_start, options.mkv_end, options.mkv_maxlen, options.mkv_minlevel, options.mkv_minlen);
+		{
+			if ((options.mkv_param[0] >= 'a' && options.mkv_param[0] <= 'z') ||
+			    (options.mkv_param[0] >= 'A' && options.mkv_param[0] <= 'Z'))
+			{
+				options.mkv_level = 0;
+				/*
+				 * Treat options.mkv_param as the name of a subsection of [Markov:]
+				 *
+				 * Should I check whether this section exists at all?
+				 * (Incremental mode also just complains about the charset being no defined.)
+				 */
+
+				if((options.mkv_statfile = cfg_get_param("Markov:", options.mkv_param, "Statsfile")) == NULL)
+				{
+					fprintf(stderr, "Statsfile not defined for Markov mode %s\n", options.mkv_param);
+					exit(1);
+				}
+				if((options.mkv_start = cfg_get_int("Markov:", options.mkv_param, "MkvStart")) == -1)
+					options.mkv_start = 0;
+				if((options.mkv_end = cfg_get_int("Markov:", options.mkv_param, "MkvEnd")) == -1)
+					options.mkv_end = 0;
+				if((options.mkv_level = cfg_get_int("Markov:", options.mkv_param, "MkvLvl")) == -1)
+				{
+					fprintf(stderr, "MkvLvl not defined for Markov mode %s\n", options.mkv_param);
+					exit(1);
+				}
+				if((options.mkv_minlevel = cfg_get_int("Markov:", options.mkv_param, "MkvMinLvl")) == -1)
+					options.mkv_minlevel = 0;
+				if((options.mkv_maxlen = cfg_get_int("Markov:", options.mkv_param, "MkvMaxLen")) == -1)
+				{
+					fprintf(stderr, "MkvMaxLen not defined for Markov mode %s\n", options.mkv_param);
+				}
+				if((options.mkv_minlen = cfg_get_int("Markov:", options.mkv_param, "MkvMinLen")) == -1)
+					options.mkv_minlen = 0;
+			}
+			else
+			{
+				options.mkv_statfile = cfg_get_param("Options", SUBSECTION_MARKOV, "Statsfile");
+
+			}
+			do_markov_crack(&database, options.mkv_level, options.mkv_start, options.mkv_end, options.mkv_maxlen, options.mkv_minlevel, options.mkv_minlen, options.mkv_statfile);
+		}
 		else
 		if (options.flags & FLG_EXTERNAL_CHK)
 			do_external_crack(&database);
diff --git a/src/mkv.c b/src/mkv.c
index f29fac6..9ca796d 100644
--- a/src/mkv.c
+++ b/src/mkv.c
@@ -238,9 +238,8 @@ static int get_progress(int *hundth_perc)
 }
 
 
-void do_markov_crack(struct db_main *db, unsigned int mkv_level, unsigned long long mkv_start, unsigned long long mkv_end, unsigned int mkv_maxlen, unsigned int mkv_minlevel, unsigned int mkv_minlen)
+void do_markov_crack(struct db_main *db, unsigned int mkv_level, unsigned long long mkv_start, unsigned long long mkv_end, unsigned int mkv_maxlen, unsigned int mkv_minlevel, unsigned int mkv_minlen, char *mkv_statfile)
 {
-	char * statfile;
 
 #ifdef HAVE_MPI
 	unsigned long long mkv_size;
@@ -267,8 +266,7 @@ void do_markov_crack(struct db_main *db, unsigned int mkv_level, unsigned long l
 			error();
 		}
 
-	statfile = cfg_get_param("Options", SUBSECTION_MARKOV, "Statsfile");
-	if(statfile == NULL)
+	if(mkv_statfile == NULL)
 	{
 		log_event("statfile not defined");
 #ifdef HAVE_MPI
@@ -314,7 +312,7 @@ void do_markov_crack(struct db_main *db, unsigned int mkv_level, unsigned long l
 	rec_restore_mode(restore_state);
 	rec_init(db, save_state);
 
-	init_probatables(path_expand(statfile));
+	init_probatables(path_expand(mkv_statfile));
 
 	crk_init(db, fix_state, NULL);
 
diff --git a/src/mkv.h b/src/mkv.h
index 0913eed..be43aa6 100644
--- a/src/mkv.h
+++ b/src/mkv.h
@@ -12,6 +12,6 @@
 /*
  * Runs the markov mode cracker.
  */
-extern void do_markov_crack(struct db_main *db, unsigned int mkv_level, unsigned long long mkv_start, unsigned long long mkv_end, unsigned int mkv_maxlen, unsigned int mkv_minlevel, unsigned int mkv_minlen);
+extern void do_markov_crack(struct db_main *db, unsigned int mkv_level, unsigned long long mkv_start, unsigned long long mkv_end, unsigned int mkv_maxlen, unsigned int mkv_minlevel, unsigned int mkv_minlen, char *mkv_statfile);
 
 #endif
diff --git a/src/options.c b/src/options.c
index e9b874a..5b9e078 100644
--- a/src/options.c
+++ b/src/options.c
@@ -178,7 +178,7 @@ static struct opt_entry opt_list[] = {
 "                          full list, use --encoding=LIST\n" \
 "--rules[=SECTION]         enable word mangling rules for wordlist mode\n" \
 "--incremental[=MODE]      \"incremental\" mode [using section MODE]\n" \
-"--markov[=LEVEL[:opts]]   \"Markov\" mode (see documentation)\n" \
+"--markov[=[SECTION|LVL]]  \"Markov\" mode, see doc/MARKOV\n" \
 "--external=MODE           external mode or word filter\n" \
 "--stdout[=LENGTH]         just output candidate passwords [cut at LENGTH]\n" \
 "--restore[=NAME]          restore an interrupted session [called NAME]\n" \
@@ -438,7 +438,9 @@ void opt_init(char *name, int argc, char **argv)
 		options.mkv_maxlen = 0;
 		options.mkv_minlevel = 0;
 		options.mkv_minlen = 0;
-		if (options.mkv_param)
+		if (options.mkv_param &&
+		   (options.mkv_param[0] < 'a' || options.mkv_param[0] > 'z') &&
+		   (options.mkv_param[0] < 'A' || options.mkv_param[0] > 'Z'))
 		{
 			token = strtok(options.mkv_param, ":");
 			if(sscanf(token, "%d-%d", &options.mkv_minlevel, &options.mkv_level) != 2)
diff --git a/src/options.h b/src/options.h
index 4dfc172..250a7a8 100644
--- a/src/options.h
+++ b/src/options.h
@@ -144,6 +144,7 @@ struct options_main {
 	unsigned int mkv_maxlen;
 	unsigned int mkv_minlevel;
 	unsigned int mkv_minlen;
+	char *mkv_statfile;
 
 /* Maximum plaintext length for stdout mode */
 	int length;
-- 
1.7.7.6


Powered by blists - more mailing lists

Your e-mail address:

Powered by Openwall GNU/*/Linux - Powered by OpenVZ