Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Date: Sat, 25 Feb 2006 21:50:42 +0100
From: "Frank Dittrich" <frank_dittrich@...mail.com>
To: john-users@...ts.openwall.com
Subject: "Charset file has changed: $JOHN/all.chr" error

Hi,

while I had running another john-1.6.3? session in another directory,
I started testing john 1.7 (using the original john.conf) with a sample
DES password file:

$ ./john --session=sec_DES /tmp/sec/pw.DES
Loaded 56 password hashes with 56 different salts (Traditional DES [64/64 BS 
MMX ])
[...]

and with a sample FreeBSD MD5 password file:

Loaded 223 password hashes with 223 different salts (FreeBSD MD5 [32/32])
$ ./john --session=sec_MD5 /tmp/sec/pw.MD5
[...]

Both sessions have been interrupted manually after some passwords
have been cracked, then they have been restored
$ ./john --restore=sec_DES
and
$ ./john --restore=sec_DES

I did, however, not run the two john 1.7 sessions simultaneoulsy.

Furthermore, I did send a signal 1 to both sessions while keeping
them running.

I didn't encounter any problems with the DES password file, but with
the FreeBSD MD5 passwords I repeatedly got
"Charset file has changed: $JOHN/all.chr" messages
(on stdout or stderr), and the session had been stopped.

After this error occured the second time, I restarted the session,
manually interrupted it twice, checked the MD5 sum of all.chr,
restarted the session, waited until the error occured again,
and double-checked the MD5 sum of all.chr:

[...]
guesses: 27  time: 0:02:08:40 94% (2)  c/s: 3501  trying: Helpping
guesses: 27  time: 0:02:08:41 94% (2)  c/s: 3501  trying: Rainbowing
guesses: 27  time: 0:02:08:42 94% (2)  c/s: 3501  trying: Archiing
Charset file has changed: $JOHN/all.chr
$ ./john --restore=sec_MD5
Loaded 196 password hashes with 196 different salts (FreeBSD MD5 [32/32])
guesses: 27  time: 0:02:08:10 94% (2)  c/s: 3494  trying: Montanaed
guesses: 27  time: 0:02:08:13 94% (2)  c/s: 3495  trying: Patchesed
Charset file has changed: $JOHN/all.chr
$ ./john --restore=sec_MD5
Loaded 196 password hashes with 196 different salts (FreeBSD MD5 [32/32])
guesses: 27  time: 0:02:08:11 94% (2)  c/s: 3494  trying: Montanaed
guesses: 27  time: 0:02:08:12 94% (2)  c/s: 3494  trying: Neutrinoed
guesses: 27  time: 0:02:08:12 94% (2)  c/s: 3495  trying: Oaxacaed
Session aborted
§ ./john --restore=sec_MD5
Loaded 196 password hashes with 196 different salts (FreeBSD MD5 [32/32])
guesses: 27  time: 0:02:08:14 94% (2)  c/s: 3494  trying: Parroted
guesses: 27  time: 0:02:08:15 94% (2)  c/s: 3494  trying: Pianosed
guesses: 27  time: 0:02:08:16 94% (2)  c/s: 3495  trying: Pyroed
Session aborted
fd@...ux:~/john-1.7/run>
fd@...ux:~/john-1.7/run> ./john --restore=sec_MD5
Loaded 196 password hashes with 196 different salts (FreeBSD MD5 [32/32])
guesses: 27  time: 0:02:08:18 94% (2)  c/s: 3495  trying: Robbied
Session aborted
fd@...ux:~/john-1.7/run> md5sum all.chr
8fec3288c3f1bc96273d86cd1447d019  all.chr
fd@...ux:~/john-1.7/run> ./john --restore=sec_MD5
Loaded 196 password hashes with 196 different salts (FreeBSD MD5 [32/32])
Charset file has changed: $JOHN/all.chr
fd@...ux:~/john-1.7/run> md5sum all.chr
8fec3288c3f1bc96273d86cd1447d019  all.chr
fd@...ux:~/john-1.7/run> ./john --incremental --session=sec_MD5 
/tmp/sec/pw.MD5
Loaded 196 password hashes with 196 different salts (FreeBSD MD5 [32/32])
guesses: 0  time: 0:00:00:01  c/s: 4578  trying: mannie
guesses: 0  time: 0:00:00:02  c/s: 4702  trying: sharks
guesses: 0  time: 0:00:00:03  c/s: 4732  trying: sames1
[...]

As you can see, the MD5 sum didn't change.

Here's an extract from the log file:

[...]
0:02:01:11 - Rule #51: '-c <*>2!?Acp' accepted as '<*>2!?Acp'
0:02:02:57 - Rule #52: '<*>2!?AlP' accepted
0:02:04:44 - Rule #53: '<*>2!?AlI' accepted
0:02:06:43 - Rule #54: '-c <*>2!?AcP' accepted as '<*>2!?AcP'
0:02:08:33 - Rule #55: '-c <*>2!?AcI' accepted as '<*>2!?AcI'
0:02:10:19 - Rule #56: '-s x**' rejected
0:02:10:19 - Rule #57: '-s-c x**MlQ' rejected
0:02:10:19 Proceeding with "incremental" mode: All
0:02:10:19 ! Charset file has changed: $JOHN/all.chr
0:02:10:19 Terminating on error
0:02:08:09 Continuing an interrupted session
0:02:08:09 Loaded a total of 223 password hashes with 223 different salts
0:02:08:09 Remaining 196 password hashes with 196 different salts
0:02:08:09 - Hash type: FreeBSD MD5 (lengths up to 15)
0:02:08:09 - Algorithm: 32/32
0:02:08:09 - Candidate passwords may be buffered and tried in chunks of 8
0:02:08:09 Proceeding with wordlist mode
0:02:08:09 - Wordlist file: ./password.lst
0:02:08:09 - 57 preprocessed word mangling rules
0:02:08:09 - Rule #54: '-c <*>2!?AcP' accepted as '<*>2!?AcP'
0:02:08:34 - Rule #55: '-c <*>2!?AcI' accepted as '<*>2!?AcI'
0:02:10:21 - Rule #56: '-s x**' rejected
0:02:10:21 - Rule #57: '-s-c x**MlQ' rejected
0:02:10:21 Proceeding with "incremental" mode: All
0:02:10:21 ! Charset file has changed: $JOHN/all.chr
0:02:10:21 Terminating on error
0:02:08:10 Continuing an interrupted session
0:02:08:10 Loaded a total of 223 password hashes with 223 different salts
0:02:08:10 Remaining 196 password hashes with 196 different salts
0:02:08:10 - Hash type: FreeBSD MD5 (lengths up to 15)
0:02:08:10 - Algorithm: 32/32
0:02:08:10 - Candidate passwords may be buffered and tried in chunks of 8
0:02:08:10 Proceeding with wordlist mode
0:02:08:10 - Wordlist file: ./password.lst
0:02:08:10 - 57 preprocessed word mangling rules
0:02:08:10 - Rule #54: '-c <*>2!?AcP' accepted as '<*>2!?AcP'
0:02:08:12 Session aborted
0:02:08:13 Continuing an interrupted session
0:02:08:13 Loaded a total of 223 password hashes with 223 different salts
0:02:08:13 Remaining 196 password hashes with 196 different salts
0:02:08:13 - Hash type: FreeBSD MD5 (lengths up to 15)
0:02:08:13 - Algorithm: 32/32
0:02:08:13 - Candidate passwords may be buffered and tried in chunks of 8
[...]

Looks like the wrong diagnosis happened immediately after john
finished wordlist rules and switched to incremental mode.
(If john's message is correct: what else could have been changed,
if the MD5 sum remained the same?
I certainly didn't change ownership or access permissions.
And: the error did not occur with the DES password file.)

When I started john --incremental on the same password file,
the error did not occur.


Every software involved is used as delivered from SuSE Linux prof. 9.2
(including online updates, of course):

Linux kernel version:
2.6.8-24.19-default #1 Tue Nov 29 14:32:45 UTC 2005 i686 athlon i386 
GNU/Linux

$ cat /proc/cpuinfo
processor       : 0
vendor_id       : AuthenticAMD
cpu family      : 6
model           : 6
model name      : AMD Athlon(TM) XP 2100+
stepping        : 2
cpu MHz         : 1725.238
cache size      : 256 KB
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 1
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 sep mtrr pge mca cmov 
pat pse36 mmx fxsr sse pni syscall mmxext 3dnowext 3dnow
bogomips        : 3416.06

$  gcc --version
gcc (GCC) 3.3.4 (pre 3.3.5 20040809)

John has been built for linux-x86-mmx.
No patches have been applied.
The only change to john's Makefile was adding -march=athlon-xp to the 
CFLAGS.

What could have gone wrong.
What additional tests could help?
Do you need more information?
(I could provide the sample password files via mail.)



While I had running another john-1.6.?? session in another directory,
I started testing john 1.7 (using the original john.conf) with a sample
DES password file:
$ ./john --session=sec_DES /tmp/sec/pw.DES
Loaded 56 password hashes with 56 different salts (Traditional DES [64/64 BS 
MMX ])

and with a sample FreeBSD MD5 password file:

Loaded 223 password hashes with 223 different salts (FreeBSD MD5 [32/32])
$ ./john --session=sec_MD5 /tmp/sec/pw.MD5
...

Both sessions have been interrupted manually after some passwords
have been cracked, then they have been restored
$ ./john --restore=sec_DES
and
$ ./john --restore=sec_DES

I did, however, not run the two john 1.7 sessions simultaneoulsy.

Furthermore, I did send a signal 1 to both sessions while keeping
them running.

I didn't encounter any problems with the DES password file, but with
the FreeBSD MD5 passwords I repeadedly got
"Charset file has changed: $JOHN/all.chr" messages
(on stdout or stderr)

After this error occured the second time, I restarted the session,
manually interrupted it twice, checked the MD5 sum of all.chr,
restarted the session, waited until the error occured again,
and ddouble-checked the MD5 sum of all.chr:

[...]
guesses: 27  time: 0:02:08:40 94% (2)  c/s: 3501  trying: Helpping
guesses: 27  time: 0:02:08:41 94% (2)  c/s: 3501  trying: Rainbowing
guesses: 27  time: 0:02:08:42 94% (2)  c/s: 3501  trying: Archiing
Charset file has changed: $JOHN/all.chr
$ ./john --restore=sec_MD5
Loaded 196 password hashes with 196 different salts (FreeBSD MD5 [32/32])
guesses: 27  time: 0:02:08:10 94% (2)  c/s: 3494  trying: Montanaed
guesses: 27  time: 0:02:08:13 94% (2)  c/s: 3495  trying: Patchesed
Charset file has changed: $JOHN/all.chr
$ ./john --restore=sec_MD5
Loaded 196 password hashes with 196 different salts (FreeBSD MD5 [32/32])
guesses: 27  time: 0:02:08:11 94% (2)  c/s: 3494  trying: Montanaed
guesses: 27  time: 0:02:08:12 94% (2)  c/s: 3494  trying: Neutrinoed
guesses: 27  time: 0:02:08:12 94% (2)  c/s: 3495  trying: Oaxacaed
Session aborted
§ ./john --restore=sec_MD5
Loaded 196 password hashes with 196 different salts (FreeBSD MD5 [32/32])
guesses: 27  time: 0:02:08:14 94% (2)  c/s: 3494  trying: Parroted
guesses: 27  time: 0:02:08:15 94% (2)  c/s: 3494  trying: Pianosed
guesses: 27  time: 0:02:08:16 94% (2)  c/s: 3495  trying: Pyroed
Session aborted
fd@...ux:~/john-1.7/run>
fd@...ux:~/john-1.7/run> ./john --restore=sec_MD5
Loaded 196 password hashes with 196 different salts (FreeBSD MD5 [32/32])
guesses: 27  time: 0:02:08:18 94% (2)  c/s: 3495  trying: Robbied
Session aborted
fd@...ux:~/john-1.7/run> md5sum all.chr
8fec3288c3f1bc96273d86cd1447d019  all.chr
fd@...ux:~/john-1.7/run> ./john --restore=sec_MD5
Loaded 196 password hashes with 196 different salts (FreeBSD MD5 [32/32])
Charset file has changed: $JOHN/all.chr
fd@...ux:~/john-1.7/run> md5sum all.chr
8fec3288c3f1bc96273d86cd1447d019  all.chr
fd@...ux:~/john-1.7/run> ./john --incremental --session=sec_MD5 
/tmp/sec/pw.MD5
Loaded 196 password hashes with 196 different salts (FreeBSD MD5 [32/32])
guesses: 0  time: 0:00:00:01  c/s: 4578  trying: mannie
guesses: 0  time: 0:00:00:02  c/s: 4702  trying: sharks
guesses: 0  time: 0:00:00:03  c/s: 4732  trying: sames1
[...]

As you can see, the MD5 sum didn't change.

Here's an extract from the log file:

[...]
0:02:01:11 - Rule #51: '-c <*>2!?Acp' accepted as '<*>2!?Acp'
0:02:02:57 - Rule #52: '<*>2!?AlP' accepted
0:02:04:44 - Rule #53: '<*>2!?AlI' accepted
0:02:06:43 - Rule #54: '-c <*>2!?AcP' accepted as '<*>2!?AcP'
0:02:08:33 - Rule #55: '-c <*>2!?AcI' accepted as '<*>2!?AcI'
0:02:10:19 - Rule #56: '-s x**' rejected
0:02:10:19 - Rule #57: '-s-c x**MlQ' rejected
0:02:10:19 Proceeding with "incremental" mode: All
0:02:10:19 ! Charset file has changed: $JOHN/all.chr
0:02:10:19 Terminating on error
0:02:08:09 Continuing an interrupted session
0:02:08:09 Loaded a total of 223 password hashes with 223 different salts
0:02:08:09 Remaining 196 password hashes with 196 different salts
0:02:08:09 - Hash type: FreeBSD MD5 (lengths up to 15)
0:02:08:09 - Algorithm: 32/32
0:02:08:09 - Candidate passwords may be buffered and tried in chunks of 8
0:02:08:09 Proceeding with wordlist mode
0:02:08:09 - Wordlist file: ./password.lst
0:02:08:09 - 57 preprocessed word mangling rules
0:02:08:09 - Rule #54: '-c <*>2!?AcP' accepted as '<*>2!?AcP'
0:02:08:34 - Rule #55: '-c <*>2!?AcI' accepted as '<*>2!?AcI'
0:02:10:21 - Rule #56: '-s x**' rejected
0:02:10:21 - Rule #57: '-s-c x**MlQ' rejected
0:02:10:21 Proceeding with "incremental" mode: All
0:02:10:21 ! Charset file has changed: $JOHN/all.chr
0:02:10:21 Terminating on error
0:02:08:10 Continuing an interrupted session
0:02:08:10 Loaded a total of 223 password hashes with 223 different salts
0:02:08:10 Remaining 196 password hashes with 196 different salts
0:02:08:10 - Hash type: FreeBSD MD5 (lengths up to 15)
0:02:08:10 - Algorithm: 32/32
0:02:08:10 - Candidate passwords may be buffered and tried in chunks of 8
0:02:08:10 Proceeding with wordlist mode
0:02:08:10 - Wordlist file: ./password.lst
0:02:08:10 - 57 preprocessed word mangling rules
0:02:08:10 - Rule #54: '-c <*>2!?AcP' accepted as '<*>2!?AcP'
0:02:08:12 Session aborted
0:02:08:13 Continuing an interrupted session
0:02:08:13 Loaded a total of 223 password hashes with 223 different salts
0:02:08:13 Remaining 196 password hashes with 196 different salts
0:02:08:13 - Hash type: FreeBSD MD5 (lengths up to 15)
0:02:08:13 - Algorithm: 32/32
0:02:08:13 - Candidate passwords may be buffered and tried in chunks of 8
[...]

Looks like the wrong diagnosis happened immediately after john
finished wordlist rules and switched to incremental mode.

When I started john --incremental on the same password file,
the error did not occur.


Every software involved is used as delivered from SuSE Linux prof. 9.2
(including online updates, of course):

Linux kernel version:
2.6.8-24.19-default #1 Tue Nov 29 14:32:45 UTC 2005 i686 athlon i386 
GNU/Linux

$ cat /proc/cpuinfo
processor       : 0
vendor_id       : AuthenticAMD
cpu family      : 6
model           : 6
model name      : AMD Athlon(TM) XP 2100+
stepping        : 2
cpu MHz         : 1725.238
cache size      : 256 KB
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 1
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 sep mtrr pge mca cmov 
pat pse36 mmx fxsr sse pni syscall mmxext 3dnowext 3dnow
bogomips        : 3416.06

$  gcc --version
gcc (GCC) 3.3.4 (pre 3.3.5 20040809)

John has been built for linux-x86-mmx.
No patches have been applied.
The only change to john's Makefile was adding -march=athlon-xp to the 
CFLAGS.

What could have gone wrong.
What additional tests could help?
Do you need more information?
(I could provide the sample password files via mail.)

Regards, Frank


Powered by blists - more mailing lists

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.