[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Tue, 28 Feb 2012 09:10:17 -0600
From: "jfoug" <jfoug@....net>
To: <john-dev@...ts.openwall.com>
Subject: RE: Re: [JtR PATCH] Support rar's -p mode by spawning external unrar process.
>From: Dhiru Kholia
>
>On Tue, Feb 28, 2012 at 2:37 AM, magnum wrote:
>>
>>We would still have this -p mode problem but with more power under the
>>hood someone might get the idea to fix that. If I got it right, this is
>> only about quickly verifying or rejecting the decrypted data, just like
>> Jim did with pkzip. I bet we can find some situations where we can
>> reject early, if we just get the full CRC in place first.
>
>In RAR format, there is no CRC stored for the compressed (decrypted)
>data. So there is no quick way to verify / reject it. The CRC stored
>is for the original uncompressed file.
The 'quick' rejection on pkzip was by design of the algo. There was a 1
byte checksum (or 2 byte checksum) that was located in 1 of 2 places. Thus,
if there were multiple files in the zip file, all with the same PW, just
that checksum can be used to eliminate almost all invalid passwords. Even
if there was only 1 file in the zip blob, this checksum would eliminate at
least 255 out of 256 candidates.
However, in zip, there was not 'quick' way either, to fully check. The crc
was for the original unencrypted, then uncompressed data.
There were some quick rejection techniques that worked 'here' also, and I
think these would work fine for rar (NOTE, I have not looked into the rar
format, so am not 100% sure).
One method was to decrypt small amount of data, then use the pkzip
algorithm, which has 'some' data validation within it (i.e. there ARE byte
streams which are not valid). The check data is mostly 'invalid' cases.
Thus, if you decrypt some, run inflate on what you have, and it 'works',
then you decrypt some more, test again, and if that works, simply continue
forward, until either an error in the data is detected (meaning the PW is
correct), or you inflate the whole blob, and the crc matches (which is a
perfect match). To do this, I used the example code of zlib, but rewrote
it. I want to do nothing more than spin through the data, 'looking' for
where zlib would emit an error. I was not storing bytes, or doing much
else. Thus my code is much thinner than the zlib, and runs quite a bit
faster, and does a hell of a lot less. But it does have a very high
probability of detecting that a password is invalid, after only checking a
small amount of decrypted data.
The problem with this method (same problem you get with rar), is you have to
have the original file available, or at least 'enough' to get a very high
probablity. You cannot pack much on a JtR input line. A tiny file 'can' be
put on an input line, all in one blob. But a larger one, cannot. I made
the line have the ability to 'link' to the original file. That way, JtR can
open the original file, and get the entire blob, when it needs to do a full
test.
I do not know the algo for rar. I would assume that the encryption is post
compression. Is that the case? I would certainly assume so, since if
encryption is done prior to compression, then there IS no compression.
One thing I also looked at, in 'speedup' within the pkzip, is to look for
certain file-magic values. That has a lot of potential. If you have 2
compress Excel files, or group of gif or jpeg files, then you easily only
have to decrypt/decompress a tiny bit of each file, to know that this
password is very likely.
I am just tossing out some of the things I worked on in the 'old' pkzip
crypt format I worked on.
Now for uncompressing rar, would the 7z lib be able to be used? Would zlib
(I think not).
As a faster klunky method (until something better is done), one could get 7z
library hooked in, vs spawning out to the unrar process. This should be
faster (possibly quite a bit faster), since there would be no process
startup/tear down. It will depend upon how efficiently the unrar and the 7z
lib can detect the wrong password. If they detect it pretty quickly, then
the startup/teardown of the process may be a very large portion of the
runtime in the spawning model.
I am glad others are digging in, and having fun. I simply do not have time
available right now, to invest in this code. The job, and dealing with
family health issues have my time pretty much used up.
Jim.
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
