[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Sat, 03 Mar 2012 03:22:40 +0100
From: magnum <john.magnum@...hmail.com>
To: john-dev@...ts.openwall.com
Subject: Re: rar format [was Re: [JtR PATCH] Support rar's -p mode
by spawning external unrar process.]
On 02/28/2012 04:10 PM, jfoug wrote:
> The problem with this method (same problem you get with rar), is you have to
> have the original file available, or at least 'enough' to get a very high
> probablity. You cannot pack much on a JtR input line. A tiny file 'can' be
> put on an input line, all in one blob. But a larger one, cannot. I made
> the line have the ability to 'link' to the original file. That way, JtR can
> open the original file, and get the entire blob, when it needs to do a full
> test.
I believe a lot of the work you did with zip2john (some of which ended
up not being used) might be perfect for rar2john. We may end up using a
lot of it.
> I do not know the algo for rar. I would assume that the encryption is post
> compression. Is that the case? I would certainly assume so, since if
> encryption is done prior to compression, then there IS no compression.
Sure. In completely unoptimised code we first derive the IVs from the
key (in CPU code this is the hard part, it's 256K SHA-1 updates in the
unstable version, 512K in Jumbo-5). Then we decrypt (AES), then inflate
(LZSS and/or PPMII) and finally we do a CRC32. In my mind a good
"simple" start would be to just put the SHA-1 stuff in OpenCL, then aim
for a streaming decrypt with combined inflate + CRC but that depends on
what code we end up using.
> One thing I also looked at, in 'speedup' within the pkzip, is to look for
> certain file-magic values. That has a lot of potential. If you have 2
> compress Excel files, or group of gif or jpeg files, then you easily only
> have to decrypt/decompress a tiny bit of each file, to know that this
> password is very likely.
I too think this might be the key to success, use file magics when
possible. Unless we end up finding good enough early reject algo's of
course. And maybe a combo of both can be used.
> Now for uncompressing rar, would the 7z lib be able to be used? Would zlib
> (I think not).
Not zlib afaik. There seem to be at least two different algorithms
involved: LZSS and PPMII. The first bit of each block tells if it's LZ
or PPM.
I did not even know there is a 7z lib but I suspect it's not low level
enough. Probably a tad better than spawning unrar but still <100 cps,
just my guess.
I suppose solid archives (I believe that's a non-default option) would
pose a problem if using "custom" inflate. Maybe these are less common.
As long as we can pick the first file in an archive it might not even
matter.
magnum
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
