Date: Mon, 05 May 2008 09:14:53 +0200 From: Simon Marechal <simon@...quise.net> To: john-users@...ts.openwall.com Subject: Re: working with large files Solar Designer a écrit : > Well, what JtR does right now is to completely rebuild the per-salt hash > table each time there's a guess for that salt (and a hash gets removed). > The obvious optimization here is to update the existing per-salt hash > table instead. If there's demand for a speedup in this area, even > though it is negligible for long-living sessions, then perhaps this > should be done. > > Alternatively, you could choose to not remove hashes for cracked > passwords at all. This means that there may be duplicate guesses if > duplicate candidate passwords are tried (but it's not much of a problem > since "john --show" will only report each guess just once). It also > means that the reported effective c/s rate will be artificially higher - > this will not correspond to any meaningful speedup, and overall things > may get slower for long-living sessions because keeping more hashes in > the tables results in a higher number of collisions. Hello, here is a patch to my tree with the changes. It should however be possible to patch vanilla john versions, except with the part about the rawMD5_fmt.c changes. Here is what it (supposedly) does: * increases the size of the largest hash table from 4096 to 65536 entries * increases accordingly the size of the hashes returned by the raw-md5 format * add a "found" marker to the db_password structure * when a password is found, the corresponding db_password structure is marked as such, and then the hash table is searched for this entry and this entry is removed * if the password was the first in the related db_salt "list", it is removed and then this list is parsed until a password that is not found is encountered. All consecutive found passwords that are parsed are then removed The two last items dramatically alter the performance when working with large files. The last item ensures that salts where all passwords have been found are cleaned. It seems to work for me (doesn't crash, seems to clean the salts properly, dramatically increase speed in the rare case you have 100000+ passwords per salt). As always with stuff I submit, it is a workaround to an immediate problem i encountered, so there are cleaner ways to do this (suggested by Solar, or changing the linked-list structures to something faster), so don't use it on something important. Feedback would however be tremendously useful. Simon Download attachment "largepasswordset.patch.gz" of type "application/x-gzip" (1607 bytes) -- To unsubscribe, e-mail john-users-unsubscribe@...ts.openwall.com and reply to the automated confirmation request that will be sent to you.
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.