Date: Mon, 28 Nov 2011 05:07:29 +0400 From: Solar Designer <solar@...nwall.com> To: john-dev@...ts.openwall.com Subject: .bss size in -jumbo magnum, Jim - .bss in 1.7.8-jumbo-8 grew to 140 MB in a 64-bit build with OpenMP: $ size ../run/john text data bss dec hex filename 873185 104696 140226432 141204313 86a9b59 ../run/john The main culprits are: $ size *.o | sort -rnk 3 | head 8318 1664 55837824 55847806 3542b7e NETNTLMv2_fmt_plug.o 6431 1088 27918480 27925999 1aa1def NETLMv2_fmt_plug.o 6230 1088 27525334 27532652 1a41d6c mskrb5_fmt_plug.o 8438 1472 10420400 10430310 9f2766 MSCHAPv2_fmt_plug.o 7500 1664 10420379 10429543 9f2467 NETNTLM_fmt_plug.o 6038 1184 2883721 2890943 2c1cbf NETLM_fmt_plug.o 4267 992 1573005 1578264 181518 NETSPLITLM_fmt_plug.o 14169 1472 713100 728741 b1ea5 mscash1_fmt_plug.o 3239 800 589872 593911 90ff7 crc32_fmt_plug.o 4332 704 270656 275692 434ec XSHA512_fmt.o I think at least top 5 of these (or maybe 7 or 9) should be changed to allocate their memory dynamically, like 1.7.9 does in MD5_fmt.c and BSDI_fmt.c. (1.7.9's DES_fmt.c and c3_fmt.c also use dynamic memory allocation, but in format-specific ways. BF_fmt.c's saved_key is relatively small (the number of buffered keys is low because this hash is so slow), and BF_std.c puts its larger data structures on thread stacks (just one instance per thread).) Since there are no format destructors - I mean, we have init(), but not the opposite - the entire 140 MB will end up getting allocated by a full --test run anyway, but at least that address space won't be allocated during normal cracking runs. Also, systems that can't afford 140 MB will be able to run the tests individually (not all tests in one invocation). So this will be an improvement. Of course, on typical systems this 140 MB is just address space and not even virtual memory unless/until someone runs the full --test. So this cleanup won't matter too much. Yet it'd be nice to make it if we can do that without performance impact and without much effort (e.g., limiting this to top 5 only). Also, this may allow for scalability to much higher thread counts. Format destructors may be added as a next step. Alexander
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.