Date: Tue, 22 Apr 2014 10:43:59 -0500 From: "jfoug" <jfoug@....net> To: <john-dev@...ts.openwall.com> Subject: New functionality added to bleeding (and a few changes we as devs need to make) There is a new memory debugging module that has recently been promoted into jumbo-bleeding branch of JtR. Any new .c files (with very few exceptions) should include this file: #include "memdbg.h" as the LAST include. With this simple change, all normal memory allocation and free calls 'can' be tracked, and upon program exit, any non-freed items will be listed. Also during runtime, there are checks for things like memory buffer overrun or underrun. By DEFAULT (as the source comes from the git tree), all memory debugging within memdbg will be turned off. To turn this on (there IS a performance hit, and for some things, that hit can be a bit), simply edit the memdbg_defines.h file. That file should be commented well enough to show which lines to uncomment to make things work. I would recommend only uncommenting MEMDBG_ON for most work. The MEMDBG_EXTRA_CHECKS does a lot more, BUT it does not free memory (until program exit). There are formats which allocate during their runtime. Those formats will run a system out of memory very quickly, and should not be run using these extra checks. There are also a few other macros to use. These are mostly used in 'main' program files (i.e. a main that starts, parses command line, and exits, like all of the *2john tools). In cases like this, there is another macro to add to the end of the main() function. That is MEMDBG_PROGRAM_EXIT_CHECKS(stderr) (or any file handle). This will be a NOOP, if the MEMDBG_ON is not set. But if it is set, then it will be a function that checks the state of memory, to make sure things were cleaned up properly. There are also more functions in there, which can be used. There are things like MEMDBG_getSnapshot and MEMDBG_checkSnapshot allow you to set a check around a function call, to see if something in it leaks. NOTE, this will only work with NATIVE compiled code. You will not detect any leaks from any included library calls, since the library was not built using memdbg. So code like this mem_handle = MEMDBG_getSnapshot(0); call_Some_function(); // Do other code. MEMDBG_getSnapshot(mem_handle); There are also some functions which do NOT use memdbg logic no matter what. These are: libc_free() libc_malloc() and libc_calloc(). These will all through and not use the memdbg functions at all. There are times when this MUST be done. For instance, if a library call allocates memory which is defined by the library as needing cleaned up by calling free to clean up that memory. This is usually BAD practice for lib writers. They should provide a free function to clean up their memory. This allows a library to be used, where the memory allocator is NOT the default malloc/free (which we get with memdbg). Not all libs are created that way, so if you call str=do_something(input); and then str later needs to be freed using free(), then in jtr, we should use the libc_free() call to free that memory. I am sure there will be a question here and there about the memory debugging code. It IS very helpful code. It has found quite a few leaks over the past couple years. However, this has always been in a parallel branch by itself, until magnum recently put this into the main bleeding trunk. Again, the easiest way to go, is to simply ignore that it is there, BUT to add a #include "memdbg.h" as the last include in your new .c files (and a call to MEMDBG_PROGRAM_EXIT_CHECKS(stderr) to the end of a main() function). That is all which is needed. I originally wrote this back in the 90's. It has caught 1000's of memory leaks, buffer over and under writes, etc. There were other 'tools' that came out later which do much the same thing (bound checker, valgrind, etc). The nice thing about this code, is you simply edit 1 line in a header, rebuild, and get all (most) of the memory checking, without having to have some external tool to do this checking. My original code was for C++. I rewrote it to C, and put it in pub domain. Enjoy. NOTE, memdbg_defines.h is in the .gitignore file. Any modifications you make to your local copy will not be propagated back to the git repository. Jim. Content of type "text/html" skipped
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.