Date: Tue, 7 Sep 2010 10:07:38 +0400 From: Solar Designer <solar@...nwall.com> To: john-users@...ts.openwall.com Subject: rants (was: NTLM and OpenCL 1.0) rembrandt, OK, let me be rude as well, for the benefit of the project. ;-) Seriously, though, I am not offended nor inconvenienced by your posting at all. I just have difficulty making use of it as constructive criticism, for the reasons I outline below. Maybe my reply will result in more-easily-usable constructive criticism and specific suggestions from you and/or others. On Tue, Sep 07, 2010 at 05:58:29AM +0200, rembrandt wrote: > On Tue, 7 Sep 2010 02:27:23 +0400 > Solar Designer <solar@...nwall.com> wrote: <moderator hat on><nitpick> Why, you did not actually quote anything I wrote. ;-) You also misused an existing thread for your rants, which were better posted into their own (new) thread. </nitpick></moderator hat off> > DO YOU WILL INCLUDE THE PATCHES into the "official" distribution... y/n Which ones? All? Most? Then no. On the other hand, you might have noticed (or didn't you?) that the jumbo patch became a lot more official lately - it is even PGP-signed. Also, you might have noticed that I included a lot more stuff into the jumbo patch, including some that was bound to break things, and it did (recall the matching salt detection issue affecting at least the Oracle and PIX hashes). So I was somewhat more open to and aggressive in merging the patches. Not enough? Maybe. I am more willing to accept criticism for not spending enough time on JtR since, let's say, version 1.6 released in Dec 1998. The project could definitely evolve much faster if I were spending more time on it. At the same time, with the relatively little time that I did put into the project, I don't think I should have spent that time substantially differently - such as on merging all patches into the official codebase, without having the time and means to make that code as clean and reliable as I'd like the official JtR to be. By having the jumbo patch (and other patches), I am giving everyone (and myself) the choice: use feature-restricted but reliable and portable code, or use feature-rich but less reliable and less portable code. Yes, I am actually making use of this choice myself. I sometimes use the official JtR, and sometimes JtR with the jumbo patch or with other patches. Even during the contest, I was using 3 different builds of JtR for this reason. I understand that it would have been better to have one perfect version/build instead, but I just did not have time to make one yet. ;-) ...and if I did, we would have even more/newer/better patches that would still exist as patches for a while. Just when I thought I had merged almost everything into jumbo, Simon came up with more of his SSE intrinsics patches, which are very nice and desirable, yet difficult or unreasonable to merge even into jumbo at this time. So another patch on top of jumbo to maintain? Or a few days of my time to achieve the same functionality and performance in a way suitable for the official JtR? I just don't have that time to put into that now, and the priorities for JtR should probably be different anyway (e.g., the contest highlighted mostly other issues). > You wont include anything new into the base-codebase...??? Not true. I am including stuff that I had time to (re-)write. Without putting a lot of time into the project, this happens to be relatively little code - but there was in fact new and changed code getting into the official JtR. Recall the generic crypt(3) support and (limited) OpenMP support for the extra-slow hashes in 1.7.6. Recall the wordlist rules engine enhancements in 1.7.4 and 1.7.5 - many of which were requested by the community, including some by JimF. I could let JimF implement some of them and they would likely end up in jumbo. Instead, I dedicated some time to implementing them myself and they're official. Of course, I could/should have done a lot more. > All those patches wont help. Did anybody tested it under OpenBSD So what do you propose? Break portability of the official releases by integrating non-portable code just to hopefully force users to submit portability corrections? To some extent, this would work. But it could also result in JtR in fact becoming less portable and staying that way. We see both kinds of things happen with other projects. I prefer to do this sort of things with the jumbo patch. If you (and others) like, I am also open to the idea of starting an entire "community edition" or whatever we call it - with the difference being that it'd be a separate repository and tarballs ready to be built, not just patches. A drawback, though, is that such a tree would likely deviate from the official and reliable and portable JtR even further, making it harder to merge things into the official tree (if this is ever to be done - not necessarily by me and for official release, but maybe for other uses). Simply providing jumbo-patched tarballs avoids this drawback indeed, but does it make much of a difference compared to what we have now? > (you remmeber our SSE-register talk years ago?! > And still I was right *smiles* :) ). You were right in exactly one thing: 16 SSE registers with AMD64. I have no idea if you knew or if you guessed that, and in fact I don't recall if you even stated it explicitly. You also made a lot of other claims, which made me treat your guesses/claims as not trustworthy. (I am being sincere, not rude. Really.) I admit there was a time (early 2006?) when I thought that AMD64 expanded only the regular register file from 8 to 16 registers (I had read about that, I've seen the code, etc.), not the SSE one (I did not see this mentioned anywhere, and I did not read a full architecture manual). In fact, they did both. When I seriously got to look into this (before May 2006), of course I found this out and implemented the proper bitslice DES code (in May 2006). > So seriously: OpenCL -> GOOD! COOL! *for Windows, Mac, Linux! Not any > BSD so far! It is the issue of the BSD! Not building a front.... > they do all care for themself and can't code a shit... if you analyze > them partly. heyho Theo.. ;)) but if it will get handled like all those > "patches" it will have no effect Alex... INCLUDE them into the base! > You need to include them somehow... *my oppinion* :) I find your arguments self-contradictory. You both want the cutting-edge stuff integrated, which will definitely not work on OpenBSD right away (speaking of your reference to the brand new OpenCL patch, which does not even improve performance yet), but you also want it to work there. Where would this get us? I am more willing to buy Minga's arguments in favor of us working on the rulesets. There are clear shortcomings of the rulesets currently included with JtR, there are specific improvements to be made, this will actually result in more passwords getting cracked in realistic uses of JtR, and all of this is do-able in a realistic amount of time. So this should be a priority. Not what you describe. > Apply the algos *jumbo, as a hint! not all but liekly most coomon.. OK, "most common" - this is becoming realistic. I do in fact intend to clean up and merge Alain's NTLM code. Just a matter of my time. > remmeber stories of you.... wont say more, back then u tpold me about > it... come on bro... I am not sure what you're referring to here. It'd be better if you do "say more" - just not rants and almost-irrelevant references, but specific reminders, suggestions, requests. > are the codes to badly?! >THEN COMMENT about it... And then what?.. I am already maybe-rightly criticized for "ridiculing" people (although this was never my intent!) when they post naively-implemented wordlist rules. If that's how people treat my recommendations on how to write their rules better, then how would they treat my "complaints" about their contributed code?.. That said, I did also point out specific issues with the code on some occasions. In many cases, the authors of the code are well-aware of its limitations (such as of portability issues), but that does not change much. For example, I am aware of some issues with my own -omp-des-4 and -omp-des-7 patches, where neither is strictly better than the other (they have pros and cons) and neither is getting merged into the official JtR just yet. There are all sorts of issues "like that" and others with contributed patches as well. Just pointing them out will rarely magically get them fixed, although, once again, I did point problems out on many occasions - and I also made a lot of fixes to code in the jumbo patch myself. And sometimes pointing problems out does help - e.g., Alain's work on making his assembly file patches hopefully work with Solaris' assembler is a result of my report of the problem to him. (Thanks, Alain!) > but providing a patch is fucked! most people test their stuff just on > linux.. they're blind....).. and the codes... and make a little > test-lap.. and if you need something i will provide it, or somebody > else... I do test the jumbo patch code on non-Linux systems on some occasions - e.g., there was a round of fixes for problems discovered on NetBSD/sparc64, which I have installed on a normally-powered-off machine at home. I don't do it for every update of the patch, though. That would distract me from other work on JtR too much, given how little time I have for it. Also, "test" is one thing, "fix all issues" is another. I was aware of the NTLM assembly code vs. Solaris assembler issue for 2-3 years, without getting around to fixing it. (Sorry!) Apparently, some recent Solaris systems are not affected, though - somehow Robert managed to avoid this issue during his Solaris builds. Also, no one actually reported this issue (I saw it myself), and no one requested it getting fixed during all this time (so few people want the jumbo patch working on older versions of Solaris?) > Maybe today I got totaly drunken... who knows... but hell Alex... I > think I am right. What is your oppion or the oppinion of any other > "core" developer worth if no single patch gets included into the > base...?! I understand that this might frustrate code contributors, and there will likely be an exception for the NTLM code soon. I care about having things in the official JtR done right to a greater extent than I do about not frustrating a contributor - that is, I would not merge a patch just to avoid frustrating someone. With some contributions, I ended up reimplementing things in somewhat different ways - e.g., recall Simon's patch for faster processing of successful guesses and JimF's patch for larger hash tables. I did not merely apply those patches, but I implemented the equivalent functionality (in this case, the performance enhancements) in ways similar in spirit but different in code. Simon's and JimF's code contributions played an important role of proving the need for such changes and letting them get tested before I reimplemented them for inclusion. I see no reason for Simon and JimF to be frustrated that their exact code did not get in, but some replacement code did. Rather, they should be happy for having contributed to this end result, which I am grateful to them for. > And to all developers: test your shit also at BSDs (and if possible > MAC!)! Most of you seam to be Linux *loosers* *yes, provocating, I know This is contradictory to your other requests/suggestions - being open to contributions, merging them all, trying not to frustrate contributors. Then you ask the contributors not to post their stuff until they've tested it on lots of systems, some of which they wouldn't want to touch? > Deeply appreaciating your work and the work fo ANYBODY who sends and > IMPROVES JOHN(!); Thank you. Alexander
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.