Date: Mon, 19 Aug 2013 09:44:27 +0200 From: Frank Dittrich <frank_dittrich@...mail.com> To: john-dev@...ts.openwall.com Subject: Re: relbench/benchmark-unify On 08/19/2013 08:59 AM, Frank Dittrich wrote: > If I have a string like "mschapv2-naive, MSCHAPv2 C/R", should I always > drop the "format-label, " part in benchmark-unify, so that I still > detect multiple implementations of the same algorithm, picking the fastest? > > But then, this would mean that "descrypt, traditional crypt(3)" would > become "traditional crypt(3)". > And "md5crypt, crypt(3) $1$" would become "crypt(3) $1$". > I'm afraid there are some cases where important information gets lost: > LastPass, sniffed sessions > MongoDB, system / network > PST, custom CRC-32 > PuTTY, Private Key > RAdmin, v2.x > > The other approach would be to keep all those format labels, and just > drop those few labels where I have different implementations of the same > algorithms, allowing the fastest one to be picked for comparison. > But that would mean even more future benchmark-unify changes if > additional implementations for existing algorithms come up. Checking CUDA and OpenCL formats on bull, I noticed one more problem. Benchmarking: md5crypt [CUDA]... DONE Benchmarking: md5crypt [OpenCL]... DONE vs. Benchmarking: md5crypt-cuda, crypt(3) $1$ [MD5 CUDA]... DONE Benchmarking: md5crypt-opencl, crypt(3) $1$ [MD5 OpenCL]... DONE May be it doesn't make much sense to compare CPU and GPU formats anyway. On the other hand, measuring how much faster a GPU implementation is compared to a CPU implementation for a given format might be useful information. And, without dropping the format labels, I won't be able to easily compare CUDA and OPenCL implementations. This is getting ugly. Ideas how to best handle the situation are most welcome. My best idea so far is to add a new option to benchmark-unify: --drop-format-labels[=0|=1] Default would be to just drop the format labels which are known to be alternative implementations: *-ng, *-naive, nt2: --drop-format-labels=0 would be to keep format labels even for those alternative implementations. --drop-format-labels or --drop-format-labels=1 would mean to drop all format labels, even if this makes "RAdmin, v2.x" a useless format name May be we need to fix those few format names which get meaningless without the format label. Any thoughts? Frank
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.