Date: Mon, 7 Nov 2011 06:24:22 +0400 From: Solar Designer <solar@...nwall.com> To: john-users@...ts.openwall.com Subject: Re: comparing two benchmarks On Mon, Nov 07, 2011 at 05:50:24AM +0400, Solar Designer wrote: > $ ./relbench.pl asis Os > Geometric mean of 158: 0.914688 real, 0.915206 virtual > Standard deviation: 0.966662 real, 0.962432 virtual > > Thus, it appears that going from -O2 to -Os made things slower by 8.5% > overall. However, the standard deviation is large, meaning that things > vary between individual benchmarks a lot. While it is true that the standard deviation is relatively large on this test, it is not as large as shown above - there was a bug in the script. To correct it, right before these lines: > $dr **= 0.5; > $dv **= 0.5; add: $dr /= $n; $dv /= $n; Then we get: $ ./relbench.pl asis Os Geometric mean of 158: 0.914688 real, 0.915206 virtual Standard deviation: 0.076904 real, 0.076567 virtual For comparison, the standard deviation for equivalent benchmarks is much lower: $ ./relbench.pl asis asis-2 Geometric mean of 158: 1.000566 real, 1.002138 virtual Standard deviation: 0.013746 real, 0.013566 virtual $ ./relbench.pl Os Os-2 Geometric mean of 158: 1.002091 real, 1.001839 virtual Standard deviation: 0.008966 real, 0.008853 virtual Alexander
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.