Openwall GNU/*/Linux - a small security-enhanced Linux distro for servers
[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Date: Sun, 08 May 2011 17:01:00 +0200
From: magnum <rawsmooth@...dband.net>
To: john-dev@...ts.openwall.com
Subject: Help! mskrb5 64-32-bit builds crashing hard while native builds does
 not?

I have a weird problem. My mskrb5 format works fine on 64-bit and 
32-bit, but not when building a 64-32 target and running it on 64-bit.

It smashes the stack. When I trace it, everything is fine until I call 
RC4_set_key() the first time, after that stack is crap (I can't step 
inside RC4_set_key, maybe I should look for a debug version of the lib)

Is there anything wrong with this code? I can't see what, but maybe it's 
something really silly as usual. Disregard the OMP pragma, it happens 
with or without OMP. But why does it NOT happen on a 32-bit machine? Is 
that a clue?


static void crypt_all(int count)
{
   HMACMD5Context ctx;
   unsigned char K3[KEY_SIZE], cleartext[PLAINTEXT_OFFSET + BINARY_SIZE];
   RC4_KEY key;
   int i;

#ifdef _OPENMP
#pragma omp parallel for default(none) private(i, ctx, K3, cleartext, 
key) shared(count, output, saved_key, saltblob)
#endif
   for (i = 0; i < count; i++) {
     // K1 is stored in saved_key[index]
     // CHECKSUM and TIMESTAMP are just defines, they are actually
     // concatenated to saltblob[]

     // K3 = HMAC-MD5(K1, CHECKSUM)
     hmac_md5(saved_key[i], CHECKSUM, CHECKSUM_SIZE, K3, &ctx);

     // RC4(K3, TIMESTAMP) decrypt part of the timestamp
     RC4_set_key(&key, KEY_SIZE, K3);
     RC4(&key, PLAINTEXT_OFFSET + BINARY_SIZE, TIMESTAMP,
         cleartext);

     // 15th byte and on is our partial binary
     memcpy(output[i], &cleartext[PLAINTEXT_OFFSET], BINARY_SIZE);
   }
}


KEY_SIZE is 16. I have tried defining K3, cleartext, saved_key and 
output as static and/or double the size they need and this does not 
change a thing.

I just upgraded my Ubuntu for newer gcc and (I guess) openssl and 
nothing changed.

Any hints welcome! This drives me crazy. All other formats works fine.

magnum

Powered by blists - more mailing lists

Your e-mail address:

Powered by Openwall GNU/*/Linux - Powered by OpenVZ