Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Date: Sun, 08 May 2011 17:01:00 +0200
From: magnum <>
Subject: Help! mskrb5 64-32-bit builds crashing hard while native builds does

I have a weird problem. My mskrb5 format works fine on 64-bit and 
32-bit, but not when building a 64-32 target and running it on 64-bit.

It smashes the stack. When I trace it, everything is fine until I call 
RC4_set_key() the first time, after that stack is crap (I can't step 
inside RC4_set_key, maybe I should look for a debug version of the lib)

Is there anything wrong with this code? I can't see what, but maybe it's 
something really silly as usual. Disregard the OMP pragma, it happens 
with or without OMP. But why does it NOT happen on a 32-bit machine? Is 
that a clue?

static void crypt_all(int count)
   HMACMD5Context ctx;
   unsigned char K3[KEY_SIZE], cleartext[PLAINTEXT_OFFSET + BINARY_SIZE];
   RC4_KEY key;
   int i;

#ifdef _OPENMP
#pragma omp parallel for default(none) private(i, ctx, K3, cleartext, 
key) shared(count, output, saved_key, saltblob)
   for (i = 0; i < count; i++) {
     // K1 is stored in saved_key[index]
     // CHECKSUM and TIMESTAMP are just defines, they are actually
     // concatenated to saltblob[]

     // K3 = HMAC-MD5(K1, CHECKSUM)
     hmac_md5(saved_key[i], CHECKSUM, CHECKSUM_SIZE, K3, &ctx);

     // RC4(K3, TIMESTAMP) decrypt part of the timestamp
     RC4_set_key(&key, KEY_SIZE, K3);

     // 15th byte and on is our partial binary
     memcpy(output[i], &cleartext[PLAINTEXT_OFFSET], BINARY_SIZE);

KEY_SIZE is 16. I have tried defining K3, cleartext, saved_key and 
output as static and/or double the size they need and this does not 
change a thing.

I just upgraded my Ubuntu for newer gcc and (I guess) openssl and 
nothing changed.

Any hints welcome! This drives me crazy. All other formats works fine.


Powered by blists - more mailing lists

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.