Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Date: Tue, 07 May 2013 23:26:50 +0400
From: Alexander Cherepanov <cherepan@...me.ru>
To: john-dev@...ts.openwall.com
Subject: Core: undefined behavior in DES_std.c and MD5_std.c

Hi, Solar!

I've tried to compile core john with clang -fsanitize=undefined and have 
got several errors (on x86_64).

in bsdicrypt:
DES_std.c:407:18: runtime error: left shift of 1 by 63 places cannot be 
represented in type 'long'

In descrypt format:
DES_std.c:1084:25: runtime error: left shift of 1 by 31 places cannot be 
represented in type 'int'
DES_std.c:1085:23: runtime error: left shift of 1 by 31 places cannot be 
represented in type 'int'
DES_std.c:1123:26: runtime error: left shift of 1 by 31 places cannot be 
represented in type 'int'

In LM format:
DES_std.c:1100:25: runtime error: left shift of 1 by 31 places cannot be 
represented in type 'int'
DES_std.c:1101:23: runtime error: left shift of 1 by 31 places cannot be 
represented in type 'int'

The problem is that trying to compute 1<<31 or (long)1<<63 leads to an 
undefined behavior (per C11, 6.5.7p4). A tentative patch (for these and 
some more) is attached but it seems a thorough check of all the file is 
due because it freely mixes signed and unsigned types. Is it really 
necessary to have ARCH_WORD signed?

BTW even "int i = 0xFFFFFFFF;" is not well-defined per C11, 6.3.1.3p3. 
(It reads: "Otherwise, the new type is signed and the value cannot be 
represented in it; either the result is implementation-defined or an 
implementation-defined signal is raised.")

Other errors are in md5crypt format:
MD5_std.c:738:2: runtime error: store to misaligned address 
0x000000f3f96a for type 'MD5_word' (aka 'unsigned int'), which requires 
4 byte alignment
MD5_std.c:740:2: runtime error: store to misaligned address 
0x000000f3f756 for type 'MD5_word' (aka 'unsigned int'), which requires 
4 byte alignment
MD5_std.c:741:2: runtime error: store to misaligned address 
0x000000f3f75a for type 'MD5_word' (aka 'unsigned int'), which requires 
4 byte alignment
MD5_std.c:742:2: runtime error: store to misaligned address 
0x000000f3f75e for type 'MD5_word' (aka 'unsigned int'), which requires 
4 byte alignment
MD5_std.c:743:2: runtime error: store to misaligned address 
0x000000f3f762 for type 'MD5_word' (aka 'unsigned int'), which requires 
4 byte alignment
MD5_std.c:745:2: runtime error: store to misaligned address 
0x000000f3f95e for type 'MD5_word' (aka 'unsigned int'), which requires 
4 byte alignment
MD5_std.c:746:2: runtime error: store to misaligned address 
0x000000f3f962 for type 'MD5_word' (aka 'unsigned int'), which requires 
4 byte alignment
MD5_std.c:747:2: runtime error: store to misaligned address 
0x000000f3f966 for type 'MD5_word' (aka 'unsigned int'), which requires 
4 byte alignment

This is probably intended and I haven't researched futher.

-- 
Alexander Cherepanov

View attachment "0001-Fix-undefined-behavior-of-signed-shifts-in-DES_std.c.patch" of type "text/x-patch" (2070 bytes)

Powered by blists - more mailing lists

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.