Openwall GNU/*/Linux - a small security-enhanced Linux distro for servers
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Mon, 05 Mar 2012 14:58:30 -0500
From: Richard Farina <sidhayn@...il.com>
To: john-dev@...ts.openwall.com
Subject: Re: CFLAGS="-march=native" sometimes unsupported

I have confirmed the offending CFLAG, added by -march=native and not by
-mtune=native as well as causing the build issue.

-mavx

The relevant section of man gcc:

       -mno-abm
           These switches enable or disable the use of instructions in
the MMX, SSE, SSE2, SSE3, SSSE3, SSE4.1, AVX, AES, PCLMUL, SSE4A, FMA4,
XOP, LWP, ABM or 3DNow! extended
           instruction sets.  These extensions are also available as
built-in functions: see X86 Built-in Functions, for details of the
functions enabled and disabled by these
           switches.

           To have SSE/SSE2 instructions generated automatically from
floating-point code (as opposed to 387 instructions), see -mfpmath=sse.

           GCC depresses SSEx instructions when -mavx is used. Instead,
it generates new AVX instructions or AVX equivalence for all SSEx
instructions when needed.

           These options will enable GCC to use these extended
instructions in generated code, even without -mfpmath=sse.  Applications
which perform runtime CPU detection must
           compile separate files for each supported architecture, using
the appropriate flags.  In particular, the file containing the CPU
detection code should be compiled
           without these options.


And my personal build error if it helps (with mpi support):

mpicc john-mpi.o DES_fmt.o DES_std.o DES_bs.o DES_bs_b.o BSDI_fmt.o
MD5_fmt.o MD5_std.o BF_fmt.o BF_std.o AFS_fmt.o LM_fmt.o trip_fmt.o
timer.o md5_go.o md5_eq.o md5.o rc4.o hmacmd5.o base64.o md4.o
dynamic_fmt.o dynamic_parser.o dynamic_preloads.o dynamic_utils.o
rawSHA224_fmt.o rawSHA256_fmt.o rawSHA384_fmt.o rawSHA512_fmt.o
XSHA512_fmt.o hmailserver_fmt.o SybaseASE_fmt.o SKEY_fmt.o ssh_fmt.o
ssh2john.o pdf_fmt.o pdf2john.o pdfcrack_common.o pdfcrack_md5.o
pdfparser.o pdfcrack.o pdfcrack_rc4.o rar_fmt.o rar2john.o zip_fmt.o
zip2john.o gladman_hmac.o gladman_pwd2key.o phpassMD5_fmt_plug.o
rawMD5go_fmt_plug.o hmacMD5_fmt_plug.o rawSHA1_fmt_plug.o
oracle_fmt_plug.o mskrb5_fmt_plug.o sapB_fmt_plug.o DOMINOSEC_fmt_plug.o
sha1_gen_fmt_plug.o NETNTLM_fmt_plug.o NT_fmt_plug.o PHPS_fmt_plug.o
XSHA_fmt_plug.o rawMD5_fmt_plug.o nt2_fmt_plug.o rawSHA_fmt_plug.o
MSCHAPv2_fmt_plug.o mssql-old_fmt_plug.o mscash1_fmt_plug.o
KRB4_fmt_plug.o IPB2_fmt_plug.o salted_sha1_fmt_plug.o rawMD4_fmt_plug.o
NS_fmt_plug.o pkzip_fmt_plug.o KRB4_std_plug.o oracle11_fmt_plug.o
NETLMv2_fmt_plug.o mssql05_fmt_plug.o pixMD5_fmt_plug.o EPI_fmt_plug.o
MYSQL_fast_fmt_plug.o KRB5_fmt_plug.o mysqlSHA1_fmt_plug.o
NETLM_fmt_plug.o NETNTLMv2_fmt_plug.o KRB5_std_plug.o PO_fmt_plug.o
mscash2_fmt_plug.o NETSPLITLM_fmt_plug.o HDAA_fmt_plug.o
lotus5_fmt_plug.o md4_gen_fmt_plug.o MYSQL_fmt_plug.o nsldap_fmt_plug.o
rawmd5u_fmt_plug.o DMD5_fmt_plug.o sapG_fmt_plug.o crc32_fmt_plug.o
mediawiki_fmt_plug.o BFEgg_fmt_plug.o plugin.o dummy.o batch.o bench.o
charset.o common.o compiler.o config.o cracker.o crc32.o external.o
formats.o getopt.o idle.o inc.o john.o list.o loader.o logger.o math.o
memory.o misc.o options.o params.o path.o recovery.o rpp.o rules.o
signals.o single.o status.o tty.o wordlist.o mkv.o mkvlib.o unicode.o
unshadow.o unafs.o undrop.o unique.o c3_fmt.o x86-64.o sse-intrinsics.o
-Wl,-O1 -Wl,--as-needed -Wl,--hash-style=gnu -lcrypto -lssl -lm -lz
-fopenmp -lcrypt -ldl -lcrypto -lssl -lm -lz -fopenmp -o ../run/john
john.o: In function `CPU_detect_or_fallback':
john.c:(.text+0xd65): undefined reference to `CPU_detect'
collect2: ld returned 1 exit status
make[1]: *** [../run/john] Error 1


my build error (without mpi support):
x86_64-pc-linux-gnu-gcc DES_fmt.o DES_std.o DES_bs.o DES_bs_b.o
BSDI_fmt.o MD5_fmt.o MD5_std.o BF_fmt.o BF_std.o AFS_fmt.o LM_fmt.o
trip_fmt.o timer.o md5_go.o md5_eq.o md5.o rc4.o hmacmd5.o base64.o
md4.o dynamic_fmt.o dynamic_parser.o dynamic_preloads.o dynamic_utils.o
rawSHA224_fmt.o rawSHA256_fmt.o rawSHA384_fmt.o rawSHA512_fmt.o
XSHA512_fmt.o hmailserver_fmt.o SybaseASE_fmt.o SKEY_fmt.o ssh_fmt.o
ssh2john.o pdf_fmt.o pdf2john.o pdfcrack_common.o pdfcrack_md5.o
pdfparser.o pdfcrack.o pdfcrack_rc4.o rar_fmt.o rar2john.o zip_fmt.o
zip2john.o gladman_hmac.o gladman_pwd2key.o phpassMD5_fmt_plug.o
rawMD5go_fmt_plug.o hmacMD5_fmt_plug.o rawSHA1_fmt_plug.o
oracle_fmt_plug.o mskrb5_fmt_plug.o sapB_fmt_plug.o DOMINOSEC_fmt_plug.o
sha1_gen_fmt_plug.o NETNTLM_fmt_plug.o NT_fmt_plug.o PHPS_fmt_plug.o
XSHA_fmt_plug.o rawMD5_fmt_plug.o nt2_fmt_plug.o rawSHA_fmt_plug.o
MSCHAPv2_fmt_plug.o mssql-old_fmt_plug.o mscash1_fmt_plug.o
KRB4_fmt_plug.o IPB2_fmt_plug.o salted_sha1_fmt_plug.o rawMD4_fmt_plug.o
NS_fmt_plug.o pkzip_fmt_plug.o KRB4_std_plug.o oracle11_fmt_plug.o
NETLMv2_fmt_plug.o mssql05_fmt_plug.o pixMD5_fmt_plug.o EPI_fmt_plug.o
MYSQL_fast_fmt_plug.o KRB5_fmt_plug.o mysqlSHA1_fmt_plug.o
NETLM_fmt_plug.o NETNTLMv2_fmt_plug.o KRB5_std_plug.o PO_fmt_plug.o
mscash2_fmt_plug.o NETSPLITLM_fmt_plug.o HDAA_fmt_plug.o
lotus5_fmt_plug.o md4_gen_fmt_plug.o MYSQL_fmt_plug.o nsldap_fmt_plug.o
rawmd5u_fmt_plug.o DMD5_fmt_plug.o sapG_fmt_plug.o crc32_fmt_plug.o
mediawiki_fmt_plug.o BFEgg_fmt_plug.o plugin.o dummy.o batch.o bench.o
charset.o common.o compiler.o config.o cracker.o crc32.o external.o
formats.o getopt.o idle.o inc.o john.o list.o loader.o logger.o math.o
memory.o misc.o options.o params.o path.o recovery.o rpp.o rules.o
signals.o single.o status.o tty.o wordlist.o mkv.o mkvlib.o unicode.o
unshadow.o unafs.o undrop.o unique.o c3_fmt.o x86-64.o sse-intrinsics.o
-Wl,-O1 -Wl,--as-needed -Wl,--hash-style=gnu -lcrypto -lssl -lm -lz
-fopenmp -lcrypt -ldl -lcrypto -lssl -lm -lz -fopenmp -o ../run/john
john.o: In function `CPU_detect_or_fallback':
john.c:(.text+0xd65): undefined reference to `CPU_detect'
collect2: ld returned 1 exit status
make[1]: *** [../run/john] Error 1
make[1]: Leaving directory
`/var/tmp/portage/app-crypt/johntheripper-1.7.9/work/john-1.7.9/src'
make: *** [linux-x86-64] Error 2
make: Leaving directory
`/var/tmp/portage/app-crypt/johntheripper-1.7.9/work/john-1.7.9/src'


I lack the ability to go any farther without help on this.  Anyone care
to step in and make suggestions for how to build this to support the
latest generations of processors with avx support?

Thanks,
Rick

On 03/05/12 09:25, Richard Farina wrote:
> There are currently two open bugs in the Gentoo bug tracker for john 1.7.9:
> 
> https://bugs.gentoo.org/show_bug.cgi?id=405941
> https://bugs.gentoo.org/show_bug.cgi?id=405941
> 
> Both of these bugs seem to revolve around CFLAGS="-march=native" not
> being supported properly for certain CPU.
> 
> Since I am about to recreate the build failures reliably, and have
> access to testers who can succeed, I figured I'd report the issue and
> see if anyone can help.
> 
> This is how -march=native expands on my system (and build fails):
> 
> /usr/libexec/gcc/x86_64-pc-linux-gnu/4.5.3/cc1 -quiet -
> -D_FORTIFY_SOURCE=2 -march=core2 -mcx16 -msahf -maes -mpclmul -mpopcnt
> -mavx --param l1-cache-size=32 --param l1-cache-line-size=64 --param
> l2-cache-size=3072 -mtune=generic -quiet -dumpbase - -auxbase-strip
> /dev/null -o /tmp/ccJ8icEQ.s
> 
> Since running -mtune=native fixes the issue, this is what that expands
> to on my system (and build succeeds):
> 
> /usr/libexec/gcc/x86_64-pc-linux-gnu/4.5.3/cc1 -quiet -
> -D_FORTIFY_SOURCE=2 --param l1-cache-size=32 --param
> l1-cache-line-size=64 --param l2-cache-size=3072 -mtune=generic -quiet
> -dumpbase - -march=x86-64 -auxbase-strip /dev/null -o /tmp/cc220dAi.s
> 
> I'm happy to test any patches or VCS versions as appropriate.  I have
> access to testers who currently can, as well as cannot, build with
> -march=native if it helps.
> 
> Thanks,
> Rick Farina

Powered by blists - more mailing lists

Your e-mail address:

Powered by Openwall GNU/*/Linux - Powered by OpenVZ