[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Mon, 05 Mar 2012 17:06:25 -0500
From: Richard Farina <sidhayn@...il.com>
To: john-dev@...ts.openwall.com
Subject: Re: CFLAGS="-march=native" sometimes unsupported
The issue stems from having CFLAGS which support my architecture but
building the generic target. This is fixed by "make linux-x86-64-avx".
Sorry for decreasing the signal to noise ratio on the list.
Thanks,
Rick
On 03/05/12 14:58, Richard Farina wrote:
> I have confirmed the offending CFLAG, added by -march=native and not by
> -mtune=native as well as causing the build issue.
>
> -mavx
>
> The relevant section of man gcc:
>
> -mno-abm
> These switches enable or disable the use of instructions in
> the MMX, SSE, SSE2, SSE3, SSSE3, SSE4.1, AVX, AES, PCLMUL, SSE4A, FMA4,
> XOP, LWP, ABM or 3DNow! extended
> instruction sets. These extensions are also available as
> built-in functions: see X86 Built-in Functions, for details of the
> functions enabled and disabled by these
> switches.
>
> To have SSE/SSE2 instructions generated automatically from
> floating-point code (as opposed to 387 instructions), see -mfpmath=sse.
>
> GCC depresses SSEx instructions when -mavx is used. Instead,
> it generates new AVX instructions or AVX equivalence for all SSEx
> instructions when needed.
>
> These options will enable GCC to use these extended
> instructions in generated code, even without -mfpmath=sse. Applications
> which perform runtime CPU detection must
> compile separate files for each supported architecture, using
> the appropriate flags. In particular, the file containing the CPU
> detection code should be compiled
> without these options.
>
>
> And my personal build error if it helps (with mpi support):
>
> mpicc john-mpi.o DES_fmt.o DES_std.o DES_bs.o DES_bs_b.o BSDI_fmt.o
> MD5_fmt.o MD5_std.o BF_fmt.o BF_std.o AFS_fmt.o LM_fmt.o trip_fmt.o
> timer.o md5_go.o md5_eq.o md5.o rc4.o hmacmd5.o base64.o md4.o
> dynamic_fmt.o dynamic_parser.o dynamic_preloads.o dynamic_utils.o
> rawSHA224_fmt.o rawSHA256_fmt.o rawSHA384_fmt.o rawSHA512_fmt.o
> XSHA512_fmt.o hmailserver_fmt.o SybaseASE_fmt.o SKEY_fmt.o ssh_fmt.o
> ssh2john.o pdf_fmt.o pdf2john.o pdfcrack_common.o pdfcrack_md5.o
> pdfparser.o pdfcrack.o pdfcrack_rc4.o rar_fmt.o rar2john.o zip_fmt.o
> zip2john.o gladman_hmac.o gladman_pwd2key.o phpassMD5_fmt_plug.o
> rawMD5go_fmt_plug.o hmacMD5_fmt_plug.o rawSHA1_fmt_plug.o
> oracle_fmt_plug.o mskrb5_fmt_plug.o sapB_fmt_plug.o DOMINOSEC_fmt_plug.o
> sha1_gen_fmt_plug.o NETNTLM_fmt_plug.o NT_fmt_plug.o PHPS_fmt_plug.o
> XSHA_fmt_plug.o rawMD5_fmt_plug.o nt2_fmt_plug.o rawSHA_fmt_plug.o
> MSCHAPv2_fmt_plug.o mssql-old_fmt_plug.o mscash1_fmt_plug.o
> KRB4_fmt_plug.o IPB2_fmt_plug.o salted_sha1_fmt_plug.o rawMD4_fmt_plug.o
> NS_fmt_plug.o pkzip_fmt_plug.o KRB4_std_plug.o oracle11_fmt_plug.o
> NETLMv2_fmt_plug.o mssql05_fmt_plug.o pixMD5_fmt_plug.o EPI_fmt_plug.o
> MYSQL_fast_fmt_plug.o KRB5_fmt_plug.o mysqlSHA1_fmt_plug.o
> NETLM_fmt_plug.o NETNTLMv2_fmt_plug.o KRB5_std_plug.o PO_fmt_plug.o
> mscash2_fmt_plug.o NETSPLITLM_fmt_plug.o HDAA_fmt_plug.o
> lotus5_fmt_plug.o md4_gen_fmt_plug.o MYSQL_fmt_plug.o nsldap_fmt_plug.o
> rawmd5u_fmt_plug.o DMD5_fmt_plug.o sapG_fmt_plug.o crc32_fmt_plug.o
> mediawiki_fmt_plug.o BFEgg_fmt_plug.o plugin.o dummy.o batch.o bench.o
> charset.o common.o compiler.o config.o cracker.o crc32.o external.o
> formats.o getopt.o idle.o inc.o john.o list.o loader.o logger.o math.o
> memory.o misc.o options.o params.o path.o recovery.o rpp.o rules.o
> signals.o single.o status.o tty.o wordlist.o mkv.o mkvlib.o unicode.o
> unshadow.o unafs.o undrop.o unique.o c3_fmt.o x86-64.o sse-intrinsics.o
> -Wl,-O1 -Wl,--as-needed -Wl,--hash-style=gnu -lcrypto -lssl -lm -lz
> -fopenmp -lcrypt -ldl -lcrypto -lssl -lm -lz -fopenmp -o ../run/john
> john.o: In function `CPU_detect_or_fallback':
> john.c:(.text+0xd65): undefined reference to `CPU_detect'
> collect2: ld returned 1 exit status
> make[1]: *** [../run/john] Error 1
>
>
> my build error (without mpi support):
> x86_64-pc-linux-gnu-gcc DES_fmt.o DES_std.o DES_bs.o DES_bs_b.o
> BSDI_fmt.o MD5_fmt.o MD5_std.o BF_fmt.o BF_std.o AFS_fmt.o LM_fmt.o
> trip_fmt.o timer.o md5_go.o md5_eq.o md5.o rc4.o hmacmd5.o base64.o
> md4.o dynamic_fmt.o dynamic_parser.o dynamic_preloads.o dynamic_utils.o
> rawSHA224_fmt.o rawSHA256_fmt.o rawSHA384_fmt.o rawSHA512_fmt.o
> XSHA512_fmt.o hmailserver_fmt.o SybaseASE_fmt.o SKEY_fmt.o ssh_fmt.o
> ssh2john.o pdf_fmt.o pdf2john.o pdfcrack_common.o pdfcrack_md5.o
> pdfparser.o pdfcrack.o pdfcrack_rc4.o rar_fmt.o rar2john.o zip_fmt.o
> zip2john.o gladman_hmac.o gladman_pwd2key.o phpassMD5_fmt_plug.o
> rawMD5go_fmt_plug.o hmacMD5_fmt_plug.o rawSHA1_fmt_plug.o
> oracle_fmt_plug.o mskrb5_fmt_plug.o sapB_fmt_plug.o DOMINOSEC_fmt_plug.o
> sha1_gen_fmt_plug.o NETNTLM_fmt_plug.o NT_fmt_plug.o PHPS_fmt_plug.o
> XSHA_fmt_plug.o rawMD5_fmt_plug.o nt2_fmt_plug.o rawSHA_fmt_plug.o
> MSCHAPv2_fmt_plug.o mssql-old_fmt_plug.o mscash1_fmt_plug.o
> KRB4_fmt_plug.o IPB2_fmt_plug.o salted_sha1_fmt_plug.o rawMD4_fmt_plug.o
> NS_fmt_plug.o pkzip_fmt_plug.o KRB4_std_plug.o oracle11_fmt_plug.o
> NETLMv2_fmt_plug.o mssql05_fmt_plug.o pixMD5_fmt_plug.o EPI_fmt_plug.o
> MYSQL_fast_fmt_plug.o KRB5_fmt_plug.o mysqlSHA1_fmt_plug.o
> NETLM_fmt_plug.o NETNTLMv2_fmt_plug.o KRB5_std_plug.o PO_fmt_plug.o
> mscash2_fmt_plug.o NETSPLITLM_fmt_plug.o HDAA_fmt_plug.o
> lotus5_fmt_plug.o md4_gen_fmt_plug.o MYSQL_fmt_plug.o nsldap_fmt_plug.o
> rawmd5u_fmt_plug.o DMD5_fmt_plug.o sapG_fmt_plug.o crc32_fmt_plug.o
> mediawiki_fmt_plug.o BFEgg_fmt_plug.o plugin.o dummy.o batch.o bench.o
> charset.o common.o compiler.o config.o cracker.o crc32.o external.o
> formats.o getopt.o idle.o inc.o john.o list.o loader.o logger.o math.o
> memory.o misc.o options.o params.o path.o recovery.o rpp.o rules.o
> signals.o single.o status.o tty.o wordlist.o mkv.o mkvlib.o unicode.o
> unshadow.o unafs.o undrop.o unique.o c3_fmt.o x86-64.o sse-intrinsics.o
> -Wl,-O1 -Wl,--as-needed -Wl,--hash-style=gnu -lcrypto -lssl -lm -lz
> -fopenmp -lcrypt -ldl -lcrypto -lssl -lm -lz -fopenmp -o ../run/john
> john.o: In function `CPU_detect_or_fallback':
> john.c:(.text+0xd65): undefined reference to `CPU_detect'
> collect2: ld returned 1 exit status
> make[1]: *** [../run/john] Error 1
> make[1]: Leaving directory
> `/var/tmp/portage/app-crypt/johntheripper-1.7.9/work/john-1.7.9/src'
> make: *** [linux-x86-64] Error 2
> make: Leaving directory
> `/var/tmp/portage/app-crypt/johntheripper-1.7.9/work/john-1.7.9/src'
>
>
> I lack the ability to go any farther without help on this. Anyone care
> to step in and make suggestions for how to build this to support the
> latest generations of processors with avx support?
>
> Thanks,
> Rick
>
> On 03/05/12 09:25, Richard Farina wrote:
>> There are currently two open bugs in the Gentoo bug tracker for john 1.7.9:
>>
>> https://bugs.gentoo.org/show_bug.cgi?id=405941
>> https://bugs.gentoo.org/show_bug.cgi?id=405941
>>
>> Both of these bugs seem to revolve around CFLAGS="-march=native" not
>> being supported properly for certain CPU.
>>
>> Since I am about to recreate the build failures reliably, and have
>> access to testers who can succeed, I figured I'd report the issue and
>> see if anyone can help.
>>
>> This is how -march=native expands on my system (and build fails):
>>
>> /usr/libexec/gcc/x86_64-pc-linux-gnu/4.5.3/cc1 -quiet -
>> -D_FORTIFY_SOURCE=2 -march=core2 -mcx16 -msahf -maes -mpclmul -mpopcnt
>> -mavx --param l1-cache-size=32 --param l1-cache-line-size=64 --param
>> l2-cache-size=3072 -mtune=generic -quiet -dumpbase - -auxbase-strip
>> /dev/null -o /tmp/ccJ8icEQ.s
>>
>> Since running -mtune=native fixes the issue, this is what that expands
>> to on my system (and build succeeds):
>>
>> /usr/libexec/gcc/x86_64-pc-linux-gnu/4.5.3/cc1 -quiet -
>> -D_FORTIFY_SOURCE=2 --param l1-cache-size=32 --param
>> l1-cache-line-size=64 --param l2-cache-size=3072 -mtune=generic -quiet
>> -dumpbase - -march=x86-64 -auxbase-strip /dev/null -o /tmp/cc220dAi.s
>>
>> I'm happy to test any patches or VCS versions as appropriate. I have
>> access to testers who currently can, as well as cannot, build with
>> -march=native if it helps.
>>
>> Thanks,
>> Rick Farina
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
