Openwall GNU/*/Linux - a small security-enhanced Linux distro for servers
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Thu, 10 May 2012 19:16:43 +0200
From: Frank Dittrich <frank_dittrich@...mail.com>
To: john-dev@...ts.openwall.com
Subject: Re: bash auto-completion for john

On 05/09/2012 07:18 PM, Solar Designer wrote:
> I see that john.bash_completion is in magnum-jumbo now (great!), but it
> is lacking a copyright and license statement.

The attached file now contains the following:
# bash completion for john and unique commands (John the Ripper)
#
# This software is Copyright  2012 Frank Dittrich
# and hereby released to the general public under the following terms:
# Redistribution and use in source and binary forms, with or without
# modification, are permitted.
#
# Minor improvements suggested by Aleksey Cherepanov have been
# incorporated.

I hope this is OK. If not, please suggest what needs to be changed.

I also added some comments how to make bash completion for john and
unique work (the new make target already takes care of this):

# This file needs to be copied to /etc/bash_completion.d/john
# alternatively, a link /etc/bash_completion.d/john will work as well.
#
# To make the new completion rules work, you need to logout and login,
# or source /etc/bash_completion instead.


I also added some information how to use the same completion logic for
other binaries (if you created different binaries with various
compilation options or for different make targets):

# To use the same completion rules not just for john, but for
# differently named binaries (say john-omp, john-sse2i, john-avx,
# john-cuda, john-gpu, ...),
# just use this command to get the current expansion rule settings:
#       complete -p |grep " john"
#
# If the output is
#       complete -F _john john
# you can use this command to activate the same completion rules
# for john-omp:
#       complete -F _john john-omp
#
# To use these completion rules permanently, you might add
#       complete -F _john john-omp
# to your ~/.bashrc file.

Other changes:

1.
For --make-charset=, completion will now list directories and .chr
files, not longer all files and directories.
(I still think I should further restrict this to just directories, to
make accidentally overwriting all.chr less likely.)

2.
New expansion logic for --salts=, --users=, --groups=, --shells=

$ ./john --salts=
COUNT   -COUNT

$ ./john --users=
-LOGIN,...  LOGIN,...   -UID,...    UID,...

$ ./john --groups=
-GID,...  GID,...

$ ./john --shells=
-SHELL,...  SHELL,...

(Pressing the [tab] key will just list these options, the command line
will not change, because I disabled expansion for --salts=-*|--salts=C*
and so on.)

3.
New expansion logic for --test, if the john version supports an optional
parameter.
(Pressing the [tab] key will just list these options, the command line
will not change, because I disabled expansion for --test=*)

$ ./john --test
--test          --test=SECONDS

(I think showing --test=SECONDS instead of --test=TIME better explains
the meaning of the optional value.)


3.
Added LIST to the list of completions for --encoding=

$ ./john --encoding=
LIST         cp1252       cp737        cp866        iso-8859-7
ansi         cp1253       cp850        iso-8859-1   koi8-r
cp1251       cp437        cp858        iso-8859-15  utf-8

Bash automatically sorts the possible options. To let LIST appear at the
beginning, I needed to temporarily set LC_ALL=C during the expansion of
--encoding=. Otherwise LIST would appear between koi8-r and utf-8.

If there are already other parameters on the command line before
--encoding=, I'll not include LIST as a completion option:

$ ./john --format=bf --encoding=
ansi         cp1253       cp850        iso-8859-1   koi8-r
cp1251       cp437        cp858        iso-8859-15  utf-8
cp1252       cp737        cp866        iso-8859-7

Even though ./john --format=bf --encoding=LIST would "work":

$ ./john --format=bf --encoding=LIST
Supported encodings within john are:
utf-8, iso-8859-1 (or ansi), iso-8859-7, iso-8859-15, koi8-r,
cp437, cp737, cp850, cp858, cp866, cp1251, cp1252, cp1253


BTW: I noticed that completion will not work (the _john function doesn't
get called) if I use
$ LC_ALL=C ./john

In this case, just the default completion logic (listing file and
directory names) works.
But this seems to be a general issue, e.g., it  also applies for
$ LC_ALL=C git


Frank

# bash completion for john and unique commands (John the Ripper)
#
# This software is Copyright © 2012 Frank Dittrich
# and hereby released to the general public under the following terms:
# Redistribution and use in source and binary forms, with or without 
# modification, are permitted.
#
# Minor improvements suggested by Aleksey Cherepanov have been
# incorporated.
#
#
# This file needs to be copied to /etc/bash_completion.d/john
# alternatively, a link /etc/bash_completion.d/john will work as well.
#
# To make the new completion rules work, you need to logout and login,
# or source /etc/bash_completion instead.
#

# To use the same completion rules not just for john, but for 
# differently named binaries (say john-omp, john-sse2i, john-avx,
# john-cuda, john-gpu, ...),
# just use this command to get the current expansion rule settings:
#       complete -p |grep " john"
#
# If the output is
#       complete -F _john john
# you can use this command to activate the same completion rules
# for john-omp:
#       complete -F _john john-omp
#
# To use these completion rules permanently, you might add
#       complete -F _john john-omp
# to your ~/.bashrc file.
#
#
# The code is still ugly, many things can probably be done more efficiently.
# Currently, grep and sed are the only external commands used.
#
#
# Trying to build a perfect completion script will be hard.
# If possible, I'd like to avoid the need to maintain this script whenever
# john gets a new option.
#
# john can either be globally installed (e.g. in /usr/bin/john),
# or it can be installed locally somewhere in a user's home directory.
# It can be an official version, or a community encanced (jumbo) version,
# with a variety of patches.
#
# FIXME: is using __expand_tilde_by_ref OK?
#
# TODO:
#       --wordlist=~user/filename or --wordlist=~/dir/file doesn't work,
#         but pressing [tab] expands this to something useful
#         Where to fix this? In john? Some bash config option?
#
#       --rules= or --single= should use names of existing [List.Rules:...] sections
#         Currently a hard coded list of section names (NT single wordlist) is used
#         Proper fix probably needs support by a new john option (--list=rules?).
#       --incremental should use names of [Incremental:...] sections
#         Currently a hard coded list is used (All Alpha Digits Alnum LanMan)
#         (New john option needed for proper fix)
#       --external should use names of [List.External:..-.] sections
#         (Just sections without a generate() function, if --wordlist, --incremental or --single
#         is present on the command line; with a generate() function, if none of these
#         options is used - WHAT IF the user adds a --wordlist option later?)
#         Currently a hard coded List is used:
#	  Filter_Alpha Filter_Digits Filter_Alnum Filter_LanMan LanMan Double Parallel Strip Keyboard
#	  DumbForce KnownForce DateTime Repeats Subsets AtLeast1-Simple AtLeast1-Generic Policy AppendLuhn
#	  (New john option needed for proper fix)
#       --rules=, --single=, --incremental=, --external= need to take into account a --config file
#         which might have been specified on the command line, correct default john.conf locations 
#         (/etc/, ~/.john/, or ./, depending on build options), and included config files...
#         FIXME: The user might have used -co: instead of --config..., the user might specify 
#         --config=... later (should we really scan config files for sections referenced by other
#         command line options, to skip config files which do not have the right sections?)
#         Currently -opt is expanded to --option or --option=...
#         FIXME: To be able to locate the correct .conf and .rec files, we might even need a new
#         john option to know the values of JOHN_SYSTEMWIDE ... (e.g. --list=build-options)
#       --restore= and --status= expansion should search for existing .rec files in the correct directory,
#         instead of $PWD.
#       --mkpc and other options which are not mentioned in the usage output are currently ignored
#
#       Should I support -option instead of --option? (currently -option (or -opt) gets replaced 
#	with --option during expansion.
#	Should I support --option:val, -option:val, or even -opt:val instead of just --option=val?
#	Should expanding an abbreviated option to its long form also be done by john itself (new option)?


# grep and sed are used to process john's usage info the list of .rec files...

# john
have grep && have sed && 
_john()
{
	local first cur options valopts compreplya compreplyb encodings formats subformats
	COMPREPLY=()
	_get_comp_words_by_ref -n = cur

#	we need to make sure we run the correct program, not some other program 
#	called john which is located somewhere in $PATH
	first="${COMP_WORDS[0]}"
#	Most options are listed at the begin of the line, but the line with the --pipe option
#	does have trailing spaces, and --stdin is mentioned after --wordlist=FILE.
#
#	all options (the '=' will be emoved for options with an optional value)
	options=""
# FIXME: How do I suppress the error message if someone tries to be clever: cd run; ./john --[tab] ???
	options="`${first}|sed -n '{ s#^ *\(--[a-z-]*=\?\(LIST\)\?\).*$#\1# }; /^--/ p'` --stdin"
	if [[ "_${options}_" == "__" ]] ; then
		compopt -F _filedir_xspec
		return 0
	fi

#	Just those options that can be used together with a value, even if that value is optional:
	valopts=`${first}|grep '^ *--[a-z\[-]*='|grep -v '^ *--subformat='|sed 's#^ *\([a-z=-]*\).*$#\1#'`
#	This is used to decide whether or not the completion should add a trailing space.
#	(That means, for a jumbo build, --rules doesn't get a trailing space, but for the john version
#	distributed by fedora16, --rules does get a trailing space during expansion.
#	The same applies for --show and single)

	case "${cur}" in
		--format=dynamic*)
			if echo "${options}" | grep "^--subformat" > /dev/null ; then
				 subformats=`${first} --subformat=LIST|sed 's#^\(User\)\?Format = \(dynamic_[0-9]*\).*$#\2#'`
				cur=${cur#*=}
				COMPREPLY=( $(compgen -W "${subformats}" -- ${cur}) )
			fi
			return 0
			;;
		--format=dy|--format=dyn|--format=dyna|--format=dynam|--format=dynami)
			if echo "${options}" | grep "^--subformat" > /dev/null ; then
				cur=${cur#*=}
				COMPREPLY=( $(compgen -W "dynamic" -- ${cur}) )
				compopt -o nospace
			fi
			return 0
			;;
		--format=*)
			cur=${cur#*=}
			formats=`${first} |sed -n '/^--format/,$ { s#^--format=[ A-Za-z]*:##; /^--/ q; s#^ *##; s#\<dynamic_n\>#dynamic#; s#[/ ]#\n#g; p }'`
			COMPREPLY=( $(compgen -W "${formats}" -- ${cur}) )
			return 0
			;;

		--restore|--status|--incremental)
			COMPREPLY=( $(compgen -W "${cur}=" -- ${cur}) )
			compopt -o nospace
			return 0
			;;
		--restore=*|--status=*)
			cur=${cur#*=}
			COMPREPLY=( $(compgen -W "$(for f in *.rec; do echo ${f%.rec};done)" -- ${cur}) )
			return 0
			;;
		--wordlist=*)
			#cur=${cur#*=}
			#_filedir expansion of --wordlist=~/te doesn't work
			# _filedir_xspec "_xspecs: bad array subscript" written to stderr
			_filedir_xspec 2> /dev/null
			return 0
			;;
 		--rules|--single)
			if echo "${valopts}" | grep "^${cur}$" > /dev/null ; then
				COMPREPLY=( $(compgen -W "${cur} ${cur}=NT ${cur}=single ${cur}=wordlist" -- ${cur}) )
			else
				COMPREPLY=( $(compgen -W "${cur}" -- ${cur}) )
			fi
			return 0
			;;
		--rules=*|--single=*)
			if echo "${valopts}" | grep "^${cur%=*}$" > /dev/null ; then
				cur=${cur#*=}
				COMPREPLY=( $(compgen -W "NT single wordlist" -- ${cur}) )
			fi
			return 0
			;;
		--external=*)
			cur=${cur#*=}
			COMPREPLY=( $(compgen -W "Filter_Alpha Filter_Digits Filter_Alnum Filter_LanMan LanMan Double Parallel Strip Keyboard DumbForce KnownForce DateTime Repeats Subsets AtLeast1-Simple AtLeast1-Generic Policy AppendLuhn" -- ${cur}) )
			return 0
			;;
		--incremental=*)
			cur=${cur#*=}
			COMPREPLY=( $(compgen -W "All Alpha Digits Alnum LanMan" -- ${cur}) )
			return 0
			;;
		--make-charset=*)
			cur=${cur#*=}
			#redirect stderr just in case __expand_tilde_by_ref
			#doesn't exist everywhere
			#(I'm a bit worried because of the __ at the begin.
			#May be this function isn't part of an "official" API.)
			__expand_tilde_by_ref cur 2>/dev/null
# FIXME:		should I just expand directories, not files, 
# FIXME:		to make overwriting existing files harder?
			_filedir "chr"
			return 0
			;;
		--markov=*)
			return 0
			;;
		--markov)
			if echo "${options}" | grep "^${cur}$" > /dev/null ; then
				COMPREPLY=( $(compgen -W "--markov --markov=LEVEL[:START[:END[:LENGTH]]]" -- ${cur}) )
			fi
			return 0
			;;
		--test)
			if echo "${valopts}" | grep "^${cur}$" > /dev/null ; then
				COMPREPLY=( $(compgen -W "--test --test=SECONDS" -- ${cur}) )
			else
				COMPREPLY=( $(compgen -W "${cur}" -- ${cur}) )
			fi
			return 0
			;;
		--show)
			if echo "${valopts}" | grep "^--show$" > /dev/null ; then
				COMPREPLY=( $(compgen -W "--show --show=LEFT" -- ${cur}) )
			else
				COMPREPLY=( $(compgen -W "--show" -- ${cur}) )
			fi
			return 0
			;;
		--show=l*)
			if echo "${valopts}" | grep "^--show$" > /dev/null ; then
				cur=${cur#*=}
				COMPREPLY=( $(compgen -W "left" -- ${cur}) )
			fi
			return 0
			;;
		--show=*)
			if echo "${valopts}" | grep "^--show$" > /dev/null ; then
				cur=${cur#*=}
				COMPREPLY=( $(compgen -W "LEFT" -- ${cur}) )
			fi
			return 0
			;;
		--users=L*|--users=U*|--users=-*|--groups=G*|--groups=-*|--shells=S*|--shells=-*|--salts=C*|--salts=-*)
			return 0
			;;
		--users=*)
			cur=${cur#*=}
			COMPREPLY=( $(compgen -W "LOGIN,... UID,... -LOGIN,... -UID,..." -- ${cur}) )
			return 0
			;;
		--groups=*)
			cur=${cur#*=}
			COMPREPLY=( $(compgen -W "GID,... -GID,..." -- ${cur}) )
			return 0
			;;
		--shells=*)
			cur=${cur#*=}
			COMPREPLY=( $(compgen -W "SHELL,... -SHELL,..." -- ${cur}) )
			return 0
			;;
		--salts=*)
			cur=${cur#*=}
			COMPREPLY=( $(compgen -W "COUNT -COUNT" -- ${cur}) )
			return 0
			;;
		--encoding=*)
			if  echo "${options}" | grep "^--encoding=" > /dev/null ; then
				encodings=`${first} --encoding=LIST 2>&1|grep -v 'Supported encodings'|sed 's#[,)]##g'|sed 's#(or ##g'`
			cur=${cur#*=}
				if [[ ${COMP_CWORD} -eq 2 || ${COMP_CWORD} -eq 3 && "_${cur}" != "_" ]] ; then
					encodings="${encodings} LIST"
					# make sure LIST will be the first option:
					LC_ALL=C
				fi
				COMPREPLY=( $(compgen -W "${encodings}" -- ${cur}) )
			fi
			return 0
			;;
		--pot=*)
			cur=${cur#*=}
			#redirect stderr just in case __expand_tilde_by_ref
			#doesn't exist everywhere
			#(I'm a bit worried because of the __ at the begin.
			#May be this function isn't part of an "official" API.)
			#                                                                         
			__expand_tilde_by_ref cur 2>/dev/null
			_filedir "pot"
			return 0
			;;
		--config=*)
			cur=${cur#*=}
			__expand_tilde_by_ref cur 2>/dev/null
			_filedir '@...nf|ini)'
			return 0
			;;
		--save-memory=*)
			cur=${cur#*=}
			COMPREPLY=( $(compgen -W "1 2 3" -- ${cur}) )
			return 0
			;;
		--regen-lost-salts=*)
			if echo "${options}" | grep "^--regen-lost-salts=" > /dev/null ; then
				cur=${cur#*=}
				COMPREPLY=( $(compgen -W "1 2 3 4 5" -- ${cur}) )
			fi
			return 0
			;;
		--subformat=l*)
			if echo "${options}" | grep "^--subformat=" > /dev/null ; then
				cur=${cur#*=}
				COMPREPLY=( $(compgen -W "list" -- ${cur}) )
			fi
			return 0
			;;
		--subformat=*)
			if echo "${options}" | grep "^--subformat=" > /dev/null ; then
				cur=${cur#*=}
				COMPREPLY=( $(compgen -W "LIST" -- ${cur}) )
			fi
			return 0
			;;
		--mem-file-size=*|--field-separator-char=*|--fix-state-delay=*|--max-run-time=*)
			return 0
			;;
		-*)
			compreplya=`compgen -W "${options}" -- ${cur}`
			if [[ "_${compreplya}_" == "__" ]] ; then
				cur="-${cur}"
				compreplya=`compgen -W "${options}" -- ${cur}`
			fi
			compreplyb=`compgen -W "${valopts}" -- ${cur}`
			COMPREPLY=( $(compgen -W "${options}" -- ${cur}) )
			if [[ "_${compreplya}" == "_${compreplyb}" ]] ; then
				compopt -o nospace
			fi
			return 0
			;;
		*)
			_filedir
			return 0
			;;
	esac
} &&
complete -F _john john

# unique
have head && have grep && have sed &&
_unique()
{
	local first filename cur usage options valopts compreplya compreplyb
        COMPREPLY=()
        _get_comp_words_by_ref -n = cur

# we need to make sure we run the correct program, not some other program 
# called unique which is located somewhere in $PATH
	first="${COMP_WORDS[0]}"
	filename=`echo  "${first}"|sed 's#^.*/\(.*\)$#\1#'`
	usage=`echo ""|${first}|grep '^Usage:'|sed 's#^Usage:\? \?[^ ]*unique *##'`
	case "_${cur}" in
		_|_${first})
			if [[ "_${usage}_" != "_OUTPUT-FILE_" ]] ; then
				COMPREPLY=( $(compgen -W "${usage}" -- "") )
			fi
			return 0
			;;
		_-cut=*|_-mem=*)
			return 0
			;;
		_-inp=*|_-ex_file=*|_-ex_file_only=*)
                        cur=${cur#*=}
			__expand_tilde_by_ref cur 2>/dev/null
			_filedir
			return 0
			;;
		_-*)
			if [[ "_${usage}_" != "_OUTPUT-FILE_" ]] ; then
				options=`echo ${usage}|sed 's# #\n#g'|grep '^\[.*\]$'|sed 's#^.\(.*\).$#\1#'|sed 's#=.*$#=#'`
				valopts=`echo "${options}"|grep '='`
				compreplya=`compgen -W "${options}" -- ${cur}`
				compreplyb=`compgen -W "${valopts}" -- ${cur}`
				if [[ "_${compreplya}" == "_${compreplyb}" ]] ; then
					COMPREPLY=( $(compgen -W "${valopts}" -- "${cur}") )
					compopt -o nospace
				else
					COMPREPLY=( $(compgen -W "${options}" -- "${cur}") )
				fi
			fi
			return 0
			;;
		_*)
			return 0
			;;
	esac
} &&
complete -F _unique unique

Powered by blists - more mailing lists

Your e-mail address:

Powered by Openwall GNU/*/Linux - Powered by OpenVZ