Openwall GNU/*/Linux - a small security-enhanced Linux distro for servers
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Sun, 16 Sep 2018 21:28:42 +0200
From: Albert Veli <albert.veli@...il.com>
To: john-users@...ts.openwall.com
Subject: Re: good program for sorting large wordlists

Hi!

On Tue, Sep 11, 2018 at 5:19 PM JohnyKrekan <krekan@...nykrekan.com> wrote:

> Hello, I would like to ask whether someone has experience with good tool
> to sort large text files with possibilities such as gnu sort. I am using it
> to sort wordlists but when I tried to sort 11 gb wordlist, it crashed while
> writing final output file after writing around 7 gb of data  and did not
> delete some temp files.


If you don't succeed with other methods, one thing that has worked for me
is splitting the wordlist into smaller parts and sorting each one
individually. Then you can merge the sorted lists together using for
instance mli2 from hashcat-utils. That will put the big list in sorted
order. But the parts must be sorted first, before merging.

This is only necessary for very large wordlists. Like in your case.

PS I think there is a tool in hashcat-utils for splitting too. Don't
remember the name. Maybe gate.

Powered by blists - more mailing lists

Your e-mail address:

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.