aircrack-ng and wordlists over 2Gb

[imported_h4x0r]

hello,

i hope i'm here right with my question.
i made a really big wordlist. then i would use it with aircrack-ng. this say me, that the file is empty. i thinked ok...check the wordlists. i cat them together....each folder seperated in a file...after them i merged the files...after every merg i tested the new file with aircrack...i found a wordlist (2.5Gb) and tested it because it looks like programmingcode.....ok aircrack sayd it's empty and i thinked this file was that bogus one that crashd my first list....but my new one got over 2GB and then aircrack says it's empty....ok next try.....i reseted to the last stand before the error occurs....i got the first list-> 1,9Gb no probs with aircrack....i was lucky and the second one got 1.9gb too and work fine....but the files arnt sorted and not uniq...and not fitted for wpa....

is there a way to sort and uniq the diffrent files without merging it?
how u manage ur lists?

thanks

[Archangel-Amael]

is there a way to sort and uniq the diffrent files without merging it?
how u manage ur lists?

thanks

Those commands you mentioned will do just that.
Try
#man sort and then #man uniq

[Gitsnik]

Those commands you mentioned will do just that.
Try
#man sort and then #man uniq

Something that I have to say because I keep seeing it show up around here, there is a -u switch to most (if not all) sort programs. It doesn't seem like much of a difference, but you will save yourself a fairly hefty amount of processing time by not executing two programs when you can only execute one, especially when you're working with gigabytes of wordlists.

OP: You probably want to look into using "cat" as well, and pipes.

[jenbo]

i always do

cat wordlist-name sort | uniq | pw-inspector -m 8 -M 63 > wordlist-name

this will remove all dupes "uniq" and format it to length 8 - 63 chars and put it into the filename "wordlist-name" or what ever you want to call it. HOWEVER if you do the sort | uniq ... You must have enough RAM to load the wordlist fully into memory as thats ware it will get processed. If you only got 1 GB ram and you do the above with a 2 GB wordlist you will only get 50 % done ! outher than this small problem its a very good way of managing WPA wordlists.

[purehate]

i always do

cat wordlist-name sort | uniq | pw-inspector -m 8 -M 63 > wordlist-name

this will remove all dupes "uniq" and format it to length 8 - 63 chars and put it into the filename "wordlist-name" or what ever you want to call it. HOWEVER if you do the sort | uniq ... You must have enough RAM to load the wordlist fully into memory as thats ware it will get processed. If you only got 1 GB ram and you do the above with a 2 GB wordlist you will only get 50 % done ! outher than this small problem its a very good way of managing WPA wordlists.

You missed what he was saying. sort -u is the same thing as sort | uniq and requires less memory because only one program is running.

[jenbo]

yes, sorry i missunderstood what he ment.