how to remove junk chracters from word list files

[benjsh]

Hi
Anyone know how I can remove junk from word list files.
It seems I got some files where there are some wierd characters

so aircrack exitet with fread: Illegal seek
and I see on the screen wierd stuff like

%2RB½ó etc..
I already can a pw-inspector -m 8 -M 63 on it to remove all short things
Anyone know how I can remove those junk characters ?

[benjsh]

I think it is binary data

[KMDave]

Some regex magic with Perl will do.

[kidFromBigD]

Some regex magic with Perl will do.

Yes; and BT4pf has some tools that can help.

For example:

Remove pesky carriage return characters from Windows .txt files:

tr -d '\015' < infile.txt > outfile.txt

This filters infile.txt removing all carriage returns (which is octal 015 on the ASCII chart) simply deleting them, writing to outfile.txt

You can play other games with the tr command, for example removing the space character from your wordlists(beware: a space character could appear in a WPA passphrase, so you may wish not to use this):

tr -d '\040' < infile.txt > outfile.txt

Where octal 040 is the space-bar ASCII character.

...and so on and so forth...

Simply use the man tr if you get stuck or care to investigate this powerful tool further.

Let us know how it goes for you.

[Gitsnik]

Surely it would be easier to use a regexp and perl -pe to remove any non-printable characters from each line, rather than hacking through tr with such complexity.

Seems like a far more efficient use of ones time.

[kidFromBigD]

Surely it would be easier to use a regexp and perl -pe to remove any non-printable characters from each line, rather than hacking through tr with such complexity.

Seems like a far more efficient use of ones time.

Agreed. IMHO, tr is good to practice with. Advance to other methods of cleaning up your passlists as your skill improves. With just a bit of practice you should be able to handle any .txt file and whip it into shape quickly and automatically.