Removing patterns from Numeric wordlist

[zombie22]

Numeric, and otherwise, wordlists have patterns that are unlikely to be utilized by software which creates the key based on 'random' digits. Repetition is an obvious one, but all patterns must go.

0000000001
0101010101
89abcdef
febf2cff (utilizing the 'f' 4/8 makes this key highly improbable to be generated randomly)

Obviously, this can drastically reduce a wordlist's size; a necessity if you want to challenge larger keys.

It seems this has been covered elsewhere but I cannot find a reference: Does anyone know of an existing program that does such a thing, or am I delusional? I realize there are numerous rudimentary ways to do this, but its seems as if there is already a program specifically for this. If not, its off to the coding board!

[thad0ctor]

everything can be done with sed, to remove a pattern:

cat filename.txt > sed '/pattern/d' | cat outfile.txt should work

also you can remove words with X amount of consecutive characters using sed, I don't know the function off the top of my head but you can find it in Tape's Word List Manipulator

[Bl4cks4b3r]

Thad0ctor what is a good way to contact you for a project idea?

[TAPE]

Yeah, further to thad0ctor's message hereabove
(by the way, your email as mentioned in your contact info in your script is not working thad0ctor)
you can narrow down the outcome of wordlists created with
for instance crunch by using a specific pattern which makes
it more relevant to yourtarget.

For details on crunch have a look at ;
http://adaywithtape.blogspot.com/201...runch-v30.html
and for simple wordlist manipulation have a look at ;
http://adaywithtape.blogspot.com/201...revisited.html

[thad0ctor]

hey tape, I forgot there is a hyphen between tha and d0ctor but I lost the login info anyways

edited

[TAPE]

Got it thanks

[zombie22]

Please, do not leave your garbage in my thread. Why bother posting if you do not fully read the OP or are posting regarding airoscript redux or any other bash front-ends(shortcuts) you have 'created'. BTW, that example is nasty, I am not sure how one can make shell scripts without having any clue about in/out redirection. No, that example should never work.I do not intend to be so rude, but damn, these replies are depressing. The OP said: "I realize there are numerous rudimentary ways to do this, but its seems as if there is already a program specifically for this.." not 'zomg how I make w0rdlistz'. I can only hope the eager few did not scare away the knowledgeable. So please, if you are one of the few who have already posted here, kindly stay away from my thread.

The appropriate solution will do much more than some inline with regex. Not to mention, parsing multiple TB in such a way would be reckless, at best. The solution I am looking for will do more than just remove g 'a pattern' , it will remove 'ALL patterns'. The target is software that creates keys based on random digits.

Like I said, it seems as if this has been covered before and a solid program exists for just this purpose. Of course if there is not, I do not mind creating something.

[Gitsnik]

Alright then here is a script that will remove duplicated letters. It's not going to remove all patterns, I'm not giving you anything in the area's of artificial networks, but you can figure out the extras for yourself.

http://gitsnik.blogspot.com.au/2011/...nch-redux.html

That will remove any duplicate characters in a line (read the page). I am not going to support it, don't ask me any questions, it works, if you can't use it that is your problem.

Have a good read around the forums, your attitude will not go over well here.

[thad0ctor]

based on what you want you'll probably have to write a new bash script along the lines of:


- crunch with your settings
- pipe the output to sed to remove X consecutive characters
- run a for loop for each line of output counting how many instances of each character there are in a line
- pipe out the characters that did not fit the requirements
- cat the output list into a word list

I'll look into making a function like this, it sounds like a fun challenge and a cool feature for my tool kit but in the meantime you should reassess you attitude towards those who are trying to help you

[Gitsnik]

It'll take a fair whack of processing power to check everything but at least you can clean it up along the way thad0ctor - nested if/ loop checks would be most appropriate I should think:

crunch | while read $line {
    if ( moreThan2Same( $line ) ) {
        break; # One condition has been failed, so we can bail. The how many instances loop is going to be quicker than consec characters, so we put it first.
    } else {
        if ( consecCharacters( $line ) > 2 ) {
            break; # 3 or more consec characters is not random enough. This gives us pattern checks with trinomials - 123, 234, 345, 456, 567, 678, 789, 890, etc.
        } else {
            # additional if/else checks.
            print $line;
        }
    }
}

If you start with the most efficient checks (string length, counting characters, then consec characters asc, consec characters desc, then whatever), you'll reduce the processing power somewhat significantly.