Page 2 of 2 FirstFirst 12
Results 11 to 19 of 19

Thread: Removing patterns from Numeric wordlist

  1. #11
    Member
    Join Date
    Dec 2011
    Posts
    86

    Default Re: Removing patterns from Numeric wordlist

    very good point gitsnik, my order was a bit out of whack and there really isn't the need to pipe it so many times. I'm interested to see how large the final product would be and how much all these computations would slow down your crunching.

    I think I figured out what I'm doing this afternoon

  2. #12
    Very good friend of the forum Gitsnik's Avatar
    Join Date
    Jan 2010
    Location
    The Crystal Wind
    Posts
    851

    Default Re: Removing patterns from Numeric wordlist

    Haha no probs, hit TAPE up if you run in to any issues, he can contact me if he needs the extra and looks like you two could come up with the answers properly anyway. Look forward to seeing it.

    Wait, did I just add more crap to a thread
    Still not underestimating the power...

    There is no such thing as bad information - There is truth in the data, so you sift it all, even the crap stuff.

  3. #13
    Member
    Join Date
    Dec 2011
    Posts
    86

    Default Re: Removing patterns from Numeric wordlist

    haha, whoops I just added more crap too. wow I suck

  4. #14
    My life is this forum thorin's Avatar
    Join Date
    Jan 2010
    Posts
    2,629

    Default Re: Removing patterns from Numeric wordlist

    Quote Originally Posted by SomeNewbThatFeelsEntitled
    So please, if you are one of the few who have already posted here, kindly stay away from my thread.
    This thread is NOT yours.

    It belongs to an open community of volunteers that owe you absolutely nothing. If you wish this thread to be yours please feel free to spin up your own domain and server, start your own forums with no users, and post your question.

    Numeric, and otherwise, wordlists have patterns that are unlikely to be utilized by software which creates the key based on 'random' digits. Repetition is an obvious one, but all patterns must go.

    0000000001
    0101010101
    89abcdef
    febf2cff (utilizing the 'f' 4/8 makes this key highly improbable to be generated randomly)
    You seem to be operating under a false premiss. Repetition does not denote any failure of randomness. In fact removing seeming patterns and repetition simply removes valid outcomes. If a value is truly random then it's just as likely to contain repetition (00000) vs all unique values (12345). The fact that your 4th example contains 4fs out of 8 does not make it any more or less likely to be generated at random.

    The same applies for sequences and other patterns. If you randomly generate values you'll eventually see words (in any and all languages even) that doesn't render the value non-random. It simply means that someone interprets them as having meaning or a pattern.

    Consider the case for dual factor authentication and use of key fobs (i.e.: RSA tokens) if the developer(s) used some kind of pattern recognition on the generated values and dropped all duplicates or sequential values they'd simply be reducing the number of values an attacker has to attempt in order to hit a correct "randomly generated" one-time-password.
    Last edited by thorin; 05-07-2012 at 08:21 AM.
    I'm a compulsive post editor, you might wanna wait until my post has been online for 5-10 mins before quoting it as it will likely change.

    I know I seem harsh in some of my replies. SORRY! But if you're doing something illegal or posting something that seems to be obvious BS I'm going to call you on it.

  5. #15
    Member
    Join Date
    Dec 2011
    Posts
    86

    Default Re: Removing patterns from Numeric wordlist

    I just finished making a customizable script that accomplishes all this and allows you to customize:

    mid and max string length
    output filename / directory
    the max number of consecutive characters allowed
    the max number of one character allowed
    the charset that will be crunched


    I'll try to post it here tomorrow or tonight at the earliest. I was planning on adding it to my word list toolkit but I don't think i'm quite ready to release v1.1 just yet.

  6. #16
    Senior Member ShadowMaster's Avatar
    Join Date
    Jul 2011
    Location
    /root
    Posts
    189

    Default Re: Removing patterns from Numeric wordlist

    On the risk of adding more crap to this thread, I'd just like to say that while I look forward to adding thad0ctor's tool to my arsenal, what is this adding that a well-written JtR rule would not have? The semblance of taking away multiple chars? ...or what?
    World Domination is such an ugly phrase. I prefer the term World Optimization.

  7. #17
    Member
    Join Date
    Dec 2011
    Posts
    86

    Default Re: Removing patterns from Numeric wordlist

    I'm pretty sure you could do the same with JTR rules but I'm just not familiar with JTR at all. Instead my script uses most bash and crunch to get the job done. In addition, you can customize everything from:

    the min and max password length
    the charset
    the maximum number of sequenctial characters
    and the max number of one character per line created

    everything works great but it really slows down crunch. granted I'm using it on a 2 core laptop currently. Once I post it I'd really like to get some input to kind of speed up the process. I'll try to get script uploaded within the next 3 hours or so.

  8. #18
    Member
    Join Date
    Dec 2011
    Posts
    86

    Default Re: Removing patterns from Numeric wordlist

    here is what I came up with. please let me know if it doesn't work I crudely cut and pasted the code from what will appear in my script:

    #!/bin/sh
    function f_random () {
    echo
    echo "What is the minimum password string length you would like to use?"
    echo
    read minlength
    echo
    echo "What is the maximum password string length you would like to use?"
    echo
    read maxlength
    echo
    echo "Enter the character set you would like to use."
    echo
    read charset
    echo
    echo "Enter the length of consecutive characters you would like to exclude."
    echo "(e.x. 3)"
    echo
    read consec
    echo
    echo "What is the maximum number of one character you would like to use?"
    echo "(e.x. 3)"
    echo
    read maxchar
    echo
    echo "What would you like the output file to be named?"
    echo "(the file extension .lst will be appended to your selection)"
    echo
    read output
    echo
    echo "Where would you like $output.lst to be placed?"
    echo "(e.x. /root/Desktop)"
    echo
    read destination
    while [ ! -d "$destination" ]
    do
    echo
    echo "Directory cannot be found or does not exist"
    echo
    sleep 1
    echo "Would you like to create a folder for the directory you selected? (y/n)"
    read newdir
    if [ $newdir = "y" ]; then
    mkdir "$destination"
    while [ ! -d "$destination" ]
    do
    echo "Folder: $destination still cannot be found, starting over..."
    sleep 2
    f_random
    done
    else
    sleep 1
    echo "Where would you like the output word list to be placed?"
    echo "(e.x. /root/Desktop/)"
    read destination
    fi
    done
    echo
    echo "Would you like to go ahead and create $output.lst in $destination? (y/n)"
    echo
    read create
    echo
    if [ "$create" = "y" ]; then
    echo
    let consec=$(($consec - 1))
    /pentest/passwords/crunch/crunch $minlength $maxlength $charset | while read line
    do
    filter=$(echo $line | sed "/\([^A-Za-z0-9_]\|[A-Za-z0-9]\)\1\{$consec,\}/d" | sed "/^$/d")
    if [ "$filter" != "" ]; then
    stopp="no"
    number=1
    filter3=$(echo $filter | fold -w1 | sort | uniq -c)
    count=$(echo $line | wc -c)
    let count=$(($count - 1))
    until [[ "$number" -gt "$count" || "$stopp" == "yes" ]]
    do
    filter4=$(echo $filter3 | awk '{print $NF ":" $'$number'}' | sed -n 's/[^:]*://p')
    let number=$(($number + 2))
    if [ "$filter4" -gt "$maxchar" ]; then
    stopp="yes"
    else
    stopp="no"
    fi
    done
    if [ "$stopp" == "no" ] && [ "$number" -gt "$count" ] ; then
    echo "Writing: $filter"
    echo $filter | cat >> "$destination"/"$output".lst
    else
    echo
    echo "Skipping string: $filter"
    fi
    else
    echo
    fi
    done
    if [ -e "$destination"/"$output".lst ]; then
    echo
    echo "$output.lst exists in $destination"
    sed -i -e '1d' "$destination"/"$output".lst
    sed -i -e '1d' "$destination"/"$output".lst
    sed -i -e '1d' "$destination"/"$output".lst
    sed -i -e '1d' "$destination"/"$output".lst
    echo
    sleep 2
    echo "Successfully created $output.lst"
    echo
    sleep 2
    echo "Returning to the main menu..."
    echo
    sleep 2
    f_menu
    else
    echo
    echo "$output.lst does not exist in $destination"
    echo
    sleep 2
    echo "Unsuccessfully created $output.lst"
    echo
    sleep 2
    echo "Starting over..."
    echo
    sleep 2
    f_random
    fi
    elif [ "$create" = "n" ]; then
    echo
    else
    echo
    fi
    }
    f_random

  9. #19
    Junior Member
    Join Date
    Jun 2012
    Posts
    42

    Default Re: Removing patterns from Numeric wordlist

    Quote Originally Posted by zombie22 View Post
    Numeric, and otherwise, wordlists have patterns that are unlikely to be utilized by software which creates the key based on 'random' digits. Repetition is an obvious one, but all patterns must go.

    0000000001
    0101010101
    89abcdef
    febf2cff (utilizing the 'f' 4/8 makes this key highly improbable to be generated randomly)

    Obviously, this can drastically reduce a wordlist's size; a necessity if you want to challenge larger keys.

    It seems this has been covered elsewhere but I cannot find a reference: Does anyone know of an existing program that does such a thing, or am I delusional? I realize there are numerous rudimentary ways to do this, but its seems as if there is already a program specifically for this. If not, its off to the coding board!
    Your first premise is incorrect. Randomness requires that all possibilities are equally likely to occur, even "simple" ones.

    Your second premise is also incorrect. Removing basic patterns from a comprehensive keyspace wordlist does not "drastically reduce a wordlist's size," depending somewhat on how you define "drastically." The patterns in your example occur very infrequently in a randomized key space. Try running crunch with the -d flag to limit repetitions and see how little it affects the resulting wordlist.

    Consider this trade-off: will the computing time saved using a truncated wordlist really be less than the time spent generating said wordlist? Is it also worth the risk that you may truncate the target passphrase?

    Unless you know with absolute certainty that the target passphrase was generated using specialized pattern-avoidance methods, I suggest you keep your dictionary complete.

Page 2 of 2 FirstFirst 12

Similar Threads

  1. Removing words from wordlist
    By CKing in forum Experts Forum
    Replies: 6
    Last Post: 04-18-2011, 05:00 AM
  2. Removing numbers from a wordlist.
    By oracx in forum Beginners Forum
    Replies: 2
    Last Post: 02-22-2010, 11:39 PM
  3. alpha-numeric wordlist
    By cain#1 in forum OLD Newbie Area
    Replies: 1
    Last Post: 06-28-2009, 03:03 AM
  4. Replies: 28
    Last Post: 10-23-2008, 10:28 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •