Comparing Wordlists and REMOVING Words Without Combining Them
Hey Folks,
I've tried searching for this, but not sure what kind of keywords I would type to find such a thing.
Is there a script, or even simpler a command line, that allows you to compare two word lists and REMOVE words from one of them?
For example, compare WorldList_01 to WordList_02 and remove from WordList_02 words that are in WordList 01, leaving you with two wordlists that are unique to each other?
My problem is that I have two wordlists that I have created.
One is a list of dates starting from 01 01 1900 (with no spaces of course) and going all the way up to 01 01 2013. I have them in 6 variations:
mmddyyyy
mmyyyydd
ddyyyymm
ddmmyyyy
yyyymmdd
yyyyddmm
I also have it "uniqued" since 01011900 can be both mmddyyyy and ddmmyyyy. (If anyone wants this date list, let me know.)
The other is a list of 8-character permutated numbers using crunch from 00000000 to 99999999. This file is about 900 megabytes big.
Now, of course, the crunch-generated list will have every single date in my date list. I would like to remove all of those dates from my date list from my crunch-generated list so that the two lists are unique compared to each other. Is there a way to do this?
Many thanks in advance!
- Sulegna -
Re: Comparing Wordlists and REMOVING Words Without Combining Them
Comparing 2 Wordlists and Removing Identical Words WITHOUT Combining The Lists
Hey Folks,
Okay, to AVOID CONFUSION, I am going to attempt to reword this post so that it is clear as to what I am searching for.
I have HONESTLY tried searching for this, but I cannot find a simple way to do this.
I would like to compare one wordlist to another, and remove words from the second wordlist WITHOUT having to combine the two lists together and using sort & uniq to remove the duplicates. I already know how to combine multiple wordlists and use sort and uniq to remove all the duplicates. This is NOT what I am trying to do. I want to keep the two files SEPARATE.
For example, compare WorldList_01 to WordList_02 and remove from WordList_02 words that are in WordList_01, leaving you with TWO unique wordlists.
My problem is that I have two wordlists that I have created.
One is a list of dates starting from 01 01 1900 (with no spaces of course) and going all the way up to 01 01 2013. I have them in 6 variations:
mmddyyyy
mmyyyydd
ddyyyymm
ddmmyyyy
yyyymmdd
yyyyddmm
I also have it "uniqued" since 01011900 can be both mmddyyyy and ddmmyyyy. (If anyone wants this date list, let me know.)
The other is a list of 8-character permutated numbers using crunch from 00000000 to 99999999. This file is about 900 megabytes big.
Now, of course, the crunch-generated list will have every single sequence of numbers in my date list. What I would like to do is remove everything listed in my date list from my crunch-generated list, WITHOUT combining the two of them, so that the two separate lists are unique compared to each other.
Is there a way to do this?
Many thanks in advance!
- Sulegna -
Re: Comparing 2 Wordlists and Removing Identical Words WITHOUT Combining The Lists
