Results 1 to 9 of 9

Thread: Help with extracting tcpdump data...?

  1. #1
    Just burned his ISO
    Join Date
    Sep 2009
    Posts
    4

    Default Help with extracting tcpdump data...?

    Hey everyone,

    I'm using tcpdump to monitor the traffic on my router - very slick! Anyway, right now I'm using wireshark with a filter like this to pull yahoo chats:

    data.data contains "Command=\"6" || data.data contains "Command=\"11"

    This gets me the right packets, but it's very tedious to extract the actual chat session - ie, I have to do a 'follow stream' in WireShark and then copy and paste all of the chat texts to another document. In addition to that, it seems like the follow stream only pulls the current session so if the session had ended and another one started later I need to find a packet from the next session and follow that stream...

    Is there a utility out there that will pull yahoo chats from these files in a nice format? Something like this:

    user1: blah blah
    user2: blah blah blah
    ...

    I have dozens of these files with huge amounts of chat data that I want to archive.

    Thanks!

  2. #2
    Just burned his ISO
    Join Date
    Sep 2009
    Posts
    4

    Default

    Oh... I've also tried exporting the selected packets, but even when I select expand all the packet data doesn't seem to be exported - only the header stuff.

  3. #3
    Member zWiReDz's Avatar
    Join Date
    Sep 2009
    Posts
    123

    Default

    Try searching around the forum already? I'm sure theres some great scripters/coders that have made something of this calibur.
    "If it's stupid but works, it's not stupid." - Murphy's Laws of combat, #2

  4. #4
    Super Moderator lupin's Avatar
    Join Date
    Jan 2010
    Posts
    2,943

    Default

    Quote Originally Posted by ajf3ajf3 View Post
    Oh... I've also tried exporting the selected packets, but even when I select expand all the packet data doesn't seem to be exported - only the header stuff.
    How were you running tcpdump to capture the data? What command line options were you using?
    Capitalisation is important. It's the difference between "Helping your brother Jack off a horse" and "Helping your brother jack off a horse".

    The Forum Rules, Forum FAQ and the BackTrack Wiki... learn them, love them, live them.

  5. #5

    Default

    msgsnarf or chaosreader may be able to reconstruct the sessions for you.

    Good Luck...

  6. #6
    Just burned his ISO
    Join Date
    Sep 2009
    Posts
    4

    Default

    I'll look into those options... I've been using this:

    ./tcpdump ip host 192.168.168.109 -w ./cap.tst -s0

  7. #7
    Super Moderator lupin's Avatar
    Join Date
    Jan 2010
    Posts
    2,943

    Default

    Quote Originally Posted by ajf3ajf3 View Post
    I'll look into those options... I've been using this:

    ./tcpdump ip host 192.168.168.109 -w ./cap.tst -s0
    Yeah OK. My first thought when you mentioned that you werent seeing all the packet contents was that you may not have captured all of the packet using tcpdump. Use of the snap length switch as you have done above should take care of that though.

    Here are some other applications you could try to see if they assist in extracting the data you need. Some are Windows only, and I haven't tested any of them personally.
    NetworkMiner packet analyzer | Get NetworkMiner packet analyzer at SourceForge.net
    NetWitness - Total Network Knowledge&#8482 - Investigator
    Xplico - Internet Traffic Decoder
    Capitalisation is important. It's the difference between "Helping your brother Jack off a horse" and "Helping your brother jack off a horse".

    The Forum Rules, Forum FAQ and the BackTrack Wiki... learn them, love them, live them.

  8. #8
    Just burned his ISO
    Join Date
    Sep 2009
    Posts
    4

    Default

    Thx guys, tried out msgsnarf... not sure it's going to do it for me, so I'll check out the chaosreader too.

    Here's what my problems are... up until around the end of June yahoo chats showed up with a protocol of YMSG. msgsnarf does pull data out of these files, but there are two problems - 1) it doesn't grab all of the chat exchange - ie, I can see more than what it pulls by looking via WireShark and 2) not a big deal, but it throws all kinds of junk chars around the texts of the chats.

    The bigger problem is that around the end of June, the yahoo chats stopped showing up with a protocol of YMSG and instead show up as TCP. I can still check these out in wireshark, but I can't use the YMSG filter - I have to use the data contains filter listed in an earlier post. For caps with this kind of chat data, msgsnarf doesn't pull ANY of the chat activity out of the file

    xplico looks really cool, but according to the status page it doesn't do yahoo chats yet:

    w ww.xplico.org/status

    (not being a dick - all help is super-appreciated! Just putting the info here in case others are trying to do the same

    NetWorkminer is also badass - you should check it out, small exe, no install... not good for the chat extract I'm looking to do though.

  9. #9
    Super Moderator lupin's Avatar
    Join Date
    Jan 2010
    Posts
    2,943

    Default

    Quote Originally Posted by ajf3ajf3 View Post
    (not being a dick - all help is super-appreciated! Just putting the info here in case others are trying to do the same
    No its OK, I wouldn't have taken that personally

    Quote Originally Posted by ajf3ajf3 View Post
    NetWorkminer is also badass - you should check it out, small exe, no install... not good for the chat extract I'm looking to do though.
    Yes, Networkminer is on my list of tools to check out when I have time or a specific need. Its a long list so I haven't got around to it yet.

    So it basically sounds like they have changed the chat protocol sometime during your monitoring period. There's no real requirement that Yahoo is trying to meet regarding interoperability so there's nothing to really stop them doing this unfortunately.

    Depending on what data you want out of the chats you might have some luck in doing a straight string extraction (using the strings command) - but this may result in you missing out on various metadata in the communication. If you were after files (binary data) transferred using the chat client you could try tcpxtract.

    Other than that though you may be stuck with writing your own parser. You may be able to find the spec for the chat protocol online somewhere or perhaps you may be able to reverse engineer this from the source of an open source chat client that can work with Yahoo.

    Hope that helps somewhat. If you find a good method come back here and share the knowledge!
    Capitalisation is important. It's the difference between "Helping your brother Jack off a horse" and "Helping your brother jack off a horse".

    The Forum Rules, Forum FAQ and the BackTrack Wiki... learn them, love them, live them.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •