Metasploit export database - insert into mysql

[kagemusha]

Hello all.
I have a bit of a problem that I have been googling for a couple of days, and so far I have reached the conclusion that it is possible to do, but can't figure out how to do it.

I want to export the metasploit postgresql database and insert it into a mysql database. Now I have managed to dump the postgresql database into a xml file and now I want to insert it into the mysql database, I have seen that there are a couple of differnt ways to do this with different modules and commands. But I am wondering, do I need to create the tables in mysql before inserting the data? In that case it would be nice to use the same structure as the metasploit database, but I cant find the init sql file for metasploit.

It might be possible to install postgresql and use the pg_dump command, but I would prefere to have a solution using the xml way since I have an other similar problem with an other xml file that I would like to push into a mysql database.

If anyone could help with this I would be very greatfull.

Cheers.

[scottm99]

My preference would be to create the database tables before trying to load the data, but it depends on how you want to do it. You can always start up postgresql from a command line, connect to the Metasploit database as the msf user, and see what tables are used. This info is in the /opt/framework/config/ database.yml file. I've never looked for the script Metasploit uses to create its database, but try doing a grep -r from /opt/framework. For the ultimate authority on the framework, the Metasploit mailing list is the place to go. HDM and other developers are often on there.

[kagemusha]

My preference would be to create the database tables before trying to load the data, but it depends on how you want to do it. You can always start up postgresql from a command line, connect to the Metasploit database as the msf user, and see what tables are used.

Yes, I would like to keep the same database structure as is used for the framework sql server. It would probably make a lot of things easier. The problem with this solution is that the postgresql server that metasploit uses is built into the framework which also means that the normal postgresql tools are not included in a basic install, they can be added by installing the postgresql package.

And I would prefere using the xml method since, as I said I have an other xml file that I would like to insert into another database.

the Metasploit mailing list is the place to go. HDM and other developers are often on there.

I understand that, but this is, at least to me, more of a question about how to insert xml into a mysql database. And a tiny bit about how metasploit populates its database.


* Sorry for the chop chop on the quote, but it seemed easier this way.

[scottm99]

Well, the developers over on the metasploit mailing list could certainly tell you how metasploit populates its database. They may be able to point you in the right direction of an export/import. If you have some programming background (especially ruby), have a look at the MSF source. Have you tried the db_export command in msfconsole? Haven't used it myself, but it will export to an XML file.

[kagemusha]

Yes, I have used the db_export command, which is from where I get the xml file that I want to push into the mysql server. And so far I have figured out that it can be done, but very unsure of how.

I have started taking contact with the metasploit people about that side. Will post any replies I get here as well.


Edit:

Ok, progress made..

1. Dump the metasploit database with db_export to an xml file.
2. Convert the xml file to a csv file.
3. Create the database structure in mysql (init scripts for metasploit postgresql database are located in /opt/framework/msf3/data/sql/migrate/ as ruby module scripts)
4. Import the csv file into mysql.

If anyone has a solution how to automate this, I would be very thankfull. There are 73 ruby scripts that populate the database, and I have no knowlage about ruby either.

Any suggestions?

[scottm99]

Do you have any programming, or SQL background? I looked at a few of the ruby modules from the path you described in step 3. They seem to be fairly straight-forward embedded SQL statements. First thing I'd do is grep that directory for the word "create" or "create_table". That will at least give you the script names that do the table creation. In my opinion, you'll have to get your steps down one-and-a-time before trying to automate this. What you're trying to do seems to be quite a project.

If you don't have any programming, or SQL background, I'm afraid you're in for a tough time You may need to get a working knowledge of ruby & SQL before going too far.

[kagemusha]

Ruby.. Never touched it. SQL.. well.. I can do enough simple select queries for my own purposes..

Just took a look at 000_create_tables.rb.. I think I get the picture. But you are right.. making something to automate this is beond my skillset. I will have to do it the old fasioned way.. ehm.. I mean manually. not paper and pencil.

Will get back to this when I have had a bit of time to play with it.

[scottm99]

I think a basic understanding of ruby, coupled with SQL knowledge, will give you the best chance of success. Good luck