Problem wit CeWL

[2901119]

upon trying to run CeWL I got this error

/pentest/passwords/cewl# ./cewl.rb 
./cewl.rb:58:in `require': no such file to load -- spider (LoadError)
	from ./cewl.rb:58:in `<main>'

I checked the docs to try to see what dependencies may be needed and installed all of the necessary gems... among those was spider so I specifically did

gem install spider

also to make sure it was installed I did a quick

/pentest/passwords/cewl# gem list -d spider

*** LOCAL GEMS ***

spider (0.4.4)
    Author: John Nagro
    Rubyforge: http://rubyforge.org/projects/spider
    Homepage: http://spider.rubyforge.org/
    Installed at: /var/lib/gems/1.9.2

    A Web spidering library

It's there but I keep getting the same error

I opened up cewl.rb but unfortunately I'm not too familiar with ruby so It didn't help much

also worth mentioning I'm on gnome 64 bit, I do have the ruby opt environment variable set in my .bashrc file and I do have exiftool installed

I also googled a bit to find a solution and came up with a couple things

1. the gems spider and arachni don't run well together, but I don't have arachni on my system so that can't be it

2. the documentation says the reason for that error is the gem is missing, but it clearly is not in my case

3. It appears a couple people posted this same issue on the backtrack 4 forums... what worked for them was passing the environment variable "export RUBYOPT=rubygems" Unfortunately that doesn't work for me either.

can anyone else verify this or am I the only one?

any ideas?

UPDATE: also when trying to run make in /pentest/telephony/warvox I get a similar error

/pentest/telephony/warvox# make
Checking for RubyGems and the Bundler gem...
-e:1:in `require': no such file to load -- bundler (LoadError)
	from -e:1:in `<main>'
make: *** [bundler] Error 1

again the gem is installed... possibly related???

[digininja]

To fix the gem problem you need to install them to your user directory, this is a bit of a hacky fix but works for now.

 gem install --user-install spider
WARNING:  You don't have /root/.gem/ruby/1.9.2/bin in your PATH,
          gem executables will not run.
Successfully installed spider-0.4.4
1 gem installed
Installing ri documentation for spider-0.4.4...
Installing RDoc documentation for spider-0.4.4...
root@root:~# irb
irb(main):001:0> require "spider"
=> true
irb(main):002:0>

However that doesn't fix CeWL as it will only run in 1.8 at the moment and not 1.9. However I do have an updated version that I've tested in 1.9.2 outside BT but not in BT, I'll try to get it into a VM at some point soon and test it, if it works I'll send it to the devs for a package update.

[wicky]

I had the same problem. It does seem that the 1.9 ruby/gems setup is not working quite right on BT5. Though, there is a simple solution to the problem, even if it doesn't solve the underlying issue.

You need to switch ruby version to 1.8 as per this post from muts: http://www.backtrack-linux.org/forum...-versions.html.

That post does mention that you need to use 1.8 for some tools such as cewl and warvox. I haven't seen a definitive list of which tools require which version but then a) I haven't looked and b) it will become quickly apparent when you run the tool and get ruby dependency errors.

Of course, the alternative, if you prefer to leave your ruby version alone is to call the required script with the specific ruby binary you need, for example, the following will work:

/usr/bin/ruby1.8 /pentest/passwords/cewl/cewl.rb -h

HTH.