Netglub

Really Open Source Information Gathering

How to compile the slave

juillet 23rd, 2010 Posted in
Juil, 23 2010
No Comments »

The slave can be compiled on GNU linux distribution only.

Step 1 : Prerequisites

The sources are not available yet

Mandatory Dependencies and package needed

Some package are mandatory to compile and run the slave :

  • build-essential
sudo aptitude install build-essential

Step 2 : Qt framework install

The slave depend on Qt framework (version 4.6 at least).

Actually, the latest release of Qt framework is the 4.6.3. The packages available on Ubuntu official repository being too old, two solutions may be used :

First Solution: Install Qt SDK.

You can download the 64bit version of Qt SDK at http://qt.nokia.com/downloads/sdk-linux-x11-64bit-cpp or download it from our local mirror (private link). netglub slave can be used on a 32bit processor too, if you made this choose, you should dowload the Qt SDK 32bit version http://get.qt.nokia.com/qtsdk/qt-sdk-linux-x86-opensource-2010.03.bin or dirrectly here (private link)

The SDK take up more disk space than compiling Qt sources, but it contains a inary version of Qt libraries which avoid a complete compile. Furthermore, the compilation is trivial via graphical wizard.

wget http://get.qt.nokia.com/qtsdk/qt-sdk-linux-x86_64-opensource-2010.03.bin
chmod +x qt-sdk-linux-x86_64-opensource-2010.03.bin 
./qt-sdk-linux-x86_64-opensource-2010.03.bin

Second solution : Compiling Qt Sources

The first solution is recomended

This solution, more laborious, is to download the Qt source (« http://get.qt.nokia.com/qt/source/qt-everywhere-opensource-src -4.6.3.tar.gz ») and to make yourself compile it

Here is a hint :

sudo aptitude install libxq11-dev libxext-dev 
tar xf qt-everywhere-opensource-src-4.6.3.tar.gz 
cd qt-everywhere-opensource-src-4.6.3 cd qt-everywhere-opensource-src-4.6.3
./configure -opensource -fast -nomake examples -nomake demos 
make 
sudo make install
To compile Qt in debug mode :

 ./configure -opensource -fast -nomake examples -nomake demos -debug -no-separate-debug-info
Now, Qt is installed, you can begin the slave install.

step 3 : install the slave :

You can now compile the slave :

cd (@netglub location)/slave 
qmake (qmake is locate in the bin directory of Qt)
make

To install the slave, you should have the root privileges :

cd (@netglub location)/slave
sudo ./install.sh

To launch the slave : ./(@netglub_location)/slave/tools/netglub_slave start

The slave will automaticaly start at each boot.
sudo ./slave -h
for help
if you want to change some configuration, you can change the file
(@netglub_location)/slave/tools/slave.conf, and launch install.sh again.
Now, the slave is successfully installed.
You may have some problems with transforms that don’t work. please refere to the followings.

step 4 : transforms dependencies.

Netglub is given with some transforms, that have some dependencies :
(This tests are done under linux, ubuntu 9.04)

This transforms are mostly based on python 2.6.2 (at least)
  • DDToIp:
  • DnsNameToWebsite:
  • DnsWebsiteToDomain:
  • DomainEmailToEmailSE:
  • DomainIpToEmailWhois:
  • DomainToDomainExpand:
  • DomainToDomainTLD:
  • DomainToDomainTopLevel:
  • DomainToEmailPGP:
  • DomainToEmailSEDomOnly:
  • DomainToLocationWhois:
  • DomainToMX:
  • DomainToNS:
  • DomainToPersonPGP:
  • DomainToWebsite:
  • EmailIpToWebsiteSE:
  • EmailToDomain:
  • EmailToEmailSamePGP:
  • EmailToPersonPGP:
  • EmailWebsiteToUrlSE:
  • IpBlockToDnsName:
  • IpToAsWhois:
  • IpToDnsName:
  • IpToDomainSharedIp:
  • IpToIpBlock:
  • IpToIpBlockCuts:
  • IpToLocation:
  • IpToWiki:
  • PersonToEmailPGP:
  • PersonToEmailSE:
  • PersonToPersonPGP:
  • PersonToWebsiteSE:
  • PhraseToUrlSE:
  • UrlToEmail:
  • UrlToUrl:
  • UrlToUrlInk:
  • UrlToWebsite:
  • WebsiteToIp:
  • WebsiteToWebsiteInk:

    • None

  • DomainToDnsNameBrute:
  • DomainToDnsNameZN:

    • libnet-ip-perl
    libnet-dns-perl

  • DomainToDnsNameSE:

    • simplejson
    you can download it from : http://pypi.python.org/pypi/simplejson/
    extract the file then run setup.py :

      tar -xzvf simplejson-(version) 
  python (@simplejson-(version))/setup.py 
  python (@simplejson-(version))/setup.py build
  sudo python (@simplejson-(version))/setup.py install
  • IpBlockToIpAddress:
  • IpBlockToPorts:
  • IpToOs:
  • IpToOsNmap:
  • IpToPorts:

    • nmap

  • IpToOsPhp:

    • nmap
    php5-cli

  • WebsiteToEntitiesNER:
  • UrlToEntitiesNER:

    • simplejson
    pyxml     (patched)
    Download PyXML-0.8.4.tar.gz from http://sourceforge.net/projects/pyxml/files/
    You may need to install python2.6-dev, for example via: sudo apt-get install python2.6-dev
    Use these commands:

    tar xvzf PyXML-0.8.4.tar.gz
  cd PyXML-0.8.4
  wget http://launchpadlibrarian.net/31786748/0001-Patch-for-Python-2.6.patch
  patch -p1 < 0001-Patch-for-Python-2.6.patch
  sudo python2.6 setup.py install
  • WebsiteToUrlMirror:
  • WebsiteToWebsiteMirror:
  • WebsiteToEmailMirror:

    • omt

  • WebsiteToPhraseWhatWeb:

    • ruby
    ruby-dev
    libopenssl-ruby

    You can use this to install most of them.

    sudo apt-get install libnet-ip-perl libopenssl-ruby ruby-dev ruby omt php5-cli nmap libnet-dns-perl libnet-ip-perl

    Leave a Reply

    You must be logged in to post a comment.