Results 1 to 7 of 7

Thread: BackTrack Modularity Guide

  1. #1
    Just burned his ISO
    Join Date
    Aug 2008
    Posts
    15

    Arrow BackTrack Modularity Guide

    BackTrack – Internals

    BackTrack Roots

    BackTrack is a great Linux distribution tailored specifically for penetration testing, security auditing, reverse engineering, and much more. This powerful distribution was build with portability in mind, and that’s why BackTrack is based on the Slax distribution, which has amazing portability features. Slax is minimalistic distribution that is completely modularized; every single part of the OS is build using modules which are layered on top of each other to form the end file system(think of a photoshop document, all the layers in the document form the final image). The aufs file system is the key component that enables this extensibility.

    The aufs file system

    The aufs file system is a completely rewritten Unionfs. It is a stackable unification filesystem, which unifies several directories into a single merged directory. What this mean is that directories are stacked upon themselves to create the final directory. Here is a simple example:

    Code:
    1: Directory1/
          File1
          File2
          File3
    2: Directory1/
          File4
          File5
          Directory2/
    
    3: Directory1/
          File1
          File2
          File3
          File4
          File5
          Directory2/
    Directories 1: Direcory1 and 2: Directory1 are merged together to form the directory 3: Directory1 . As you can see using this method of stacking we can build a entire filesystem(the directories being different root’s). Each layer of this merged directory can be packaged into modules and then at runtime merged. That’s exactly how BackTrack is build. The / directory is build using the following modules located in the BT3/base directory: bin.lzm, etc.lzm, home.lzm, lib.lzm, pentest.lzm, sbin.lzm, etc… Each module contains a small portion of the entire / directory. At boot time all those modules are merged together to form the entire system. The neat thing about using this type of file system is that we can build a base system, and then without modifying it, we can customize it layering other modules on top of it. That exactly why BackTrack has a BT3/modules directory. All customizations to the base file system are made using the modules in that directory(for example the nvidiadriver.lzm module add NVidia driver support to the system).
    Another key feature of aufs is that changes to the base file system(the final merged directory) can be recorded in a special directory. This special directory contains all the modification made to the base system. This is great because if the distribution was running off of a cdrom, you can record all the changes you have made and then save them to a module, which you can then load at the next startup.
    So what is a module? A module is simply a special file system stored in a file. The file system used is called squashfs and is compressed using the lzm algorithm (hence the .lzm extension).

    The squashfs file system

    Squashfs is a compressed read-only file system, which compresses files, inodes and directories. This file system is used when space is a priority, like putting modules on to a CD that has limited capacity. Squashfs is very popular in the embedded market, where space scarce. Usually squashfs uses the gzip compression format, but lzm support was added by the author of the Slax distribution because it has a higher rate of compression.

    Modularity using modules

    Modularity of the system is achieved by being able to load and unload modules dynamically. Because the system is layered, we can add or remove functionality from the system without modifying the base. This is especially helpful when running the OS from a read-only medium like CD or DVD. Also we can shadow(hide) files or directories with our own. So for example you want to add a new app to BackTrack but don’t want to reburn the iso image, well you can easily create a temporary root directory, populate it with the files you want, and then turn it into a module and load it. Here is a example where we want to install 2 cool scripts, one that sets up the system while the other is a utility we wrote:

    Code:
    # mkdir tmpRoot
    #cd tmpRoot
    # mkdir –p etc/rc.d/
    # mkdir –p usr/local/bin
    # cp /tmp/myUtility usr/local/bin
    #cp /tmp/rc.local etc/rc.d/
    # cd ..
    # dir2lzm tmpRoot myScripts.lzm
    # activate myScripts.lzm
    Now I’m going to explain these commands. First we created are root directory, this directory will emulate the location of the files on the real system(think of it like a chroot). Next go into the root and create the directory structure where we want our file to be in. Then we copy the 2 files to their appropriate location. Once we have finished we use dir2lzm to create a module named myScripts.lzm .
    Optionally we also loaded the new module into the live system using the activate command.
    The net effect o loading this module is that our live system will have 2 new files added. Now if we add this module to the BT3/modules directory, every time the system starts up, our script files will also be loaded, and because the loading of modules is before the system initialization, the rc.local script will be executed at boot time. On thing to note is, if there was a rc.local file before the module was loaded, when our modules gets loaded it will overshadow the previous rc.local file.

    Persistent Changes

    As I explained before changes made to live system are recorded in a special directory called changes. This directory is usually located at /mnt/live/memory/changes/. If you look into that directory you will the all the files that were modified since the filesystem was loaded. This directory is a replica of the root file system, but with only modified files/directories. So for example if we create the file /etc/inputrc and change the file /etc/HOSTNAME, the /mnt/live/memory/changes/ will have those 2 file with their entire directory structure replicated:
    Code:
    /mnt/live/memory/change/
       etc/HOSTNAME
       etc/inputrc
    An easy way to store the changes we have made is to use the dir2lzm command. We can turn the /mnt/live/memory/changes/ directory into a module and then added into our BT3/modules directory. So when we boot next time all our changes will be there. One thing to note, if a module is loaded no changes will occur in the changes/ directory. So if you want to keep you changes from boot to boot using this method you will have to create new modules, because you cant append to moduels. One solution to that would be extracting all the modules you want to merge, together into one module, into a single directory using the lzm2dir command, and then creating a new module from the merged directory using dir2lzm command.
    Saving changes using the method described above is tedious and manual, fortunately BackTrack has the ability save our changes using the changes=<changes_location> kernel parameter. Appending this to this to the kernel at boot time tell BackTrack to load and save the changes to that place. Here is a entry within syslinux.cfg(/boot/syslinux/syslinux.cfg) with the changes kernel parameter:

    LABEL pchanges
    MENU LABEL BT3 Graphics mode with Persistent Changes
    KERNEL /boot/vmlinuz
    APPEND vga=0x317 initrd=/boot/initrd.gz ramdisk_size=6666 root=/dev/ram0 rw changes=/changes/slaxsave.dat autoexec=xconf;kdm
    This kernel parameter can also be added at boot time in the boot loader by pressing TAB and appending changes=<changes_location>.

  2. #2
    Just burned his ISO
    Join Date
    Aug 2008
    Posts
    15

    Default

    <changes_location> can be several types of paths, it can be a file(loop file to be exact with a filesystem), a directory or a device. Paths can be specified using the /dev/XXX/Some/Path convention, where /dev/XXX is a valid filesystem device which has /Some/Path on it. An alternative way to specify the changes location is using /mnt/XXX instead of /dev/XXX, but internally /mnt/XXX is converted to /dev/XXX so there is no reason to specify it that way. Also we can just specify the device, /dev/XXX or /mnt/XXX (the conversion mentioned above is also done). We can specify the file: /dev/XXX/changes_file or /dev/XXX/Some/Path/changes_files. A neat feature of the system enables you to omit the device and just specify the path(changes=/Some/Path or changes=/Some/Path/changes_file), BackTrack will automatically try to find the path on all available devices, trying first on the boot source. The path/file specified should be a posix compatible file system, although fat32 could be used(using the posixovl system overlay) but currently there are problems with mounting. Here are some examples:

    changes=/dev/sda1/
    changes=/dev/sda1/BTChanges/
    changes=/mnt/sda1/
    changes=/mnt/sda1/BTChanges/
    changes=/BTChanges/
    changes=/dev/sda1/ChangesFile.dat
    changes=/ChangesFile.dat
    One very important note, all the paths specified must have a directory called changes created in them, otherwise it will fail, this include the loop file! Example:

    changes=/dev/sda1/BTChanges/
    /mnt/sda1/BTChanges/changes/

    changes=/dev/sda1/
    /mnt/sda1/changes/
    A simple way of creating a loop file is the following:
    dd if=/dev/zero of=/Path/To/changes_file bs=1024 count=<SIZE of File>
    mkfs.ext2 /Path/To/changes_file
    mount –o loop /Path/To/changes_file /tmp
    mkdir /tmp/changes/
    umount /tmp
    Note the before last command, we are creating the mandatory changes directory!



    Modules
    I’ve been talking about modules, but I haven’t really explained where they are usually located, or where one should place their newly created modules. Well at the root of the CD image, there is a BT3 folder, this is where all the files of the OS are stored.
    BT3/base - contains all modules of the base root filesystem.
    BT3/modules – contains all modules that customize and add new features, every module in this directory is loaded.
    BT3/optional – contains other modules that are not loaded automatically at boot time unless, specified with the load=XXXX kernel parameter.
    BT3/rootcopy – directory tree which is copied on top of the base filesystem after all modules have been loaded
    As I said the base and modules directories are loaded at boot time, base first modules second. Then if the load=SomeModueName kernel parameter is appended at boot time, that module located in the BT3/optional directory is also loaded. Here is a example of loading the cubez.zlm module(which is located in the optional directory) - this is a extract of boot/syslinux/syslinux.cfg file:

    LABEL cubez
    MENU LABEL BT3 Graphics mode (Compiz) - Experimental
    KERNEL /boot/vmlinuz
    APPEND vga=0x317 initrd=/boot/initrd.gz ramdisk_size=6666 root=/dev/ram0 rw chexpand=256 load=cubez autoexec=xconf;cubez;startx
    Then all the files from the BT3/rootcopy are copied into the live root filesystem.
    If for some reason you have a module which wasn’t loaded at boot time and you want to load it, you have 2 command line utilities for that: activate and uselivemod. Both commands do exactly the same thing with on little exception, the activate command after it loads the module into the filesystem, it the checks the /etc/rc.d/ directory for any files that module has added, and if it finds a executable script it executes it with start as it’s argument. If you want to remove a loaded module, rmlivemod is available to do just that.
    Now how do we create those modules you may ask… Well we have 3 commands to create a module and 1 command to extract a modules to a directory. Here are the 3 commands:

    dir2lzm – converts a directory into a module
    deb2lzm – converts a debian package into a module
    tgz2lzm – converts a slackware TGZ package into a module
    And here is the command to extract those modules to directories:
    lzm2dir - converts a lzm module into a directory structure
    These commands are pretty self explanatory….. There is also 2 utilities that these 3 commands use, it’s good to now that they exist:
    mksquashfs – create a squashfs filesystem from a directory/ies
    unsquashfs – copy files/directories from squashfs filesystem to directory

    The Linux Live scripts and boot process

    <To Be added>



    Any correction, or suggestions are welcome. Thanks for reading

  3. #3
    Good friend of the forums
    Join Date
    Feb 2010
    Posts
    328

    Default

    ok I sort of get lzm files now ..

    I think the changes is not specific to backtrack or slax but maybe grub etc in general. ( new to linux boot loaders )

    here are some links on 'changes' that helped me:

    Remote Exploit Forums - View Single Post - New Method For Saving Changes to USB
    http://forums.remote-exploit.org/sho...7&postcount=15


    TUTORIAL: live install with changes, swap and data partitions - Remote Exploit Forums
    http://forums.remote-exploit.org/showthread.php?t=7844

  4. #4
    Just burned his ISO
    Join Date
    Aug 2008
    Posts
    15

    Default

    The Linux Live scripts and boot process

    The liblinuxlive script is the brain child of the creator of Slax and it is the actual work horse of the system’s modularity. It is both used in boot process as well as in the utilities that manipulate the modules. This script file is basically an aggregation of function that do a specific tasks(find a file, mount a module, create a module, etc..).

    The boot process of the system starts in the initrd file, just like in many other Linux distributions. This file is used by the kernel to initially setup the environment for booting the OS. It is actually a small filesystem compressed in gzip. Now normally the use of the initrd image is to enable the kernel to load the most basic kernel modules (filesystem modules) in order to mount the base filesystem. But in case of BackTrack as well as Slax, the initrd image is where all the magic happens. It is this image that is responsible for constructing the entire filesystem and loading all our modules. Basically an initrd.gz image is small Linux OS which its sole purpose is to setup and mount the initial root filesystem.

    When the kernel is loaded into memory by the boot loader, the next thing that happens is the initrd.gz image is loaded. Then the linuxrc script is executed, and then at the end of this script the initd process is executed which actually setups up our system for runtime. I’m going to focus my attention to the linuxrc script for now, because it is in this file where the mounting of the aufs filesystem is done.
    If you would like to take a peak into the initrd.gz file, here’s how you can mount it:

    Code:
    mkdir /mnt/initrd
    gunzip –c /boot/initrd.gz > /tmp/initrd
    mount –o loop /tmp/initrd /mnt/initrd
    I’m going to give you a annotated summary of the boot process done by the linuxrc script. The first thing that happens at boot time is the mounting of /proc and /sys directories, and remounting the root filesystem as read-write (now the root filesystem is actually a ramdisk /dev/ram0). Once that is done mtab is linked to /proc/mounts so that umount –a can be done.

    Ok, now that the system is partially setup the liblinuxlive script is loaded as its going to be heavily used during the boot setup process. Before anything else happens basic kernel modules are loaded, these are essential filesystem drivers (aufs, iso, etc..). Next usr.lzm module is loaded, which has essential utilities used during boot setup process (ex. posixovl – used to overlay vfat filesystem with posix compatible features). Then other kernel filesystem modules are loaded.

    The hunt for the BackTrack system files is done next, and I said hunt, because the files are actually searched for every time you boot. If the from= kernel parameter is supplied at boot time, all available devices are mounted and checked to see if path supplied has a BT3/base ,BT3/modules and BT3/optional folders with modules in them, if the path has those folders that means we found our system files needed for boot. Otherwise the same search is done but looking for a directory named BT3 with a file called livecd.sgn in it, if found that mean we found the system files. As you can see modularity is also built into the booting of the OS, you could theoretically have the BackTrack system files on another medium and it would still boot. You can use the from= kernel parameter for example to boot a thumb drive with BackTrack on in, on a computer that doesn’t have usb booting capabilities. Just pop in the BackTrack cd, append from= parameter in the boot loader, and voila you’re thumb drive is booted.

    Continuing on, after the system files have been found the next thing on the agenda is the finding changes folder. If the changes= kernel parameter is supplied, the path supplied is searched for on all available devices, exactly like the system files above. If the path is found, then posix filesystem compatibility is checked, if that fails, posixovl filesystem overlay will be applied on the path (it makes the path compatible to posix standards). Otherwise changes are written to ram using a tempfs filesystem, if the ramsize= kernel parameter is specified that’s how big the tempfs filesystem will be, otherwise it’s 60% of the size of the ram installed.

    A quick note about how paths can be specified and how path searching is done. Paths can be specified in 2 ways, they can be relative(no device specified) or absolute(device specified).

    Code:
    Relative:
     /Some/Path
     /
    Absolute:
     /dev/XXX/Some/Path
     /mnt/XXX/Some/Path
     /dev/XXX
    Relative paths will be searched for by mounting all devices and trying to find the path on the device, while absolute paths are checked only on the specified device. Now devices can be specified in 2 ways, using /dev/XXX or /mnt/XXX, both naming conventions are the same, because internally /mnt/XXX will be translated into /dev/XXX.

    Next the boot process checks to see if the toram or copy2ram kernel parameters were specified at boot time, and if they were then all filesystem modules (BT3/base, BT3/modules, and modules in BT3/optional supplied with the load= kernel parameter) are copied into ram.

    Next comes the loading of the base modules from BT3/base, and then the modules from BT3/modules. Optionally if the load= kernel parameter was supplied, then those modules from BT3/optional are also loaded. The last thing that is done is the copying of the BT3/rootcopy folder contents onto the root filesystem.

    The setup of the bootup is nearly done, the only things left to do is cleanup. Unused kernel modules are unloaded, /usr and /sys directories are umounted. /boot is mounted (BT3/boot from the boot media). The root is changed using chroot and initd is executed from the filesystem that was setup.


    Thanks for reading, any suggestions or comments welcomed.

  5. #5
    Just burned his ISO
    Join Date
    Aug 2008
    Posts
    15

    Default

    Quote Originally Posted by operat0r View Post
    ok I sort of get lzm files now ..

    I think the changes is not specific to backtrack or slax but maybe grub etc in general. ( new to linux boot loaders )
    Actually the changes kernel parameter is used by the Linux Live scripts(liblinuxlive). If you wanted you can add you own kernel parameters. If you check /proc/cmdline you can see all the parameters that where specified at boot time.

    If you wanted a script to do different things based on the parameters supplied at boot time you could easily do this by checking /proc/cmdline. So no it's not actually a boot loader thing Hope that clears up the misunderstanding.

  6. #6
    Senior Member
    Join Date
    Jan 2006
    Posts
    1,334

    Default

    Makes a nice change to see such a detailed and informative post

    Thanks for taking the time to contribute this

    It should really take its place in the "Tutorials & Guides" section, so I'll move it there........

  7. #7
    Junior Member
    Join Date
    Aug 2007
    Posts
    85

    Default

    Thanks vladsmail


    really interesting Topic
    How you spend your time is more important than how you spend your money. Money mistakes can be corrected, but time is gone forever. David Norris

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •