On November 22nd, I discovered two vulnerabilities in sites based on Drupal Core 7.9 with default configuration.  These were:

• an automatic remote phishing vulnerability (automated email sent from drupal user’s website can contain links to an attacker’s site!)
  Suggested CVSS v2.0: AV:N/AC:M/Au:N/C:P/I:P/A:N/E:POC/RL:U/RC:C (What’s that?)
  Suggested Drupal Security Risk Level: Moderately Critical (3 of 5)
• a potential XSS vulnerability (High Access Complexity… attacker must have MITM or control of a Proxy)
  Suggested CVSS v2.0: AV:A/AC:H/Au:N/C:P/I:P/A:N/E:POC/RL:U/RC:C
  Suggested Drupal Security Risk Level: Less Critical (2 of 5)

The technical details of this vulnerability have been removed until further notice from the Drupal security team 😉

I decided to write a simple web spider in order to learn Python, and to generate a list of urls for webserver benchmarking & stress testing… and so Spyder was born. It is written in Python 3.

Github:

https://github.com/trinitronx/Spyder

Spyder:

a simple web spider written in Python 3

When called on a url, it will spider the pages and any links found up to the depth specified.
After it's done, it will print a list of resources that it found.
Currently, the resources it tries to find are:

images   -  any images found on the page (ie: <img src="THIS"/>)
styles   -  any external stylesheets found on the page.  CSS included via '@import' is currently only supported if within a style tag!
(ie: <link rel="stylesheet" src="THIS"/>  OR <style>@import url('THIS');</style> )
scripts  -  any external scripts found in the page (ie: <script src="THIS"> )
links    -  any urls found on the page.  'Fragments' are discarded. (ie: <a href="THIS#this-is-a-fragment"> )
emails   -  any email addresses found on the page (ie: <a href="mailto:THIS"> )

An example script for doing something like this, 'www-benchmark.py', is included.  It uses apache benchmark as an example.
Eventually I'll be experimenting with 'siege' for benchmarking & server stress-testing.

NOTE: Currently the spider can throw exceptions in certain cases (mainly character encoding stuff, but there are probably other bugs too)
Getting *working* character encoding detection is a goal, and is sorta-working... ish?  Help in this area would be appreciated!
Filtering the results by domain is almost working too

Recently I found myself with shell access on a host without a git client installed, and also without the necessary build tools to compile it (gcc, make, etc…).  However, I did have access to a machine with the same processor architecture (in fact, the same exact processor).  If you mange to find yourself in this situation, what you need to do is compile git statically on the machine which does have gcc and make installed.

What’s static mean?

In the compilation process, the compiler usually must link in shared dynamic libraries.  In Windows, these are called .dll files, and in linux they are usually .so files.  These libraries are simply collections of compiled functions that can be reused for many different programs that require them to do a specific task.  By sharing these libraries, the computer can save RAM and hard drive space by only requiring one copy of a specific library to be present for many programs that have been compiled for it.

In order to avoid unexpected behavior, a program must sometimes be compiled with a specific version of a dynamic library in mind.  This isn’t always true, but in order to ensure portability and expected behavior it’s important.  In linux, your package manager takes care of making sure these version dependencies are satisfied correctly.  However, this can be a problem when you’re stuck on a machine for which you have no control over.  You can’t know for sure what version of a specific library is installed, or when it will be upgraded.  You could build your program on another machine with the same processor architecture, and with the same libraries and then just copy it over, but that leaves room for breakage down the line in case your target machine’s libraries are upgraded, or if any of the libraries on the target machine are compromised or replaced by malicious versions.  Here’s where statically building comes in handy.

How to build git with static linking

This example assumes you already have access to a machine with build tools already installed.  This build machine is also assumed to have the same processor architecture as your target machine.  You can find the latest stable release of git at: http://git-scm.com

Here are the steps to take:

1) On your build machine, get the source code for git, unpack it, and go into the source directory:

$ wget http://kernel.org/pub/software/scm/git/git-1.7.2.2.tar.bz2
$ tar -jxvf git-1.7.2.1.tar.bz2
$ cd git-1.7.2.2

2) Configure git to install to a predetermined directory, with static linking.  (Replace /home/myuser/git-static
with whatever path you want):

$ mkdir /home/myuser/git-static
$ ./configure --prefix=/home/myuser/git-static CFLAGS="${CFLAGS} -static"

3) Make git and install it to your directory

$ make
# Optional: make man pages and documentation
# Assumes you have asciidoc and other required programs on your build machine
$ make doc
# Install to your target directory
$ make install

4) Assuming all went well, now you can pack it up into a tarball for transfer to your target machine.

$ cd /home/myuser/git-static/
$ tar -cjvf git-static.tar.bz2 ./*

5) Copy it over to your target machine however you can, and unpack it to your home directory there with tar:

$ cd ~
$ tar -jxvf git-static.tar.bz2

# Check that you can use git.
# If you can't, make sure that your ~/bin directory exists in your environment's $PATH
$ git --version

6) Enjoy!