Comparing two directory structures

 Sometimes when I'm copying important directory structure (with cp -rp) I want to make sure that each and every file was copied correctly, so I compare source directory structure with it's copy this way:

DIRR=<copied-dir-structure>; diff -u <(cd <path-where-source-dir-lies>; find "$DIRR" -type f -exec ls -al {} \;) <(cd <path-where-copied-dir-lies>; find "$DIRR" -type f -exec ls -al {} \;)

SSH local port forwarding

Recently I advised on serverfault.com how to do local port forwarding with ssh:

On remote machine with sshd server, start a service that you would like to give access to:

 echo "hello" | nc -l -p 2222

On local machine initiate port forwarding:

 ssh -L 1234:localhost:2222 root@remoteserver.com

try it (from local machine):

 nc localhost 1234

If you will see "hello" that means port forwarding worked as expected.

 Use cases for this kind of functionality:
 - tunneling otherwise insecure TCP traffic
 - accessing services behind firewall when only sshd access is available

Hardening LAMP stack (part 1)

This article begins series of posts which are meant to create very SECURE but also easy to manage web server on Linux box. I start from default Debian 6.0 Stable installation and move on configuring various parts of the system.

We start from:

APACHE - PUTTING IT IN CHROOT JAIL

# compile apache2
  ./configure --prefix=/usr/local/apache --disable-userdir --disable-include
 make
 make install

# prepare new root environment for apache
  mkdir -p /srv/chroot/apache/

# create /usr/local directory in new environment
  mkdir -p /srv/chroot/apache/usr/local

# move built and installed apache to new environment
  mv /usr/local/apache /srv/chroot/apache/usr/local

# link it under old installation directory for easy updates
  ln -s /srv/chroot/apache/usr/local/apache /usr/local/apache

# stisfy all apache's library dependencies by copying it to new /lib directory
  mkdir -p /srv/chroot/apache/lib

 ldd /chroot/apache/usr/local/apache/bin/httpd

 (copy it to new /lib)

# copy strace (and it's dependencies) for debug purposes (remove it in production)
  mkdir -p /srv/chroot/apache/bin

 cp `which strace` /srv/chroot/apache/bin

 ldd `which strace`

 (copy it to new /lib)

# first launch (with strace), probably something will be missing
 chroot /srv/chroot/apache /bin/strace /usr/local/apache/bin/httpd

# name resolution
  mkdir -p /srv/chroot/apache/etc

 cp /etc/nsswitch.conf /srv/chroot/apache/etc/  #make sure that access to passwd is set to 'files'

 cp /lib/libnss_files.so.2 /srv/chroot/apache/lib

# dns name resolution
  cp /lib/libnss_dns.so.2 /srv/chroot/apache/lib

 cp /etc/hosts /srv/chroot/apache/etc

 cp /etc/resolv.conf /srv/chroot/apache/etc

# create special devices which apache uses
  mkdir /srv/chroot/apache/dev

 mknod -m 666 /srv/chroot/apache/dev/null c 1 3

 mknod -m 666 /chroot/apache/dev/zero c 1 5

 mknod -m 644 /chroot/apache/dev/random c 1 8

# create /tmp directory
  mkdir /srv/chroot/apache/tmp

 chmod +t /srv/chroot/apache/tmp

 chmod 777 /srv/chroot/apache/tmp

# create /etc/passwd & /etc/group files
  echo "www-data:x:33:33:Apache:/:/sbin/nologin" > /srv/chroot/apache/etc/passwd

 echo "www-data:x:33:" > /srv/chroot/apache/etc/group

# prepare /var directory in new environment (in jail) for test-site
  mkdir -p /srv/chroot/apache/var/www

 cd /srv/chroot/apache/var/www

 mkdir -p test-site/bin

 mkdir -p test-site/cgi-bin

 mkdir -p test-site/data

 mkdir -p test-site/htdocs

 mkdir -p test-site/logs

# link it for easier updates of future web applications

 ln -s /srv/chroot/apache/var/www /var/www

# second launch, now it should start, if not, use strace to find out why
 chroot /srv/chroot/apache /usr/local/apache/bin/httpd

APACHE - PRELIMINARY HARDENING 

# proper files & directories privilages
  chown -R root:root /usr/local/apache

 find /srv/chroot/apache/usr/local/apache -type d | xargs chmod 755

 find /srv/chroot/apache/usr/local/apache -type d | xargs chmod g-s

 find /srv/chroot/apache/usr/local/apache -type f | xargs chmod 644

 find /srv/chroot/apache/usr/local/apache/bin -type f | xargs chmod 744

# configuration and logs can be read only by root
  chmod -R go-r /srv/chroot/apache/usr/local/apache/conf

 chmod -R go-r /srv/chroot/apache/var/www/site-test/logs

Software flaw #1: Signed conversion vulnerability

To better understand this kind of flaw let's consider following code:

#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
#include <fcntl.h>
#include <unistd.h>

int read_from_file()
{
    char size;
    char buf[32];
    buf[1] = 0;
    int fd;

    if((fd = open("fileSubverted", O_RDONLY)) == -1) {
        perror("[open file]");
        return -1;
    }

    if(read(fd, &size, sizeof(char)) == -1) {
        perror("[read size]");
        return -1;
    }
    if(size > 32) {
        fprintf(stderr, "Too big size\n");
        return -1;
    }

    unsigned char s = size;
    if(read(fd, buf, s) == -1) {
        perror("[read content]");
        return -1;
    }

    return 0;
}

int main()
{
    if(read_from_file() == -1)
        exit(1);

    return 0;
}
 

* the above code should work without an explicit casting of size variable to unsigned int type (the bolded line) but in my environment it doesn't - without this cast I get EDEFAULT (Bad address) error from read function.

Problem lies in implicit conversion of 'size' variable form char type to size_t (unsigned int) type, if user provides negative value for 'size' (let's say -2) it will pass the 'if(size > 32)' check and it will be converted to size_t type while invoking read(...) function which will be 0x000000fe in 2's complement representation. Provided that file is bigger than 32 bytes, 'buf' buffer overflow will occur.

Attack:

Commands below will generate file that will cause application to crash (Segmentation fault), due to buffer overflow which overwrites return address from read_from_file function:

echo -e '\xfe'`perl -e 'print "A"x250'` > maliciousFile

or in pure Bash:

echo -e '\xfe'`printf 'A%.0s' {1..255}` > maliciousFile

Countermeasures:

1) Best in this situation is to declare size variable as unsigned char

2) Change validation condition to: if( size < 0 || size > 32) { //generate error }

Simple intrusion detection using standard UNIX commands

After I'm done with configuring and hardening Linux server I'm doing "reference of the server" by writing output from several crucial commands to my laptop:

ps -x -o user,command

netstat -natuw
find / -uid 0 -perm -4000 -print

find / -size +10000k -print

crontab -u root -l

Then using this simple script on regular basis, I have chance to detect compromised boxes:

#!/bin/bash

function usage
{
    echo "Usage: $0 <hostname>"
}

function getSrvAddr
{
    case $1 in
        'hostname1' ) echo "ssh user@srv1.addr"
            ;;
        'hostname2' ) echo "ssh user@srv2.addr"
            ;;
        * ) echo -n ""
    esac
}

HOSTNAME=$1
REFERENCE_PATH=~/lab/configs/myServers/${HOSTNAME}
SERVER=`getSrvAddr "$HOSTNAME"`
CMDS=(
    "ps -x -o user,command"
    "netstat -natuw"
    "find / -uid 0 -perm -4000 -print"
    "find / -size +10000k -print"
    "crontab -u root -l"
)

if [ -z "$SERVER" ]; then
    usage
    exit 1
fi

i=0
for FILE in ${REFERENCE_PATH}/*; do
    echo
    echo "########################################################################"
    echo  ${SERVER} ${CMDS[$i]}
    echo "########################################################################"
    echo
    diff -w -u <(sort $FILE) <(${SERVER} ${CMDS[$i]} 2> /dev/null | sort)
    i=$((i+1))
done

Basically, what the script does is compare server's initial output from several UNIX commands with it's current output. Using this script I can easily extend it for more commands and more servers. It's very simple method of intrusion detection (and by no mean 100% reliable!) but it's good addition to other mechanisms that should be in place.

What works in software security testing

... at least in opinion's of two companies that treat software security more and more seriously (Microsoft) and lives from software (in)security (Matasano Chargen):

Here's an interesting series of articles from Microsoft

Here's much shorter article from Matasano

Introduction to fuzzing

Fuzzing is an effective technique for finding potential security bugs in software. I will show two examples of fuzzing using SPIKE and SPIKEfile frameworks.

SPIKE is framework for fuzzing network protocols. Let's say that we want to o fuzz UDP server that communicates using very simple protocol:

[length][string]

So in the packet there are sent: length (32 bit integer) and string (of earlier sent length). To fuzz it we create following SPIKE script (mz.spk):

s_blocksize_string("fileformat", 4);

s_block_start("fileformat");

s_string(" ");

s_string_variable("abc");

s_block_end("fileformat");

Now we can execute generic_send_udp fuzzer from SPIKE framework:

$ ./generic_send_udp 127.0.0.1 81 mz.spk 0 0 10

We can observe packets sent by the fuzzer with tcpdump:

$ sudo tcpdump -i lo -n -X udp port 81

SPIKEfile is fuzzer based on SPIKE for file formats fuzzing. Let's consider badly vulnerable, sample application that opens a file (with format identical with packet format that we discussed above):

 int main(int argc, char **argv)

{

    FILE *f = fopen(argv[1], "r");

    int num;

    char array[650] = {0};

    fscanf(f, "%d %s", &num, array);

    fclose(f);

    printf("num: %d, array: %s\n", num, array);

    return 0;

}

As we can see, data from the file is copied to array without any length checking. We will fuzz it the same way as previous (using the same SPIKE script) but with SPIKEfile fuzzer:

$ ./SPIKEfile -f file.mz mz.spk 0 0 "./vuln_app %FILENAME%"

On each crash SPIKEfile will generate report with stack dump.

This two simple examples show philosophy behind fuzzing using SPIKE based fuzzers. When we have more complicated network protocol or file format, our SPIKE script is much longer (the better we now fuzzed protocol/file format the more effective fuzzing becomes) but the approch is the same.