Saturday, December 22, 2007

Standard ML New Jersey on OpenSUSE 10.3

I needed a version of Standard ML New Jersey to compile PADS but the smlnj version from binary RPM for OpenSUSE was configured to use files in it's build directory.

First, I followed the install instructions. I downloaded the files from the SourceForge file list, and went through the install process but the build was convoluted by assembly source problems.

Next, I took a whack at building from the source RPM.

I searched for "smlnj" at the OpenSUSE Build Service .
Downloaded the source RPM and copy it to /usr/src/packages/SRPMS.

Installed the source RPM with rpm -Uvh smlnj-110.65-10.src.rpm. This will result in smlnj.spec in /usr/src/packages/SPECS and /usr/src/packages/SOURCES.

Build the RPM:
cd to the /usr/src/packages/SPECS and build the SRPM and RPM with rpmbuild -ba smlnj.spec.
On my system this results in:

  • /usr/src/packages/SRPMS/smlnj-110.65-10.src.rpm

  • /usr/src/packages/RPMS/i586/smlnj-100.65-10.i586.rpm

  • /usr/src/packages/RPMS/i586/smlnj-debuginfo-100.65-10.i586.rpm

Install the RPM:
cd to /usr/src/packages/RPMS/i586 and install the binary RPMS with rpm -Uvh smlnj-100.65-10.i586.rpm.

The end result... AHRGGG! Not surprisingly, the install is still looking for library files in /usr/src/packages/BUILD/smlnj-110.65/sml.boot.x86-unix.

Finally, I extracted the source from the RPM (rpm2cpio smlnj-110.65-10.src.rpm | cpio -ivmud), applied the patches and built the whole thing in /usr/local/lib/smlnj then added /usr/local/smlnj/bin to the PATH. Now PADS builds.

Friday, December 21, 2007

My code wasteland

Nearly forgot about the google code project hosting. My code wasteland is at

Actually, I need to fire it up again for another project because my school's CVS is down for maintenance (and probably will be down the entire Christmas holiday.)

Thursday, December 13, 2007

Approximating context-free grammars with regular grammars.

Got a version of the Mohri and Nederhof algorithm working with SWI-prolog. The prolog source (attached here) is from Implementation of Regular Approximation of Context-Free Grammars Through Transformation by Tapani Raiko at Helsinki University of Technology.

Raiko explains how to use the program in his write-up.

A similar idea is implemented by Tanaka Akira in abnf converter. It's a ruby program to convert Augmented BNF to Regexp.

libpcap for cygwin

After much fruitless googling I came accross a modifed version 0.7 of libpcap that was ported by Nevil Brownlee to the cygwin environment. It allows you to use the unix libpcap API in your code and then translates the calls into equivalent WinPcap calls.

The port was conducted by CAIDA Metrics Working Group project to support the port of NeTraMet to MS Windows.

Source is in libpcap-0.7n.tar.gz at the NeTraMet downloads page.

The code is a bit dated (early 2002) and doesn't sync up with the current version of libpcap (v0.9.8). I don't really need it at the moment but it might be worth looking at later.

Wednesday, December 12, 2007

ANSI C way to check if a file is readable

I needed a dead simple ANSI C compatible way to check if a file was readable. So here it is:

int can_read_file(const char * filename) {
FILE *file=NULL;
if ((file = fopen(filename, "r"))) {
return 1;
return 0;

You'll have to #include stdio.h

Bro v1.2.1 autotools tweaks

Here are a few tweaks to the Bro 1.2.1 autotools files so it builds in a separate build directory.

In added the line below so I could see if broccoli was on or off:
echo "  - Building Broccoli:      "${BLD_ON}$broccoli${BLD_OFF}

In src/ changed AM_CFLAGS to:
#MED:2007-12-11: binpac.h is generated from in src/binpac/lib
AM_CFLAGS = -I. -I$(top_srcdir)/src/binpac/lib -I$(top_srcdir)/src -I$(srcdir) -I$(top_builddir) -I$(top_builddir)/src/binpac/lib

In src/ and aux/adtrace/ changed the LDFLAGS back to AM_LDFLAGS.

In src/binpack/ added the line:
#MED:2007-12-11: binpac.h is generated from in src/binpac/lib
libbinpac_a_CPPFLAGS = -I$(top_builddir)/src/binpac/lib -I$(top_srcdir)/src/binpac/lib

Haven't looked at the broccoli build problems yet. I just configure with --disable-broccoli.

Monday, December 3, 2007

Graphics conversion hacks

I had to convert a slew of images into png and eps so they would work in a LaTeX document. The following proved useful to me:

  • img2eps to covert gif, jpeg, png, tiff, and xpm to eps.

  • giftopng to convert gifs to png. find . -iname "*.gif" -exec giftopng {} \;

  • And a short script to convert jpg to png:

    • #!/bin/bash

    • filename=${1%.jpg}

    • jpegtopnm "$1" | pnmtopng > "$filename.png"

I'm sure there are better ways but these worked for me.

Sunday, December 2, 2007

Installing parallel versions of GCC.

I needed a GCC.version 3.3 c/c++ compiler to build Mical so I googled and found the following guides on how to install parallel versions of GCC.:
I decided to build the compiler using a modified approach. First I downloaded gcc-3.3.6.tar.bz2. Extracted the source in ~/tmp and made a build directory. Then from the build directory configured with: ../gcc-3.3.6/configure --prefix=/opt/gcc/3.3.6 --enable-languages=c,c++. Did make; sudo make install | tee install.log.

Next to test it out I downloaded mical-0.1.0.tar.gz to ~/tmp. I extracted the files and then did export PATH=/opt/gcc/3.3.6/bin:$PATH; configure;make. I didn't do a make install. I'm not comfortable installing something that will need the parallel version of GCC in /local or /opt yet. But I did plink with the algorithm tests and they seemed to work correctly.

For some reason the PATH approach above didn't seem to stick so now I'm building mical by specifying the version of gcc/g++ to as a parameter to the configure script:
configure CC=/opt/gcc/3.3.6/bin/gcc CXX=/opt/gcc/3.3.6/bin/g++.

Saturday, December 1, 2007

Adventures With NetDude.

I've been looking for a tool that would allow me to trim a large pcap file down to specific application level connections and wanted to give NetDude a shot. My "fun" started with getting and build from source because there are not RPMs in a repository for openSUSE at this time.

Building NetDude
First I downloaded the netdude-0.4.8a.tar.gz file and the supporting libnetdude-0.10a.tar.gz and libpcapnav-0.8.tar.gz.

After extracting the arcived files I used configure; make; sudo make install | tee install.log in the following order: libpcapnav, libnetdude, netdude.

Next on to the plugins. I downloaded Essentials Pack, Appdemux, and TCP Filter plugins.

Libnetdude Plugins
AppDemux depends on several of the plugins in the Essentials
Pack and TCP Filter depends on Conntrack and Trace-Set from the Essentials Pack so the first order of business is to get the Essentials Pack built and installed.

I used configure; make; sudo make install | tee install.log in the following order: libnetdude-plugin-essentials, libnetdude-tcpfilter-plugin, libnetdude-appdemux-plugin

Netdude Plugins
The only NetDude plugin at the time of writing is the Traffic Analyzer. Unfortunately for me the version I downloaded did not build without errors. Since this was a quick hack I didn't spend time fixing the build so I was unable to try it out.

On to doing something useful with this...
I was able to demux TCP connections (transport level) with the demux plugin using lndtool (lndtool -r demux -0 wk3.01 -p we3.01_Mon.inside.tcpdump). I'm not sure that I like the way the traces are de-muxed. That is they don't directly translate into a format that I can input into a grammatical inference system. Also, I need to check on the parameters used to decide flow membership. It would probably help me out if I tweaked the demux plugin so it was parameterized.

Sadly the AppDemux plugin isn't working for me and that is the feature I really need at the moment. So my quest for a way to demux application level protocols continues. Going to re-evaluate tcpflow.

Does anyone have other flow reconstruction tools they can recommend?