Saturday, December 22, 2007

Standard ML New Jersey on OpenSUSE 10.3

I needed a version of Standard ML New Jersey to compile PADS but the smlnj version from binary RPM for OpenSUSE was configured to use files in it's build directory.

First, I followed the install instructions. I downloaded the files from the SourceForge file list, and went through the install process but the build was convoluted by assembly source problems.

Next, I took a whack at building from the source RPM.

I searched for "smlnj" at the OpenSUSE Build Service .
Downloaded the source RPM and copy it to /usr/src/packages/SRPMS.

Installed the source RPM with rpm -Uvh smlnj-110.65-10.src.rpm. This will result in smlnj.spec in /usr/src/packages/SPECS and /usr/src/packages/SOURCES.

Build the RPM:
cd to the /usr/src/packages/SPECS and build the SRPM and RPM with rpmbuild -ba smlnj.spec.
On my system this results in:

  • /usr/src/packages/SRPMS/smlnj-110.65-10.src.rpm

  • /usr/src/packages/RPMS/i586/smlnj-100.65-10.i586.rpm

  • /usr/src/packages/RPMS/i586/smlnj-debuginfo-100.65-10.i586.rpm



Install the RPM:
cd to /usr/src/packages/RPMS/i586 and install the binary RPMS with rpm -Uvh smlnj-100.65-10.i586.rpm.

The end result... AHRGGG! Not surprisingly, the install is still looking for library files in /usr/src/packages/BUILD/smlnj-110.65/sml.boot.x86-unix.

Finally, I extracted the source from the RPM (rpm2cpio smlnj-110.65-10.src.rpm | cpio -ivmud), applied the patches and built the whole thing in /usr/local/lib/smlnj then added /usr/local/smlnj/bin to the PATH. Now PADS builds.






Friday, December 21, 2007

My code wasteland

Nearly forgot about the google code project hosting. My code wasteland is at http://code.google.com/p/mark-e-deyoung/.

Actually, I need to fire it up again for another project because my school's CVS is down for maintenance (and probably will be down the entire Christmas holiday.)

Thursday, December 13, 2007

Approximating context-free grammars with regular grammars.

Got a version of the Mohri and Nederhof algorithm working with SWI-prolog. The prolog source (attached here) is from Implementation of Regular Approximation of Context-Free Grammars Through Transformation by Tapani Raiko at Helsinki University of Technology.

Raiko explains how to use the program in his write-up.

A similar idea is implemented by Tanaka Akira in abnf converter. It's a ruby program to convert Augmented BNF to Regexp.

libpcap for cygwin

After much fruitless googling I came accross a modifed version 0.7 of libpcap that was ported by Nevil Brownlee to the cygwin environment. It allows you to use the unix libpcap API in your code and then translates the calls into equivalent WinPcap calls.

The port was conducted by CAIDA Metrics Working Group project to support the port of NeTraMet to MS Windows.

Source is in libpcap-0.7n.tar.gz at the NeTraMet downloads page.

The code is a bit dated (early 2002) and doesn't sync up with the current version of libpcap (v0.9.8). I don't really need it at the moment but it might be worth looking at later.

Wednesday, December 12, 2007

ANSI C way to check if a file is readable

I needed a dead simple ANSI C compatible way to check if a file was readable. So here it is:


int can_read_file(const char * filename) {
FILE *file=NULL;
if ((file = fopen(filename, "r"))) {
fclose(file);
return 1;
}
return 0;
}


You'll have to #include stdio.h

Bro v1.2.1 autotools tweaks

Here are a few tweaks to the Bro 1.2.1 autotools files so it builds in a separate build directory.

In configure.in added the line below so I could see if broccoli was on or off:
echo "  - Building Broccoli:      "${BLD_ON}$broccoli${BLD_OFF}

In src/Makefile.am changed AM_CFLAGS to:
#MED:2007-12-11: binpac.h is generated from binpac.h.in in src/binpac/lib
AM_CFLAGS = -I. -I$(top_srcdir)/src/binpac/lib -I$(top_srcdir)/src -I$(srcdir) -I$(top_builddir) -I$(top_builddir)/src/binpac/lib


In src/Makefile.am and aux/adtrace/Makefile.am changed the LDFLAGS back to AM_LDFLAGS.

In src/binpack/Makefile.am added the line:
#MED:2007-12-11: binpac.h is generated from binpac.h.in in src/binpac/lib
libbinpac_a_CPPFLAGS = -I$(top_builddir)/src/binpac/lib -I$(top_srcdir)/src/binpac/lib

Haven't looked at the broccoli build problems yet. I just configure with --disable-broccoli.

Monday, December 3, 2007

Graphics conversion hacks

I had to convert a slew of images into png and eps so they would work in a LaTeX document. The following proved useful to me:

  • img2eps to covert gif, jpeg, png, tiff, and xpm to eps.

  • giftopng to convert gifs to png. find . -iname "*.gif" -exec giftopng {} \;

  • And a short script to convert jpg to png:

    • #!/bin/bash

    • filename=${1%.jpg}

    • jpegtopnm "$1" | pnmtopng > "$filename.png"




I'm sure there are better ways but these worked for me.

Sunday, December 2, 2007

Installing parallel versions of GCC.

I needed a GCC.version 3.3 c/c++ compiler to build Mical so I googled and found the following guides on how to install parallel versions of GCC.:
I decided to build the compiler using a modified approach. First I downloaded gcc-3.3.6.tar.bz2. Extracted the source in ~/tmp and made a build directory. Then from the build directory configured with: ../gcc-3.3.6/configure --prefix=/opt/gcc/3.3.6 --enable-languages=c,c++. Did make; sudo make install | tee install.log.

Next to test it out I downloaded mical-0.1.0.tar.gz to ~/tmp. I extracted the files and then did export PATH=/opt/gcc/3.3.6/bin:$PATH; configure;make. I didn't do a make install. I'm not comfortable installing something that will need the parallel version of GCC in /local or /opt yet. But I did plink with the algorithm tests and they seemed to work correctly.

For some reason the PATH approach above didn't seem to stick so now I'm building mical by specifying the version of gcc/g++ to as a parameter to the configure script:
configure CC=/opt/gcc/3.3.6/bin/gcc CXX=/opt/gcc/3.3.6/bin/g++.

Saturday, December 1, 2007

Adventures With NetDude.

I've been looking for a tool that would allow me to trim a large pcap file down to specific application level connections and wanted to give NetDude a shot. My "fun" started with getting and build from source because there are not RPMs in a repository for openSUSE at this time.

Building NetDude
First I downloaded the netdude-0.4.8a.tar.gz file and the supporting libnetdude-0.10a.tar.gz and libpcapnav-0.8.tar.gz.

After extracting the arcived files I used configure; make; sudo make install | tee install.log in the following order: libpcapnav, libnetdude, netdude.

Next on to the plugins. I downloaded Essentials Pack, Appdemux, and TCP Filter plugins.

Libnetdude Plugins
AppDemux depends on several of the plugins in the Essentials
Pack and TCP Filter depends on Conntrack and Trace-Set from the Essentials Pack so the first order of business is to get the Essentials Pack built and installed.

I used configure; make; sudo make install | tee install.log in the following order: libnetdude-plugin-essentials, libnetdude-tcpfilter-plugin, libnetdude-appdemux-plugin

Netdude Plugins
The only NetDude plugin at the time of writing is the Traffic Analyzer. Unfortunately for me the version I downloaded did not build without errors. Since this was a quick hack I didn't spend time fixing the build so I was unable to try it out.

On to doing something useful with this...
I was able to demux TCP connections (transport level) with the demux plugin using lndtool (lndtool -r demux -0 wk3.01 -p we3.01_Mon.inside.tcpdump). I'm not sure that I like the way the traces are de-muxed. That is they don't directly translate into a format that I can input into a grammatical inference system. Also, I need to check on the parameters used to decide flow membership. It would probably help me out if I tweaked the demux plugin so it was parameterized.

Sadly the AppDemux plugin isn't working for me and that is the feature I really need at the moment. So my quest for a way to demux application level protocols continues. Going to re-evaluate tcpflow.

Does anyone have other flow reconstruction tools they can recommend?

Friday, November 30, 2007

Building ARPACK on openSUSE 10.3

I needed an installation of ARPACK for FreeMat and couldn't find a convenient way to get it installed on an openSUSE box I was using. So I googled for a bit and came up with the following hack.
  • Make sure fortran is installed: rpm -q gcc-fortran.
  • Download the RPM from ATrpms.
  • Extract files from rpm using (see ):
    rpm2cpio arpack-2.1.7.src.rpm | cpio -ivmud
  • Edit the .spec file so the BuildRequires statement for gfortran is for gcc-fortran instead of gcc-gfortran. (That took several "rpm -q --provides gcc-gfortran" not found queries before I realized the package is named gcc-fortran on openSUSE.)
  • Build the RPM: rpmbuild -ba arpack.spec
  • Install the RPMs. In my case the RPMS were in /usr/src/packages/RPMS/i586. I installed the arpack-2.1-7.i586.rpm, then arpack-devel-2.1-7.i586.rpm and finally arpack-static-2.1-7.i586.rpm.




Unfortunately, FreeMat still does not pick up the version of ARPACK built above when running configure. I suspect it's something to do with the differences between f77 and gfortran. Check here for some fortran relevant autoconf macros.

FreeMat checks the fortran configuration with AC_PROG_F77 and AC_F77_LIBRARY_LDFLAGS in it's configure.in and uses several AC_F77_XXX macros in acinclude.m4 and checks for ARPACK with AC_F77_FUNC(znaupd).

Have to get back to this one later.

Sunday, November 25, 2007

-mtune=native : GCC 4.2.x command line option

Other optimization options are explained in detail at:
Using the GNU Compiler Collection Section 3.10

  • To use it on an automake/autoconf project in KDevelop:














Qfsm configure.in hacks

After a bit of effort I finally tweaked the configure.in for Qfsm so it works with a separate build directory. I needed this so I could build the project inside KDevelop.

The solution I used was to hack a large portion of the autoqt m4 macros into Qfsm's configure.in. A similar approach was used here.

A better solution would probably be to update autoqt so it works with a dual Qt3/Qt4 installation. There was also some useful info at Qtnode. Also, FreeMat's configure.in might be a good source for autoconf macros that are Qt4 related.

par2cmdline fix for gcc 4.x

Stumbled across this E-mail which includes a patch that gets the par2cmdline from Parchive compiling with gcc 4.x. Up to this point I've used QuickPar running under wine.

What about BloGTK?

What about BloGTK? I didn't try it because it wasn't in an openSUSE repository.

Test Post from drivel

This is a test post from drivel. Drivel didn't have the correct server at startup. I actually found the server name for the Blogger API by trying out Gnome Blog. (The server name is http://www.blogger.com/api/RPC2).

Test Post from Gnome Blog

Test from Gnome Blog

This is a test post from Gnome Blog.

Bro v1.2.1 and new ClamAV versions

As mentioned here, the version 1.2.1 stable source does not build correctly if ClamAV is installed. Bro 1.2.1 is using the cl_scannbuf function which is not exported in newer versions of ClamAV.

To work around this I edited the configure.in file so it checks for the cl_scanbuff function and does not configure ClamAV support if it is not exported by ClamAV.

In the Bro configure.in I edited the ClamAV checks to read:
# Libclamav
have_libclamav=Yes
AC_CHECK_HEADERS([clamav.h],,have_libclamav=No)
AC_CHECK_LIB(clamav,cl_scanbuff,,have_libclamav=No)
#AC_CHECK_LIB(clamav,cl_retdbdir,,have_libclamav=No)

Then re-generated the configure file.

One possible "real fix" would be to re-write the FileAnalyzer functions to create temporary files and use cl_scanfile to scan the temporary file.