Mark E. DeYoung's Miscellaneous Stuff

Extract files from executable installation package/Windows installers

2012-01-15T09:44:00.001-06:00

I needed to extract a couple of files that were embedded into an executable installation package. After a few fruitless googles on recording Windows installers, and automated software removal I finally came across Universal Extractor. I used it to extract the files from an InstallShield package in an .exe file.

KODU Game Lab

2011-12-28T19:04:00.001-06:00

I was looking for an approachable, kid friendly, programming environment and found KODU Game Lab from Microsoft Research. Seems fun so far. Also has an online community web site (KODU Game Lab Community) where you can share games. The system is built around Microsoft’s XNA for Windows PCs and XBox 360s. The GUI works with a keyboard and mouse on a Windows 7 PC but seems like it would be a better fit with an XBox 360 Controller for a Windows PC.

Games with constructible/destructible environments

2011-11-27T08:57:00.001-06:00

If you enjoy Minecraft you might want to check out Ace of Spades or Voxatron. Both are voxel based constructible/destructible environments.

A# (ADA for .NET) with Visual Studio Integration

2011-10-27T04:59:00.001-05:00

I decided I really needed to build RAPTOR from source. The program is written in C# and and Ada targeting .NET (A#). Dr. Carlisle released his last version of the A# compiler and a Visual Studio 2005 plugin via SourceForge in May 2006. I installed the tool set into Visual Studio 2005 as described here on Windows 7 64-bit. Although the integration installed it did not work after I patched VS2005 for Windows 7 (KB928957, KB929470).

Since 2006 maintenance of the A# code was taken over by AdaCore. They have released a downloadable version of GNAT that targets .NET 2.0 in the GNAT GPL packages at their LIBRE page. Look for the package that targets dotnet-windows. At the time of writing it was in a file named “gnat-gpl-2011-dotnet-windows-bin.exe” dated 11 May 2011.

The dotnet-windows package includes Visual Studio integration. Appears to be built around ProjectAggregator2 technology from the now defunct Visual Studio Integration Program (VSIP) SDK. It successfully integrated with Visual Studio 2010 when I installed GNAT. It might be possible to install the GNAT integration into the Visual Studio 2010 Shell (Integrated) Redistributable Package if you don’t have a full up VS2010 install. My understanding is that VS2010 Shell can be used to support languages like F# and Python.

Anyways, after I installed GNAT GPL for dotnet I was able to open the raptor.sln in Visual Studio and start tracking down dependencies (missing dlls, files, x64 issues, etc.).

Java Decompiler project

2011-08-21T10:39:00.001-05:00

I wanted to check out some compiled Java byte code from a .class file. Unfortunately, the tool I historically used (JAD via the DJ Java Decompiler) didn’t handle Java version 5 byte code. Luckily for me a project by Emmanuel Dupuy has released a similar tool called JD-GUI.

MySQL convert time zone from local to UTC

2010-12-05T19:01:00.001-06:00

Had to convert a slew of times form local time to UTC time in an MySQL database on a Windows box. I had pushed a large number of Windows event logs into a MySQL database with LogParser.

While the event log files have the time in UTC format I found that LogParser renders the time as system local time of the computer where it is running. The program doesn’t have any way to know the time zone of the computer where the event log files originated. I should have originally used TO_UTCTIME in the LogParser queries.

So… on to using MySQL CONVERT_TZ. Unfortunately, my Windows install of MySQL did not include time zone information needed by the CONVERT_TZ function. I had to download and install the Time zone description tables for the function to work correctly.

The CONVERT_TZ function works with POSIX defined time zone names To avoid the ambiguity of common Time zone abbreviations. For example I used

UPDATE table_x SET TimeGenerated=CONVERT_TZ(TimeGenerated, ‘America/Chicago’,’UCT’);

To convert a US Central time to UCT/Zulu time.

UPDATE table_x SET TimeGenerated=CONVERT_TZ(TimeGenerated, ‘America/New_York’,’UCT’);

To convert a US Eastern time to UCT/Zulu time.

Now back to event correlation.

Exchange Server Setup Progress.log time is in system local time

2010-11-30T19:55:00.001-06:00

I need to know what time was recorded in the “Exchange Server Setup Progress.log” by Exchange 2003 setup. This is one of those little factoids that Google failed me on….

So I set up a Windows 2003 server in VMWare and ran the Exchange 2003 setup. It turns out that the log times are recorded in system local time (not Zulu time).

Here’s a couple of slightly related links that talk about the log file and events recorded by the setup program for Exchange 2003:

Tips on Troubleshooting Using the Exchange Server Setup Progress Log

Post-Installation Steps for Exchange Server 2003

Play FLAC with Windows Media Player

2010-11-12T05:35:00.001-06:00

Just tried out the xiph.org DirectShow plugin to play some FLAC in Windows Media Player. Seemed to work for playback but I’m not sure how to do tag editing yet.

Windows MultiPoint Mouse SDK

2010-11-11T06:27:00.001-06:00

While I’m not a huge fan of thin client type solutions like the MultiPoint server the Windows MultiPoint Mouse SDK seems like it has some possibilities. The tic-tac-toe sample in the SDK is pretty fun. Would probably add some fun to other simple desktop games if they supported multiple mice.

You can also get the SDK as a Visual Studio 2010 Extension.

Log Parser Charts on 64-bit Windows Boxes

2010-11-09T20:08:00.001-06:00

I need to generate some charts from Windows Event Log files on a couple of 64-bit Win2008 and Win7 boxes and got the following error from Log Parser:

LogParser.exe : Error creating output format "CHART": This output format requires a licensed Microsoft Office Chart Web Component to be installed on the local machine

A quick web search revealed that Log Parser requires a copy Microsoft Office Web Components (OWC) to generate charts. I downloaded a copy, installed, and Log Parser started producing charts.

The OWC install includes some programming references that can be located with the help of KB319793. The documentation is in “C:\Program Files (x86)\Common Files\microsoft shared\Web Components\10\1033” on my Win7 x64 box. Or you can check out the ChartSpace Object Model documentation on MSDN.

After spelunking the LogParser and OWC docs it appears OWC supports GIF, JPG and PNG output. Unfortunately, LogParser only supports GIF and JPG charts. I’d really prefer a scalable vector format.

HDMI CEC interface for PC

2010-11-06T13:14:00.001-05:00

The state of HDMI Consumer Electronics Control (CEC) interfaces for PCs seems pretty dismal. Neither ATI or nVidia currently have CEC support built into their graphics devices with HDMI interfaces. So you can’t directly send or receive CEC data over the CEC-line (pin 13/17).

One bright spot is the RainShadow HDMI-CEC to USB bridge/converter. Supposedly it’s also been hacked into a regular video card.

Classic DOS and Windows games…

2010-11-05T15:34:00.001-05:00

Well if you’re pining for the glory days of DOS games go get D-Fend Reloaded (a great DOSBox front-end). Then head over to RGB Classic Games.

Or you could check out GOG.com for some classic DOS and Windows games at a reasonable price.

Convert Apple Disk Image (.dmg) to ISO Image (.iso)

2010-11-05T15:20:00.001-05:00

Found a couple of ways to turn a Apple Disk Image (.dmg) into an ISO Image (.iso). First was dmg2img, second was DMGExtractor. I ended up using DMGExtractor probably because the author posted the source at SourceForge.

I wanted to use the GUI interface so I added the –gui flag to the Windows Batch file in the bin directory.

@echo off
setlocal
set DMGX_CP="%~dp0..\lib\dmgextractor.jar"
java -cp %DMGX_CP% org.catacombae.dmgextractor.DMGExtractor -startupcommand dmgx -gui %1 %2 %3 %4 %5 %6 %7 %8 %9
endlocal

Create 32-bit ODBC DSN on 64-bit Windows

2010-11-03T10:28:00.002-05:00

Needed to create a 32-bit compatible ODBC DSN to output Log Parser results from a 64-bit version of Windows to a MySQL instance.
Log Parser gave me an architecture mismatch error:

Error connecting to ODBC Server
SQL State: IM014
Native Error: 0
Error Message: [Microsoft][ODBC Driver Manager] The specified DSN contains
an architecture mismatch between the Driver and Application

NOTE: Log Parser did not work well with the more modern 32-bit version 5.1 ODBC Connector so I had to use the 32-bit version 3.51 ODBC Connector.
Then used the [WindowsDir]\SysWOW64\odbcad32.exe to create a 32-bit System DSN. (See the MS KB942976 for more detail.)
And finally, started pushing event logs into the MySQL database with the DSN created above. (Where X is the server name and XXX is the DSN name.)

LogParser “SELECT * INTO Events FROM *.evtx” –server:X –i:EVT –o:SQL –dsn:XXX –fixColNames:ON –maxStrFieldLen:2048

Or if you prefer from PowerShell. (Where X is the server name and XXX is the DSN name.)

dir *.evtx | foreach-object {LogParser “SELECT * INTO Events FROM ‘$_.’” –server:X –i:EVT –o:SQL –dsn:XXX –fixColNames:ON –maxStrFieldLen:2048}

I used the PowerShell option because the event logs were rather large (~500K events each) so the connector kept running out of RAM with the *.evtx source. Also, I needed a specific timeframe from the event logs so I added a where clause to get:

dir *.evtx | foreach-object {LogParser “SELECT * INTO Events FROM '$_.' WHERE TimeGenerated > TIMESTAMP('2010-07-05 00:00:00','yyyy-MM-dd hh:mm:ss') AND TimeGenerated < TIMESTAMP('2010-07-21 00:00:00','yyyy-MM-dd hh:mm:ss')” –server:X –i:EVT –o:SQL –dsn:XXX –fixColNames:ON –maxStrFieldLen:2048}

Ogawa’s Software Evolution Storylines

2010-10-09T06:26:00.001-05:00

Nice software visualization experiment from Michael Ogawa.

Windows 7 Activation Error 0x8007232B

2010-09-25T15:50:00.002-05:00

If you get the error 0x8007232B when you try to activate Windows 7 installed from volume licensed media you can run the following from an elevated powershell window:

slmgr.vbs –ipk [product key]
slmgr.vbs –ato

I installed the Elevation PowerToys for Windows Vista to simplify getting an elevated powershell window. Unfortunately, the powertoys didn't seem to work with Win7 so I had to start the powershell by ctrl-clicking the link and selecting run as Administrator.

Offline spelunking of Active Directory - setup

2010-09-21T19:23:00.001-05:00

Needed to extract some historical information from Active Directory (AD) backups stored in NTBackup BKF files. First challenge was extracting the NTDS.DIT from the BKF. How do I get that snapshot loaded up so I can query the backup? I didn’t find a way to directly query for results from the NTDS.DIT without loading it into a running Domain Controller (DC) instance.

My approach is loosely modeled on the technique outlined at technet to “mount” a snapshot in VMWare Workstation on a server configured as a DC that closely models the forest structure of the targeted AD backup. I’d prefer to use a clone or image from the actual DCs but I can not disturb the online production environment.

I set up a VMWare team consisting of a Win 2008 R2 x64 server configured as a DC and a Vista x86 32-bit Workstation with Windows Server 2008 Remote Server Administration Tools; Active Directory Explorer and some custom/hand built ADSI based tools. I’ll be pushing tests of the custom tools from Visual Studio on the host to the virtualized Vista box in the team because I don’t want to directly mess with the virtualized DC. Now to get the directory backup loaded so I can do some offline spelunking.

Lightweight Portable Security – access .mil portal from public computer

2010-09-21T19:19:00.001-05:00

SPI has put out a bootable live Linux ISO with CAC support. It’s called “Lightweight Portable Security”. Very nice (in my opinion) if you need to get onto a .mil portal like the AF Portal from a public computer. Downloading it now to setup in a virtual machine and USB CAC reader. Hopefully it works out.

General Robotics, Automation, Sensing and Perception (GRASP) Laboratory

2010-09-20T17:29:00.001-05:00

Recommend surfing the very cool robotics research from the GRASP lab.

MP3 tagging - foobar2000

2010-09-20T17:13:00.001-05:00

Just noticed how jacked up my MP3 tags were when I copied some mp3s over to my phone. Lifehacker had some recommendations for MP3 tagging for Windows boxes. I ended up going with foobar2000.

Install MySQL Connector/J for use from Eclipse

2010-09-19T15:42:00.001-05:00

I already had a JDK, Eclipse, and MySQL installed but I needed to install the MySQL Connector/J so I could use it with the Eclipse SQL Explorer plugin to access the database with JDBC instead of the JDCB-ODBC bridge. There are many ways to get the job done. Here is what I did:

First I downloaded MySQL Connector/J; created a directory “C:\Devel\Java\JRE\ext”; extracted the binary jar for the driver from the download and copied the mysql-connector-java-3.1.14-bin file into “C:\Devel\Java\JRE\ext”.

Started Eclipse; in the preferences dialog edited the settings for the Installed JREs.

Specifically, I used the “Add External JARs…” to add the MySQL connector I put in C:\Devel\Java\JRE\ext.

First select the installed JRE you want to edit.

Next “Add External JARS…”

Then you should see the driver show up in the “JRE system libraries” list.

Now I have a working MySQL Connector/J JDBC driver ready for use with Eclipse projects. Yay!

Clear resume state from a paused VMWare VM

2010-09-17T18:36:00.001-05:00

Had a team that I copied from another location with saved resume state that was not compatible with the new host workstation. As mentioned here I deleted the *.vmem and *.vmss files while VMWare was off then resumed the team on the new host.

bkfjtools hack and slash for english

2010-09-17T05:51:00.001-05:00

I needed a quick way to extract some files out of an NTbackup “BKF” file and didn’t have time to go buy a tool. A couple seconds of googling revealed the bkfjtools program. Unfortunately, I don’t read Japanese that well. So I did a quick hack and slash conversion to English using google translate on the strings for the bkfexp tool. Source is here. Probably should clean it up for real language support and commit back. Probably won’t happen due to time constraints.

Should have googled more than two seconds….Another technique to use NTBackup on Win Vista/7 is described here. But what’s the fun in that?

Google Code Blog: Google Relaunches Instantiations Developer Tools -...

2010-09-17T01:46:00.000-05:00

This looks like a good add on for Eclipse:

Google Code Blog: Google Relaunches Instantiations Developer Tools -...: "(Cross-posted from the Google Web Toolkit blog) In early August, Google acquired Instantiations, a company known for its focus on Eclipse J..."

Flame Tornado!

2010-09-16T19:26:00.001-05:00

Flame Tornado! That is just cool. Or maybe hot.

Live View – no Virtual Server 2.x support

2010-09-06T13:12:00.001-05:00

Wanted to use Live View to check out a couple of dd images from a set of workstations. Primary goal was to extract VHD files from the workstations dd images for a separate analysis task but also needed to check some “live” system behavior.

I’m using Windows Vista x64 at the moment and it requires digitally signed drivers. Unfortunately, this means Virtual Server 1.x series will not function correctly on my workstation because the 1.x series does not include signed drivers for all necessary virtualized devices. No network and/or no USB device.

Being to lazy to build a 32-bit box I decided to see if I could get Live View to work with Virtual Server 2.x series. Actually, it wasn’t really laziness (piles of spare parts just don’t want to build a box right now) so much as I don’t have a USB KVM and my desk is already loaded up with monitors. A little bit of research on the Live View forums indicated that the 2.x series is not supported at this time. Unfortunately, Virtual Server and Workstation can not co-exist on the same workstation and I need Virtual Server for another piece of the analysis workflow. Bummer for me, no Virtual Server 2.x goodness for this project unless I want to start code spelunking Live View or go the VMWare Workstation route on a separate workstation.

“Summon Laptop”. So I downloaded and installed the following: VMWare Workstation 7.x; Virtual Disk Development Kit (VDDK); Live View Public installer. And now I have Live View installed on a separate workstation. Next step in the plan was to use this install of Live View to copy dd images on external drive to vmdk files on an external drive. Then plug that drive into the workstation with Virtual Server for actual artifact extraction and analysis. We’ll see how that works out in a bit.

The User Profile Service failed the logon. User profile cannot be loaded – or – Working for the Computer instead of the Computer working for me (yet again).

2010-09-06T08:22:00.002-05:00

I really hate it when I end up working for the computer instead of the computer working for me. Not 100% sure about the root cause, suspect the operator – me; but I got a “The User Profile Service failed the logon. User profile cannot be loaded” error when logging in on a Windows Vista x64 box.
Googled a bit and came across MS KB 947215. It includes a Fix-It that did not work for me so I moved on to a manual process. I’m working for the computer yet again.
I made a copy of the “corrupted” User profile to an external drive before deleting it. The profile was about 250 Gig so the copy process did not seem responsive.

Was going to kill the copy it but happened to check the disk space on the external drive and it was slowly getting filled up by the copy of the “corrupted” User profile. That was about 11.5 hours with the “Copy to dialog” not responding.
Next went to clean up some of the mess left on the %UserProfiles% before re-creating the profile. There were several temporary files that couldn’t be deleted due to permissions so got a Recovery Disk. Booted up the recovery disk and got a command window up to delete the stubborn temp files. Renamed the profile directory; and finally rebooted so I could recreate the profile and fix up the permissions on files. New SID means permissions are still a bit hosed. I'll need to go back and fix those later.

What a pain. Finally back up and running.

Edith Sitwell

2010-09-06T07:20:00.001-05:00

Eccentricity is not, as dull people would have us believe, a form of madness. It is often a kind of innocent pride, and the man of genius and the aristocrat are frequently regarded as eccentrics because genius and aristocrat are entirely unafraid of and uninfluenced by the opinions and vagaries of the crowd.

Edith Sitwell
English biographer, critic, novelist, & poet (1887 - 1964)

I’m no genius and I’m no aristocrat but that is a keen observation. Kind of a bummer when people you’d like to meet are already dead.

Setting up IIS on Windows Vista x64 for VMWare Server 2.x install

2010-09-05T20:56:00.002-05:00

I had originally configured IIS to use Client Certificate Mapping Authentication (or IIS Client Certificate Mapping Authentication). Unless your authentication provider is set up for certificates you won’t need this mapping on. I’m working on a standalone workstation in a workgroup so I’m not set up for certificates in this case. This www.iis.net post discusses ISS authentication and certificate mapping in a bit more detail.
I used the following process to turn Client Certificate mapping off:
Access “Uninstall or change a program”.

Access the “Turn Windows features on or off” Using the “Control Panel -> Programs and Features”

Uncheck the "Client Certificate Mapping Authentication" and "IIS Client Certificate Mapping Authentication" if they are selected. Then apply your settings (press “OK”).

You’ll get the following dialog:

Warning: Make sure that you wait for the configuration settings process to complete!
This took about 10 to 15 minutes on my workstation. It’s also likely that you’ll need to reboot after making your changes. First time around I got impatient and rebooted before the changes were completely applied which led to a whole slew of problems (User profile got trashed, but that’s another story).
I did generate a self-signed certificate for the Default Site after turning off the Certificate Mapping so I could access the site through https but not require a client certificate.

Disk2vhd – migrate a “real” Windows box to virtual

2010-09-03T06:11:00.001-05:00

I won’t hash out the details which are available elsewhere but Disk2vhd looks like a solid way to migrate a running Windows box to a Virtual PC or VirtualBox.

Might also work to grab an image of a running box for offline analysis. With Windows 7 you can mount a VHD using built in features. Otherwise you’ll probably need something like Gizmo Drive.

Building 32-bit MySQL clients with 64-bit MySQL install

2010-09-03T05:53:00.002-05:00

Had to build a 32-bit MySQL client (MySQL-python) on a 64-bit Windows Vista hosted install of 64-bit MySQL.
I used setuptools to get the initial download of MySQL-python. “easy_install –b temp mysql-python”. But installation failed due to 64-bit MySQL libraries (and the registry setting in site.cfg).

First, I downloaded the no-install 32-bit version of MySQL extracted it out and copied the libraries directory to “C:\Program Files\MySQL\lib32”. Now I have the 32-bit libraries in “C:\Program Files\MySQL\lib32” and the 64-bit libraries in “C:\Program Files\MySQL\lib”.
Then tweaked the MySQL-python site.cfg so the registry_key was correct (registry_key = SOFTWARE\MySQL AB\MySQL Server 5.1) .
Next tweaked the setup_windows.py so it would use lib32 instead of the 64-bit libraries (library_dirs = [ os.path.join(mysql_root, r'lib32\opt') ]
Finally, it was a simple run of “python setup.py clean”; “python setup.py install” to get MySQL-python installed.

Purely a hack but it got the job done.

Virtual Drive Software for Vista

2010-09-02T06:34:00.001-05:00

I’ve used VirtualCloneDrive for mounting ISO images but needed to mount some Virtual PC virtual hard drive (VHD) images which it unfortunately does not do. I’m using Vista on the moment so I don’t have the cool VHD mounting feature in Windows 7. So after about 2 minutes of googling came across a post describing a process that uses Gizmo Drive to mount VHDs. Going to work with that for the moment.

Convert Windows 2003/XP event logs (.evt file) to Vista/7/2008 event logs (.evtx file)

2010-08-17T14:11:00.008-05:00

Recently had to convert a slew of Windows 2003/XP .evt files to Vista/7/2008 .evtx files so I could do some offline log analysis. First problem was that some of the .evt files were corrupt. They’d been directly copied off of running Windows 2003 servers (from the %SystemRoot%\System32\Config) to a local drive. I used fixevt to fix the logs and a powershell script wrapper around wevtutil to convert them.
After downloading fixevt the repair and conversion came down to two simple commands in powershell:

fixevt *.evt
Get-ChildItem . -recurse -include "*.evt" | foreach-object {wevtutil epl $_.FullName ($_.FullName + "x") /lf:true}

You can also use wevtutil to trim down an evtx file to a specific date range. For example:

wevtutil.exe epl Security Test.evtx /q:"*[System[TimeCreated[@SystemTime>='2010-11-01T05:00:01.000Z' and @SystemTime>='2010-11-11T05:59:59.999Z']]]"

Will output the Security log events between 1 Nov 2010 and 11 Nov 2010.

You'll need to substitute actual > and < for the &gt and &lt. Also, note that the time is in Zulu. The times in evtx files are in Zulu the event viewer and many other event apps tend to translate the time into local time

Now off to the fun of event correlation….

Delete your Facebook account

2010-05-02T21:11:00.001-05:00

If you’ve finally had enough of the Facebook backslide on privacy (nice graphic that covers the changes) and their evil interfaces you can try to delete your account with this link:

http://www.facebook.com/help/contact.php?show_form=delete_account

After "Deleting", do not log in for at least two weeks to test if the account is really gone. Otherwise Facebook could reactivate the account.

Southpark sums Facebook up.

CMake, autotools and Eclipse integration

2010-03-28T09:03:00.001-05:00

Found a couple of Eclipse plugins that help work with CMake build system:

CMake Editor – works with Eclipse 3.4+
CMakeBuilder – works with Eclipse 3.2.

Also, if you prefer GNU autotools, there is the Linux Tools Project. This appears to be the only surviving autotools plugin for Eclipse. I’ve had mixed success using this for projects with a cygwin backend on Windows.

Windows Icon editors

2010-03-21T09:56:00.001-05:00

Did some spelunking to find some free Icon editors for Windows icons. So far I found:

IcoFX - a freeware icon editor. It works with Windows (XP, Vista, 7) icons formats and supports transparency.
@icon sushi - a freeware image to icon converter and mask editor for the creation of Windows (XP, Vista) icon formats.
JDraw - described as “a pixel oriented graphics editor designed especially for small to medium-sized pictures used to decorate web pages”. It is an open source Java program that works with (animated) GIF, ICO and PNG formats. Unfortunately, JDraw did not work with multiple images in the ICO files I wanted to edit.
PNG (ICO) icon create utilite - a “web-app” that produces simple flat PNG and ICO files.

IcoFX worked best for me because it supported the Windows icons I was editing. YMMV.

Rock Paper Scissors Spock Lizard

2010-03-16T09:36:00.001-05:00

Bored with classic Rock Paper Scissors?

Step it up a notch with Rock Paper Scissors Spock Lizard as seen on TV. Or go with the instructions from the inventor or you could go with a tee-shirt.

Or if you want to go to the extreme try RPS-15 or RPS-25.

Open Source Routers

2010-02-20T09:05:00.002-06:00

These projects are still percolating… Right now the Netgear WNR3500L looks like a good “high-end” open source router. Anyways don’t forget to search Amazon and Newegg for other Open Source Routers.

I actually ended up getting a Cisco/Linksys WRT160NL and installing DD-WRT on it. Plan is to recycle this into one of the other projects once I'm done using it as a network device.

BackTrack 4 released

2010-02-15T09:02:00.001-06:00

BackTrack 4 released in January. Not sure how I missed that until now.

Reset Windows XP/Vista passwords – NTpasswd / Hiren's BootCD

2010-02-10T03:59:00.002-06:00

Every once and a while I have to revive a Windows box that I haven’t used for a while and I can’t recall the passwords. Fortunately, NTpasswd can reset the passwords. If you prefer a bootable USB thumb drive it’s pretty painless to create one with unetbootin.

Another option is to use Hiren's BootCD which includes a plethora of useful recovery tools.

Photo processing tools and toys - Photosynth

2010-02-08T18:45:00.001-06:00

Microsoft Photosynth, based on Photo Tour by University of Washington and Microsoft Research, is an interesting way to composite a large picture collection (20-300 recommended) of a scene or subject. To create a Synth you need to install a Windows Live Photo Gallery plugin that interacts with Microsoft’s Photosynth backend web service. While you can view public Synths at Photosynth, you’ll need a Windows Live account to create and share your own Synths. Also, recommend that you check out the “How to Shoot” guide.

My Synths are rather boring so far. Really need a larger collection of input photos to create a more complete Synth. Might be fun to get a bunch of people together to create an input photo collection of a scene or subject. Maybe a good use case for “Crowdsourcing”? It’s hard to leverage the power of crowdsourcing with an un-crowd of one.

Photo processing tools and toys – Editing, Batch Processing, Panoramas, and publishing to the web

2010-02-07T13:12:00.001-06:00

I’m not much of an artist or photographer but I do enjoy playing around with the photos I’ve taken over the years. Since I lack talent I have to resort to tools. Because I’m “frugal” I tend to use free and preferably open source photo editing/processing tools and toys. Some days I use Linux boxes, some times Windows boxes, and very infrequently Apple boxes (they’re just to expensive to be worth it to me). I like cross-platform tools and toys but that’s not the main driver here. These are some of the photo editing/processing tools and toys that I’ve experimented with.

Editing: You could go for a full blown application like the gimp which rivals/surpasses PhotoShop in power, features and complexity. On the simpler side Google Picasa and Microsoft Windows Live Photo Gallery (WLPG) provide automated photo touch up. For stand alone editing on Windows I like Paint.NET. It has just enough power for an amateur like me. For quick touchups I usually use WLPG. The Pinta Project looks like a promising implementation of Paint.NET feature set for Linux and Macintosh.

Batch Processing: ImageMagick is a very robust set of command line image manipulation tools. If you’re into scripting it’s pretty straight forward to string together some bash scripts or powershell scripts to do batch processing and manipulation. Check the Usage page for examples. I installed from the binary Windows setup (at the moment ImageMagick-6.5.9-2-Q16-windows-dll.exe).

Panoramas: Hugin is one of several panorama tools that uses the PanoTools framework. The PanoTools framework supports re-projecting and blending multiple source images into a panoramic image (or some other unusual remappings). With Hugin You can assemble a mosaic of photographs into a panorama and stitch a series of overlapping pictures. Check out some examples and the tutorials. Personally, I didn’t get over the learning curve for Hugin so I went looking for something easier to use.

Microsoft ICE for Windows XP and Windows Vista is dead simple to use. I just dragged and dropped fifteen source images into the main window of the program and let it do all the work. It selected five images and generated the panorama below. Source images were taken standing at the end of a bridge going off of Harbor Island in Tampa Florida.

Seems that the technology from Microsoft ICE is also included as part of Microsoft Windows Live Photo Gallery. See: Creating photo panoramas with Windows Live Photo Gallery. ICE is more in the sweet spot for my interests. ICE gives me the illusion that I’m creatively in control without overtaxing my brain like Hugin. Also, ICE integrates into WLPG under the Extras on the toolbar. Look for “Extras –> Create Image Composite…” in WLPG after you install ICE.

Publishing to the web: I don’t normally use a dedicated photo sharing service but instead share photos with Google Picasa Web, Microsoft Windows Live, or Facebook. Using a client side program like Google Picasa or Microsoft Windows Live Photo Gallery (WLPG) simplifies publishing photos to a web backend. Both applications are, of course, tied to their respective vendor’s online service but you can get plugins to add support for other server backends. There is a Picasa Facebook plugin and WLPG Facebook plugin. Or if you prefer, PicasaPublisher eases the pain of publishing photos to Picasa Web from WLPG. Also, WLPG plugins are available for several server backends.

Trying out some mosaic generators and 2D to 3D programs but everything I’ve found so far is pretty limited. More on that later.

File and Folder renamer

2010-02-07T06:09:00.001-06:00

Needed to rename a batch of photos on a Windows box and found Metamorphose File -n- Folder Renamer. Only a minor glitch but had to install as Administrator. Don’t know why Vista didn’t detect it as an install program on it’s own. Other than that it did the trick.

Journals for failure and negative results

2010-02-07T03:41:00.001-06:00

A couple of online journals to document research failures so others don’t tread the dead-end path: Journal of Serendipitous and Unexpected Results (JSUR) and Journal of Interesting Negative Results in Natural Language Processing and Machine Learning (JI-R). JSUR is new and ironically it appears that JI-R has only one publication from April 2008. I guess that could mean JI-R is a failure. Maybe I should submit an article to JSUR to document that?

cygpath quoting for directories with spaces in their name

2010-01-17T11:17:00.002-06:00

Had to experiment with quoting with cygpath to patch up some scripts under cygwin.
I was trying to properly quote an environment variable defined on the Windows side:

MSSDK=C:\Program Files\Microsoft SDKs\Windows\v7.0

Just using

cygpath –u $MSSDK

resulted in a multi-line return of:

C:/Program
Files/Microsoft/
SDKs/Windows/v7.0

cygpath –u “$MSSDK”

returned a single line:

/cygdrive/c/Progarm Files/Microsoft SDKs/Windows/v7.0

and finally

echo \”`cygpath –u “$MSSDK”`"\”

returned the directory properly quoted:

“/cygdrive/c/Program Files/Microsoft SDKs/Windows/v7.0”

Hurray. Back to script editing…

Boost for Visual Studio 2008 and Qt

2010-01-16T08:21:00.001-06:00

Needed Boost for a Windows project. After struggling with the build and install for a bit I wimped out and used the BoostPro Binary Installer for Visual C++. I wanted to use this with a Qt project so I added the lines below to the qtvars.bat file:

@if not "%BOOST_ROOT%" == "" (
    echo -- Adding boost from %BOOST_ROOT%
    set "PATH=%BOOST_ROOT%\lib;%PATH%"
    set "INCLUDE=%BOOST_ROOT%;%INCLUDE%"
    set "LIB=%BOOST_ROOT%\lib;%LIB%"
)

and defined BOOST_ROOT to be the directory where I installed boost.

Install HP OfficeJet Pro 8500 Network Access Software on Windows Vista x64

2010-01-04T15:22:00.003-06:00

Bought an HP OfficeJet Pro 8500 to support wireless printing and scanning. This thing was an incredible pain in the butt to get installed. The error dialog I got is shown below.

I had to manually install the Microsoft Visual C++ 2005 Redistributable Package (x64), install the Microsoft Visual C++ 2005 Redistributable Package (x86) and disable my anti-virus software during the install so the HP installer could complete.

Another gripe…Yahoo Toolbar as a default part of the installer? HP that really does not set the tone as this being a professional office product when you are shilling for Yahoo.

Finally, had to manually install a Critical update to fix Thai fonts.

Make VideoLAN – VLC media player the Default DVD Player in Windows Vista

2010-01-03T21:44:00.001-06:00

I’ve been searching for a while for a way to make VideoLAN – VLC media player the default DVD player in Windows Vista (32 and 64 bit). After, several minutes of fruitless searching I finally came across a lifehacker reference to Autoplay Repair and a guide explaining usage.

To use VLC I created an entry for DVDs with Autoplay Repair

See the VLC documentation for command line parameters. I used dvd://%L for now.

Then ran the Vista AutoPlay tool to make sure that VLC is the default.

Inserted a DVD and was off to the movies.

Flintstone Vitamins Character Recognition Guide

2010-01-01T05:39:00.002-06:00

Just in case your kid’s ask you which character a Flintstone Vitamin is here’s the Flintstone Vitamins Character Recognition Guide.

Fred Flintstone	Wilma Flint stone	Pebbles Flintstone
Barney Rubble	Betty Rubb le	Bamm-Bamm Rubble
Dino	The Great Gazoo

Which OS for Wii (embedded PowerPC) homebrew project?

2009-12-31T13:36:00.001-06:00

Googling about for an open source OS that would work for a Wii (embedded PowerPC 740/750 work alike) homebrew project and I’ve narrowed my choices down two categories: a real-time OS or an embedded Linux based OS.

While I’m more comfortable with the toolchains and feature sets provided by a “full blown” embedded Linux, BlueCat Linux for example, I like the configurability and architectural layering philosophy of eCos, while RTEMS has cool points for being used in space projects.

My main concerns with eCos are: it does not have a USB stack (only USB Slave code is included); no Bluetooth stack; and does not appear to support dynamic loadable libraries. Probably, need to just go check out the GameCube/Wii Linux scene.

Using mixed toolchain – GNU build system with Visual C/C++ compilers

2009-12-31T07:39:00.001-06:00

I’ve been searching for an automated way to use the GNU build system to build autoconf based GNU libraries like libiconv using Visual C/C++ compilers instead of GCC. The end goal is to create Windows DLLs that I can integrate into Visual C/C++ projects.

Found two alternatives that I want to check out: simacros from Systems in Motion; and parity by Markus Duft and Michael Haubenwallner. The two have a similar goal and similar approach. Both include source to shim/proxy software that translates GNU GCC command line options into Visual C/C++ command line options; and both require modified autotools.

A big difference is that simacros uses cygwin for it’s execution environment and parity is designed for the Windows Subsystem for UNIX-based Applications (SUA). Another difference is ease of access. Parity is readily available as a stand alone SourceForge project. I found simacros is a bit difficult to extract out of the various web links and downloads at the Coin3d website.

Based on initial code spelunking and the ease of getting to the parity source code I’m leaning toward porting parity to cygwin.

DVD Flick – easy way to make simple video DVDs

2009-12-29T07:06:00.001-06:00

DVD Flick is a straight forward way to make simple video DVDs from all the random AVI clips I collected over the holiday. Now I can haunt everyone with DVDs of family Christmas videos. Source available at SourceForge project page.

How does your development language of choice rank?

2009-12-28T10:04:00.001-06:00

LangPop.com develops rankings for various development language. At the moment my interpretation is the c, c++, and Java trifecta still dominates.

No more Qtopia

2009-12-27T23:14:00.001-06:00

Apparently, Qtopia was killed off in late 2008, re-released as Qt Extended, then dropped by Nokia/Trolltech early 2009. Finally, source was “liberated” by a fork as Qt Extended Improved. I was considering it for the GUI layer of an embedded PowerPC project with an Embedded Linux (or maybe eCos) substrate.

Scratch Programming Language for Learning/Play

2009-12-22T04:03:00.001-06:00

During some early morning googling to see if anyone had started a port of Java as Wii homebrew I came across Scratch. Described as: “Scratch is a new programming language that makes it easy to create your own interactive stories, animations, games, music, and art -- and share your creations on the web.”

EnvMan Windows Environment Variable Editor

2009-12-22T03:27:00.001-06:00

I wanted a convenient way to edit Environment Variables on a Windows box. Some googling brought me to EnvMan at SourceForge. Worked for me. YMMV.

eBook library managers - eLibrary and calibre

2009-10-05T09:42:00.001-05:00

If you’re collection of electronic books is growing unmanageable you might want to check out eLibrary or calibre.

My favorite feature of eLibrary is source file renaming after it downloads the book information. The program is Windows only at this time and is focused on library management and reading on the computer.

Another option is calibre. Calibre is also an open source project hosted on SourceForge.net. Calibre supports file format conversion for several popular ebook readers as well as a built-in ebook reader.

Install TAP for Wippien on Vista x64

2009-10-02T19:49:00.002-05:00

I wanted to try out Wippien on 64-bit Vista. Had to download the 64-bit TAP driver then run the “hdwwiz.exe” to manually install the TAP driver.

Unfortunately, Wippien didn't work out the way I hoped it would. Still looking for a VPN solution that I can run on my home network for remote access. Probably will need to look at using a hardware solution like a VPN concentrator.

Multi-core/Many-core OS research

2009-09-28T08:45:00.002-05:00

The Barrelfish Operating System is research effort addressing how to build OS for multi-core/many-core and heterogeneous core computers. AMD also has some related many-core publications.

Windows Vista on Acer Aspire One (A-150)

2009-07-19T21:27:00.001-05:00

Windows XP on my wife’s Acer Aspire One finally died an ungraceful malware induced death and instead of re-installing I figured I’d give Vista a chance. First step was to get an external USB DVD drive. I booted a 32-bit Vista SP1 DVD from an ASUS DR-08D1S-U and the basic install went without a hitch.

Next the adventure of locating drivers. Out of the box Vista SP1 did not support the wireless NIC, onboard NIC, video, or sound. Not a big surprise. Unfortunately, Acer does not provide Vista drivers.

The first thing I usually try in the situation is the onboard NIC. It was listed as a “Realtek RTL8101 Family PCI-E Fast Ethernet NIC (NDIS 6.0)”. At the time of this post a driver is available from the Realtek driver download site. Followed the menus shown to get to PCI Express Software.

The driver that worked for me was version 6.225 dated 2009/7/17 in the file named Install_Vista_6225_0709.zip.

After installing the onboard NIC is listed as “Realtek PCIe FE Family Controller”.

Since I had a wired connection to the network I activated Vista and started Windows Update. Let that chug away downloading and installing for an hour or so. After all the updates the onboard wireless NIC, sound and video are working.

Porting fracplanet to Qt4 (actually Qt3Support)

2009-06-21T21:46:00.001-05:00

I decided to try a quick hack-and-slash port of fracplanet to a more recent version of Qt. Since I’m using a windows box I installed the Qt SDK for Windows. Also, downloaded and installed the version of MinGW from trolltech as part of the install. I installed Qt to C:\Devel\Qt\4.5.1 and MinGW to C:\Devel\Qt\MinGW

First step ran qt3to4 on fracplanet.pro, then qmake VERSION_NUMBER=0.4 fracplanet.pro, and make. I’m missing dependencies like boost so I didn’t expect this to get very far. It didn’t.

First problem was the VERSION_NUMBER=0.4. At line 32 in control_about.cpp their isn’t a automatic conversion from QChar to double. So next I did qmake VERSION_NUMBER=1 fracplanet.pro, and make.

Got to line 55 of control_about.cpp and had to change it to:

license->setTextFormat(Qt::PlainText);

Did make again and now the boost problem. Since this was supposed to be a quick hack-and-slash I didn’t bother building boost. Found and installed Boost-1.34.1-mingw.exe into C:\Devel\Boost-1.34.1 and added the following directives to fracplanet.pro:

LIBS += -lboost_program_options-mt
INCLUDEPATH += C:\Devel\Boost-1.34.1\include
LIBPATH += C:\Devel\Boost-1.34.1\lib
LIBPATH += C:\Devel\MinGW\lib

Then redid qmake VERSION_NUMBER=1 fracplanet.pro, and make.

Next problem was a mission qhbuttongroup.h in control_render.h at line 27. Appears that the qhbuttongroup was rolled into qbuttongroup at some point. qt3to4 didn’t pick this one up so I mannually changed qhbuttongroup.h to q3buttongroup.h. Same in control_terrain.h.

Copied usage_text.h out of a previous build with cygwin to get around figuring out how to actually build the documentation.

Changed line 113 in fracplanet_main.cpp to

progress_dialog->setLabelText(QString(progress_info.c_str()));

Added the following before the use of glDrawRangeElements in triangle_mesh_viewer_display.cpp

PFNGLDRAWRANGEELEMENTSPROC glDrawRangeElements = NULL;
glDrawRangeElements = (PFNGLDRAWRANGEELEMENTSPROC)wglGetProcAddress("glDrawRangeElements");

Then redid qmake VERSION_NUMBER=1 fracplanet.pro, and make. And the end result… A working copy of fracplanet built against Qt 4 compiled with MingW.

OK, technically it’s still Qt 3 since it uses Qt3Support. Also, has to be run from the command line at the moment. Might fix it up and package it later.

Building fracplanet with cygwin

2009-06-21T08:36:00.001-05:00

Took a whack at building fracplanet with cygwin on a Windows box yesterday afternoon. It was too hot outside to mow the lawn.

Some things that I had to do to get it built:

Edit the fracplanet.pro file so it could find the installed version of boost. I added the following:

LIBS += -lboost_program_options-gcc-mt
INCLUDEPATH += /usr/include/boost-1_33_1

Edit the line 103 in /usr/include/boost-1_33_1/boost/random/mersenne_twister.hpp to

x[j]=gen;

instead of

x[j]=gen();

Had to go back and install libglut for cygwin.

After that the program compiled and linked.

I ended up using Xming instead of the X server bundled with cygwin. Of course I had to make sure to start Xming before bash and set the display by hand (export DISPLAY=localhost:0.0).

Here is my first planet.

And here it is rendered by POV-Ray for Windows.

MSN Web Messenger

2009-06-21T08:20:00.001-05:00

Started playing with the Ubuntu Netbook Remix on an Acer Aspire One and “needed” a quick way to get on Live without using a native linux client. After a bit of searching came across the MSN Web Messenger.

Homeworld source

2009-05-25T21:27:00.004-05:00

Just came across the WikiPedia page for Commercial video games with freely available source code
and noticed that Homeworld is on the list. It's a classic that I used to play.

SDL oriented source is available with restrictions through HomeworldSDL and a Windows SDL version is available here.

Team Mediaportal and dynamic IP

2009-03-20T16:20:00.002-05:00

At the moment the computer I'm running Team Mediaportal on gets a new dynamic IP every couple of days so the Mediaportal looses track of the TV Service.

My hack to get around this is to manually stop the TVService; update the SQL table with the server information with the computers current IP; then manually start the TVService.

OpenLayers and Google Maps

2009-03-02T17:28:00.002-06:00

Here is my attempt at using the Google Maps API through OpenLayers as presented at Linux.com.

Unfortunately, that didn't work as planned so on to the next hack...

Team Mediaportal and Schedules Direct plugin

2009-03-02T06:05:00.002-06:00

Picked up a Hauppauge WinTV-HVR-1600 card at Circuit City. Since they were "discounting" for their going out of business sale with tax it was the same price as most online vendors. I plan on eventually building a MythTV box but for the moment I'm using it under Windows XP using Team Mediaportal with a Schedules Direct plugin. The plugin that worked for me can be found here.

Places I've been...

2009-03-01T19:28:00.000-06:00

States I've visited in the USA

create your own personalized map of the USA

Countries I've visited

create your own visited country map

OpenWRT supported 802.11n routers

2009-03-01T10:04:00.001-06:00

The OpenWRT project is making strides on recent 802.11n capable routers based on AtherosA9100 chip sets.

As mentioned on Slashdot some of the less expensive routers that use that chip set:

Planex (PCI) MZK-W04NU, 32MB RAM and 8MB flash, USB port, 10/100 Ethernet
Trendnet TEW-652BRP, 32MB RAM and 4MB flash, 10/100 Ethernet
Trendnet TEW-632BRP, 32MB RAM and 4MB flash, 10/100 Ethernet
D-Link DIR-615 revision C1 (ONLY!), 32MB of RAM and 4MB flash, 10/100 Ethernet
TP-Link TL-WR941N WR941ND, 32MB RAM and 4MB flash, 10/100 Ethernet

Could use these for some interesting projects. Maybe a Kismet Drone (or see here)? Or how about a WiFi robot that implements SRV-1 Control Protocol ?

The Planex MZK-W04NU looks especially interesting since it has a USB connection. It's current status is discussed on the OpenWRT Forums.

Flickr tool to detect astronomical things of interest

2009-02-19T17:56:00.000-06:00

A neat Flickr tool to detect astronomical things of interest in photos. Blogged about by others here. This is just darn cool. See here for some pictures.

Dying Freeagent Xtreme 1.5 TB External

2009-02-14T08:06:00.001-06:00

Picked up a Seagate Freagent Xtreme 1.5 TB External at BestBuy last week. What a piece of dung. It lasted 7 days before it died. Not sure if I'll be able to recover the photos I copied over to it. If you see one of these on sale run away. It's not worth the headache.

Fortunately, it appears that only the USB connection died on the external case. I was able to recover my data by using the Firewire connection. Didn't have a chance to try the eSATA connection since I'm away from home but my faith in this thing is very low. Back to BestBuy it goes!

Windows Live Writer

2009-01-07T15:46:00.001-06:00

I’m temporarily stuck using only a Window’s box so I thought I’d try out Windows Live Writer to see if it plays nice with blogspot.

Seems to work OK.

Gnash and Protocol Reverse Engineering

2009-01-05T17:14:00.000-06:00

Techniques used by Rob Savoye to reverse engineer RTMP for Gnash will be presented at FOSDEM.

Automated Protocol Format Recovery and Automated Protocol Control Flow Recovery

2008-12-29T21:11:00.000-06:00

Some recent (2008) research results on "Automatic Protocol Format Reverse Engineering through Context-Aware Monitored Execution" and "Automatic Network Protocol Analysis" both look like some promising extensions of the work done for Microsoft Discoverer and (apparently defunct) Protocol Informatics approaches for protocol format recognition. A circa 2007 paper titled "Polyglot: Automatic Extraction of Protocol Message Format Using Dynamic Binary Analysis" also describes an approach to protocol recovery. Interesting to me because I did some work on automated recovery of control flows (see).

Might be interesting to combine the two (automated format recovery and automated control flow recovery). Unfortunately, it appears that the approach used in Microsoft Discoverer are going to be patented.

An overarching approach is what I call Dynamic Protocol Reverse Engineering (DPRE). This idea is derived from protocol conformance checking. I'd like to end up with an automated/semi-automated processing pipeline. An overview of a DPRE Framework is depicted here:

Conceptual DPRE Framework

Above, we have some of the ground work built for for automated model recovery. First we collect a "pile-of-packets" for the protocol implementation under inspection. Process the packets to recover an implementation model. Next the recovered models could be processed by automated verification tools to check their performance characteristics. A list of several formal verification methods and tools is located here. After model generation and model verification we could perform vulnerability assessment and generation of targeted effects (i.e. exploits). Finally, the targeted effects could be used to test a protocol implementation in a hostile environment.

A cool Qt hack in 3D

2008-12-05T15:40:00.001-06:00

Here is a cool Qt hack in 3D - Widgets enter the third dimension: WolfenQt

Network Reconissance/Forensics Tools

2008-11-17T16:30:00.000-06:00

A few network reconnaissance/forensic tools:

NetworkMiner a Network Forensic Analysis Tool (NFAT) for Windows. Good for recon'ing a pcap file to see what is in there.

LAN Crawler a crawler and indexer of public network files shared via SMB shares.

Etherwatch monitors Ethernet traffic, looking for images, and Google™ search terms, and displays these on screen in a mosaic format.

Burn a CD ISO to a DVD

2008-09-13T17:26:00.000-05:00

Downloaded a CD sized ISO image on a Windows XP box and only had blank DVD-R around. I didn't want to use an ISO mount tool and my installed version of Nero Express wouldn't burn the image to a DVD-R

I'm sure there are many ways to do this but InfraRecorder worked for me.

Grammatical Inference

2008-07-25T19:21:00.003-05:00

Grammatical inference or Grammar Induction focuses on learning formal language rules from a series of observations of samples from the language under consideration.

If you are interested in learning some grammatical inference basics the video links below are well worth a look.

Grammatical Inference: a Tutorial

Colin de la Higuera

Also, there are several articles available through citeseer. Search on grammar inference or grammatical inference.

Finally, Amazon has several Grammatical Inference conference proceedings.

Protocol Reverse Engineering

2008-06-30T17:19:00.000-05:00

Some reverse engineering ideas/resources are located here.

Another attempt at protocol reverse engineering to BNF is online at Rent A Coder.

A recent attempt at classifying encrypted traffic in SSH tunnels here.

How to specify the default installation prefix in configure.in/configure.ac?

2008-02-09T16:10:00.001-06:00

Add the AC_PREFIX_DEFAULT(/usr/local) macro in configure.in/configure.ac. Where you replace /usr/local with the prefix you want to use.

See (here) for details.

Augmented BNF (ABNF)

2008-01-01T17:11:00.000-06:00

While looking for Augmented BNF (ABNF) source for some of the common Internet protocols I came ac cross a perl script by Bill Fenner to extract ABNF from RFCs (here).

I used it to generate ABNF for RFC4234, RFC2821, and RFC2822 to use with APG an ABNF Parser Generator.

Standard ML New Jersey on OpenSUSE 10.3

2007-12-22T13:39:00.002-06:00

I needed a version of Standard ML New Jersey to compile PADS but the smlnj version from binary RPM for OpenSUSE was configured to use files in it's build directory.

First, I followed the install instructions. I downloaded the files from the SourceForge file list, and went through the install process but the build was convoluted by assembly source problems.

Next, I took a whack at building from the source RPM.

I searched for "smlnj" at the OpenSUSE Build Service .
Downloaded the source RPM and copy it to /usr/src/packages/SRPMS.

Installed the source RPM with rpm -Uvh smlnj-110.65-10.src.rpm. This will result in smlnj.spec in /usr/src/packages/SPECS and /usr/src/packages/SOURCES.

Build the RPM:
cd to the /usr/src/packages/SPECS and build the SRPM and RPM with rpmbuild -ba smlnj.spec.
On my system this results in:

/usr/src/packages/SRPMS/smlnj-110.65-10.src.rpm

/usr/src/packages/RPMS/i586/smlnj-100.65-10.i586.rpm

/usr/src/packages/RPMS/i586/smlnj-debuginfo-100.65-10.i586.rpm

Install the RPM:
cd to /usr/src/packages/RPMS/i586 and install the binary RPMS with rpm -Uvh smlnj-100.65-10.i586.rpm.

The end result... AHRGGG! Not surprisingly, the install is still looking for library files in /usr/src/packages/BUILD/smlnj-110.65/sml.boot.x86-unix.

Finally, I extracted the source from the RPM (rpm2cpio smlnj-110.65-10.src.rpm | cpio -ivmud), applied the patches and built the whole thing in /usr/local/lib/smlnj then added /usr/local/smlnj/bin to the PATH. Now PADS builds.

My code wasteland

2007-12-21T14:11:00.000-06:00

Nearly forgot about the google code project hosting. My code wasteland is at http://code.google.com/p/mark-e-deyoung/.

Actually, I need to fire it up again for another project because my school's CVS is down for maintenance (and probably will be down the entire Christmas holiday.)

Approximating context-free grammars with regular grammars.

2007-12-13T12:09:00.000-06:00

Got a version of the Mohri and Nederhof algorithm working with SWI-prolog. The prolog source (attached here) is from Implementation of Regular Approximation of Context-Free Grammars Through Transformation by Tapani Raiko at Helsinki University of Technology.

Raiko explains how to use the program in his write-up.

A similar idea is implemented by Tanaka Akira in abnf converter. It's a ruby program to convert Augmented BNF to Regexp.

libpcap for cygwin

2007-12-13T09:36:00.000-06:00

After much fruitless googling I came accross a modifed version 0.7 of libpcap that was ported by Nevil Brownlee to the cygwin environment. It allows you to use the unix libpcap API in your code and then translates the calls into equivalent WinPcap calls.

The port was conducted by CAIDA Metrics Working Group project to support the port of NeTraMet to MS Windows.

Source is in libpcap-0.7n.tar.gz at the NeTraMet downloads page.

The code is a bit dated (early 2002) and doesn't sync up with the current version of libpcap (v0.9.8). I don't really need it at the moment but it might be worth looking at later.

ANSI C way to check if a file is readable

2007-12-12T08:15:00.000-06:00

I needed a dead simple ANSI C compatible way to check if a file was readable. So here it is:

int can_read_file(const char * filename) {
FILE *file=NULL;
if ((file = fopen(filename, "r"))) {
fclose(file);
return 1;
}
return 0;
}

You'll have to #include stdio.h

Bro v1.2.1 autotools tweaks

2007-12-12T03:36:00.001-06:00

Here are a few tweaks to the Bro 1.2.1 autotools files so it builds in a separate build directory.

In configure.in added the line below so I could see if broccoli was on or off:

echo "  - Building Broccoli:      "${BLD_ON}$broccoli${BLD_OFF}

In src/Makefile.am changed AM_CFLAGS to:

#MED:2007-12-11: binpac.h is generated from binpac.h.in in src/binpac/lib
AM_CFLAGS = -I. -I$(top_srcdir)/src/binpac/lib -I$(top_srcdir)/src -I$(srcdir) -I$(top_builddir) -I$(top_builddir)/src/binpac/lib

In src/Makefile.am and aux/adtrace/Makefile.am changed the LDFLAGS back to AM_LDFLAGS.

In src/binpack/Makefile.am added the line:

#MED:2007-12-11: binpac.h is generated from binpac.h.in in src/binpac/lib
libbinpac_a_CPPFLAGS = -I$(top_builddir)/src/binpac/lib -I$(top_srcdir)/src/binpac/lib

Haven't looked at the broccoli build problems yet. I just configure with --disable-broccoli.

Graphics conversion hacks

2007-12-03T17:42:00.000-06:00

I had to convert a slew of images into png and eps so they would work in a LaTeX document. The following proved useful to me:

img2eps to covert gif, jpeg, png, tiff, and xpm to eps.

giftopng to convert gifs to png. find . -iname "*.gif" -exec giftopng {} \;

And a short script to convert jpg to png:
- #!/bin/bash
- filename=${1%.jpg}
- jpegtopnm "$1" | pnmtopng > "$filename.png"

I'm sure there are better ways but these worked for me.

Installing parallel versions of GCC.

2007-12-02T03:55:00.000-06:00

I needed a GCC.version 3.3 c/c++ compiler to build Mical so I googled and found the following guides on how to install parallel versions of GCC.:

Keeping multiple versions of gcc and installing them

Installing Alternate GCC Compiler on Fedora

How to install multiple versions of GCC

I decided to build the compiler using a modified approach. First I downloaded gcc-3.3.6.tar.bz2. Extracted the source in ~/tmp and made a build directory. Then from the build directory configured with: ../gcc-3.3.6/configure --prefix=/opt/gcc/3.3.6 --enable-languages=c,c++. Did make; sudo make install | tee install.log.

Next to test it out I downloaded mical-0.1.0.tar.gz to ~/tmp. I extracted the files and then did export PATH=/opt/gcc/3.3.6/bin:$PATH; configure;make. I didn't do a make install. I'm not comfortable installing something that will need the parallel version of GCC in /local or /opt yet. But I did plink with the algorithm tests and they seemed to work correctly.

For some reason the PATH approach above didn't seem to stick so now I'm building mical by specifying the version of gcc/g++ to as a parameter to the configure script:
configure CC=/opt/gcc/3.3.6/bin/gcc CXX=/opt/gcc/3.3.6/bin/g++.

Adventures With NetDude.

2007-12-01T18:26:00.000-06:00

I've been looking for a tool that would allow me to trim a large pcap file down to specific application level connections and wanted to give NetDude a shot. My "fun" started with getting and build from source because there are not RPMs in a repository for openSUSE at this time.

Building NetDude
First I downloaded the netdude-0.4.8a.tar.gz file and the supporting libnetdude-0.10a.tar.gz and libpcapnav-0.8.tar.gz.

After extracting the arcived files I used configure; make; sudo make install | tee install.log in the following order: libpcapnav, libnetdude, netdude.

Next on to the plugins. I downloaded Essentials Pack, Appdemux, and TCP Filter plugins.

Libnetdude Plugins
AppDemux depends on several of the plugins in the Essentials
Pack and TCP Filter depends on Conntrack and Trace-Set from the Essentials Pack so the first order of business is to get the Essentials Pack built and installed.

I used configure; make; sudo make install | tee install.log in the following order: libnetdude-plugin-essentials, libnetdude-tcpfilter-plugin, libnetdude-appdemux-plugin

Netdude Plugins
The only NetDude plugin at the time of writing is the Traffic Analyzer. Unfortunately for me the version I downloaded did not build without errors. Since this was a quick hack I didn't spend time fixing the build so I was unable to try it out.

On to doing something useful with this...
I was able to demux TCP connections (transport level) with the demux plugin using lndtool (lndtool -r demux -0 wk3.01 -p we3.01_Mon.inside.tcpdump). I'm not sure that I like the way the traces are de-muxed. That is they don't directly translate into a format that I can input into a grammatical inference system. Also, I need to check on the parameters used to decide flow membership. It would probably help me out if I tweaked the demux plugin so it was parameterized.

Sadly the AppDemux plugin isn't working for me and that is the feature I really need at the moment. So my quest for a way to demux application level protocols continues. Going to re-evaluate tcpflow.

Does anyone have other flow reconstruction tools they can recommend?

Building ARPACK on openSUSE 10.3

2007-11-30T06:23:00.000-06:00

I needed an installation of ARPACK for FreeMat and couldn't find a convenient way to get it installed on an openSUSE box I was using. So I googled for a bit and came up with the following hack.

Make sure fortran is installed: rpm -q gcc-fortran.
Download the RPM from ATrpms.
Extract files from rpm using (see ):
rpm2cpio arpack-2.1.7.src.rpm | cpio -ivmud
Edit the .spec file so the BuildRequires statement for gfortran is for gcc-fortran instead of ~~gcc-gfortran~~. (That took several "rpm -q --provides gcc-gfortran" not found queries before I realized the package is named gcc-fortran on openSUSE.)
Build the RPM: rpmbuild -ba arpack.spec
Install the RPMs. In my case the RPMS were in /usr/src/packages/RPMS/i586. I installed the arpack-2.1-7.i586.rpm, then arpack-devel-2.1-7.i586.rpm and finally arpack-static-2.1-7.i586.rpm.

Unfortunately, FreeMat still does not pick up the version of ARPACK built above when running configure. I suspect it's something to do with the differences between f77 and gfortran. Check here for some fortran relevant autoconf macros.

FreeMat checks the fortran configuration with AC_PROG_F77 and AC_F77_LIBRARY_LDFLAGS in it's configure.in and uses several AC_F77_XXX macros in acinclude.m4 and checks for ARPACK with AC_F77_FUNC(znaupd).

Have to get back to this one later.

-mtune=native : GCC 4.2.x command line option

2007-11-25T17:56:00.000-06:00

Other optimization options are explained in detail at:
Using the GNU Compiler Collection Section 3.10

To use it on an automake/autoconf project in KDevelop:

In Eclipse with the CDT:

Qfsm configure.in hacks

2007-11-25T14:31:00.000-06:00

After a bit of effort I finally tweaked the configure.in for Qfsm so it works with a separate build directory. I needed this so I could build the project inside KDevelop.

The solution I used was to hack a large portion of the autoqt m4 macros into Qfsm's configure.in. A similar approach was used here.

A better solution would probably be to update autoqt so it works with a dual Qt3/Qt4 installation. There was also some useful info at Qtnode. Also, FreeMat's configure.in might be a good source for autoconf macros that are Qt4 related.

par2cmdline fix for gcc 4.x

2007-11-25T10:37:00.000-06:00

Stumbled across this E-mail which includes a patch that gets the par2cmdline from Parchive compiling with gcc 4.x. Up to this point I've used QuickPar running under wine.

What about BloGTK?

2007-11-25T10:11:00.000-06:00

What about BloGTK? I didn't try it because it wasn't in an openSUSE repository.

Test Post from drivel

2007-11-25T09:24:00.000-06:00

This is a test post from drivel. Drivel didn't have the correct server at startup. I actually found the server name for the Blogger API by trying out Gnome Blog. (The server name is http://www.blogger.com/api/RPC2).

Test Post from Gnome Blog

2007-11-25T09:21:00.000-06:00

Test from Gnome Blog

This is a test post from Gnome Blog.

Bro v1.2.1 and new ClamAV versions

2007-11-25T08:52:00.000-06:00

As mentioned here, the version 1.2.1 stable source does not build correctly if ClamAV is installed. Bro 1.2.1 is using the cl_scannbuf function which is not exported in newer versions of ClamAV.

To work around this I edited the configure.in file so it checks for the cl_scanbuff function and does not configure ClamAV support if it is not exported by ClamAV.

In the Bro configure.in I edited the ClamAV checks to read:
# Libclamav
have_libclamav=Yes
AC_CHECK_HEADERS([clamav.h],,have_libclamav=No)
AC_CHECK_LIB(clamav,cl_scanbuff,,have_libclamav=No)
#AC_CHECK_LIB(clamav,cl_retdbdir,,have_libclamav=No)

Then re-generated the configure file.

One possible "real fix" would be to re-write the FileAnalyzer functions to create temporary files and use cl_scanfile to scan the temporary file.