The problem

Have you ever named a file something like…

main.cpp.GOOD

… to indicate a “known-good” copy of `main.cpp`?

This simplistic method for storing file metadata is great because it’s filesystem-agnostic – the metadata (e.g. “this is known-good source code”) “follows” the file wherever it goes – from ext3 to FAT32 to NTFS, etc.

However, it has a few limitations:

1. Certain characters like “/”, ” “, and “?”, while possible, become a hassle later when you need to manipulate the file by name at a shell.
2. Adding, editing, and removing metadata at a shell involves lots of line-editing acrobatics especially if you want to maintain the file’s orginal extension e.g. `main.WORKING.cpp`.

To work around these limitations I wrote a small utility called `fnmeta.py` (for “filename metadata”). It uses Base64 encoding to store metadata, thus allowing you store arbitrary binary data. Beware the limits on filename length imposed by the filesystem. For the filesystems I use, the lowest common denominator is 255 characters.

If you ask `fnmeta.py` to store the string “foo” as metadata for the file `bar.baz`, it will rename `bar.baz` to `bar.TAG.ENCODED_METADATA.TAG.baz` where…

• “TAG” is the word “fnmeta” followed by a number indicating which version of fnmeta was used to encode the metadata. “fnmeta” gives a hint at why there’s a big chunk of non-english text in the filename and the number facilitates future updates to the encoding scheme.
• “ENCODED_METADATA” is base64_encode(“foo”)

Here’s an example showing how I used `fnmeta.py` to store the URL from which I downloaded a PDF named `xapp374.pdf`:

$ fnmeta.py xapp374.pdf
please enter the new metadata: www.xilinx.com/support/documentation/application_notes/xapp374.pdf

$ ls xapp374*
xapp374.fnmeta1.d3d3LnhpbGlueC5jb20vc3VwcG9ydC9kb2N1bWVudGF0aW9uL2FwcGxpY2F0aW9uX25vdGVzL3hhcHAzNzQucGRm.fnmeta1.pdf

$

Later on, I retrieved the metadata like this:

$ fnmeta.py xapp374.fnmeta1.d3d3LnhpbGlueC5jb20vc3VwcG9ydC9kb2N1bWVudGF0aW9uL2FwcGxpY2F0aW9uX25vdGVzL3hhcHAzNzQucGRm.fnmeta1.pdf
www.xilinx.com/support/documentation/application_notes/xapp374.pdf

$

`fnmeta.py` is licensed under the GPL and can be found here.

Well too bad -- it’s just not possible to ascertain details that are hidden from the camera’s view!

You can get pretty close, though, with some software I discovered via an article in the Linux Journal.

GDV uses the Visualization Toolkit (VTK) to display what are called “surface plots”. I wanted to record a simple screencast of myself rotating a surface plot by hand but I couldn’t convince VTK to do “live” previews. As a workaround I whipped up this shell script which rotates the surface plot in small increments, taking screenshots along the way:

# simulate multiple click-and-drag mouse events along predetermined

# coordinates. after each of these click-and-drag events, take a screenshot
# with imagemagick's "import" utility.

# you should generate coordinates at which to click with...
#  (while :; do xdotool getmouselocation | \
#   awk '{print substr($1,3) " " substr($2,3)}'; done) | uniq
# ... and save them to the file at COORDS_PATH

# you can assemble the screenshots into a movie with...
#  ffmpeg -r 30 -b 300k -i %05d.jpg -b 1157kb out.mp4

COORDS_PATH=~/prog/sh/screenshot_rect_while_dragging.dat
SHOTS_DIR=/media/humid_data/tmp/shots
# head -n1 ${COORDS_PATH} | read PREV_X PREV_Y
# PUZZLE: why doesnt the above work? instead we'll use...
PREV_X=343; PREV_Y=135
I=1
mkdir ${SHOTS_DIR}
while read CUR_X CUR_Y
do
  echo moving to ${CUR_X} ${CUR_Y} as step ${I}
  # move to pos
  # greets to fellow RIT alum jordan sissel, xdotool's author
  xdotool mousemove ${PREV_X} ${PREV_Y}
  # mouse down
  xdotool mousedown 1
  # move to pos
  xdotool mousemove ${CUR_X} ${CUR_Y}
  # mouse up
  xdotool mouseup 1
  # give the app some time to finish rendering
  sleep .3s
  # take screenshot
  import -crop 640x480+38+130 -window root -quality 100 \
  ${SHOTS_DIR}/$(printf "%05d" ${I}).jpg

  I=$((I+1))
  PREV_X=${CUR_X}
  PREV_Y=${CUR_Y}
done < ${COORDS_PATH}

Next I assembled the screenshots into a video with Openshot, a young but very stable and featureful nonlinear video editor. Here’s the result:

www.youtube.com/watch?v=a4UCOrIZb2s

For some more image-processing fanciness, see this slick video demo of the “structural editing” tools planned for Photoshop CS5. There’s interesting discussion (incl. some NSFW comments) at this proggit thread. For instructions on feeding “heightmaps” to a 3D printer, see these instructions.

“A5/1 [the encryption scheme used in most cellular voice calls] has operated unchanged for the last 21 years but it has now reached its cryptographic end-of-life, engulfed by the march of Moore’s Law. However, the operational end-of-life of A5/1 may still be decades away as there are approximately 2 billion GSM subscribers, commanding about 80% of the global mobile market. This would be a tough product recall indeed. A5/1 is well-positioned to become the NT of the mobile crypto world, and I see the makings of a long tail of GSM vulnerability.” - Dr. Luke O’Connor at NoTricks: Another crack at open Rainbow Tables for A5/1

The ability to intercept and decrypt GSM cell phone conversations is now well within the reach of hobbyists.¹²

My friend Scott and I were discussing this sobering fact one night and we began wondering if any systems exist which provide end-to-end encryption for this insecure link.

Sure, there are plenty of solutions for people with access to cellular dataconnections, but what can voice call participants use to foil eavesdroppers?

We didn’t find any low-cost systems so we decided to create our own. OKCrypto is the Linux-based encrypting software modem that we’ve made. It consists of two components: the modem component and crypto component.

We needed the ability to send data before we could try sending  encrypteddata, so the first step was to design a simple software modem.

Rather than executing the modem code on the cell phones themselves, I decided to host the modem code on a the sender and receiver’s Linux systems. This design provides two benefits:

1. The modem code and crypto code has access to the rich Linux API.
2. Sensitive code and data are isolated from the both the cell phone itself and from the cell infrastructure. We have clean separation between trusted (PC) and untrusted (cell) environments.

Thus, the “real work” in OKCrypto is done on the PC – the cell phones simply allow the PCs to talk to each other. See figure 1 for an overview of the hardware involved in the system.

Conceptually, the modem is similar to the analog Plain Old Telephone Service (POTS) modems from the Bad Old Days before broadband service became popular. Instead of an audio coupler, we’re using Bluetooth to connect our PC to our phone line. Instead of custom hardware, our modem is comprised of some glue scripts and the software packages they connect – all running on a Linux PC.

The modem doesn’t provide an asynchronous full-duplex communication link like traditional modems. This modem’s operation is simpler: The sender’s modem dials the number of, and subsequently transmits a pre-prepared chunk of data to, the recipient’s modem. It then hangs up.

The modem uses Dual-tone multi-frequency (DTMF) signaling to encode the data it transmits. I chose DTMF because I was familiar with it and because Debian provides a package for Multimon. OKCrypto uses two utilities that Multimon provides:

1. gen – a DTMF generation utility (digits->wav file)
2. multimon – a DTMF detection utility (wav file->digits)

I’d been working on a few Bluetooth security projects at the time, so the Hands-Free Profile (HFP)³ came immediately to mind as a convenient way to transfer audio (and any data we’ve encoded in the audio) between a PC and a cell phone during a call.

-flickr size=”small” float=”right”-4153181752-/flickr-In most cases HFP is used to connect a a Bluetooth phone to a Bluetooth headset so that the headset can be used to make calls via the phone. See figure 2.

-flickr size=”small” float=”right”-4153181518-/flickr-In the PC world, HFP is typically used to connect a desktop computer to a bluetooth headset. See figure 3. In that configuration, the PC fulfills the first of two roles mandated by the HFP specification: the Audio Gateway (AG) role. The headset fulfills the hands-free (HF) role.

Support for the HF role in bluez, the Linux Bluetooth stack (pronounced “blue-zee”), is just now maturing⁴ so I went searching for a userspace implementation of the HFP protocol stack.

One of the best, chan_mobile, is distributed as an add-on to the popular open-source private branch exchange (PBX) Asterisk system. If you configure chan_mobile to use your cell phone, Asterisk can make both outbound calls and receive inbound calls with the phone.

Asterisk is the largest software component of the OKCrypto modem. It not only provides a reliable HFP HF role implementation which works with a wide array of modern phones (see above), but also many essential telephony operations:

1. Recording audio during a call. OKCrypto uses Asterisk’s built-in voicemail capabilities.
2. Transmitting audio during a call.
3. Pausing for a given time period.
4. Logging phone call details.

See figure 5 for an overview of the software components in the OKCrypto system. Note that the same software is used on both the sender and receiver’s PCs.

Let’s look at how we can use this modem to send a 16-byte binary file over a cellular voice connection.

First, the sender and recipient will need to perform some set-up steps:

1. 1. Acquire a Linux-supported computer and Bluetooth adapter. I found that using a virtual machine introduces latency that bluez/btusb cannot tolerate.
2. Acquire a HFP-capable cell phone.
3. Install Linux, SoX, Python, multimon, Asterisk, the Asterisk “add-ons”, GPG, and the OKCrypto scripts.
4. Pair cell phone with computer. Grant HFP access.
5. Configure chan_mobile
6. Start Asterisk Next, the sender can issue this command at a shell to send the file ‘/tmp/foo’ to the recipient at 585-555-3258.

   $ ./bin_to_int_seq.py /tmp/foo | ./ast_send.sh 5855553258 [...]
   $ cksum /tmp/foo 668417501 16 /tmp/foo

   She’ll notice her PC making a call with her phone, silently transmitting the data, and disconnecting. Behind the scenes, the OKCrypto scripts will…

7. Convert the bytes in /tmp/foo to a series of decimal digits.
8. Encode the digits as DTMF tones with gen from the multimon package. 1. Increase the pitch (time-independent) to prevent any intermediary systems (esp. Asterisk) from interpreting the tones.
9. Queue the final audio file for transmission by Asterisk.

The recipient will hear his phone ring once before his PC answers the call, records the audio, and disconnects. When the call is complete, Asterisk will invoke one of the OKCrypto scripts to…

1. Decrease the pitch to yield the original DTMF tones.
2. Decode the DTMF tones to a series of decimal digits.
3. Convert the digits to a series of bytes which is written to ‘/tmp/bar’.

   $ tr -d '\n' < /tmp/newest_vm.txt | ./int_seq_to_bin.py /tmp/bar [...]
   $ cksum /tmp/bar 668417501 16 /tmp/bar

Now that we can reliably send data, let’s make sure that it’s encrypted first. This turns out to be one of the simplest components of the system – many good crypto APIs are available.

I use GnuPG:

$ gpg --symmetric --force-mdc --cipher-algo AES256 filetoencrypt 

The `–force-mdc` option provides integrity checking – useful for handling transmission errors. Consider these GPG options carefully and make sure they fit your requirements.

Here are a few ideas we’re pursuing for the future of this project:

• Moving the modem code to the cell phone. This would simplify the setup but potentially risk security.
• Improving modem error rate. Data is often erroneously duplicated during transmission.
• Increasing the modem throughput. The current code averages a meager 10 bytes/second.
• Hiding the data within a steganographic channel in a normal voice conversation.
• Incorporating GPG into the OKCrypto scripts.
• Packaging the system as a LiveCD/Live flash drive.
• Implementing key exchange.
• Building an embedded device dedicated to OKCrypto.

I gained experience in the follow areas during the design, implementation. and testing:

• GSM, CDMA crypto. I suspect the cellular phone industry would make a great case study in protocol security by obscurity.
• Cellular voice codecs used by large carriers. Trivia: your calls only require ~10kbps.⁵
• Bluetooth HFP specification and available implementations.
• Debugging latency tolerances in virtual machine USB “passthrough” subsystems.
• The Twilio telephony API. I used Twilio when I only had one cell phone to debug with.
• Asterisk administration.

I’ve described a method to securely transmit data over any of the widely-available cell voice networks. The implementation requires only commodity hardware, open-source software, and minimal setup.

Be aware that transmitting data by “automated means” may violate the terms of your cellular service contract. I disclaim all liability. This information is provided for educational purposes only.

You can download the OKCrypto system here: http://www.unsyncopated.com/corral/okcrypto_v0.1.tar.gz

It is licensed under the LGPL.

• You’ll find lots of links to cell security architecture articles, Bluetooth HFP implementations, and Asterisk administration web pages on the wiki at Crypto Phone/Stacked Linux-based CPhone Brainstorming

Update 1/13/09 - Check out some technical notes on our recent progress with a faster and more reliable modem: OKCrypto/Progress Report for 01-09-2010

1. Open-Source Effort to Hack GSM John Blau – IEEE Spectrum Magazine – December 2009 issue
2. Subverting the security base of GSM Karsten Nohl – Hacking at Random – 8/15/2009
3. Bluetooth Hands-Free Profile (HFP) 1.5 – Bluetooth Special Interest Group – 11/25/2005
4. Comment #1 on Maemo bug #2754 – Johan Hedberg – 1/25/2009
5. Adaptive Multi-Rate audio codec – Wikipedia – 11/24/2009

• You can earn money when people download your files via their affiliate program.
• Your audience can access your files via HTTPS.
• An honest dedication to free speech and free software.
• You can upload your files via FTP-over-explicit-SSL (FTPES). Their support page doesn’t list their FTPES host key fingerprint but you ”can” just barely see it at 0m36s on their FileZilla support page. Update: they’ve posted the fingerprint in their support forum. See my notes on uploading with lftp(1) for more info on securing your CLI FTP transactions.

Their servers are also hooked up to especially fat pipes. I get 5 megabytes/sec sustained with my Thinkpad T30′s wired NIC on RIT’s library network. I got 10 megabytes/sec with my Rimuhosting VPS in their Level(3) and Abovenet-connected datacenter.

I found good.net via a link in the McGrew Security blog to good.net’s mirror of DarkOz‘s giant collection of security conference videos – the “Hacker Media Archive”.

The Archive’s 25th Chaos Communication Congress (25C3) videos alone occupy nearly 40GB…

$ curl --silent http://avondale.good.net/dl/bd/25c3/video_h264_720x576/ |
awk --assign i=0 '/.mp4"/{i=i+substr($8, 1, length($8)-1);} END {print i}'

39175

In addition to the CCC videos, you can find footage and materials from DEFCON, HOPE, Black Hat, CodeCon, DeepSec, HITB, NOTACON, PhreakNIC, REcon, Shmoocon, and ToorCon. Phew!

1. If you’re looking for a more ”active” filesharing tool, one that syncs files across machines for you, try Dropbox as recommended by an aspiring grocer friend of mine. They’ve even got a Linux client.
2. Yes, the above manhole cover photo was taken by the Doc Searls of Linux Journal fame.

$ stream.sh 1370

I usually edit small scripts like that with `nano(1)`. Rather than type the entire path to the script when I need to edit it (`nano ~/prog/sh/stream.sh`) I just say `nano $(which stream.sh)` i.e. “edit whichever file is in the `$PATH` with that name”.

It’s actually a brain-saver with more verbose names, I promise `:]`.

“Step by Step” by Chronos on di.fm‘s chillout station as recommended by DJ JMULV

• An article on Zsh(1) completion from Linux Magazine
• mikas blog – Blog Archive – mika’s advent calendar – day 19: zsh completion

As is usually the case, I had to work sweat a little to make it work. My problem was that my zsh prompt was too fancy. The solution was to toss this conditional in at the end of my .zshrc:

if [ $TERM = "dumb" ]; then
   unsetopt zle
   export PS1="%% "
fi

This works by disabling the zsh line editor (“zle”) and setting a plain vanilla prompt (“PS1″) that tramp can recognize when it logs in.

TRAMP is a little slow on my machine but it sure beats firing up a new Emacs instance every time I want to edit `/etc/superdooper.conf`

If someone somehow managed to get ahold of my shell’s history file, they’d get a lot of juicy info like:

• What system and web accounts I have.
• The names I used for my accounts which might be helpful in the cryptoanalysis of my pwsafe database.

So what I’ve done is set up zsh, my shell, to not save invocations of pwsafe in my history. With the “HIST_IGNORE_SPACE” option, you can tell zsh to ignore commands that begin with a space. I did this for a while but found myself forgetting to type the space sometimes. The trick is to set up an alias which includes a leading space:

alias pwsafe=" pwsafe"

So now whenever I run something like

pwsafe -l ubuntu.forums -up

it won’t get saved to the history file.

• The Z-Shell Manual – 16.2.4: History options

This happened to me last night when I was watching ”Casino”. The parking lot attendant from a certain scene looked just like a character from ”Fear and Loathing in Las Vegas”.

To the *nix shell, Robin!</Bruce Wayne Voice>

I wrote a short script to download the IMDb “profile” pages for all the actors in movie X and list which ones also acted in movie Y. This is, effectively, a set intersection operation.

I could’ve registered for IMDb’s 14-day trial of their $20/mo “Pro” service which allows advanced searches on their database, but I wanted a challenge.

Here are the important pieces of the script:

# grab (only) the profile pages for each actor who performed in title
# 'tt0112641' ('casino'). recurse, but only 1 level deep into links.
wget --recursive --level=1 --wait=1 --include_directories=/names
'http://www.imdb.com/title/tt0112641/fullcredits'

# determine if any of the retrieved profile pages contain a reference
# to 'fear and loathing'
find . -type f -print | xargs grep --files-with-matches
--ignore-case 'fear and loathing'

The script gave me ~10 results most of which were off-screen roles e.g. “set designer”. One, however, was the dude I was looking for: Brian LeBaron!. He acted as a parking attendant in both Casino and Fear and Loathing.

Update: There are now much more elegant ways to interact with IMDb. See, for example, IMDbPY

Update: More retrospection: the “combine” utility in moreutils (via deb-a-day) might’ve been handy.

The problem

You’ve piped the lengthy output of some command to your pager, less. As you’re scrolling through the output you realize you’d like to save it to disk. The output took a long time to generate, so you don’t want to re-run the original command just to redirect the output to a file.

Your first instinct might be to press ‘v’ to edit the output with $EDITOR. less will balk “Cannot edit standard input (press RETURN)”.

The solution is to use less’ “|” command key to pipe the output to tee which will write it to a file. See the “Commands” section in LESS(1) for the full syntax of this command. Pressing the ‘h’ command key for “help” will give an abbreviated syntax under its “MISCELLANEOUS COMMANDS” section:

"|Xcommand   Pipe file between current pos & mark X to shell command."

You want to pipe the entire buffer to a file so first press “g” to go to the first line. Next, press the “|” key followed by “$” to indicate you want to pipe right up until the end of the buffer. “$” is a predefined mark for the end of the file. Lastly, type “tee” followed by a space and the name of the file you’d like to save to.

So, from anywhere in a less buffer, you can type the following to save the buffer to a file named foo.txt:

g|$tee foo.txt

you’ll see “|done (press RETURN)”.

• LESS(1) –log-file
• git-diff-tree(1) –pickaxe-regex

cp /etc/slop/gunk.conf /etc/slop/gunk.conf_before_v2_upgrade

Since the first portion of the second argument is always the same as the first argument (the green text), it bugged me to have to do tab completion for it. Luckily, zsh and bash provide an event designator for the current command line (ie the one you’re currently typing). It is !# and it works just like the other events (eg !! is the previous command). See Zsh Documentation – History Expansion. Bash’s syntax is very similar.

So now when I want to make a crude backup like above, I use something like this:

cp /etc/slop/gunk.conf !#:1_before_v2_upgrade