Storing file metadata with fnmeta.py


The problem

Have you ever named a file something like…

main.cpp.GOOD

… to indicate a “known-good” copy of `main.cpp`?

This simplistic method for storing file metadata is great because it’s filesystem-agnostic – the metadata (e.g. “this is known-good source code”) “follows” the file wherever it goes – from ext3 to FAT32 to NTFS, etc.

However, it has a few limitations:

  1. Certain characters like “/”, ” “, and “?”, while possible, become a hassle later when you need to manipulate the file by name at a shell.
  2. Adding, editing, and removing metadata at a shell involves lots of line-editing acrobatics especially if you want to maintain the file’s orginal extension e.g. `main.WORKING.cpp`.

The solution

To work around these limitations I wrote a small utility called `fnmeta.py` (for “filename metadata”). It uses Base64 encoding to store metadata, thus allowing you store arbitrary binary data. Beware the limits on filename length imposed by the filesystem. For the filesystems I use, the lowest common denominator is 255 characters.

If you ask `fnmeta.py` to store the string “foo” as metadata for the file `bar.baz`, it will rename `bar.baz` to `bar.TAG.ENCODED_METADATA.TAG.baz` where…

  • TAG” is the word “fnmeta” followed by a number indicating which version of fnmeta was used to encode the metadata. “fnmeta” gives a hint at why there’s a big chunk of non-english text in the filename and the number facilitates future updates to the encoding scheme.
  • ENCODED_METADATA” is base64_encode(“foo”)

An example

Here’s an example showing how I used `fnmeta.py` to store the URL from which I downloaded a PDF named `xapp374.pdf`:


$ fnmeta.py xapp374.pdf
please enter the new metadata: www.xilinx.com/support/documentation/application_notes/xapp374.pdf

$ ls xapp374*
xapp374.fnmeta1.d3d3LnhpbGlueC5jb20vc3VwcG9ydC9kb2N1bWVudGF0aW9uL2FwcGxpY2F0aW9uX25vdGVzL3hhcHAzNzQucGRm.fnmeta1.pdf

$

Later on, I retrieved the metadata like this:


$ fnmeta.py xapp374.fnmeta1.d3d3LnhpbGlueC5jb20vc3VwcG9ydC9kb2N1bWVudGF0aW9uL2FwcGxpY2F0aW9uX25vdGVzL3hhcHAzNzQucGRm.fnmeta1.pdf
www.xilinx.com/support/documentation/application_notes/xapp374.pdf

$

The code

`fnmeta.py` is licensed under the GPL and can be found here.


Write a Comment

Take a moment to comment and tell us what you think. Some basic HTML is allowed for formatting.

Reader Comments

Be the first to leave a comment!