<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>Tyler Oderkirk&#039;s Blog &#187; Syntax highlighter</title>
	<atom:link href="http://unsyncopated.com/blog/index.php/category/plugins-used/syntax-highlighter/feed/" rel="self" type="application/rss+xml" />
	<link>http://unsyncopated.com/blog</link>
	<description>&#34;Scared money don&#039;t[sic] make money&#34;</description>
	<lastBuildDate>Thu, 27 Jan 2011 05:02:58 +0000</lastBuildDate>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.org/?v=3.0.1</generator>
		<item>
		<title>Adding a dimension to your photos</title>
		<link>http://unsyncopated.com/blog/index.php/2010/03/09/adding-a-dimension-to-your-photos/</link>
		<comments>http://unsyncopated.com/blog/index.php/2010/03/09/adding-a-dimension-to-your-photos/#comments</comments>
		<pubDate>Tue, 09 Mar 2010 20:13:47 +0000</pubDate>
		<dc:creator>Tyler</dc:creator>
				<category><![CDATA[Flickr]]></category>
		<category><![CDATA[Shell programming]]></category>
		<category><![CDATA[Syntax highlighter]]></category>
		<category><![CDATA[Youtube]]></category>

		<guid isPermaLink="false">http://unsyncopated.com/blog/index.php/2010/03/10/adding-a-dimension-to-your-photos/</guid>
		<description><![CDATA[Have you ever wanted to transform a run-of-the-mill photograph into a complete 3D scene automagically like Jack Black does in this mildly-NSFW 35MiB 2min clip from Enemy of the State? Well too bad -- it&#8217;s just not possible to ascertain details that are hidden from the camera&#8217;s view! You can get pretty close, though, with [...]]]></description>
			<content:encoded><![CDATA[<p>Have you ever wanted to transform a run-of-the-mill photograph into a complete 3D scene automagically like Jack Black does in <a href="http://unsyncopated.com/corral/enemy_of_the_state_trimmed.mpeg">this mildly-NSFW 35MiB 2min clip</a> from <a href="http://www.imdb.com/title/tt0120660/">Enemy of the State</a>?</p>
<p>Well too bad -- it&#8217;s just not possible to ascertain details that are hidden from the camera&#8217;s view!</p>
<p>You can get pretty close, though, with some software I discovered via <a href="http://www.linuxjournal.com/magazine/new-projects-fresh-labs-14">an article in the Linux Journal</a>.</p>
<p>						<div class="flickr-gallery image right"><a href="http://www.flickr.com/photos/55502932@N00/4421405150"><img class="flickr small" title="Loctite Epoxy Bottle" alt="Loctite Epoxy Bottle" src="http://farm5.static.flickr.com/4002/4421405150_632821eaef_m.jpg" /></a></div>
					The software is called Discrete Geometry Viewer (DGV) and I&#8217;ll illustrate the salient feature with this photo I took of a cylindrical epoxy bottle:</p>
<p>GDV uses <a href="http://www.vtk.org/">the Visualization Toolkit (VTK)</a> to display what are called &#8220;surface plots&#8221;. I wanted to record a simple screencast of myself rotating a surface plot by hand but <a href="http://groups.google.com/group/discrete-geometry-viewer/browse_thread/thread/77df519c03d8eb6f">I couldn&#8217;t convince VTK to do &#8220;live&#8221; previews</a>. As a workaround I whipped up this shell script which rotates the surface plot in small increments, taking screenshots along the way:</p>
<pre class="brush: bash;">
# simulate multiple click-and-drag mouse events along predetermined

# coordinates. after each of these click-and-drag events, take a screenshot
# with imagemagick's &quot;import&quot; utility.

# you should generate coordinates at which to click with...
#  (while :; do xdotool getmouselocation | \
#   awk '{print substr($1,3) &quot; &quot; substr($2,3)}'; done) | uniq
# ... and save them to the file at COORDS_PATH

# you can assemble the screenshots into a movie with...
#  ffmpeg -r 30 -b 300k -i %05d.jpg -b 1157kb out.mp4

COORDS_PATH=~/prog/sh/screenshot_rect_while_dragging.dat
SHOTS_DIR=/media/humid_data/tmp/shots
# head -n1 ${COORDS_PATH} | read PREV_X PREV_Y
# PUZZLE: why doesnt the above work? instead we'll use...
PREV_X=343; PREV_Y=135
I=1
mkdir ${SHOTS_DIR}
while read CUR_X CUR_Y
do
  echo moving to ${CUR_X} ${CUR_Y} as step ${I}
  # move to pos
  # greets to fellow RIT alum jordan sissel, xdotool's author
  xdotool mousemove ${PREV_X} ${PREV_Y}
  # mouse down
  xdotool mousedown 1
  # move to pos
  xdotool mousemove ${CUR_X} ${CUR_Y}
  # mouse up
  xdotool mouseup 1
  # give the app some time to finish rendering
  sleep .3s
  # take screenshot
  import -crop 640x480+38+130 -window root -quality 100 \
  ${SHOTS_DIR}/$(printf &quot;%05d&quot; ${I}).jpg

  I=$((I+1))
  PREV_X=${CUR_X}
  PREV_Y=${CUR_Y}
done &lt; ${COORDS_PATH}
</pre>
<p>Next I assembled the screenshots into a video with <a href="http://www.openshotvideo.com/">Openshot</a>, a young but very stable and featureful nonlinear video editor. Here&#8217;s the result:</p>
<p>	<!-- Smart Youtube -->
	<span class="youtube">
		<object width="425" height="362">
			<param name="movie" value="http://www.youtube-nocookie.com/v/a4UCOrIZb2s&amp;rel=1&amp;color1=d6d6d6&amp;color2=f0f0f0&amp;border=1&amp;fs=1&amp;hl=en&amp;autoplay=0&amp;showinfo=0&amp;iv_load_policy=3&amp;showsearch=0" />
			<param name="allowFullScreen" value="true" />
			<embed wmode="transparent" 
				src="http://www.youtube-nocookie.com/v/a4UCOrIZb2s&amp;rel=1&amp;color1=d6d6d6&amp;color2=f0f0f0&amp;border=1&amp;fs=1&amp;hl=en&amp;autoplay=0&amp;showinfo=0&amp;iv_load_policy=3&amp;showsearch=0" 
				type="application/x-shockwave-flash" 
				allowfullscreen="true" 
				width="425" 
				height="362">
			</embed>
			<param name="wmode" value="transparent" />
		</object>
	</span><a href="http://www.youtube.com/watch?v=a4UCOrIZb2s">www.youtube.com/watch?v=a4UCOrIZb2s</a></p>
<p>For some more image-processing fanciness, see <a href="http://cs5.org/?p=147">this slick video demo of the &#8220;structural editing&#8221; tools planned for Photoshop CS5</a>. There&#8217;s interesting discussion (incl. some NSFW comments) at <a href="http://www.reddit.com/r/programming/comments/9o4uu/improved_image_editing_in_photoshop_cs5_demo/">this proggit thread</a>. For instructions on feeding &#8220;heightmaps&#8221; to a 3D printer, see <a href="http://www.thingiverse.com/thing:2078">these instructions</a>.</p>
]]></content:encoded>
			<wfw:commentRss>http://unsyncopated.com/blog/index.php/2010/03/09/adding-a-dimension-to-your-photos/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
<enclosure url="http://unsyncopated.com/corral/enemy_of_the_state_trimmed.mpeg" length="35631104" type="video/mpeg" />
		</item>
		<item>
		<title>Sending data via cellular voice connections with Linux</title>
		<link>http://unsyncopated.com/blog/index.php/2009/11/19/sending-data-via-cellular-voice-connections-with-linux/</link>
		<comments>http://unsyncopated.com/blog/index.php/2009/11/19/sending-data-via-cellular-voice-connections-with-linux/#comments</comments>
		<pubDate>Thu, 19 Nov 2009 10:45:03 +0000</pubDate>
		<dc:creator>Tyler</dc:creator>
				<category><![CDATA[Flickr]]></category>
		<category><![CDATA[Footnotes]]></category>
		<category><![CDATA[Linux]]></category>
		<category><![CDATA[Python programming]]></category>
		<category><![CDATA[Security]]></category>
		<category><![CDATA[Shell programming]]></category>
		<category><![CDATA[Syntax highlighter]]></category>
		<category><![CDATA[Table of Contents]]></category>

		<guid isPermaLink="false">http://unsyncopated.com/blog/index.php/2009/11/20/sending-data-via-cellular-voice-connections-with-linux/</guid>
		<description><![CDATA[&#8220;A5/1 [the encryption scheme used in most cellular voice calls] has operated unchanged for the last 21 years but it has now reached its cryptographic end-of-life, engulfed by the march of Moore&#8217;s Law. However, the operational end-of-life of A5/1 may still be decades away as there are approximately 2 billion GSM subscribers, commanding about 80% of [...]]]></description>
			<content:encoded><![CDATA[<div>
<p style="padding-left: 30px;">&#8220;A5/1 [the encryption scheme used in most cellular voice calls] has operated unchanged for the last 21 years but it has now reached its cryptographic end-of-life, engulfed by the march of Moore&#8217;s Law. However, the operational end-of-life of A5/1 may still be decades away as there are approximately 2 billion GSM subscribers, commanding about 80% of the global mobile market. This would be a tough product recall indeed. A5/1 is well-positioned to become the NT of the mobile crypto world, and I see the makings of a long tail of GSM vulnerability.&#8221; - <a href="http://ch.linkedin.com/pub/luke-o-connor/1/bb6/152">Dr. Luke O&#8217;Connor</a> at <a href="http://lukenotricks.blogspot.com/2009/09/another-crack-at-open-rainbow-tables.html">NoTricks: Another crack at open Rainbow Tables for A5/1</a></p>
<div class='toc wptoc'>
<h2>Contents</h2>
<ol class='toc-odd level-1'>
	<li>
		<a href="#Introduction">Introduction</a>
	</li>
	<li>
		<a href="#The_modem__Hardware_components">The modem / Hardware components</a>
	</li>
	<li>
		<a href="#Transmitting_audio_from_a_PC_to_a_cell_phone__Software_components">Transmitting audio from a PC to a cell phone / Software components</a>
	</li>
	<li>
		<a href="#Demonstration__OKCrypto_scripts">Demonstration / OKCrypto scripts</a>
	</li>
	<li>
		<a href="#Encryption">Encryption</a>
	</li>
	<li>
		<a href="#Further_work">Further work</a>
	</li>
	<li>
		<a href="#What_I_learned">What I learned</a>
	</li>
	<li>
		<a href="#Conclusion__Download">Conclusion / Download</a>
	</li>
	<li>
		<a href="#Further_reading">Further reading</a>
	</li>
</ol>
</ol>
</ol>
</div>
<div class='wptoc-end'>&nbsp;</div>
<span id="Introduction"><h3>Introduction</h3></span>
<p>The ability to intercept and decrypt GSM cell phone conversations is now well within the reach of hobbyists.<sup>1</sup><sup>2</sup></p>
<p>My friend Scott and I were discussing this sobering fact one night and we began wondering if any systems exist which provide end-to-end encryption for this insecure link.</p>
<p>Sure, there are plenty of solutions for people with access to cellular dataconnections, but what can voice call participants use to foil eavesdroppers?</p>
<p>We didn&#8217;t find any low-cost systems so we decided to create our own. OKCrypto is the Linux-based encrypting software modem that we&#8217;ve made. It consists of two components: the modem component and crypto component.</p>
<span id="The_modem__Hardware_components"><h3>The modem / Hardware components</h3></span>
<p>We needed the ability to send data before we could try sending  encrypteddata, so the first step was to design a simple software modem.</p>
<p>Rather than executing the modem code on the cell phones themselves, I decided to host the modem code on a the sender and receiver&#8217;s Linux systems. This design provides two benefits:</p>
<ol>
<li>The modem code and crypto code has access to the rich Linux API.</li>
<li>Sensitive code and data are isolated from the both the cell phone itself and from the cell infrastructure. We have clean separation between trusted (PC) and untrusted (cell) environments.</li>
</ol>
<p>Thus, the &#8220;real work&#8221; in OKCrypto is done on the PC &#8211; the cell phones simply allow the PCs to talk to each other. See figure 1 for an overview of the hardware involved in the system. 						<div class="flickr-gallery image right"><a href="http://www.flickr.com/photos/55502932@N00/4153181728"><img class="flickr small" title="Fig. 1 - Hardware components" alt="Fig. 1 - Hardware components" src="http://farm3.static.flickr.com/2534/4153181728_a0fe8e3a63_m.jpg" /></a></div>
					</p>
<p>Conceptually, the modem is similar to the analog Plain Old Telephone Service (POTS) modems from the Bad Old Days before broadband service became popular. Instead of an audio coupler, we&#8217;re using Bluetooth to connect our PC to our phone line. Instead of custom hardware, our modem is comprised of some glue scripts and the software packages they connect &#8211; all running on a Linux PC.</p>
<p>The modem doesn&#8217;t provide an asynchronous full-duplex communication link like traditional modems. This modem&#8217;s operation is simpler: The sender&#8217;s modem dials the number of, and subsequently transmits a pre-prepared chunk of data to, the recipient&#8217;s modem. It then hangs up.</p>
<p>The modem uses Dual-tone multi-frequency (DTMF) signaling to encode the data it transmits. I chose DTMF because I was familiar with it and because Debian provides a package for <a href="http://www.baycom.org/~tom/ham/linux/multimon.html">Multimon</a>. OKCrypto uses two utilities that Multimon provides:</p>
<ol>
<li>gen &#8211; a DTMF generation utility (digits-&gt;wav file)</li>
<li>multimon &#8211; a DTMF detection utility (wav file-&gt;digits)</li>
</ol>
<span id="Transmitting_audio_from_a_PC_to_a_cell_phone__Software_components"><h3>Transmitting audio from a PC to a cell phone / Software components</h3></span>
<p>I&#8217;d been working on a few Bluetooth security projects at the time, so the Hands-Free Profile (HFP)<sup>3</sup> came immediately to mind as a convenient way to transfer audio (and any data we&#8217;ve encoded in the audio) between a PC and a cell phone during a call.</p>
<p>-flickr size=&#8221;small&#8221; float=&#8221;right&#8221;-4153181752-/flickr-In most cases HFP is used to connect a a Bluetooth phone to a Bluetooth headset so that the headset can be used to make calls via the phone. See figure 2.</p>
<p>-flickr size=&#8221;small&#8221; float=&#8221;right&#8221;-4153181518-/flickr-In the PC world, HFP is typically used to connect a desktop computer to a bluetooth headset. See figure 3. In that configuration, the PC fulfills the first of two roles mandated by the HFP specification: the Audio Gateway (AG) role. The headset fulfills the hands-free (HF) role.</p>
						<div class="flickr-gallery image right"><a href="http://www.flickr.com/photos/55502932@N00/4152393267"><img class="flickr small" title="Fig. 4 - PC ~ Cell phone" alt="Fig. 4 - PC ~ Cell phone" src="http://farm3.static.flickr.com/2737/4152393267_95526e2aaf_m.jpg" /></a></div>
					For our purposes, we need the PC to appear to be a bluetooth headset to a cell phone &#8211; the PC must fulfill the HF role. See figure 4. This is similar to how call-handling in-vehicle GPS/stereo units work.</p>
<p>Support for the HF role in <a href="http://www.bluez.org/">bluez</a>, the Linux Bluetooth stack (pronounced &#8220;blue-zee&#8221;), is just now maturing<sup>4</sup> so I went searching for a userspace implementation of the HFP protocol stack.</p>
<p>One of the best, chan_mobile, is distributed as an add-on to the popular open-source private branch exchange (PBX) <a href="http://www.asterisk.org/">Asterisk</a> system. If you configure chan_mobile to use your cell phone, Asterisk can make both outbound calls and receive inbound calls with the phone.</p>
<p>Asterisk is the largest software component of the OKCrypto modem. It not only provides a reliable HFP HF role implementation which works with a wide array of modern phones (see above), but also many essential telephony operations:</p>
<ol>
<li>Recording audio during a call. OKCrypto uses Asterisk&#8217;s built-in voicemail capabilities.</li>
<li>Transmitting audio during a call.</li>
<li>Pausing for a given time period.</li>
<li>Logging phone call details.</li>
</ol>
<p>See figure 5 for an overview of the software components in the OKCrypto system. Note that the same software is used on both the sender and receiver&#8217;s PCs.						<div class="flickr-gallery image right"><a href="http://www.flickr.com/photos/55502932@N00/4153379126"><img class="flickr small" title="Fig. 5 - Software components" alt="Fig. 5 - Software components" src="http://farm3.static.flickr.com/2521/4153379126_22a0fbc569_m.jpg" /></a></div>
					
<span id="Demonstration__OKCrypto_scripts"><h3>Demonstration / OKCrypto scripts</h3></span>
<p>Let&#8217;s look at how we can use this modem to send a 16-byte binary file over a cellular voice connection.</p>
<p>First, the sender and recipient will need to perform some set-up steps:</p>
<ol>
<li>1. Acquire a Linux-supported computer and Bluetooth adapter. I found that using a virtual machine introduces latency that bluez/btusb cannot tolerate.</li>
<li>Acquire a HFP-capable cell phone.</li>
<li>Install Linux, <a href="http://sox.sourceforge.net/">SoX</a>, Python, multimon, Asterisk, the Asterisk &#8220;add-ons&#8221;, GPG, and the OKCrypto scripts.</li>
<li>Pair cell phone with computer. Grant HFP access.</li>
<li>Configure chan_mobile</li>
<li>Start Asterisk Next, the sender can issue this command at a shell to send the file &#8216;/tmp/foo&#8217; to the recipient at 585-555-3258.
<pre class="brush: bash;">$ ./bin_to_int_seq.py /tmp/foo | ./ast_send.sh 5855553258 [...]
$ cksum /tmp/foo 668417501 16 /tmp/foo</pre>
<p>She&#8217;ll notice her PC making a call with her phone, silently transmitting the data, and disconnecting. Behind the scenes, the OKCrypto scripts will&#8230;</li>
<li>Convert the bytes in /tmp/foo to a series of decimal digits.</li>
<li>Encode the digits as DTMF tones with gen from the multimon package. 1. Increase the pitch (time-independent) to prevent any intermediary systems (esp. Asterisk) from interpreting the tones.</li>
<li>Queue the final audio file for transmission by Asterisk.</li>
</ol>
<p>The recipient will hear his phone ring once before his PC answers the call, records the audio, and disconnects. When the call is complete, Asterisk will invoke one of the OKCrypto scripts to&#8230;</p>
<ol>
<li>Decrease the pitch to yield the original DTMF tones.</li>
<li>Decode the DTMF tones to a series of decimal digits.</li>
<li>Convert the digits to a series of bytes which is written to &#8216;/tmp/bar&#8217;.
<pre class="brush: bash;">$ tr -d '\n' &lt; /tmp/newest_vm.txt | ./int_seq_to_bin.py /tmp/bar [...]
$ cksum /tmp/bar 668417501 16 /tmp/bar</pre>
</li>
</ol>
<span id="Encryption"><h3>Encryption</h3></span>
<p>Now that we can reliably send data, let&#8217;s make sure that it&#8217;s encrypted first. This turns out to be one of the simplest components of the system &#8211; many good crypto APIs are available.</p>
<p>I use <a href="http://www.gnupg.org/">GnuPG</a>:</p>
<pre class="brush: bash;">$ gpg --symmetric --force-mdc --cipher-algo AES256 filetoencrypt </pre>
<p>The `&#8211;force-mdc` option provides integrity checking &#8211; useful for handling transmission errors. Consider these GPG options carefully and make sure they fit your requirements.</p>
<span id="Further_work"><h3>Further work</h3></span>
<p>Here are a few ideas we&#8217;re pursuing for the future of this project:</p>
<ul>
<li>Moving the modem code to the cell phone. This would simplify the setup but potentially risk security.</li>
<li>Improving modem error rate. Data is often erroneously duplicated during transmission.</li>
<li>Increasing the modem throughput. The current code averages a meager 10 bytes/second.</li>
<li>Hiding the data within a steganographic channel in a normal voice conversation.</li>
<li>Incorporating GPG into the OKCrypto scripts.</li>
<li>Packaging the system as a LiveCD/Live flash drive.</li>
<li>Implementing key exchange.</li>
<li>Building an embedded device dedicated to OKCrypto.</li>
</ul>
<span id="What_I_learned"><h3>What I learned</h3></span>
<p>I gained experience in the follow areas during the design, implementation. and testing:</p>
<ul>
<li>GSM, CDMA crypto. I suspect the cellular phone industry would make a great case study in protocol security by obscurity.</li>
<li>Cellular voice codecs used by large carriers. Trivia: your calls only require ~10kbps.<sup>5</sup></li>
<li>Bluetooth HFP specification and available implementations.</li>
<li>Debugging latency tolerances in virtual machine USB &#8220;passthrough&#8221; subsystems.</li>
<li>The <a href="http://www.twilio.com/">Twilio</a> telephony API. I used Twilio when I only had one cell phone to debug with.</li>
<li>Asterisk administration.</li>
</ul>
<span id="Conclusion__Download"><h3>Conclusion / Download</h3></span>
<p>I&#8217;ve described a method to securely transmit data over any of the widely-available cell voice networks. The implementation requires only commodity hardware, open-source software, and minimal setup.</p>
<p>Be aware that transmitting data by &#8220;automated means&#8221; may violate the terms of your cellular service contract. I disclaim all liability. This information is provided for educational purposes only.</p>
<p>You can download the OKCrypto system here: http://www.unsyncopated.com/corral/okcrypto_v0.1.tar.gz</p>
<p>It is licensed under the LGPL.</p>
<span id="Further_reading"><h3>Further reading</h3></span>
<ul>
<li>You&#8217;ll find lots of links to cell security architecture articles, Bluetooth HFP implementations, and Asterisk administration web pages on the wiki at <a href="http://www.unsyncopated.com/wiki/Crypto%20Phone/Stacked%20Linux-based%20CPhone%20Brainstorming">Crypto Phone/Stacked Linux-based CPhone Brainstorming</a></li>
</ul>
<p>Update 1/13/09 - Check out some technical notes on our recent progress with a faster and more reliable modem: <a href="https://www.unsyncopated.com/wiki/OKCrypto/Progress%20Report%20for%2001-09-2010">OKCrypto/Progress Report for 01-09-2010</a></p>
</div>
<ol class="footnotes">
<li id="footnote_0_317" class="footnote"><a href="http://spectrum.ieee.org/telecom/wireless/open-source-effort-to-hack-gsm">Open-Source Effort to Hack GSM</a> John Blau &#8211; IEEE Spectrum Magazine &#8211; December 2009 issue</li>
<li id="footnote_1_317" class="footnote"><a href="https://har2009.org/program/attachments/119_GSM.A51.Cracking.Nohl.pdf">Subverting the security base of GSM</a> Karsten Nohl &#8211; Hacking at Random &#8211; 8/15/2009</li>
<li id="footnote_2_317" class="footnote"><a href="http://www.bluetooth.com/Bluetooth/Technology/Works/HFP.htm">Bluetooth Hands-Free Profile (HFP) 1.5</a> &#8211; Bluetooth Special Interest Group &#8211; 11/25/2005</li>
<li id="footnote_3_317" class="footnote"><a href="https://bugs.maemo.org/show_bug.cgi?id=2754#c1">Comment #1 on Maemo bug #2754</a> &#8211; Johan Hedberg &#8211; 1/25/2009</li>
<li id="footnote_4_317" class="footnote"><a href="http://en.wikipedia.org/w/index.php?title=Adaptive_Multi-Rate_audio_codec&amp;oldid=327719647">Adaptive Multi-Rate audio codec</a> &#8211; Wikipedia &#8211; 11/24/2009</li>
</ol>
]]></content:encoded>
			<wfw:commentRss>http://unsyncopated.com/blog/index.php/2009/11/19/sending-data-via-cellular-voice-connections-with-linux/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Creating a local mirror of Ubuntu&#8217;s most popular packages</title>
		<link>http://unsyncopated.com/blog/index.php/2009/11/04/creating-a-local-mirror-of-ubuntus-most-popular-packages/</link>
		<comments>http://unsyncopated.com/blog/index.php/2009/11/04/creating-a-local-mirror-of-ubuntus-most-popular-packages/#comments</comments>
		<pubDate>Wed, 04 Nov 2009 22:25:55 +0000</pubDate>
		<dc:creator>Tyler</dc:creator>
				<category><![CDATA[Linux]]></category>
		<category><![CDATA[Syntax highlighter]]></category>
		<category><![CDATA[Table of Contents]]></category>

		<guid isPermaLink="false">http://unsyncopated.com/blog/index.php/2009/11/04/creating-a-local-mirror-of-ubuntus-most-popular-packages/</guid>
		<description><![CDATA[The problem You want to create local mirrors of the apt repositories that you use but you don&#8217;t have enough hard drive space to mirror every package. Or maybe you have a slow link and you don&#8217;t want to spend time downloading packages that you&#8217;re unlikely to need. The solution Only mirror packages whose popularity [...]]]></description>
			<content:encoded><![CDATA[<span id="The_problem"><h3>The problem</h3></span>
<p>You want to create local mirrors of the apt repositories that you use but you don&#8217;t have enough hard drive space to mirror every package. Or maybe you have a slow link and you don&#8217;t want to spend time downloading packages that you&#8217;re unlikely to need.</p>
<span id="The_solution"><h3>The solution</h3></span>
<p>Only mirror packages whose popularity (as reported by popcon&#8217;s &#8220;installed&#8221; metric) matches a certain threshold.</p>
<span id="The_explanation"><h3>The explanation</h3></span>
<p>I&#8217;ve been hacking without a network connection recently and one of the biggest pain points is not having access to my distro&#8217;s software package repository.</p>
<p>For example, while writing some Python screen-scraping code last week I realized I didn&#8217;t have the Python library I wanted to parse some HTML with &#8211; <a href="http://www.crummy.com/software/BeautifulSoup/">Beautiful Soup</a>. Rather than postpone my work on the script until I found a weefee signal, it would have been nice to simply install the package from a local mirror of the repository.</p>
<p>I soon discovered two common tools that can be used to create a local mirror of a repository &#8211; Frans Pop&#8217;s <a href="http://alioth.debian.org/~fjp/log/posts.html">debmirror</a> and Dmitriy Khramtsov&#8217;s <a href="http://apt-mirror.sourceforge.net/">apt-mirror</a>.</p>
<p>I chose apt-mirror, skimmed <a href="http://popey.com/blog/2006/10/24/Creating_an_Ubuntu_repository_mirror_with_apt-mirror/">Alan Pope&#8217;s handy step-by-step guide</a> and kicked off the mirror script&#8230;</p>
<pre class="brush: bash;">
$ sudo -u apt-mirror apt-mirror

[...]
52.7 GiB will be downloaded into archive.
Downloading 75 archive files using 10 threads...
</pre>
<p>ACK! That&#8217;s a lot of gibibytes.</p>
<p>Eventually I&#8217;d like a complete mirror, but for now, I only want the packages I&#8217;m likely to need. <a href="http://www.frontier.com">My broadband connection</a> isn&#8217;t as &#8220;broad&#8221; as I would like.</p>
<p>The <a href="http://popcon.debian.org/">Debian Popularity Contest</a> (&#8220;popcon&#8221;) came to mind and sure enough, Ubuntu also provides <a href="http://popcon.ubuntu.com/by_inst">a flat text file</a> containing the names of all packages sorted by the frequency with which they&#8217;re installed by users.</p>
<p>I downloaded this file and hacked up the primary apt-mirror perl script to consult the file, only mirroring binary and source packages if they meet a chosen popularity threshold.</p>
<p>Here&#8217;s the meat from <a href="http://unsyncopated.com/corral/apt-mirror-0.4.5-1ubuntu2_popcon.patch">a patch that applies cleanly to apt-mirror version 0.4.5-1ubuntu2</a>:</p>
<pre class="brush: perl;">

sub should_process {
# print &quot;should_process()\n&quot;;
my $pkg_name = shift;
my $section_name = shift;
my @popular_pkgs = @{ $_[0] };

# if the pkg isn't in the 'game' section...
if($section_name !~ /game/){
my %is_popular;
for (@popular_pkgs) { $is_popular{$_} = 1 };

if( $is_popular{$pkg_name} ) {
# print &quot;processing popular pkg: &quot; . $pkg_name . &quot;\n&quot;;
return 1;
} else {
# print &quot;skipping unpopular pkg: &quot; . $pkg_name . &quot;\n&quot;;
return 0;
}
} else {
# print &quot;skipping game pkg: &quot; . $pkg_name . &quot;\n&quot;;
return 0;
}
}

# [...]

# open our popcon database
my $db_path = &quot;/home/tz/Desktop/by_inst&quot;;
open(FILE,$db_path) or die &quot;Can't open popcon db: $!&quot;;
my @data=; # beware record separator ($/) tweak below
close FILE;
my $num_comment_lines = 11;
my $threshold = 3000;
my $cur_line;
my @popular_pkgs;
# for each of the first $threshold lines, grab pkg name
foreach $cur_line (@data[$num_comment_lines .. ($num_comment_lines + $threshold)]) {
# print &quot;cur_line: $cur_line&quot;;
my @tokens = split / +/, $cur_line;
# print &quot;pkgname: &quot; . $tokens[1] . &quot;\n&quot;;
push( @popular_pkgs, $tokens[1] );
}

# [...]

if( should_process( $lines{&quot;Package:&quot;}, $lines{&quot;Section:&quot;},@popular_pkgs ) ) {
add_url_to_download($uri . &quot;/&quot; . $lines{&quot;Directory:&quot;} . &quot;/&quot; . $file[2], $file[1]);
}
</pre>
<p>Tweak the path to the flat file (`$db_path`) and the threshold (`$threshold`!) to suit your needs.<br />
<!-- echo "As you can see, I also modified the\nscript to skip games. Games tend to be\nlarge and there aren't many that I use\noften, except perhaps cowsay(1) :]" |cowsay -f gnu --></p>
<pre> ________________________________________
/ As you can see, I also modified the    \
| script to skip games. Games tend to be |
| large and there aren't many that I use |
\ often, except perhaps cowsay(1) :]     /
 ----------------------------------------
    \               ,-----._
  .  \         .  ,'        `-.__,------._
 //   \      __\\'                        `-.
((    _____-'___))                           |
 `:='/     (alf_/                            |
 `.=|      |='                               |
    |)   O |                                  \
    |      |                               /\  \
    |     /                          .    /  \  \
    |    .-..__            ___   .--' \  |\   \  |
   |o o  |     ``--.___.  /   `-'      \  \\   \ |
    `--''        '  .' / /             |  | |   | \
                 |  | / /              |  | |   mmm
                 |  ||  |              | /| |
                 ( .' \ \              || | |
                 | |   \ \            // / /
                 | |    \ \          || |_|
                /  |    |_/         /_|
               /__/</pre>
<span id="Future_improvements"><h3>Future improvements</h3></span>
<ul>
<li>Download popcon db file, rather than expect that it already exists on disk.</li>
<li>Read desired popularity threshold from mirror.list rather than using a hard-coded value.</li>
<li>Read desired sections as above.</li>
<li>Speed holes! My perl-fu is weak.</li>
</ul>
<span id="Tips"><h3>Tips</h3></span>
<ul>
<li>I found a faster mirror half-way through creating my local mirror. Renaming `/var/spool/apt-mirror/{mirror,skel}/${OLD_MIRROR` to `/var/spool/apt-mirror/{mirror,skel}/${NEW_MIRROR}` was sufficient.</li>
<li>If you try to install a package from your local mirror which doesn&#8217;t exist, you&#8217;ll get a 404 error &#8211; nothing catastrophic happens.</li>
<li>Beware <a href="http://sourceforge.net/projects/apt-mirror/forums/forum/197335/topic/1747995">permissions issues</a>. Avoid running apt-mirror as root rather than the prescribed `apt-mirror` user.</li>
<li>debmirror has a `&#8211;exclude-deb-section` option</li>
</ul>
]]></content:encoded>
			<wfw:commentRss>http://unsyncopated.com/blog/index.php/2009/11/04/creating-a-local-mirror-of-ubuntus-most-popular-packages/feed/</wfw:commentRss>
		<slash:comments>2</slash:comments>
		</item>
		<item>
		<title>Downloading photos from Facebook with Linux</title>
		<link>http://unsyncopated.com/blog/index.php/2009/11/02/downloading-photos-from-facebook-with-linux/</link>
		<comments>http://unsyncopated.com/blog/index.php/2009/11/02/downloading-photos-from-facebook-with-linux/#comments</comments>
		<pubDate>Tue, 03 Nov 2009 02:37:49 +0000</pubDate>
		<dc:creator>Tyler</dc:creator>
				<category><![CDATA[Linux]]></category>
		<category><![CDATA[Python programming]]></category>
		<category><![CDATA[Syntax highlighter]]></category>

		<guid isPermaLink="false">http://unsyncopated.com/blog/index.php/2009/11/03/downloading-photos-from-facebook-with-linux/</guid>
		<description><![CDATA[Tommy Murphy recently released photograbber, a simple Python/tkinter app that uses the Facebook API to&#8230; 1. Find all the photos that you&#8217;ve been tagged in and&#8230; 2. Download each photo to a chosen directory. It worked fine for me after I fixed a trivial tkinter bug. Right now I&#8217;m adapting photograbber to download entire albums [...]]]></description>
			<content:encoded><![CDATA[<p><a href="http://eat.ourbunny.com/">Tommy Murphy</a> recently released <a href="http://code.google.com/p/photograbber">photograbber</a>, a simple Python/tkinter app that uses the Facebook API to&#8230;</p>
<p>1. Find all the photos that you&#8217;ve been tagged in and&#8230;<br />
2. Download each photo to a chosen directory.</p>
<p>It worked fine for me after I <a href="http://code.google.com/p/photograbber/issues/detail?id=5">fixed a trivial tkinter bug</a>.</p>
<p>Right now I&#8217;m adapting photograbber to download entire albums for me. Here&#8217;s the crucial code that requests a collection of &#8220;photo&#8221; records using <a href="http://wiki.developers.facebook.com/index.php/FQL">Facebook Query Language</a> (FQL):</p>
<pre class="brush: python;">

photos = self.facebook.fql.query(&quot;SELECT pid, aid, src_big FROM &quot; \
&quot;photo WHERE pid IN (SELECT pid FROM photo_tag WHERE subject=&quot; + \
str(self.facebook.uid) + &quot;)&quot;)
</pre>
<p>Changing the query to&#8230;</p>
<pre class="brush: python;">

&quot;SELECT pid, aid, src_big FROM photo WHERE aid IN (SELECT aid &quot; \
&quot;FROM album WHERE owner IN (SELECT uid FROM user WHERE name=\&quot;&quot; + \
FriendName + &quot;\&quot;) AND name=\&quot;&quot; + AlbumName + &quot;\&quot;)&quot;
</pre>
<p>&#8230; did the trick. Caveat coder: the album IDs that you see in your browser while surfing Facebook <em>aren&#8217;t</em> the same IDs that you should use in your FQL queries. If you slip up, you might receive this misleading message: &#8220;FacebookError: Error 600: An unknown error occurred in FQLPhotoTable::get_ids_for_queries:  should never have a pid or aid without a uid&#8221;</p>
<p>Keep <a href="http://www.facebook.com/terms.php">Facebook&#8217;s terms of service</a> in mind when interacting with their site &#8211; you don&#8217;t want a &#8220;cease and desist&#8221; letter like <a href="http://www.vincentcheung.ca/blog/facedown/cease-and-desist/">Vincent Cheung received for his &#8220;FaceDown&#8221; application</a>.</p>
<p>If all else fails, you might have some luck with a PHP script called &#8220;FBCMD&#8221; as illustrated <a href="http://dieter.plaetinck.be/a_script_that_pulls_photos_from_facebook">here</a>. There&#8217;s also a .NET standalone app: <a href="http://rajitss.com/albumdownloader/index.php">The RajitSS Album Downloader</a>.</p>
<p>I&#8217;ll spare you my internet balkanization rant &#8211; Fred Vogelstein made the argument far better in the June issue of Wired: <a href="http://www.wired.com/techbiz/it/magazine/17-07/ff_facebookwall?currentPage=all">Great Wall of Facebook &#8211; The Social Network&#8217;s Plan to Dominate the Internet &#8211; and Keep Google Out</a>.</p>
]]></content:encoded>
			<wfw:commentRss>http://unsyncopated.com/blog/index.php/2009/11/02/downloading-photos-from-facebook-with-linux/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Secure browser-based communications</title>
		<link>http://unsyncopated.com/blog/index.php/2009/04/18/secure-browser-based-communications/</link>
		<comments>http://unsyncopated.com/blog/index.php/2009/04/18/secure-browser-based-communications/#comments</comments>
		<pubDate>Sat, 18 Apr 2009 20:35:12 +0000</pubDate>
		<dc:creator>Tyler</dc:creator>
				<category><![CDATA[Flickr]]></category>
		<category><![CDATA[Footnotes]]></category>
		<category><![CDATA[Security]]></category>
		<category><![CDATA[Syntax highlighter]]></category>

		<guid isPermaLink="false">http://unsyncopated.com/blog/index.php/2009/04/19/secure-browser-based-communications/</guid>
		<description><![CDATA[Introduction I&#8217;ve often wanted to securely send a message to someone who didn&#8217;t have a serious software package like GPG installed. Rather than ask them to install software they might not use ever again, I employed the scheme described below &#8211; it only requires a web browser with Javascript support. Our friends Alice and Bob [...]]]></description>
			<content:encoded><![CDATA[<span id="Introduction"><h3>Introduction</h3></span>
<p>I&#8217;ve often wanted to securely send a message to someone who didn&#8217;t have a serious software package like <a href="http://www.gnupg.org/">GPG</a> installed. Rather than ask them to install software they might not use ever again, I employed the scheme described below &#8211; it only requires a web browser with Javascript support.</p>
<p style="text-align: left;">Our friends Alice and Bob have agreed to help me illustrate this scheme for secure browser-based communications. Eve declined my invitation. She&#8217;s busy <a href="http://www.willhackforsushi.com/Home/Entries/2009/2/3_Decrypting_DebIan-Vulnerable_SSH_Traffic.html">decrypting debian-vulnerable SSH sessions off the wire</a>.</p>
<p style="text-align: left;"><div class='toc wptoc'>
<h2>Contents</h2>
<ol class='toc-odd level-1'>
	<li>
		<a href="#Introduction">Introduction</a>
	</li>
	<li>
		<a href="#The_scheme">The scheme</a>
	</li>
	<li>
		<a href="#Potential_pitfalls">Potential pitfalls</a>
	</li>
	<li>
		<a href="#Conclusion">Conclusion</a>
	</li>
	<li>
		<a href="#An_aside:_Damn_I_shouldnt_have_killed_that_Emacs_buffer">An aside: "Damn! I shouldn't have killed that Emacs buffer!"</a>
	</li>
	<li>
		<a href="#See_also">See also</a>
	</li>
</ol>
</ol>
</ol>
</div>
<div class='wptoc-end'>&nbsp;</div></p>
<span id="The_scheme"><h3>The scheme</h3></span>
<p>						<div class="flickr-gallery image right"><a href="http://www.flickr.com/photos/55502932@N00/3456154555"><img class="flickr small" title="Secure browser-based communication" alt="Secure browser-based communication" src="http://farm4.static.flickr.com/3585/3456154555_a31501c097_m.jpg" /></a></div>
					<br />
Alice wants to send Bob a secret message. Bob doesn&#8217;t want to fiddle around with any stand-alone crypto software in order to read the secret message. Bob <em>is</em>, however, saavy with his web browser.</p>
<ol>
<li>Alice points her browser to an SSL-secured web page (&#8220;https://foo.com/aes.html&#8221;) which contains a bit of HTML and an inline <a href="http://www.movable-type.co.uk/scripts/aes.html">Javascript implementation of the AES encryption standard</a>.</li>
<li>Alice uses the form on the web page to encrypt the secret message (&#8220;linux rulz&#8221;) with the password she chose (&#8220;thelegendofdrunkenmaster&#8221;). The encryption is performed locally &#8211; the secret message does not enter the network nor interact with any software besides the browser&#8217;s Javascript engine.</li>
<li>Alice sends the message encrypted message (&#8220;MGfrSbm5ubmqsnbYtoa9cgeYfA==&#8221;) to Bob via a second channel &#8211; e.g. by reading it to him over the telephone. Alice also gives Bob a hint about the password (&#8220;it&#8217;s the name of <a href="http://en.wikipedia.org/wiki/Drunken_Master_II">the movie</a> we watched last week &#8211; without any spaces&#8221;).</li>
</ol>
<ol>
<li>Bob visits the same secured page with &#8221;his&#8221; web browser. He types the encrypted message and the password into the form.</li>
<li>The decrypted message appears on his screen and he compliments Alice on her astute observation.</li>
</ol>
<span id="Potential_pitfalls"><h3>Potential pitfalls</h3></span>
<ul>
<li>The password &#8220;hint&#8221; should be chosen carefully. If it isn&#8217;t, a passive attacker (Eve, when she&#8217;s done submitting Valgrind &#8220;uninitialized data&#8221; warning reports to the Debian project<sup>1</sup> ) has a chance at brute-forcing the password. Substituting an ad-hoc &#8220;hint&#8221; system for well-designed key-exchange mechanisms should be done with caution!</li>
<li>Alice and Bob should devote an entire browser instance to this task and &#8221;only&#8221; to this task. They should close the browser and reboot after handling the decrypted message. If they don&#8217;t, malicious browser components, malicious websites, or OS paging could subvert their efforts by revealing the decrypted message to an attacker. I recommend that they use <a href="http://www.google.com/googlebooks/chrome/">a browser</a> that was &#8221;designed&#8221; with security in mind.</li>
<li>The webserver that serves the encryption/decryption page should be managed by a trusted party. If it isn&#8217;t, an active attacker could obtain the decrypted message. Barring port-forwarding and self-signed SSL key hassles, Alice herself could run a <a href="http://www.yeraze.com/2008/03/webservers-with-python-ssl-cac-authentication/">small webserver</a> devoted to the purpose.</li>
<li><a href="http://www.movable-type.co.uk/scripts/aes.html">Chris Veness&#8217; Javascript</a> emits Base64 encoded ciphertext. Base64&#8242;s default alphabet may lend itself to transcription errors if the telephone is chosen as a second channel e.g. &#8220;3&#8243; is heard as &#8220;E&#8221;. In rare cases, this could go undetected by the decoding scheme&#8217;s error detection and result in the generation of incorrect plaintext e.g. &#8220;linux drulz&#8221;. For &#8221;one&#8221; clever (ref: genetic algorithms!) solution to the transcription problem, see <a href="http://en.wikipedia.org/wiki/PGP_word_list">The PGP word list</a>.</li>
</ul>
<span id="Conclusion"><h3>Conclusion</h3></span>
<p>I&#8217;ve presented a scheme for secure communications between two parties where the only application software required is a Javascript-capable web browser and a SSL-capable web server.</p>
<p>Using a second channel and carefully-chosen password &#8220;hint&#8221;, this scheme allows access to high-grade cryptography with minimal &#8220;new software&#8221; requirements.</p>
<span id="An_aside:_8220Damn_I_shouldn8217t_have_killed_that_Emacs_buffer8221"><h3>An aside: &#8220;Damn! I shouldn&#8217;t have killed that Emacs buffer!&#8221;</h3></span>
<p>I began writing some code for this article in an Emacs scratch buffer. I then proceeded to accidentally kill the buffer before cutting its contents to my kill-ring.</p>
<p>I figured that maybe I could find the &#8220;lost&#8221; source code in Emacs&#8217; heap so I forced a core dump with `killall &#8211;signal ABRT emacs`<sup>2</sup><sup>3</sup>.</p>
<p>Next I tried to search for a snippet of the source code (the word &#8220;alice&#8221;) in the core dump with the `strings(1)` utility. The values for `CUR_ENC` are from the `strings(1)` manpage.</p>
<pre class="brush: bash;">

$ for CUR_ENC in s S b l B L; do strings --encoding \
${CUR_ENC} core | grep --ignore-case alice; done

$
</pre>
<p>No dice! If I had to guess, I&#8217;d say that Emacs uses some funky elisp mechanism for allocating memory for the contents of buffers, hence the failure of this naive `strings | grep` search. Suggestions welcome `;]`.</p>
<p><strong>Update 5/12/09</strong> &#8211; I should have tried <a href="http://bitbucket.org/haypo/hachoir/wiki/hachoir-grep">hachoir-grep</a> &#8211; it&#8217;s UNICODE-aware.</p>
<span id="See_also"><h3>See also</h3></span>
<ul>
<li><a href="http://lists.immunitysec.com/pipermail/dailydave/2009-May/005730.html">A DailyDave discussion on sources of entropy is Javascript-land</a></li>
<p><!-- detacord --></ul>
<ol class="footnotes">
<li id="footnote_0_245" class="footnote">Ooooooh. Twice in one blog post? I only do it because I love ya, Debian `:]`</li>
<li id="footnote_1_245" class="footnote">I&#8217;d started emacs in a shell where I&#8217;d previously run `ulimit -c unlimited`, thus enabling core dumps. Dog help you if you haven&#8217;t done so and need a core dump. </li>
<li id="footnote_2_245" class="footnote">Ubuntu disables `/dev/mem` (booooo hissss) IIRC so I didn&#8217;t try that route.</li>
</ol>
]]></content:encoded>
			<wfw:commentRss>http://unsyncopated.com/blog/index.php/2009/04/18/secure-browser-based-communications/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>A Flickr macro for MoinMoin</title>
		<link>http://unsyncopated.com/blog/index.php/2009/02/24/a-flickr-macro-for-moinmoin/</link>
		<comments>http://unsyncopated.com/blog/index.php/2009/02/24/a-flickr-macro-for-moinmoin/#comments</comments>
		<pubDate>Tue, 24 Feb 2009 20:59:50 +0000</pubDate>
		<dc:creator>Tyler</dc:creator>
				<category><![CDATA[Flickr]]></category>
		<category><![CDATA[Python programming]]></category>
		<category><![CDATA[Syntax highlighter]]></category>

		<guid isPermaLink="false">http://unsyncopated.com/blog/index.php/2009/02/25/a-flickr-macro-for-moinmoin/</guid>
		<description><![CDATA[Would you like to include Flickr photos in your MoinMoin wiki pages? Here&#8217;s a MoinMoin macro that allows you to do just that. The code is a little ugly, but I was inspired by this affirmation of the &#8220;release early, release often&#8221; philosophy to make the source available. Slide on over to my wiki to [...]]]></description>
			<content:encoded><![CDATA[<p>						<div class="flickr-gallery image right"><a href="http://www.flickr.com/photos/wka/1627089248"><img class="flickr small" title="Flickr logo II" alt="Flickr logo II" src="http://farm3.static.flickr.com/2260/1627089248_a53be86ce7_m.jpg" /></a></div>
					<br />
Would you like to include Flickr photos in your MoinMoin wiki pages? Here&#8217;s a MoinMoin macro that allows you to do just that.</p>
<p>The code is a little ugly, but I was inspired by <a href="http://www.contrast.ie/blog/fail-early-fail-often-and-learn/">this affirmation of the &#8220;release early, release often&#8221; philosophy</a> to make the source available.</p>
<p>Slide on over to my wiki to see the <a href="http://unsyncopated.com/wiki/MoinMoin_Flickr_Macro">my design notes for this macro</a>.</p>
<pre class="brush: python;">

#!/usr/bin/env python

&quot;&quot;&quot;
    MoinMoin - Flickr Macro

    A macro to embed Flickr photos on wiki pages.

    Usage: &lt;&gt;

    Parameters:

      photo_id: The ID of the photo you want to embed. You can find
                this within most Flickr URLS.
      size:     One of &quot;Square&quot;, &quot;Thumbnail&quot;, &quot;Small&quot;, &quot;Medium&quot;,
                or &quot;Original&quot;

    @copyright: 2009 by MoinMoin:TylerOderkirk
    @license: GNU GPL.
&quot;&quot;&quot;

from MoinMoin import wikiutil
import flickrapi

def execute(macro, args):

    argParser = wikiutil.ParameterParser(&quot;%(photo_id)i%(size)s&quot;)
    argDict = argParser.parse_parameters(args)
    # print argDict
    # photoid
    photoID = argDict[1][&quot;photo_id&quot;] # &quot;444769890&quot;
    desiredSize = argDict[1][&quot;size&quot;] # &quot;Medium&quot;
    # size Square, Thumbnail, Small, Medium, Original
    # TODO: add img caching/backup
    # TODO: logging as to not overuse API key.
    # TODO: store api_key somewhere else. wikiconfig.py?

    api_key = 'BEEEEEEEEEEEEEEEF'

    flickr = flickrapi.FlickrAPI(api_key)
    sizes = flickr.photos_getSizes(photo_id=photoID)
    info = flickr.photos_getInfo(photo_id=photoID)

    photoSourceURL = &quot;asdf&quot;
    for i in sizes.sizes[0].size:
        # print i['label']
        if i['label'] == desiredSize:
            photoSourceURL = i['source']

    # TODO: ensure this is an URL of type 'photopage'. ref
    # http://www.flickr.com/services/api/flickr.photos.getInfo.html
    photoPageURL = info.photo[0].urls[0].url[0].text
    photoDescription = info.photo[0].description[0].text
    # TODO: XSS here?
    photoTitle = info.photo[0].title[0].text
    photoAuthorUsername = info.photo[0].owner[0]['username']

    # print photoSourceURL, photoPageURL, photoDescription

    # TODO: wrap/truncate long titles. maybe put credits on new line?
    # TODO: potentially include description instead of title

    # TODO: use moin markup for table instead of raw html
    # ref. http://moinmo.in/MacroMarket/MiniPage
    # ref. http://unsyncopated.com/BrainSolvent/MoinMoin Flickr
    #       Macro?action=diff&amp;rev2=29&amp;rev1=28
    #  pypaper.py
    # Thanks to NirSoffer for the CSS
    # ref. http://moinmo.in/HelpOnTables
    return '
&lt;div&gt;
'\
     '
'\
     '&lt;a href=&quot;%s&quot;&gt;&lt;img title=&quot;%s&quot; src=&quot;%s&quot; alt=&quot;%s&quot; /&gt;'\
     '&lt;/a&gt;
%s (Flickr image by'\
     '%s)
&lt;table style=&quot;float: right; font-size: 0.85em; '\&amp;lt;br /&amp;gt;
     'background: #eeeeee; margin: 0 0 1em 1em;&quot;&gt;
&lt;tbody&gt;&lt;/tbody&gt;
&lt;tbody&gt;&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
' % (photoPageURL,
     photoTitle, photoSourceURL,photoTitle, photoTitle,
     photoAuthorUsername )

if __name__ == &quot;__main__&quot;:
    print( execute( 0, &quot;photo_id=2088619558,size=Small&quot; ) )4153379126
</pre>
<span id="Caveats"><h3>Caveats</h3></span>
<ul>
<li>Beware copyrighted photos. Luckily, Flickr allows you to specify license criteria on their &#8220;advanced search&#8221; page. I use <a href="http://yubnub.org/kernel/man?args=flcc">yubnub&#8217;s &#8220;flcc&#8221; command</a> to do this.</li>
<li>If the macro experiences an error at runtime and you have MoinMoin tracebacks enabled the tracebacks will contain your Flickr API key. Patches welcome `;]`</li>
</ul>
<p><strong>Update 3-1-09</strong>: Flickr&#8217;s API <a href="http://www.flickr.com/help/forum/en-us/91205/page2/#reply604216">just went down</a>. Seems like a good time to implement exception handling `:]`. Email me for the updated code.</p>
]]></content:encoded>
			<wfw:commentRss>http://unsyncopated.com/blog/index.php/2009/02/24/a-flickr-macro-for-moinmoin/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>good.net is fast.com</title>
		<link>http://unsyncopated.com/blog/index.php/2009/02/24/goodnet-is-fastcom/</link>
		<comments>http://unsyncopated.com/blog/index.php/2009/02/24/goodnet-is-fastcom/#comments</comments>
		<pubDate>Tue, 24 Feb 2009 16:34:57 +0000</pubDate>
		<dc:creator>Tyler</dc:creator>
				<category><![CDATA[Flickr]]></category>
		<category><![CDATA[Footnotes]]></category>
		<category><![CDATA[Security]]></category>
		<category><![CDATA[Shell programming]]></category>
		<category><![CDATA[Syntax highlighter]]></category>

		<guid isPermaLink="false">http://unsyncopated.com/blog/index.php/2009/02/24/goodnet-is-fastcom/</guid>
		<description><![CDATA[good.net is a stripped-down file-hosting service (cf. drop.io/sendthisfile.com)1 that has four &#8220;new-to-me&#8221; features:2 You can earn money when people download your files via their affiliate program. Your audience can access your files via HTTPS. An honest dedication to free speech and free software. You can upload your files via FTP-over-explicit-SSL (FTPES). Their support page doesn&#8217;t list [...]]]></description>
			<content:encoded><![CDATA[						<div class="flickr-gallery image right"><a href="http://www.flickr.com/photos/docsearls/2440194966"><img class="flickr small" title="2008_04_biketrail_infrastructure_016" alt="2008_04_biketrail_infrastructure_016" src="http://farm3.static.flickr.com/2133/2440194966_ec0face197_m.jpg" /></a></div>
					<br />
<a href="http://good.net/">good.net</a> is a stripped-down file-hosting service (cf. <a href="http://www.drop.io">drop.io</a>/<a href="http://www.sendthisfile.com">sendthisfile.com</a>)<sup>1</sup> that has four &#8220;new-to-me&#8221; features:<sup>2</sup></p>
<ul>
<li>You can earn money when people download your files via <a href="http://good.net/affiliate/">their affiliate program</a>.</li>
<li>Your audience can access your files via HTTPS.</li>
<li>An honest <a href="https://good.net/about/">dedication to free speech and free software</a>.</li>
<li>You can upload your files via FTP-over-explicit-SSL (FTPES). <a href="http://good.net/support/">Their support page</a> doesn&#8217;t list their FTPES host key fingerprint but you &#8221;can&#8221; just barely see it at 0m36s on their <a href="http://good.net/support/filezilla.html">FileZilla support page</a>. <strong>Update:</strong> they&#8217;ve posted <a href="https://forums.good.net/phpBB/viewtopic.php?f=4&amp;t=6&amp;sid=a6c80c76bc9054ac1a3a51e7f313b3a1">the fingerprint in their support forum</a>. See <a href="http://unsyncopated.com/wiki/Secure good.net FTP uploads with lftp">my notes on uploading with lftp(1)</a> for more info on securing your CLI FTP transactions.</li>
</ul>
<p>Their servers are also hooked up to especially fat pipes. I get 5 megabytes/sec sustained with my Thinkpad T30&#8242;s wired NIC on RIT&#8217;s library network. I got <em>10</em> megabytes/sec with my <a href="http://www.rimuhosting.com">Rimuhosting</a> VPS in their <a href="http://rimuhosting.com/datacenters.jsp#a3">Level(3) and Abovenet</a>-connected datacenter.</p>
<span id="Backstory"><h3>Backstory</h3></span>
<p>						<div class="flickr-gallery image right"><a href="http://www.flickr.com/photos/55502932@N00/3308794151"><img class="flickr small" title="Openmoko guru Harald Welte at 25C3" alt="Openmoko guru Harald Welte at 25C3" src="http://farm4.static.flickr.com/3374/3308794151_8d3c79147a_m.jpg" /></a></div>
					
<p>I found good.net via a link in <a href="http://www.mcgrewsecurity.com/2008/09/11/black-hat-usa-2008-and-defcon-16-audio-available/">the McGrew Security blog</a> to <a href="https://avondale.good.net/dl/bd/">good.net&#8217;s mirror</a> of <a href="http://darkoz.com/">DarkOz</a>&#8216;s <em>giant</em> collection of security conference videos &#8211; the &#8220;Hacker Media Archive&#8221;.</p>
<p>The Archive&#8217;s <a href="https://avondale.good.net/dl/bd/25c3/video_h264_720x576/">25th Chaos Communication Congress (25C3) videos</a> alone occupy nearly 40GB&#8230;</p>
<pre class="brush: bash;">

$ curl --silent http://avondale.good.net/dl/bd/25c3/video_h264_720x576/ |
awk --assign i=0 '/.mp4&quot;/{i=i+substr($8, 1, length($8)-1);} END {print i}'

39175
</pre>
<p>						<div class="flickr-gallery image right"><a href="http://www.flickr.com/photos/55502932@N00/3309623054"><img class="flickr small" title="Tor Project co-founder Roger Dingledine at 25C3" alt="Tor Project co-founder Roger Dingledine at 25C3" src="http://farm4.static.flickr.com/3039/3309623054_268ec04272_m.jpg" /></a></div>
					<br />
I&#8217;ve included two screen captures from the <a href="http://creativecommons.org/licenses/by-nc-nd/2.0/">freely-licensed</a> h264-encoded 720&#215;576 25C3 videos at right.</p>
<p>In addition to the CCC videos, you can find footage and materials from DEFCON, HOPE, Black Hat, CodeCon, DeepSec, HITB, NOTACON, PhreakNIC,  REcon, Shmoocon, and ToorCon. Phew!
<ol class="footnotes">
<li id="footnote_0_128" class="footnote">If you&#8217;re looking for a more &#8221;active&#8221; filesharing tool, one that syncs files across machines for you, try <a href="http://www.getdropbox.com">Dropbox</a> as recommended by an <a href="http://www.imdb.com/name/nm1168932/bio">aspiring grocer</a> friend of mine. They&#8217;ve even got a Linux client.</li>
<li id="footnote_1_128" class="footnote">Yes, the above manhole cover photo was taken by <a href="http://blogs.law.harvard.edu/doc/">the Doc Searls of Linux Journal fame</a>.</li>
</ol>
]]></content:encoded>
			<wfw:commentRss>http://unsyncopated.com/blog/index.php/2009/02/24/goodnet-is-fastcom/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Editing files as root with Emacs and Tramp</title>
		<link>http://unsyncopated.com/blog/index.php/2007/09/09/editing-files-as-root-with-emacs-and-tramp/</link>
		<comments>http://unsyncopated.com/blog/index.php/2007/09/09/editing-files-as-root-with-emacs-and-tramp/#comments</comments>
		<pubDate>Mon, 10 Sep 2007 00:50:42 +0000</pubDate>
		<dc:creator>Tyler</dc:creator>
				<category><![CDATA[Emacs]]></category>
		<category><![CDATA[Shell programming]]></category>
		<category><![CDATA[Syntax highlighter]]></category>

		<guid isPermaLink="false">http://unsyncopated.com/blog/?p=723</guid>
		<description><![CDATA[Did you know that you can use an existing non-privileged instance of Emacs to open files as root? Yep, Emacs includes a module called TRAMP which can use sudo to read and write files. As is usually the case, I had to work sweat a little to make it work. My problem was that my [...]]]></description>
			<content:encoded><![CDATA[<p>Did you know that you can use an existing non-privileged instance of Emacs to open files as root? Yep, Emacs includes a module called <a href="http://www.emacswiki.org/cgi-bin/wiki/TrampMode">TRAMP</a> which can use sudo to read and write files.</p>
<p>As is usually the case, I had to work sweat a little to make it work. My problem was that my zsh prompt was too fancy. The solution was to toss this conditional in at the end of my .zshrc:</p>
<pre class="brush: bash;">

if [ $TERM = &quot;dumb&quot; ]; then
   unsetopt zle
   export PS1=&quot;%% &quot;
fi
</pre>
<p>This works by disabling the zsh line editor (&#8220;zle&#8221;) and setting a plain vanilla prompt (&#8220;PS1&#8243;) that tramp can recognize when it logs in.</p>
<p>TRAMP is a little slow on my machine but it sure beats firing up a new Emacs instance every time I want to edit `/etc/superdooper.conf`</p>
]]></content:encoded>
			<wfw:commentRss>http://unsyncopated.com/blog/index.php/2007/09/09/editing-files-as-root-with-emacs-and-tramp/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Determining who acted in two known films</title>
		<link>http://unsyncopated.com/blog/index.php/2007/08/18/determining-who-acted-in-two-known-films/</link>
		<comments>http://unsyncopated.com/blog/index.php/2007/08/18/determining-who-acted-in-two-known-films/#comments</comments>
		<pubDate>Sat, 18 Aug 2007 06:13:06 +0000</pubDate>
		<dc:creator>Tyler</dc:creator>
				<category><![CDATA[Flickr]]></category>
		<category><![CDATA[Shell programming]]></category>
		<category><![CDATA[Syntax highlighter]]></category>

		<guid isPermaLink="false">http://unsyncopated.com/blog/index.php/2009/02/28/determining-who-acted-in-two-known-films/</guid>
		<description><![CDATA[Have you ever seen someone in a movie and thought to yourself &#8220;Hey, weren&#8217;t they in movie Y too?&#8221; This happened to me last night when I was watching &#8221;Casino&#8221;. The parking lot attendant from a certain scene looked just like a character from &#8221;Fear and Loathing in Las Vegas&#8221;. To the *nix shell, Robin!&#60;/Bruce [...]]]></description>
			<content:encoded><![CDATA[<p>						<div class="flickr-gallery image right"><a href="http://www.flickr.com/photos/55502932@N00/3316942898"><img class="flickr small" title="Just take the ticket!" alt="Just take the ticket!" src="http://farm4.static.flickr.com/3401/3316942898_1c48506dd5_m.jpg" /></a></div>
					Have you ever seen someone in a movie and thought to yourself &#8220;Hey, weren&#8217;t they in movie Y too?&#8221;</p>
<p>This happened to me last night when I was watching &#8221;Casino&#8221;. The parking lot attendant from a certain scene looked just like a character from &#8221;Fear and Loathing in Las Vegas&#8221;.</p>
<p>To the *nix shell, Robin!&lt;/Bruce Wayne Voice&gt;</p>
<p>I wrote a short script to download the IMDb &#8220;profile&#8221; pages for all the actors in movie X and list which ones also acted in movie Y. This is, effectively, a <a href="http://math.comsci.us/sets/intersection.html">set intersection</a> operation.</p>
<p>I could&#8217;ve registered for IMDb&#8217;s 14-day trial of their $20/mo &#8220;<a href="http://pro.imdb.com/about/a2z">Pro</a>&#8221; service which allows advanced searches on their database, but I wanted a challenge.</p>
<p>Here are the important pieces of the script:</p>
<pre class="brush: bash;">

# grab (only) the profile pages for each actor who performed in title
# 'tt0112641' ('casino'). recurse, but only 1 level deep into links.
wget --recursive --level=1 --wait=1 --include_directories=/names
'http://www.imdb.com/title/tt0112641/fullcredits'

# determine if any of the retrieved profile pages contain a reference
# to 'fear and loathing'
find . -type f -print | xargs grep --files-with-matches
--ignore-case 'fear and loathing'
</pre>
<p>						<div class="flickr-gallery image right"><a href="http://www.flickr.com/photos/55502932@N00/3317046420"><img class="flickr small" title="They're just checking in now" alt="They're just checking in now" src="http://farm4.static.flickr.com/3358/3317046420_c7167bd824_m.jpg" /></a></div>
					<br />
wget will, by default, obey a site&#8217;s robots.txt file. IMDb.com&#8217;s robots.txt says you&#8217;ll get fined 1 cent for every request to their server that causes a denial of service! So, be nice and only download from the crawl-able portions of their site (and do it slowly with `&#8211;wait`, otherwise you might get temporalily blocked with HTTP 500 errors).</p>
<p>The script gave me ~10 results most of which were off-screen roles e.g. &#8220;set designer&#8221;. One, however, was the dude I was looking for: <a href="http://www.imdb.com/name/nm0494040/">Brian LeBaron</a>!. He acted as a parking attendant in both Casino and Fear and Loathing.</p>
<p><strong>Update: </strong>There are now <em>much</em> more elegant ways to interact with IMDb. See, for example, <a href="http://imdbpy.sourceforge.net/">IMDbPY</a></p>
<p><strong>Update:</strong> More retrospection: the &#8220;combine&#8221; utility in <a href="http://debaday.debian.net/2007/04/15/moreutils-a-collection-of-useful-command-line-tools/">moreutils (via deb-a-day</a>) might&#8217;ve been handy.</p>
]]></content:encoded>
			<wfw:commentRss>http://unsyncopated.com/blog/index.php/2007/08/18/determining-who-acted-in-two-known-films/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
	</channel>
</rss>

