No, Mr. Bond, I expect you to fund my kickstarter

Amazon S3 - a large, far away, widely distributed gadget

Fri, 17 May 2013 23:51:58 -0400

Amazon S3 counts as a gadget, right? :-) I’ve been using it professionally for a while, and of course many of the services we take for granted until us-east-1 goes down use it too. Turns out that you can hook it in to a homebrew website without very much work…

The other day I traced a period of terrible performance (8s network latency getting out of the house) to a visit from Googlebot-Video/1.0 fetching an old AVI file from a post-hoc image stabilization project (now made mostly redundant by youtube’s builtin stabilization feature.) The file was about 50M, and anyone interested in the project really wants the less-compressed original, shoving it to youtube really doesn’t help… but it turns out that that’s tiny by Amazon S3 standards, and the free tier covers it just fine.

There were a surprisingly small set of steps; I’m posting them here with the actual domains involved, since they’re visible and public anyway, you just need to convert them to your own needs…

Create yourself an AWS account. (You already shop at amazon, right? Just log in and create one…)
“Better get a bucket.” Go to the console web page, pick S3, hit “create bucket”. Name it something obvious; in my case, avi.thok.org although something more generic like s3.thok.org would have been a common choice. Do this first, because the bucket namespace is global and isn’t checked against DNS registration at all, so there’s a very faint chance someone already has a bucket of that name; at this stage, if you find a collision you can just pick a different name, like s3-namespaces-can-you-speak-it.thok.org.
Install s3cmd (just git clone the github version and run it from the checkout - the one in ubuntu doesn’t actually handle puts with redirects.)
1. Configure it: s3cmd --configure and get the Access and Secret keys from the console under “security”; don’t bother to configure encryption or https because these are files that are already available by http, you don’t want to deal with certificates, and you’ll check the md5sums later.
2. Copy your files. Note that s3 doesn’t have directories per se; you just put the path with slashes in place as you go, so s3cmd put --no-encrypt kicx1440.avi s3://avi.thok.org/me/publish/europython/day2/kicx1440.avi works just fine, without having to do anything about me/publish directly.
3. Make them world-readable. By default S3 is, correctly, private; s3cmd setacl --acl-public s3://avi.thok.org/me/publish/europython/day2/kicx1440.avi makes that single file public. At this point, there’s a long convoluted url that will fetch this file, and you could stop here and just change the html that points to it, but let’s handle this cleanly…
Edit your DNS zone and add avi IN CNAME s3.amazonaws.com. Carlton Bale gets credit for having the first google hit that actually said this would work. Once you’ve pushed this through, curl -L -v -I http://avi.thok.org/me/publish/europython/day2/kicx1440.avi works - note carefully, the -I gets curl to do a HEAD (-H was already taken?) so you get back headers, not 100m of video. You should see the Location header taking you over to S3, and then a convincing ETag (md5sum of the file, in this particular case) and Content-Length.
Edit your apache config and add RewriteRule ^/(me/.*\.avi)$ http://avi.thok.org/$1 [R,L] To pick this apart:
1. RewriteRule is the apache swiss-army-knife of URL mangling.
2. The first bit is a regular expression that matches the entire “path” (^ for start, $ for end) and grabs everything after the leading slash (thus the slash is outside the grouping parentheses.) Within this part of the path, it has to start with me/ and end with .avi but can have anything at all in between; if we wanted literally all AVI files, we’d drop the me/ part, but I have some small ones elsewhere on the site that I didn’t want to bother hunting down and uploading.
3. The second bit is the new URL - avi.thok.org to point to the CNAME we set up above, $1 is the first set of parentheses in the match (so, me/xxx.avi.)
4. Finally, the last bit is what to do with this big of hatchet work; R says to make it a redirect (and because our result starts with http it automatically becomes an “external” redirect, in this case a 302, ie. “don’t try to fetch this url, just tell the client to go away and find it themselves.” You can’t get theyah from heah, but you can get there from over there… the L is for “last” and just says to stop trying and don’t do any more rewriting on this particular result.
Don’t forget to actually /etc/init.d/apache2 reload or however your system spells that. At this point, you can curl -L -v -I http://www.thok.org/me/publish/europython/day2/kicx1440.avi (note that we’re actually starting with the primary domain here, where the original problem started) and follow our HTTP/1.1 302 Found and then amazon’s HTTP/1.1 307 Temporary Redirect and the bandwidth problem (remember the bandwidth problem? This song’s about a bandwidth problem) is now gone.

Future refinements:

use [R=307] and make the first hop a Temporary Redirect as well. Not sure if that’s correct, yet, but given that this all started with a search engine bot that wasn’t aware of the human-readable “slow (home)” and “fast (MIT)” alternate links, it’s worth looking into.
if thok.org were more of a CMS, automatically noticing avi files and pushing them to amazon would be a good transparent trick. For a total of five files on a home website? Not actually worth the trouble, even if the logs say I have at least a month before the bot comes around again :-)
actually process the logs by object size and see if anything else should get this treatment; in practice, these files got noticed so they’re the right starting point, and this isn’t anything you’d mistake for a major site.

"Chromebook Air" and a trip to Santa Clara

Wed, 03 Apr 2013 02:12:20 -0400

Put my mobility and gadget obsessions to the test by going to Pycon 2013 in Santa Clara. Didn’t take the X220t Thinkpad, just grabbed a Samsung Chromebook a couple of days before leaving (and installed crouton on it.) Crouton gives me a chrooted Ubuntu Precise environment, into which I poured my normal desktop coding and photography workflow. I added a 32G SD card with a couple of months of photography and my latest master index, which let me process pictures as I went; the one other useful addition to my pocket pouch was a dual-end mini-micro USB cable because most of my cameras are Canon, and so I still need Mini-USB when everything else is MicroUSB. I also happen to prefer short cables for reducing pocket clutter… but made up for it by carrying a 2ft A-A cable in the bag with the camera chargers.

The other trick was to remind myself that I was, after all, going to Silicon Valley, where they already have all the gadgets, and though “I can’t Amazon-Prime things to my hotel” is a very #firstworld definition of “roughing it”, clearly making several visits to Fry’s Electronics would be just as good… and in fact, when I “accidentally” took a thousand pictures my first day in town (at Point Reyes) the next thing I picked up was a 64G SD card to upgrade my local picture backup… and a knife to open the package with :-)

I did prepare the Chromebook before leaving, with an old Apple sticker; rather than causing comment, I think it served more as camouflage, at least at a distant glance. On closer inspection, of course, the $250 Chromebook is relatively flimsy (hold it up by a corner and the touchpad isn’t clickable anymore, for example) and the screen is very much not IPS (let alone Retina) which isn’t really a problem in conference and hotel settings, but was kind of unpleasant on the flight. Interestingly, I didn’t need to plug in power the entire length of the BOS to SFO flight, nor the return, even with (Gogo) wifi turned on the whole time.

That brings up “why crouton, instead of just installing ubuntu?” - first of all, I do like the idea of a limited self-sysadmining laptop, and have made on-and-off use of the CR-48 since they shipped, though I haven’t been able to make the leap to programming on it (I did try koding, and it’s just Not Emacs) and second, I hadn’t figured out how the “12 free Gogo coupons” was implemented, and figured it was easier to just use the ChromeOS-side browser for that. Just a little thing, but crouton worked well enough that it didn’t really get in my way.

Another nice thing about the Chromebook was that I didn’t especially worry about it; if it got damaged, the SD card would probably be fine, and I could probably just hit BestBuy and grab another one :-) There are a lot of workplaces (not mine, sadly) where being able to salvage a business trip by expensing a laptop that costs less than two nights in a hotel room could be a huge win.

I even picked up a cheap HDMI cable at Fry’s - having not noticed that the conference hotel TVs were (coincidentally Samsung branded) analog panels, with VGA inputs, no digital ones. (The “vacation” part of the trip did have HDMI-capable TVs in the hotel rooms, but part of sucessfully shooting 4000 pictures in a week was getting out and shooting and using the laptop for backup and picking out highlights, and deciding to leave serious tagging and uploading until I got home when I had time to do research and identification (after all, there were a couple of inches of snow on the ground when I got home; California was beautiful, New England hasn’t actually managed spring yet.)

Other trip gadgets: I had a rental car, so I got a small, cheap, and hard-to-recommend allegedly-2.1amp 12V-to-USB adapter that couldn’t keep my Gnote at equilibrium, let alone charge it, while using it as a navigation system. Still looking for a sane answer there. More helpful was the 12V camera-battery charger with little clip adapters for each different battery type, that let me top up the camera I’d used most on a given day; while it was a nice and performant gadget, I would have been better off actually being disciplined and keeping a charged backup battery for each camera every night. (Since it’s 12V and 120V, it might be sensible to bring only that charger next time, for the slight benefit over the multiple 120V chargers I’d otherwise carry, though on closer inspection, one of those was also 12V capable already.)

Finally, the trip was to go to Pycon 2013, so I brought back more linux boxes than I went with :-) So far the only interesting thing I’ve done with the RasPi is to hook it up to a tiny keyboard and project my trip-report slides at work. The idea that you can reasonably “rummage around on your desk to find a linux box” the way you used to rummage around looking for a spare ethernet cable is highly entertaining :-)

Programming an Open Source Flashlight

Fri, 01 Feb 2013 00:39:00 -0500

After a year and a half, the HexBright Kickstarter actually delivered. Personally I think it was entirely worth the wait - the developer is clearly kind of obsessive and it’s wonderful the way the internet supports (and/or exploits :-) that. Mechanically, it is very impressive, I’ve been carrying it for a couple of weeks - it’s definitely my large flashlight of choice though I’m still wearing my Fenix E11 as well.

I’ve only done moderately well at using the stream of technology that KickStarter has supplied me, so it was notable that I was able to get “up and coding” for the HexBright in a couple of hours… On an ubuntu box,

$ apt-get install arduino
$ git clone https://github.com/dhiltonp/hexbright.git
$ less hexbright/README.md

and follow the instructions. (Note that if you’re on precise rather than quantal or later, you want to grab the precise-backports version of arduino-core - 1.0.1 is the lowest tested version, though there are some patches that sort of work with 1.0, at least they work well enough to build and upload tactical.ino, a basic brightness-and-flashing firmware.)

The tactical firmware is a good starting point; the main loop is only around 35 lines of code, and if you just want to do custom flashing patterns (sliding rate changes, or morse code or something) you’ve got the right starting points at hand. I just made the pulse rate drop by 10ms every 1/2s which is obnoxious to look at but does visibly do something :-)

My next step is to drop the arduino GUI and just use avr-gcc and avrdude directly (since that’s all the GUI does anyway.) There’s an arduino-mk package that I suspect will help with that. As someone who worked at Cygnus on G++ as an embedded systems cross-compiler in the mid 90’s, though, I’m a little boggled that I’m using a standard prepackaged 32 bit compiler to upload code into my flashlight :-)

Another mobility path (Asus Transformer TF300T)

Tue, 25 Dec 2012 12:45:27 -0500

Woot had the Asus Transformer TF300T on sale cheap recently, and I’d always been enamored of the form factor - I’d played around with Android/x86 on the EEEpc briefly, even without a touch screen, and I’m certainly one of the people stalling on getting a MacBook Air because it really should have a screen which is a detachable (retina) iPad :-) Likewise the Motorola Atrix was pretty tempting when it first came out (I got to poke at one and conclude that it wasn’t quite there yet but was a good direction, it certainly wasn’t enough to switch me over to AT&T at the time.) The TF300T is an interesting pile of neat features:

runs Jelly Bean (shipped with ICS, over-the-net upgrade immediately on unboxing; slight chance that this implies a possible upgrade to the next Android too…)
decent mac-ish keyboard that solidly latches on - the assembled Transformer really does feel like a laptop
extra battery in the keyboard
SD slot and USB port (meets my “can use to caption pics from the dSLR” target)

It has a touchpad, which is a negative as I’ve pointed out before - but there’s a top-row key to turn it off entirely, so it doesn’t end up bothering me (and yet I can turn it back if I must have it… or I can just touch the screen which I keep forgetting :-)

Oddly the Jelly Bean upgrade was more trouble than benefit: my two primary “content production” (as opposed to consuming/surfing) apps, Flickr and Tumblr, report themselves as “not compatible with your device.” I’m actually writing this on the TF300T, using the website and Chrome - but I’m at home with solid network, and making use of tumblr’s “save as draft” feature - and I’m still worrying about not actually having persistence or auto-save. (Still going through the exercise, of course; writing speed is certainly 100% anyway, the keyboard is Just Fine as long as I don’t think about it too much :-) (Update: The Tumblr app now has tablet support, though it’s still inconsistent about rotation.)

Having carried the Transformer for another week, I’ve taken a few more steps:

installed a 32G microSD card (in the “tablet”, there’s an empty SD slot in the “keyboard” too)
used rsync for android and some scripting to keep the last 90 days of photography on the card (mostly to be able to show it off from the tablet, but that may become more)
picked up the relevant “micro HDMI” cable, I’m typing this in front of a Toshiba 21.5” LCD (another Woot! - $99, great price for another monitor “just to have around”, usually I use it on VGA with an old thinkpad, but it works pretty well with this - sometimes a little glitchy in “scale” mode, no problems at all in “crop” mode which just drops the bottom-of-screen Jelly Bean control bar.)
also picked up the SD and USB adaptors that plug in to the tablet itself - they’re redundant with using the keyboard slab, but seemed like they might be convenient and would be easier to get ahold of now rather than down the road.

It’s definitely working out as “mobile typewriter” - just slightly less overhead than the GNote+bluetooth approach, at the cost of making one pocket a lot bulkier. Was also useful for getting some quick cat pictures up on Flickr after all, even without the native app - the Web UI uploader is just fine, stick the card into the keyboard (haven’t tried USB-to-camera yet), select the image from /Removable/SD, add some tags and a description, done. Winter is upon us (no really, just because it was 55F two days in a row, doesn’t mean, umm…) so I won’t be getting as many excuses to go out “in the field” and want to immediately push wildlife shots to Flickr, but I might do a few more for practice anyhow.

Another thing the transformer is letting me experiment with is the whole idea of a “MacBook Air with a removable iPad screen” - the TF300T is a fine prototype of that, and it turns out that

having the touchpad does help, especially with the button to turn it off
reaching up and touching UI elements on the screen isn’t that bad after all - as long as it’s a sparse thing, I wouldn’t want to do something screen-UI-intensive that way, but most apps with that property are games anyway :-)
having the extra battery “mileage” is the real benefit of such a combination
noone at Google is sitting down and trying to work this way, there are still a bunch of ways in which the UI is a touch UI first and other paths are clunky (though I haven’t found any that are entirely missing.)

I haven’t tried SSH or VNC from here yet; oddly, I’ve kept finding plenty to do that doesn’t need either of them; it’s not that I don’t need to do those things, but if I’m forcing myself to spend time using the transformer, I can use the time well, just a little selectively. (I really need to come up with a decent programming solution - in fact, I should try Koding again, see if it works better from here than from the CR-48 ChromeBook…)

Spark WIFI, MakeyMakey, paper computing

Mon, 26 Nov 2012 03:12:02 -0500

Spark WIFI light fixture looks plausible, if only because the Belkin WEMO already exists at a comparable price point. (The Spark is indoor only, and what I actually want is either an outdoor-socket-capable one, or an indoor light switch replacement, ideally one that can be dropped in as one of a pair of three-way switches, which would actually be tricky…) sponsored, not yet funded

I finally used my Makey Makey - basically a very sensitive closed-circuit detector that comes with a pile of alligator clips, and pretends to be a USB keyboard - the reference standard demo is the banana piano - anyway, I just moved into a new office, and traded up from a well-worn Comfy Chair to a standing desk (with advanced pneumatics, very easy to move up and down.) Stories about standing desks run the gamut from “it changed my life and I need a new wardrobe” to “meh, everyone here just leaves them at the lowest setting and uses chairs”. Clearly I need DATA… and a quick prototype showed I could hang some (zinc-plated) chains from the desktop, clipped to the Makey Makey, and have them trigger a key press whenever the desk was lowered enough for the chains to pool up on the base. That part was easy - the problem is that because it acts as a keyboard, it’ll “hit space” when it gets a connection, but I want to treat it as a distinct channel… libusb was a messy failure, but in the end, the Python EVDEV bindings and in particular InputDevice.grab came through, barring one memory leak that I may be able to work around. (Github link to follow soon…)

Finally, the classic “choose your own adventure” book is sort of like following a program, or at least walking a tree - well, in that spirit, there’s a KickStarter for doing Hamlet in that form - and if that weren’t cool enough, the list of “Amazing People Who Are Doing Pictures For This Book” is about 2/3 comic artists that I read, and 1/3 ones that I probably should :-) If you haven’t gone over there to look yet, do it for this quote: “But I’ll warn you: Shakespeare’s choices didn’t lead to the best ending for the characters.”. Paper and e-book versions. (Ok, that “paper computing” bit was kind of a stretch, I just needed an excuse to included it here, it’s Just That Cool.) sponsored, funded, still aiming for stretch goals

Android Phone as PDA/Laptop Replacement

Sun, 11 Nov 2012 13:09:00 -0500

I’ve had a dream for a long time of carrying a useful computing environment on me, without being encumbered by it. The first interesting piece of hardware was the Cambridge Z88 (the TRS-80 Model 100 was plausible, but expensive and pretty large; it was the laptop of its day, and at least a decade after for some niches… but the Z88 was the size of an issue of Byte Magazine and ran for days on some AA batteries, by virtue of tricks like turning the CPU off between keystrokes.) I took it on various trips, fought with uploading and downloading over the serial port… In the end it was difficult to develop for, even harder to develop on, and laptops ended up covering the “real work” niche - eJournal or space age typewriter replacement didn’t catch on that broadly.

They did, however, catch on narrowly, there were fascinating glimpses in the mid to late 90s of how various (particularly UK) science fiction writers actually used science fictional working environments; Charlie Stross had various early mini-laptops, Warren Ellis pioneered the “hang out in the pub and drink and smoke over a PalmPilot and foldout keyboard” model - which became my benchmark for portability, the idea that you could pull two or maybe three pieces of hardware out of your pockets (small enough pieces that people didn’t suspect that you were smuggling or shoplifting) and suddenly be able to Do Real Work without needing an office or other chunk of Corporate (ie. Too Expensive for Individuals) Infrastructure. (I didn’t actually care about being “live”, though even PalmPilots had web browsers; from day 1 it was clear that the Net was as much a source of simulated productivity as actual, and that Real Work involved getting ideas out of the brain and onto “paper”.)

The years rolled by, producing a spectrum of Palms - the Sony ones with built in camera and keyboard marked the end of the 20th century, just in time to get entirely displaced by cell phones. This was actually a horrible time for the PDA space - the entire hardware-in-your-pocket space turned around and started building telephones, which certainly provided value to a much broader range of people - almost literally everyone as we can see today, “not having a cellphone” is just strange in 2012 - but the design effort was now going into antennas and batteries, screens got worse, keyboards got far worse (when T9 looks like a good sophisticated option, you know you’ve gone very far down a strange path.) At the same time, laptops were getting smaller and cheaper, skimming off the less demanding majority of mobile writers, luring them with nice screens and the ability to run modern (not to say good) word processing tools, and all you had to do was carry a backpack, and stay near an outlet. (I’m no exception - being able to run a compiler while hanging out with people was a big deal; in the early 2000’s, being able to fix bugs in the taxi on the way to supporting a sales call for your startup was a huge deal.)

Then Apple changed everything.

Note that I’ve never owned an iPhone (though I have an iPad) - I still have to acknowledge that the iPhone kicked an amazing competitive “arms-race” back in the right direction. All of a sudden, portable devices with long battery life, gigahertz processors, and lots of storage were The Thing, and an entire industry pivoted from trying to interest people in flippier flip-phones to trying to catch up with the idea that you had an awesome computer in your pocket, with a sophisticated development ecosystem (ok, that took longer, but it was inevitable) and oh, by the way, it takes care of making phone calls too, when you’re not using it for everything else in your (online) life.

Very rapidly, my niche was back under the big tent with everybody else and billions in R&D were going my way. Sitting in a coffeeshop and poking at your phone was a mundane thing. Writing and publishing to the internet was suddenly something everybody was doing (and while I’ll sneer at facebook and foursquare updates right along with the rest of my disdainfully elitist peers - it’s still writing and, to the extent that it’s pushed out specifically for others to read, still publishing, and it still needed the tools I wanted.)

I almost missed it.

I’ve kept an eye on the tools and toys, of course. Poked at various ways of hooking keyboards up to phones, played with various design sketches for better ways to connect the hardware (I’ve got pictures of a wood-and-styrofoam mockup that hooks an IBM Model M clicky keyboard up to an original model Oqo - which I used as my “work laptop” for weeks, but entirely failed to be more portable than the Toughbook I’d been using beforehand - the Toughbook actually had a handle and hooks for a shoulder strap right on it) but I never really sat down and did it, I just poked at the gadgets.

Turns out that “poking at gadgets” is almost as much of a distraction from Real Accomplishment as arguing on the internet. I got a hint at this a while back when laser printers first got cheap - I plugged one in to my linux box, and printed, and it just worked. (I did end up tweaking the default from 300dpi to 600dpi… 6 months later even that much wasn’t necessary; I guess I’m not a victim of the ESR curse.) Having done that, I noticed that the documentation said the printer worked on Solaris too, and almost went and tried it to see if it really did - which led a friend of mine to comment “it’s like a cat, with a mouse that’s already dead - you’ve just got to bat it around a little and see if it will start being fun again?”

This also took a while to sink in. Part of the problem is that I’m actually very good at poking at gadgets, and I work in a field where that’s encouraged, or at least tolerated as a sign that you’re the sort of person they need to have around to get the computers to actually be useful. This usefulness leads directly to a decent income, which (combined with decent shopping skills) means no end to the stream of gadgets to poke at (as you’ve seen in this blog already, KickStarter is the biggest distraction I’ve had since Amazon came along.) The end result is that I’ve basically felt no pressure at all to get back to the original goal of “being usefully portable”. At the same time, I’ve gotten relatively picky about working environments; I learned on typewriters so I’ve hated most modern non-IBM keyboards, and I find mice to be a dead end that we just haven’t escaped from, so I turn trackpads off - the Mouse Stick is just minimal enough, it’s nearby when I must use a mouse, and at the same time my hands don’t have to leave the keyboard to use it; this is rather incompatible with the huge level of compromise that goes in to making portable keyboards, let alone pocketable ones.

Thus it is that I’m sitting in front of a Galaxy Note (the I-717 phone, not the larger tablets) looking at a very readable 25 lines of this posting, and typing on a Motorola KZ450 - a bluetooth keyboard with Mouse Stick - that is itself only barely thicker than the phone, yet has enough travel that I’m typing full speed and full accuracy relative to my ThinkPad (with the caveat that typing English is much easier than typing Code, but this keyboard looks up to the challenge.) As far as writing goes, there are no compromises here - I’m not thinking in letters, I’m basically thinking in sentences and while I do expect to go back and edit, the “data entry” part of the path is not a limiting factor. There’s no interruption between intending to write and writing and having the words appear on-screen; I have actual flow going on, and I’ve been writing for an hour and a half without feeling like I’m doing a special exercise or straining or contorting myself - I’m just writing. At my stopping point, I’ll save-as-draft (in the Android Tumblr app I’m using), turn off the keyboard, put the GNote back in my shirt pocket, unfold the stand back into “keyboard cover” configuration and clip it back on to the keyboard - stuff the keyboard into a jacket pocket, have a last sip of tea, and walk away.

(In the interests of full disclosure - this is a test run; I’ve had the keyboard for a week and kept failing to sit down and use it, so I’m at a worktable at home, the tea is a late-night decaf k-cup brew, the network is my local wifi. That said, the test was a complete success; the only “challenges” were figuring out that I needed to hit “back” to get the on-screen keyboard to pop down, and that save-as-draft wasn’t a missing button, it was a setting under options that changes the “publish” button to one that just says “save”. Also I have no idea how to do a word count here, but a quick scroll and sample tells me I’ve gotten around 1500 words down. NaNoWriMo it isn’t - but it is something and I’m going to have to sink my teeth into it for real now…)

Update: I’m at Pycon.CA and although I did bring two laptops and the iPad, they’re back at the hotel - I’m actually attending with only what I have in my pockets. Of course, I’m wearing a ScottEVest (winter model from a couple of years back - the newer models have fewer pockets and lower build quality, unfortunately) so I have a lot in my pockets; during the talks I primarily take notes in S/Memo, using a ThinkPad wacom-stylus, which is basically the electronic equivalent (in size and look) of a Moleskine; I occasionally hop out into Chrome to check out a URL mentioned in a talk, or hop into Plume to broadcast a choice quote on the conference hashtag. Since we just have chairs, and not tables, that’s been about the limit during lectures; I’m skipping the OpenStack lecture (no discredit to the speaker, I went through the slides in advance, and it looks like it isn’t going to cover anything I didn’t pick up in a major cloud service evaluation I did for work a year and a half ago, so it’s a good time to take a break) to hang out at the “bar” which has tables and decent chairs. I have the keyboard set up, and an Energizer “Energi To Go” battery pack boosting the Galaxy Note, and have only had one person stop by to comment on the setup. (I’m not alone in the bar; four people with conventional laptops, and one organizer reading a paperback book.) Overall it has been a pretty effective conference and blogging setup. The weak point in the short term has been photography; simple “tweet this picture” doesn’t work at all, plume uploading to twitter just gives “media too large”, so instead if I want real time pictures I shoot them with the pureview 808, share-to-flickr, wait for the upload to finish, then come back to the Android flickr app, pick that picture and hit share-to-twitter, add the comment and post. This is still kind of silly :-) and fails to support completely casual photo streaming. (I should try just shooting from the flickr app on the Note, but the 808 is a lot more almost-camera-like.)

I’m still pulling all of the cameras back to the laptop and managing the archive with kphotoalbum there. The next big step will probably be to use rsync-for-android (which actually works on ICS, it failed entirely under Gingerbread) to push/pull the most recent year of images onto the phone, and thus use it to synchronize the devices and push them back to my archive; then I can cherry pick things to twitter, and later pull things back to the laptop when I want to do my conventional kphotoalbum-based tag-heavy workflow. The other “next step” is to look into using git-annex to manage the diverse photo pools, but I’m stalling on that until the work from the kickstarter project goes mainstream.

Update 2: hit “edit” in the web interface, and preview shows proper markdown rendering, so re-saved. Will have to try to use the mobile website instead of the app for future writeups, I’m more likely to remember to do tags that way…

Welcome to the future!

Pycon.CA and Upverter

Sat, 10 Nov 2012 14:04:46 -0500

I’m at Pycon.CA this weekend (the first native Canadian python conference, in Toronto.) It’s more about code than gadgets, but I want to call attention to one of the sponsors, “Upverter” - I’d kind of ignored them previously as “some cloud thing” but it turns out that they’re actually rather “gadget-interesting” - they’re basically trying to be web based circuit design, github, and tinkercad combined. Haven’t had a chance to see how complete it is, but they’re definitely putting a lot of effort into it - I would not be surprised if sometime soon we see a kickstarter gadget where the electronics is designed in Upverter and the mechanics in TinkerCAD… The browser has been the universal platform for a while, but I don’t think anyone saw it becoming the Universal *Design* platform, for the construction of real-world devices - perhaps the next generation of Raspberry Pi class of devices will be designed and built in a browser on a Raspi-class machine :-)

KawaiiDuino

Sat, 06 Oct 2012 00:23:25 -0400

TinyDuino and TinyLily The former just smaller than a quarter, the latter just smaller than a dime, yet they have pluggable modules (“shields”) for a variety of features; the “Robotics Kit” pledge level is a couple of TinyLilys with a bunch of motor-drivers (1.8A H-bridge chips on washable circuit boards (pledged, funded, still available)

They’d pair really well with RadioBlocks which are a bit bigger (no handy coins for scale) and do 802.15.4 mesh networking using Open Source firmware, and draws 14mA at full power - so you can run it off of the data pins of another device, as long as one of them is set to “1”. (pledged, funded, less than an hour to go!)

It’s nice to see the modern “legos of computing” actually getting down to the size of lego bricks, and getting cheap too. Not quite at the level of “stick some coin cells, chips, and humidity+sunlight sensors into ping-pong balls and shake a bag of them over your garden” but definitely heading in the right direction, where it’s not completely ludicrous to think about “ubiquitous computing” ideas like that at the hobbyist level…

KickSoftware: bugs, sheet music, soundtracks

Sat, 08 Sep 2012 01:54:12 -0400

BugKick is an AGPL’ed bug tracker (with a hosted option.) Not that we need another bug tracker (especially given my Codes Well With Others ranting) and especially not one written in PHP… but the kickstarter itself is only aiming for a tiny amount of funding to do a couple of months of polish work (and presumably to find out if anyone cares) which would be a novel data point in the kickstarter software funding curves. (not yet funded; not backed)

Liederboard is an HTML5 sheet-music scratchpad with visions of user generated content :-) It made the initial funding target, has delivered some features already, and has a decent tech blog about the progress and goals. Not open source, but an interesting example of reaching a niche audience with a somewhat unique piece of software, and putting it all together and actually funding it. (funded to initial target, long list of stretch goals available; not backed)

Mission: Escape uses the GPS in an iPhone to track your driving, and produce an on-the-fly soundtrack for your commute. That just sounds like a lot of fun :-) (not yet funded, not backed)

KickScience: Sensors

Sat, 01 Sep 2012 01:38:50 -0400

The “tricorder” may come sooner via Kickstarter than via the Ansari/Qualcomm tricorder X-prize competition, a variety of well-connected sensors have gotten funding recently. The Public Laboratory Spectrometer is the most recent (still open, until October, but already past 200% funding) project to do optical analysis of materials; what makes it interesting is that it’s taking advantage of the cellphone’s builtin camera - and that they’ve already got a community of crowd-sourced observations and presentation tools, and some very close-to-home measurements (baseline measurements of various coffees at Tosci’s :-) They’ve still got a way to go to connect spectra to actual materials, though they’ve come up with wonderfully clever tricks like using the Mercury lines from ubiquitous CFL bulbs as calibration points. (Backed)

I mentioned the IO Rodeo Educational Colorimeter Kit last week, but forgot to point out that even though they KickStarter is done, the kit is available in their online store along with a bunch of other Science! gear, like optical sensors for measuring the wing-beat rates of flies…

In an entirely different sensing range, we have the Safecast pocket open source radiation sensor - designed by Bunnie and more sensitive than most commonly available devices, with sample logging, USB download, and local display of Alpha/Beta/Gamma events. Designed in response to the Fukushima event, it could also provide interesting data in conjunction with the EPA “Where You Live” program - I happen to live a couple of miles from the Starmet Superfund Site and the more “eyes” watching such things, the better.

Mobile robot platforms using cellphones as the communication link (and local “brain”) are almost mundane at this point - adding simple but sophisticated sensors to them enables a lot of interesting environmental science, with locally developed data…

Hacking gadgets, working gadgets, science! gadgets

Sat, 25 Aug 2012 00:26:59 -0400

The DigiSpark wins hands down as the cutest little Arduino ever, and almost cheap enough to be disposable, which really reduces the potential fear of “screwing one up” in an electronics project (on the software side, of course, you just reset them, so that’s no big deal.) It’s really nice to see such a range of scales for adding “a little bit” of computing to something, while still having enough power to use some modern abstractions (USB, interpreted code) instead of banging out the bits by hand (uphill, both ways, in the snow… ehem.) Backed, made target, ends 10 September.

Scanbox really qualifies more for that other tech blog that I keep failing to start in that you could slip it in a backpack or briefcase and have a nice scanning “platform” for your phone - not necessarily enabling new things you couldn’t already do with your phonecam, but taking them to a surprisingly “real” level of quality, helping “doing day to day business with only my cellphone” move further out of Dancing Bear territory. (Backed, already funded, should be available commercially at some point.)

The IO Rodeo Educational Colorimeter Kit actually arrived and looks quite nice; I haven’t had time to assemble it, and in any case I expect I’ll end up playing with it some and then passing it on to a high school science teacher friend, if I don’t find some way to use it directly for tea and or chocolate experiments myself before the end of the year.

(I’m tempted to make a spreadsheet out of the Kickstarter Transactions page, just to add a “reward arrived” column; other recent arrivals include OpenBeam, the iZen Bamboo Keyboard, the eStylo and touchfire, and I have printrbot tracking info :-)

Giving Open Source Android apps a Kick

Sun, 13 May 2012 00:54:53 -0400

Corbin Champion has a solid start with addi (an Octave-like Android app that has an optional port-of-real-Octave engine, though the other mode is more functional at this time) and is asking for funding to make it a really solid Octave/Gnuplot for Android. He’s got a convincing record of actually doing the work as open source, too.

I know a number of researchers who’d get good use out of this - but few of them have the budget to throw at something like this. I like the idea, though, so I’ve tossed some money in… plus I’d like to encourage the model, as a low friction way for a project to declare that they have a plan and a way to use money, if any gets sent their way, with well understood semantics for how the money collection actually happens (the reduction in bike-shedding alone might be enough of a friction reduction to make it worthwhile :-)

Colorimeters, Colorful LED displays, and Kicksaver

Sat, 05 May 2012 02:12:56 -0400

The IO Rodeo Colorimeter kit is building a basic easy-to-assemble (no soldering!) Open Source Hardware colorimeter - a basic scientific measurement tool, which uses different frequencies of light to measure properties of a liquid sample. The design looks very student-friendly, and is a good start on understanding that instruments aren’t magic…

This 8-digit 7 segment display board is a nice module for old-school lots-of-digits output - if you were doing your own version of a DeLorean back-to-the-future dashboard, it’d be a good component to have :-) I’ll note that in practice, a $60 refurb android tablet might be near-useless as a portable appliance, but it would make a great embedded display with graphs and whatever simulated-digit output you want… but I like LEDs so I backed it anyway.

Given how terrible Kickstarter is at actually helping me find interesting gadget projects (hey Amazon! buy them and force them to use your recommendation engine! :-) I owe credit for finding the LED display project to (KickSaver)[http://www.kicksaver.net/] - not actually a search engine, but certainly an interesting browsing alternative - you give them a price threshold, they give you kickstarter projects that need that amount to kick them over to successful funding. (Got a better idea? Fork Kicksaver on github and let me know what you came up with…)

The importance of feedback (return of the HexBright)

Mon, 23 Apr 2012 23:02:00 -0400

I’ve heard a lot of grumbling about the HexBright Kickstarter which raised $260k/$31k in July 2011, and the developer got trapped doing design improvements - worthy ones, from what I can tell, but there was a severe shortage of communication about the process.

Well, he’s finally re-emerged and started posting details of battery tests, lens and end cap samples, etc. Yay! While I’m personally in favor of erring on the side of adding more coolness to the design, it’s really worth communicating about it (which may not help convince the people who want the gadgets now but helps keep them from getting too upset with the whole process…) While there aren’t formal standards for any of this, I’d like to think it’s a Kickstarter Best Practice to communicate extensively once you start the work; see PrintrBot for an extreme example :-)

"CordCruncher" headphone cable tangle-preventer

Tue, 17 Apr 2012 22:37:10 -0400

The CordCruncher is a bit lower-tech than usually catches my attention - it was mentioned in passing in a comment on the Pebble E-paper phone-display (record-setting kickstarter, almost $4 million at this writing with a month to go) - skip the marketing video until you’ve watched the “How To Crunch” one down below (or skip to the 0:30 mark to see the first bit where someone uses the product instead of just looks pretty near it.)

The unfortunate bit is that they’re actually integrated with the headphones, and not a separate product. (Probably easier to sell that way, though.) Still seems a really clever way to protect a thin cable from tangling and looping around things, so maybe a later version will come along that can be retrofit. (I don’t actually use earbuds, so this is one of the few kickstarters mentioned here that I won’t be backing - but I still think it’s clever, and it’s already at 2.5x goal.)

Bluetooth Voltmeter/data-logger, cheap enough to toss around, kickstarting...

Thu, 22 Mar 2012 01:46:00 -0400

“i-Voltmeter” is probably a terrible name for getting the attention of the kind of people who actually want cheap sensors for experimentation, i-Stuff is usually shiny and locked down, whereas this voltmeter is described as Open Source hardware, and really, it’s pretty straightforward to talk to bluetooth devices in general, most of them are “serial ports without wires” and that’s actually good enough!

On top of that, if you actually make it further down the web page, it becomes a lot clearer that “voltmeter” (or ohmmeter) is actually one of the more flexible sensors you can have, since it can usefully hook up to most environmental sensors (light, temperature, reed switch) with nothing more than alligator clips. It’s “good glue” for this kind of thing; it has a different “scaling shape” than things like the Twine box - the Twine can itself hook up to a bunch of sensors and then you talk to it wirelessly, here you can have the sensors themselves farther from a computer (and farther from each other, even outside if needed.)

Unfortunately it’s a high-starting-point ($65k) project, apparently because of bluetooth radio certification (compare, though, the Ubertooth One OSHW firmware-replaceable bluetooth security analysis tool only needed $16K to get off the ground, but I’m guessing it’s in enough of an “experimental” category that it didn’t need as much certification/overhead to get built.

Still, this kickstarter is open until 13 April 2012, so if you’re interested in More Cheap Sensors (one of the interesting bottlenecks in household robotics in particular :-) it’s worth a look.

update This kickstarter was canceled early, on 2012-04-05.

Lytro "Light Field" camera - interesting but limited tech, unusable software

Sun, 18 Mar 2012 17:40:00 -0400

I’ve taken around 300 pictures with the Lytro (“shiny red” 16G model.) since it arrived 3 days ago. Initial impressions:

the lens and aperture are impressive for the size of the device - but to get a narrow enough depth of field for The Effect to work with, you still need to be relatively close to your subject. There’s a reason the demo pictures are “flower closeup with stuff behind it” and “focus on one friend with other friends not too much farther away” - that’s basically the only kind of composition that gets any benefit from this hardware.
actually there’s one exception to that: shots through a screen or through a rain-spattered train window. You can click on raindrops and get a “mood” shot with rain and blurred detail, or click on the distance and actually see the intended subject. (That said, many pocket cameras have an explicit “window” or “aquarium” mode that fixes the problem…) (update: see http://www.flickr.com/photos/eichin/6849258944/ and http://www.flickr.com/photos/eichin/6995382171/ )
the ability to recover a “bad” shot is less valuable than you might think, if you’ve got any experience “riding” the autofocus on a modern camera. (And remember, if it wasn’t obvious, this doesn’t help at all with motion blur, just focus-point blur.)
the screen is unusable in “sufficient daylight to want sunglasses”, compared to the Canon S100 for example. This is tempered by the shape making it surprisingly easy to point the device at your subject, though it’s very different than pointing any other device.
the touchscreen is too small to do anything more than basic “did I get The Effect at all?” testing.
the progressive rendering it does on pan when zoomed in actually obscures the refocus effect - it took me a bit to learn to wait and then refocus, otherwise I’d be fooled into thinking there was a difference that was only due to it “catching up” on rendering the image at all.
on a more positive note, it feels like the fastest small camera I’ve ever used - this may just be because there’s no focus delay; though it does have basic autofocus, it clearly doesn’t wait for it to take a shot (after all, it doens’t need to :-) The lack of review-delay also helps it feel fast, though that’s kind of a bug (first camera I’ve seen in a decade that didn’t have a review-timeout option.)
the mechanical design is very cool - you hold it entirely differently from other cameras, it has a very science-fictional feel to it - “artifact of a different design-timeline” sort of thing. One grip I tried almost felt like using a paintbrush to “dip” pictures off of a palette… definitely worth fooling around with. I found myself holding the device away from me when taking shots far more than I do with pocket cameras.
the user-experience design is bad:
- no affordance at all for the optical zoom. (When I saw the early videos, I even assumed you twisted the whole thing somehow, maybe with some kind of piezo torsion sensor between the rubber grip and the metal body.)
- half a dozen “swipes” to get the zoom from one end of the range to the other; looks dorky, feels dorky, is tedious - if I swipe and hold it should just keep moving until I let go! (and if you’re trying to get The Effect outdoors, you’re going to be going to full zoom often.)
- glorious symmetry means even with the lanyard, it’s hard to find the shutter orientation by feel - frankly they should have just put trigger buttons on all four sides and been done with it.
- at best an “accidental” affordance for actually looking at recent shots… if you notice that the shot “slides” off to the left, it might occur to you that you can “swipe” it back from the left to look at it again.

As for the specific horror of the Mac(-only) software (and the reason I’m writing this post without having any pictures to attach to it, it’s still not finished running):

300 pictures took 20+ minutes to pull down an initial “backup”, which produced three opaque 250M files and a small database.
after that, it started chewing on images. I first thought “ok, I’m going out to take more pictures while it does that”, pulled the camera - and got an error, and all of a sudden it only had 16 pictures and wasn’t working on the others. What?
when I plugged it back in, it started showing the other pictures and chewing on them again - and the existing ones are down at the bottom labelled “new story”, I haven’t figured out if I can undo that yet - because 40 minutes later, it has only processed a third of them. (update: drag and drop worked here.)

Also, the license has terms that I would never agree to for a normal camera… or even normal software:

it informs you that they will be collecting data for performance improvements. You do not opt-in or have an opt-out option - it’s just part of the license
“software updates will retroactively mess with your older pictures, and it’s not our problem”
this license expires in 75 years (not something I’ve seen in other licenses, and even for digital photography, it kind of horrifies me)
there’s a spelling error in the first couple of paragraphs :-)

If it weren’t for the fact that I want to screw around with an alternative approach to photography, and am already putting up with a lot just for early access to the toys, I’d return the device just for the license terms; at least in the short term, I’ll put up with it, but it’s very much not something I can recommend to others. (Fortunately, the device is also so limited on the photography side that I can tell people they’re not missing much…)

Most of the result of shooting with it is that I want the tech in a dSLR, so I can use it with a 250mm lens, which is where I find focus and depth of field actually matter. Basically the only shots I’ve ever wanted to refocus were extreme zoom with very narrow depth of field, and it’s always been “missed the subject”, not “want to swap subject and background”.

All of that said, I suspect a non-photographer (say, someone who only takes pictures with a cellphone) would have a different perspective, and possibly get more out of the device - as a substitute for learning (and doing) in-camera composition.

zDevice - let skynet into your home

Tue, 24 Jan 2012 14:59:52 -0500

zDevice (alternate link) is in the same space as the wildly successful Twine box - a “micro appliance” with WiFi and a bunch of sensor/actuator interfaces. Both are trying to be easy to configure; the zDevice looks like it wins more on openness (talks to a local server instead of a “cloud” one, for one thing.) The zDevice kickstarter project is actually including the rest of the kit needed for some “real” applications, including a smart garage door opener - it’s hard to “sell” a platform, but starting with a use case gets you the attention of people who realize that they really want something close to that, and with the “bias for tweaking” that these things encourage, you can capture the people who want to make something physical from an idea but don’t think they can (or simply don’t have the time to) start from scratch.

The downside of “priming” the reader with existing projects is the fear that it means that people will look at the device too narrowly. I think in practice that too many options will confuse people up front, and there’s plenty of opportunity for followup once it’s off the ground.

I’ve had my own list of projects in mind for years but never really wanted to put even a mini-tower PC in every room, just to drive some sensors. The EEEpc almost made the perfect platform, except they ended up being too useful for other things - and they kept putting out newer models every couple of months with increases in price and feature set, instead of pushing the price down on the original version - at $100 each for an EEEpc 701 (fanless, ssd, webcam), you could put one in every room, enabling things like

lights/heat/music “follow” you around
entertainment that changes depending on how many people are in the room
basic whole house environment monitoring (given data, you can conclude things like “insulating this one room better would be the most effective single change I can make” or “gee, the cat spends a lot of time in this one room, I should vacuum it more often”)
handy USB power/sync everywhere - come back from a ski trip, and while you’re making hot chocolate, you’ve already plugged the camera in and it’s getting pulled in to your gallery and backed up. (Of course, with an Eye-Fi your camera should just start backing up as soon as you get in range of the house wireless network…)
audio/visual “panic button” - voice recognition is hard, screaming detection should be easier :-)
the usual “warm up the house, I’m on the way home from work/travel” features (that should be part of a basic heating system, it’s 2012 already!)

If one of these devices gets really popular, and the price goes below even the current kickstarter prices, then I hope to see more kickstarter projects for experimental sensors - localizing microphones/vibration sensors, imaging thermal sensors, airflow detectors of some kind? Ad-hoc home environmental automation would be a fun field to be in.

And given a cheap netbook and a pile of sensors, the obvious next step is to drive them around :-) The OCULUS telepresence robot (hat tip to ladyada for reminding me of it) is still a little mispositioned in that it’s mostly about you remotely driving a camera around, instead of the netbook observing and reacting and reporting - after all, the first useful thing you could do with one isn’t “torment the cat from ground level”, it’s “Use Bundler and SLAM to produce a full interior model of your house, so you can finally figure out how large a ball-pit you can install”.

Or figure out if there’s room for the shark tank under the trap door after all…

Useful HDMI hardware (despite the industry's best efforts) - the NeTV

Fri, 06 Jan 2012 00:41:19 -0500

Bunnie just announced that the NeTV was actually available from Adafruit. It’s unusual to see a consumer product where most of the documentation is presented at a crypto hacker conference - though if that’s your thing, it’s also worth taking a look at the Ubertooth One, one of last year’s KickStarter successes, for a similarly near-consumer device which first appeared at ShmooCon.

This device takes something that was a simple task back in the late 80’s - specifically, overlaying content on a live video stream - and making it possible again, despite the modern advances that have gotten in the way. Back then, a low-end Amiga desktop computer could “genlock” (clock synchronize) with an NTSC video signal, and then overlay its own video output on it, producing a new combined NTSC signal - without the need for expensive studio hardware. (Legend has it that these machines were used for the early episodes of Babylon 5, before it had a budget…) Thirty years later, due to accumulated fear of the “analog hole”, HDCP-compliant HDMI video devices perform a key exchange to confirm that both ends of the connection are promising to be limited in the way the studios desire. Turns out that, as with DVD, “crypto” isn’t magic, and cheap crypto really isn’t magic. The NeTV box doesn’t decrypt the signal at all; after intercepting the session and helping itself to the key, it generates precisely timed, properly encrypted, replacement pixels - from any source you want, using a webkit browser as the renderer.

I think this is going to be an honestly useful gadget; Bunnie mentions obvious cases like “overlaying the televised debates with real twitter feeds instead of sanitized news-station feeds”, but it would also support a bunch of the old TiVo hacks like “display caller id of that phone” or “alert me that the laundry is done” - little things that don’t deserve any continuous visual space, but should just appear and go away at will.

That said, I’d bring it up here entirely out of respect for the techno-wizardry involved :-)