CHDK for Computational Photography

H. G. Dietz
http://aggregate.org/hankd/

Department of Electrical and Computer Engineering
Center for Visualization & Virtual Environments
University of Kentucky, Lexington, KY 40506-0046

Initial release: October 22, 2013

This document should be cited using something like the bibtex entry:

@techreport{chdk20131022,
author={Henry Gordon Dietz},
title={{CHDK For Computational Photography}},
month={October},
year={2013},
institution={University of Kentucky},
howpublished={Aggregate.Org online technical report},
URL={http://aggregate.org/DIT/CHDK/}
}

For quite some years now, I, Professor Hank Dietz, and my students have been using CHDK (Canon Hack Development Kit) to do evil things inside Canon PowerShot cameras. CHDK has always been uniquely useful in allowing commodity cameras to run arbitrary native-compiled C code, and it's slowly been getting better. The thing that hasn't gotten much better is the documentation and generally rough structure of the CHDK wiki... so I've created this page to provide a hopefully more orderly tour through CHDK and our work using it.

Getting Started with CHDK

The latest User Manual for CHDK provides a good overview of the basic features, but not so much about programming. It at least should motivate you to try CHDK.

Getting CHDK Into Your Camera

Here's a simple way to get CHDK into the majority of Canon PowerShots, which take SD cards:

Stick an unprotected SD card in your camera and select the Canon menu option to low-level Format it. This ensures the card has a format your camera can access.
Take the card and mount it on your internet-connected computer.
Grab the appropriate zip version of CHDK from the CHDK downloads page. The "unstable new development version" is usually a safe choice, but there is a reason the other one is called "stable."
Unzip that version on your SD card.
Unmount and remove the card, slide the write-protect tab into the locked position, and put the card in your camera.
Turn the camera on by pressing the play button -- NOT power.
Hit the Menu button, then scroll down to Firm Update and say OK -- this will boot CHDK. Note that CHDK never permanently installs itself; it is always loaded alongside the standard firmware in RAM and leaves no trace in the camera after power is turned off (unless you explicitly write files into the camera's non-volatile memory).
After the CHDK splash screen, tap the Play button to enable the CHDK menus -- this will show Default Script in the lower left of the rear LCD panel.
Hit the Menu button; in the CHDK menu, scroll to Miscellaneous Stuff, then SD Card, then Make Card Bootable; then OK.
Turn the camera off and then on again using the power button; CHDK should automatically boot. It will only do so if the card is write protected, but it otherwise ignores the hardwrae write protect switch happily writing files to the SD card while CHDK is running.

The default choice of using the Play button to enter the CHDK menu is very portable, but awkward. On most cameras, using the Video Record or WiFi button makes the most sense. Whatever button you assign, you can still obtain the original functionality by holding the button for a couple of seconds rather than tapping it.

Accessing Captured Images

You can access images captured by CHDK either by removing the SD card and using a card reader or by using a USB cable with the camera. The Canon firmware supports the usual (driverless) treatment of the camera as a USB mass storage device, but it doesn't show the whole card: only the DCIM directory and subdirectories. Conveniently, it does allow both reading and writing despite the SD card being marked as write protected.

The images use standard formats, especially JPEG (files ending in .JPG) and "Digital Negative" (files ending in .DNG). These files are understood by most image editing software, although a few older Canon PowerShots use unusual CFAs (color filter arrays other than the standard Bayer RGB pattern) and are not supported by some raw converters (e.g., Adobe's DNG Converter).

Using CHDKPTP Remote Control / In-Circuit Emulation

Although Canon PowerShots generally have some support for the picture transfer protocol (PTP) understood by lots of software, CHDK implements a much more powerful extension of PTP known as CHDKPTP. That is also the name of the program that you run on your computer to interact with the camera: CHDKPTP. When it works, this provides an extremely powerful mechanism for remote control and in-circuit emulation (ICE). Not only can it move files back and forth and control camera functions, but it also can run Lua scripts on the camera with the option to have Lua scripts in-camera interact with Lua code on the computer. There's even a GUI version that gives a camera live view.

When it doesn't work, CHDKPTP can be a nightmare to debug. There used to be some odd code dependencies, but the current distribution includes all it needs. The catch is that PTP is a standard, well-known, protocol, and other programs may thus see your camera as a PTP device and claim it. If that happens, CHDKPTP often will still work, but it may be running V-E-R-Y S-L-O-W-L-Y -- perhaps 1000X slower than normal, with it taking many seconds even for a simple GUI button press to register.

On Linux systems, the culprit is usually gphoto and/or gvfs, with gvfs being particularly ponderous because it runs as background processes trying to mount PTP cameras. The easiest fix is to simply not install gvfs, but it is possible to edit the rules for PTP devices; it will involve adding a file like /etc/udev/rules.d/99-usb-storage-remove.rules with a rule like:

ACTION=="add", ENV{GPHOTO2_DRIVER}=="PTP", ENV{DRIVER}!="", PROGRAM!="check_gui_user_group.sh gnome-session myusbaccessgroup", RUN+="/bin/sh -c 'echo -n %k >/sys%p/subsystem/drivers/%E{DRIVER}/unbind'"

CHDK Scripting

CHDK actually supports three programming methods:

BASIC scripts: the original scripting support
Lua scripts: the best-supported scripting environment
Compiled C code modules: the way to do things that are computationally too demanding to suffer the overhead of Lua interpretation, but it requires rebuilding CHDK from source using an appropriate cross compiler

Of those, Lua is the right answer for 99% of camera applications. After CHDK version 1.4, there are even hooks so Lua code can directly play with pixels in the raw image buffer, which used to be the main reason to use C code modules.

CDHK Lua Scripting

The CHDK Scripting Cross Reference Page is a pretty good starting point, with various links for Lua programming. Be aware that there are differences between CHDK Lua and "generic" Lua (as described here):

CHDK numbers are 32-bit integers, not floats. There are work-arounds using integers scaled by 1000, i.e., 0.25 would be 250. There is now some float support, but it's likely to be slow on many cameras.
Lua modules and coroutines are not implemented. However, there are ways to control how code is scheduled -- for example, set_yield(-1,-1) essentially gives Lua code scheduling priority over other tasks.

The CHDK Lua Reference Card

I like reference cards. However, CHDK is pretty huge to describe on two sides of a single-page reference card. After three days of work, here's my current best attempt: chdkluaref.ps and chdkluaref.pdf. This reference card doesn't tell you much about using CHDK, but it's a good reference for programming CHDK cameras using Lua scripting. This card is also now available from the CHDK Lua Reference Card page at the CHDK Wiki.

It's worth noting that I had to leave-out quite a bit of useful info to fit things on one card. I expect to further edit-down the card to the most useful and portable subset, thus making space for an example script, etc. The only obvious indication I've done an update might be the date marked on the card; the above link will always get you the latest version.

CHDK Lua Scripts

swarmgame.lua (20160912): This is a very simple game run in record mode. There is a swarm of randomly colored dots that chases a single, larger, white dot that you control. The trick is how you control it, which is by pointing the camera at an evenly-lit surface and using a flashlight to make a spot on that surface -- the camera finds the bright spot and makes the white dot follow it. Eventually, one of the dots in the swarm will catch the dot you control, but the goal of the game is to maximize how long it takes for that to happen.

candyDetector.lua (20170224): This is a very simple motion detection script that was used with flash enabled to catch people trying to grab a candy from a jar. It's actually pretty easy to defeat it by acting very quickly after someone else has triggered it, but it makes a cute demo for kids...

beat.lua (20171005): This is a simple metronome script allowing you to set the tempo in beats/minute and accenting of each Nth beat. When run, it shows a color elipse and plays a sound with each beat.

kobre.lua (20180817): This is a very crude implementation of an algorithm to detect heartbeats by observing subtle changes in skin color, typically of a person's face. It's very fragile, but does work in well-controlled situations. Output is BPM (beats per minute) display on screen.

gmfl.lua (20200128): Guess my focal length script. Uses motion detection to measure camera shake at various focal lengths, then randomly changes focal length and tries to guess the focal length used based only on the observed camera shake. Works best with cameras like the SX530 HS when image stabilization is turned off.

The only thing set in stone is our name.