In this project, you're going to make use of your Canon PowerShot used remotely either via USB or the FlashAir SD card. That is known as tethered remote control... and it's called that even when there is no physical wire tethering the camera to a computer. We're nearly out of time, so we're keeping the project simple... but it's still pretty cool. You'll be capturing images to make time-lapse movie consisting of at least 30 frames and identifying faces in them.
Making a time-lapse movie doesn't sound too hard, does it? Well, it sort of is. You see, all of the better codecs involve not just compressing one frame at a time, but interpolating between frames. To do that, one needs to have key frames both before and after the frame currently being compressed. Not a big deal, but that does tend to mean you need more memory than the PowerShots make available. There's also the little issue that most cameras don't allow you to run programs inside them at all.
Basically, you are to write a program/script to run under Linux (or a campatible environment) that will capture frames at regularly-spaced intervals from your ELPHA115. The PowerShot images should be captured at a rate of one frame every 15 seconds for a total of 30 images. I don't care if you individually trigger and transfer the images from the PowerShot or if you simply tell the PowerShot when to start and then transfer all the files from it after the complete sequence is captured... but, either way, you need to have the triggering and image fetching all done by a script running on the PC interacting with a CHDK script via 802.11 WiFi communication with the FlashAir card or via chdkptp or even via gphoto.
One of the little surprises is that most modern cameras actually do face recognition. It's mostly used to determine focus points, but your PowerShot cameras also record how many faces they see, and at what image coordinates, as part of each JPEG captured. Each time the latest PowerShot image contains a detected face, you'll extract a close-up of the face from that image and include it as an inset in each of the images taken. That would actually be somewhat difficult if there wasn't already software that essentially can do it for you....
You should scale the image resolution to a manageable size for video, i.e., 640x480 instead of 16MP. That's all most webcams deliver anyway. Do that scaling in-camera on the PowerShot to save on file size and to speed-up file transfer via WiFi. Of course, any inset image must be small enough to fit within the base image to be used as a video frame, but that shouldn't be a problem. If you wish, you may impose an arbitrary limit on inset image size, such as 320x240 pixels maximum -- one quarter of the frame. In any case, once the capture interval is over and your script has collected all the images, your program is to combine them into a movie in any standard format. For each frame that contains a face, superimpose it as described below; for frames that do not contain a face, continue to superimpose the last face found (or none, if no face has yet been seen in any frame). There is a lot of open choice here -- the goal is to make sure you understand the basics of tethered camera control, I'm not asking for anything fancy.
There are a few tricks here. So, let's talk about what you need to know....
You don't need to write any C code for this project (unless you want to), but you do need to write a script that will invoke all the other programs as needed. You can use Python, Borne Shell, etc. for that -- it's up to you. This is most naturally done using Linux, but can be done on windows instead.
It's not too bad (now that I've found a way that works). Read my page on use of FlashAir. In addition, here is a CHDK Lua script, wishoot.lua, to take a shot:
--[[
@title WiFi Shoot
@chdk_version 1.4
--]]
wifile = io.open("A/X/CMD", "r")
num = 0 + wifile:read()
wifile:flush()
wifile:close()
repeat
print("Was:", num)
repeat
sleep(500)
wifile = io.open("A/X/CMD", "r")
shot = 0 + wifile:read()
wifile:flush()
wifile:close()
print("Is:", shot)
until (shot ~= num)
num = shot
press("shoot_half")
repeat sleep(50) until get_shooting() == true
us = tv96_to_usec( get_tv96() )
iso = get_iso_real()
press("shoot_full")
sleep(500)
release("shoot_full")
repeat sleep(50) until get_shooting() == false
release("shoot_half")
ex = get_exp_count()
jpeg = string.format('%s/IMG_%04d.JPG', get_image_dir(), ex%10000)
wifile = io.open("A/X/RESP", "w")
wifile:write(jpeg, " is exposure ", num, ": ", us, "us @ ISO ", iso, "\n")
wifile:flush()
wifile:close()
until (false)
And here is a little shell script, shoot.sh, to talk to it:
#!/bin/sh expr `cat CMD` + 1 >CMD echo "Taking shot" `cat CMD` curl -s --form "file=@CMD" http://192.168.0.1/upload.cgi >/dev/null sleep 10 curl -s http://192.168.0.1/X/RESP -o RESP cat RESP
In some ways, it is more elegant to use USB tethering -- but you don't have to do this if you use wifi tethering, as described above. USB control can be done using either chdkptp or via gphoto. The big advantage to these methods is that they do not require a script on the camera... but they do need a USB cable. On the other hand, wireless tether is cooler and I gave you the code for it above.
chdkptp is an incredibly powerful way to implement tethered control of a CHDK PowerShot. It's also not the easiest thing to install, and it needs to be able to find the Lua modules that it uses to do way more than you'd expect it would be doing with Lua. Most importantly for this project, it allows shooting images and sending them back to a host via USB without ever storing them on the SD card in the camera -- which is an important distinction given that SD cards start to fail after as few as 10,000 write cycles.
The command you need to use is rs, also called remoteshoot. A complete list of the commands is in USAGE.TXT, which is about as close as you'll get to a real manual for this tool. It's worth noting that the GUI is particularly difficult to run under Linux... not that you'll want to use a GUI for this assignment.
gphoto is really straightforward to use to capture a photo. The catch is that the command structure is very ad hoc, and what cameras support varies widely. It is documented here, but the basic list of gphoto commands is:
Usage: gphoto2 [-?qvalLnPTDR] [-?|--help] [--usage] [--debug]
[--debug-loglevel=STRING] [--debug-logfile=FILENAME] [-q|--quiet]
[--hook-script=FILENAME] [--stdout] [--stdout-size] [--auto-detect]
[--show-exif=STRING] [--show-info=STRING] [--summary] [--manual]
[--about] [--storage-info] [--shell] [-v|--version]
[--list-cameras] [--list-ports] [-a|--abilities] [--port=FILENAME]
[--speed=SPEED] [--camera=MODEL] [--usbid=USBIDs] [--list-config]
[--list-all-config] [--get-config=STRING] [--set-config=STRING]
[--set-config-index=STRING] [--set-config-value=STRING] [--reset]
[--keep] [--keep-raw] [--no-keep]
[--wait-event=COUNT, SECONDS, MILLISECONDS or MATCHSTRING]
[--wait-event-and-download=COUNT, SECONDS, MILLISECONDS or MATCHSTRING]
[--capture-preview] [--show-preview] [-B|--bulb=SECONDS]
[-F|--frames=COUNT] [-I|--interval=SECONDS] [--reset-interval]
[--capture-image] [--trigger-capture] [--capture-image-and-download]
[--capture-movie=COUNT or SECONDS] [--capture-sound]
[--capture-tethered=COUNT, SECONDS, MILLISECONDS or MATCHSTRING]
[--trigger-capture] [-l|--list-folders] [-L|--list-files]
[-m|--mkdir=DIRNAME] [-r|--rmdir=DIRNAME] [-n|--num-files]
[-p|--get-file=RANGE] [-P|--get-all-files] [-t|--get-thumbnail=RANGE]
[-T|--get-all-thumbnails] [--get-metadata=RANGE] [--get-all-metadata]
[--upload-metadata=STRING] [--get-raw-data=RANGE]
[--get-all-raw-data] [--get-audio-data=RANGE] [--get-all-audio-data]
[-d|--delete-file=RANGE] [-D|--delete-all-files]
[-u|--upload-file=FILENAME] [--filename=FILENAME_PATTERN]
[-f|--folder=FOLDER] [-R|--recurse] [--no-recurse] [--new]
[--force-overwrite] [--skip-existing]
Many of those will not work on any particular camera, including your PowerShot. There are various different config options, etc. that you will want need to look through by actually hooking-up the camera. The gphoto2 -a command is a good starting point. Incidentally, yes, the current gphoto is called gphoto2 because for a time there were two mutually-incompatible versions around.
As noted above, even lowly little cameras like your Canon PowerShots actually can recognize faces (if you enable face detection). Most digital cameras can, and they record the face information in the EXIF data. The bad news is that each camera brand (and in some cases, different models within a brand) uses different EXIF fields to store the relevant info. Fortunately for us, there is a wonderful little piece of free software that knows how to read that info for us....
The real key is a free program called ExifTool that was written by Phil Harvey. It knows how to extract a mind-bogglingly-diverse set of field values from an image file. It is actually structured as both a stand-alone program and a Perl library. He also wrote a little example code called facetest.pl that uses the library to extract face data and create a copy of each image with the faces boxed. I've slightly modified that program for you so that it simply creates a new image file with just the extracted face image: getface.pl file.jpg will create a copy of the face extracted from the file as a file with the same name in the subdirectory called tmp. If no face was recognized and tagged in the EXIF, it doesn't make a file. Note that the ExifTool library and ImageMagick (described below) must be installed for their Perl scripts to work.
ImageMagick is a powerful library for image processing -- and a suite of command-line tools that serve as a demonstration of the library and make ImageMagick very easy to use in scripts. The functionality of the tools overlap each other considerably, and one can do most things in convert. In particular, a command like:
convert -size 640x480 xc:black \ image1.jpg -geometry 640x480 -composite \ image2.jpg -geometry 100x80+529+0 -composite \ image3.jpg
Will create a black canvas of 640x480 pixels, scale image1.jpg to fit it, scale image2.jpg to 100x80 and put it in the upper right corner, and then write the resulting composite image to image3.jpg.
mencoder is a video encoder co-developed with mplayer. Command lines look very scary, but if you want to do it and it involves video encoding, odds are mencoder can help. For example:
echo "Mencoder Pass 1..." mencoder mf://*.jpg -mf w=320:h=240:fps=1:type=jpg -vf scale=320:240 -ovc lavc -lavcopts vcodec=mpeg4:keyint=120:vbitrate=1600:vpass=1 -o /dev/null echo "Mencoder Pass 2..." mencoder mf://*.jpg -mf w=320:h=240:fps=1:type=jpg -vf scale=320:240 -ovc lavc -lavcopts vcodec=mpeg4:keyint=120:vbitrate=1600:vpass=2 -o /dev/null 2>/dev/null |tail -1 echo "Mencoder Pass 3..." mencoder mf://*.jpg -mf w=320:h=240:fps=1:type=jpg -vf scale=320:240 -ovc lavc -lavcopts vcodec=mpeg4:keyint=120:vbitrate=1600:vpass=3 -o movie.mpg 2>/dev/null |tail -1
That sequence of commands will produce a silent video running at 1 FPS, which is 15X the rate the images were captured. That means each image will stay up for 1 second.
You will be submitting source code for your tethered control program or script, the CHDK Lua script, a make file (which does whatever is necessary), and a simple "implementor's notes" document, formatted roughly as described here. For this project, you definitely have design decisions to talk about in those notes -- such as how you coordinated the captures between the computer and the PowerShot.
For full consideration, your project should be submitted no later than the end of finals week, Friday, December 15, 2017. Submit your .tar or .tgz file here: