How to Build a Flipbook Maker: Part 2
In Part 1, we assembled the hardware and created the event handlers for the Flipbook Maker. Now let's flesh out the software foundation and create a state machine to manage the Flipbook Maker's behavior.
Storing Images
First, the Flipbook Maker needs a place to store the images captured by the camera. The .NET Micro Framework's ArrayList class is great for this - it behaves like an array but it's dynamically resizable so it doesn't take up more memory than it needs at any given time. We'll set an upper limit of 60 images, though, to prevent running out of memory.
To use ArrayList, first add System.Collections to your program's using list.
using System.Collections;
Then create an ArrayList object and supporting objects inside the Program Class:
Image cameraImage;
bool pictureTakingInProgress = false;
We'll also create two TimeSpans representing slow and fast playback speeds.
TimeSpan PLAYBACK_SPEED_NORMAL = new TimeSpan(0, 0, 0, 0, 1000);
TimeSpan PLAYBACK_SPEED_FAST = new TimeSpan(0, 0, 0, 0, 300);
These timespans are specified in milliseconds, so the slower one equates to one frame per second and the faster one to 3.33 frames per second.
Finally, one last declaration inside the Program class:
GT.StorageDevice SDCard;
This keeps a reference handy to the object that manages reads/writes to the SD card.
The event handlers for SD card mount/unmount simply need to update this reference:
void sd_SDCardMounted(GTM.GHIElectronics.SDCard sender, StorageDevice SDCard)
{
this.SDCard = SDCard;
}
void sd_SDCardUnmounted(GTM.GHIElectronics.SDCard sender)
{
this.SDCard = null;
}
Flipbook Maker Playback States
The Flipbook Maker is built as a state machine where the device does different things in different states. The relevant states are:
enum PlaybackMode { Forward, ForwardFast, Backward, BackwardFast, Stopped, Recording, Initializing };
PlaybackMode currentMode = PlaybackMode.Initializing;
These are pretty self-descriptive: the Flipbook Maker starts in initialization mode. After initialization, Stopped is the default playback state: this shows the current frame (or nothing if there are no frames yet).
The active playback states let you watch your movie in forward or reverse mode, and at the slow and fast playback intervals that we defined above.
Finally, the Recording mode helps you pose your shots. While you hold the Button down, the camera will stream a live feed to the display. This is Recording mode. When you release the button, the last image becomes the newest frame in the movie. The Flipbook Maker the returns to Stopped mode.
Implementing the Playback Timer
After the Flipbook Maker boots, two timers start running. Here's the implementation for the playback timer. It first checks whether there are frames to display and that the camera is ready (the camera takes a few second to start up and it also needs a second or two to process each picture it takes).
Then, if the current playback mode is Forward or ForwardFast, it increments the index to the current frame. If the playback mode is Backward or BackwardFast, it decrements the current frame index. In each case, it checks whether the index should wrap to the start or end of the picture array.
void playbackTimer_Tick(GT.Timer timer)
{
if (frames.Count == 0) return;
if (!camera.CameraReady) return;
if (currentMode != PlaybackMode.Stopped || currentMode != PlaybackMode.Recording)
{
if (currentMode == PlaybackMode.Forward || currentMode == PlaybackMode.ForwardFast)
{
currentFrame++;
if (currentFrame > frames.Count - 1)
currentFrame = 0;
}
if (currentMode == PlaybackMode.Backward || currentMode == PlaybackMode.BackwardFast)
{
currentFrame--;
if (currentFrame < 0)
currentFrame = frames.Count - 1
}
cameraImage.Bitmap = (Bitmap)frames[currentFrame];
}
}
Taking Pictures
The Button is used for capturing pictures. Holding the button down causes live video to stream from the camera to the display. Releasing the button (or simply pressing it without holding it down) causes the camera to take a picture and add it to the ArrayList.
The ButtonPressed event handler checks that the camera is ready to take pictures and that there is room in memory to store more frames. Then it stops the playback and potentiometer timers - this prevents any confusing playback state changes between the end of the ButtonPressed method and the camera's BitmapStreamed callback.
void button_ButtonPressed(GTM.GHIElectronics.Button sender, GTM.GHIElectronics.Button.ButtonState state)
{
if (!camera.CameraReady) return;
if (frames.Count <= MAXIMUM_FRAMES)
{
if (playbackTimer.IsRunning) playbackTimer.Stop();
if (potentiometerCheckTimer.IsRunning) potentiometerCheckTimer.Stop();
currentFrame++;
currentMode = PlaybackMode.Recording;
cameraImage.Bitmap = new Bitmap((int)display.Width, (int)display.Height); // Give the camera and screen a new bitmap to stream to.
camera.StartStreamingBitmaps(cameraImage.Bitmap);
}
}
Now the camera is streaming bitmaps. Each time an image is sent to the mainboard, it's sent to the display if the user is holding the button down. When the button is released, the image is inserted into the array of frames. Then the potentiometer checker is restarted.
void camera_BitmapStreamed(GTM.GHIElectronics.Camera sender, Bitmap bitmap)
{
if (currentMode == PlaybackMode.Initializing)
{
numBitmapsThrown++;
if (numBitmapsThrown == BITMAPS_TO_THROW)
{
currentMode = PlaybackMode.Stopped;
camera.StopStreamingBitmaps();
}
return;
}
cameraImage.Invalidate();
if (button.IsPressed) return; //carry on streaming pictures to the screen while the button is pressed
frames.Insert(currentFrame, cameraImage.Bitmap);
camera.StopStreamingBitmaps();
potentiometerCheckTimer.Start();
}
See that bit which executes if the Flipbook Maker is initializing? The Camera module takes a few frame to callibrate its white balance. The Flipbook Maker throws away the first few bitmaps because they tend to look washed out. Here's a bit of support code which goes into the Program class:
// Camera will calibrate its brightness by taking some pictures,
// which we throw away.
Bitmap throwawayBitmap = new Bitmap(320, 240);
int numBitmapsThrown = 0;
const int BITMAPS_TO_THROW = 5;
Implementing the Potentiometer Timer
The last thing we'll cover today is the implementation of the Potentiometer timer, which allows the user to play back the movies they've created. If the potentiometer is near the middle, nothing is played back (currentMode == PlaybackMode.Stopped). When the potentiometer is turned right or left, playback starts forward or backward respectively. The amount which the potentiometer is turned determines the playback speed - far to the right is PlaybackMode.ForwardFast, and far to the left is PlaybackMode.BackwardFast.
This event handler reads the Poteniometer's position relative to center. A value of 0.0 is as far left as the Potentiometer goes, and a value of 1.0 is as far right as it goes. 0.5 is exact center.
If the FlipbookMaker is initializing, the throwaway bitmaps are taken (and thrown away). If the camera is currently taking a picture, no playback events happen. The rest of the handler maps various Potentiometer readings to PlaybackModes, and the PlaybackTimer is set accordingly.
void potentiometerCheckTimer_Tick(Gadgeteer.Timer timer)
{
double potValue = potentiometer.ReadPotentiometerPercentage();
if (currentMode == PlaybackMode.Initializing)
{
if (camera.CameraReady)
{
camera.StartStreamingBitmaps(throwawayBitmap);
}
return;
}
if (pictureTakingInProgress) return;
if (potValue < 0.40 && potValue > 0.2)
{
if (currentMode != PlaybackMode.Backward)
{
currentMode = PlaybackMode.Backward;
playbackTimer.Interval = PLAYBACK_SPEED_NORMAL;
if (!playbackTimer.IsRunning) playbackTimer.Start();
}
}
else if (potValue < 0.15)
{
if (currentMode != PlaybackMode.BackwardFast)
{
currentMode = PlaybackMode.BackwardFast;
playbackTimer.Interval = PLAYBACK_SPEED_FAST;
if (!playbackTimer.IsRunning) playbackTimer.Start();
}
}
else if (potValue > 0.60 && potValue < 0.8)
{
if (currentMode != PlaybackMode.Forward)
{
currentMode = PlaybackMode.Forward;
playbackTimer.Interval = PLAYBACK_SPEED_NORMAL;
if (!playbackTimer.IsRunning) playbackTimer.Start();
}
}
else if (potValue > 0.85)
{
if (currentMode != PlaybackMode.ForwardFast)
{
currentMode = PlaybackMode.ForwardFast;
playbackTimer.Interval = PLAYBACK_SPEED_FAST;
if (!playbackTimer.IsRunning) playbackTimer.Start();
}
}
else if (potValue < 0.55 && potValue > 0.45)
{
if (currentMode != PlaybackMode.Stopped)
{
currentMode = PlaybackMode.Stopped;
if (playbackTimer.IsRunning) playbackTimer.Stop();
}
}
}
The Flipbook Maker to So Far and What's Next
In Part 3, we'll implement the WPF-based UI for the touchscreen menu. In the process, we'll also enable the Flipbook Maker to save images to disk, delete frames which didn't turn out quite right, or clear the entire movie and start over.
For convenience, here's the entire program to date:
using System;
using System.Collections;
using System.Threading;
using Microsoft.SPOT;
using Microsoft.SPOT.Presentation;
using Microsoft.SPOT.Presentation.Controls;
using Microsoft.SPOT.Presentation.Media;
using Microsoft.SPOT.Touch;
using Gadgeteer.Networking;
using GT = Gadgeteer;
using GTM = Gadgeteer.Modules;
using Gadgeteer.Modules.GHIElectronics;
namespace FlipbookMaker
{
public partial class Program
{
GT.StorageDevice SDCard;
enum PlaybackMode { Forward, ForwardFast, Backward, BackwardFast, Stopped, Recording, Initializing };
PlaybackMode currentMode = PlaybackMode.Initializing;
ArrayList frames = new ArrayList();
int currentFrame = 0;
const int MAXIMUM_FRAMES = 60;
TimeSpan PLAYBACK_SPEED_NORMAL = new TimeSpan(0, 0, 0, 0, 1000);
TimeSpan PLAYBACK_SPEED_FAST = new TimeSpan(0, 0, 0, 0, 300);
GT.Timer playbackTimer = new GT.Timer(new TimeSpan(0, 0, 0, 0, 200));
GT.Timer potentiometerCheckTimer = new GT.Timer(new TimeSpan(0, 0, 0, 0, 100));
// Camera will calibrate its brightness by taking some pictures,
// which we throw away.
Bitmap throwawayBitmap = new Bitmap(320, 240);
int numBitmapsThrown = 0;
const int BITMAPS_TO_THROW = 5;
Image cameraImage;
bool pictureTakingInProgress = false;
// This method is run when the mainboard is powered up or reset.
void ProgramStarted()
{
// Set up event handlers for modules
button.ButtonPressed += new GTM.GHIElectronics.Button.ButtonEventHandler(button_ButtonPressed);
camera.BitmapStreamed += new GTM.GHIElectronics.Camera.BitmapStreamedEventHandler(camera_BitmapStreamed);
sdCard.SDCardMounted += new SDCard.SDCardMountedEventHandler(sdCard_SDCardMounted);
sdCard.SDCardUnmounted += new GTM.GHIElectronics.SDCard.SDCardUnmountedEventHandler(sdCard_SDCardUnmounted);
// Set up event handlers for timers, and start timers running
playbackTimer.Tick += new GT.Timer.TickEventHandler(playbackTimer_Tick);
playbackTimer.Start();
potentiometerCheckTimer.Tick += new Gadgeteer.Timer.TickEventHandler(potentiometerCheckTimer_Tick);
potentiometerCheckTimer.Start();
// Use Debug.Print to show messages in Visual Studio's "Output" window during debugging.
Debug.Print("Program Started");
}
void sdCard_SDCardMounted(GTM.GHIElectronics.SDCard sender, GT.StorageDevice SDCard)
{
this.SDCard = SDCard;
}
void sdCard_SDCardUnmounted(GTM.GHIElectronics.SDCard sender)
{
this.SDCard = null;
}
void button_ButtonPressed(GTM.GHIElectronics.Button sender, GTM.GHIElectronics.Button.ButtonState state)
{
if (!camera.CameraReady) return;
if (frames.Count <= MAXIMUM_FRAMES)
{
if (playbackTimer.IsRunning) playbackTimer.Stop();
if (potentiometerCheckTimer.IsRunning) potentiometerCheckTimer.Stop();
currentFrame++;
currentMode = PlaybackMode.Recording;
cameraImage.Bitmap = new Bitmap((int)display.Width, (int)display.Height); // Give the camera and screen a new bitmap to stream to.
camera.StartStreamingBitmaps(cameraImage.Bitmap);
}
}
void playbackTimer_Tick(GT.Timer timer)
{
if (frames.Count == 0) return;
if (!camera.CameraReady) return;
if (currentMode != PlaybackMode.Stopped || currentMode != PlaybackMode.Recording)
{
if (currentMode == PlaybackMode.Forward || currentMode == PlaybackMode.ForwardFast)
{
currentFrame++;
if (currentFrame > frames.Count - 1)
currentFrame = 0;
}
if (currentMode == PlaybackMode.Backward || currentMode == PlaybackMode.BackwardFast)
{
currentFrame--;
if (currentFrame < 0)
currentFrame = frames.Count - 1;
}
cameraImage.Bitmap = (Bitmap)frames[currentFrame];
}
}
void potentiometerCheckTimer_Tick(Gadgeteer.Timer timer)
{
double potValue = potentiometer.ReadPotentiometerPercentage();
if (currentMode == PlaybackMode.Initializing)
{
if (camera.CameraReady)
{
camera.StartStreamingBitmaps(throwawayBitmap);
}
return;
}
if (pictureTakingInProgress) return;
if (potValue < 0.40 && potValue > 0.2)
{
if (currentMode != PlaybackMode.Backward)
{
currentMode = PlaybackMode.Backward;
playbackTimer.Interval = PLAYBACK_SPEED_NORMAL;
if (!playbackTimer.IsRunning) playbackTimer.Start();
}
}
else if (potValue < 0.15)
{
if (currentMode != PlaybackMode.BackwardFast)
{
currentMode = PlaybackMode.BackwardFast;
playbackTimer.Interval = PLAYBACK_SPEED_FAST;
if (!playbackTimer.IsRunning) playbackTimer.Start();
}
}
else if (potValue > 0.60 && potValue < 0.8)
{
if (currentMode != PlaybackMode.Forward)
{
currentMode = PlaybackMode.Forward;
playbackTimer.Interval = PLAYBACK_SPEED_NORMAL;
if (!playbackTimer.IsRunning) playbackTimer.Start();
}
}
else if (potValue > 0.85)
{
if (currentMode != PlaybackMode.ForwardFast)
{
currentMode = PlaybackMode.ForwardFast;
playbackTimer.Interval = PLAYBACK_SPEED_FAST;
if (!playbackTimer.IsRunning) playbackTimer.Start();
}
}
else if (potValue < 0.55 && potValue > 0.45)
{
if (currentMode != PlaybackMode.Stopped)
{
currentMode = PlaybackMode.Stopped;
if (playbackTimer.IsRunning) playbackTimer.Stop();
}
}
}
void camera_BitmapStreamed(GTM.GHIElectronics.Camera sender, Bitmap bitmap)
{
if (currentMode == PlaybackMode.Initializing)
{
numBitmapsThrown++;
if (numBitmapsThrown == BITMAPS_TO_THROW)
{
currentMode = PlaybackMode.Stopped;
camera.StopStreamingBitmaps();
}
return;
}
cameraImage.Invalidate();
if (button.IsPressed) return; //carry on streaming pictures to the screen while the button is pressed
frames.Insert(currentFrame, cameraImage.Bitmap);
camera.StopStreamingBitmaps();
potentiometerCheckTimer.Start();
}
}
}