Being an Evil Genius with F# and .NET
A couple weeks ago I was in glorious Sandusky, Ohio presenting at CodeMash. CodeMash is a community driven conference that’s less about product announcements and typical “rah rah” and more about getting smart people together to talk about what they are passionate about – coding.
Rather than doing the prototypical “Intro to F# Talk” I figured I go with something a bit more fun and relevant to the every day developer. Sure F# is neat and everything – but why bother to learn a new programming language unless you can use it to do something meaningful. Well, in addition excelling at both functional and object-oriented programming, F# is ideal for world domination.
------------------------------------------------------
To be honest, this may be the best talk abstract I’ll ever write.
Being an Evil Genius with F# and .NET
In today's competitive global economy it's increasingly difficult to find quality henchmen to aid you in taking over the Earth. Fortunately the .NET platform has plenty of technological gems which can help you
in your quest.
Speech recognition APIs to automate your demands for ransom. The Parallel Extensions to .NET for distributing control of your nanobots. Even image recognition to identify CIA spies and kill them on sight!
This code-focused talk will show you the libraries you can use to make your next high-powered plot for world domination a success. (F# knowledge is not required, though will help any aspiring evil genius grep the demos.)
Someone once told me that at the beginning of a talk you should introduce yourself. Well, I’m much like you – except that my ambition isn’t to build a house or write the great American novel. It’s to take over the world!
I’ve been very fortunate in my career as an evil genius and 2007 was a hallmark year. I had just leased a new volcano lair, had a growing army of minions, and even got a hold of real nuclear missile. The movies these days make it look like just anybody with three billion dollars can get a hold of one of these babies – not true! I had to get my minions to stage a raid on an ex-Soviet munitions dump and then kidnap a scientist to rebuild the flight systems. Hard work, the work was well worth it.
But the global economic disaster struck me just like everyone else. As embarrassing as it is to admit this, I couldn’t keep up the payments on my volcano and it got foreclosed on. Then my minions were all ambushed and killed by some CIA commandos. And worst of all I had to give up my nuclear missile.
Saddened, I trudged on.
Then it dawned on me. Maybe being an effective evil genius doesn’t mean volcano lairs or orbiting space lasers, perhaps you can still terrorize the Earth with software.
And that brings us to the meat of my CodeMash talk, being an evil genius with F# and .NET. The talk isn’t so much about F# as it is about being an evil genius. But we all know F# is the premier .NET language for aspiring villains, so along the way I’ll highlight some language constructs that are especially helpful when plotting to take over the world.
The first problem you’ll encounter is how to actually strike your foes? What if they are behind a wall, in a bunker, or inside of a tank?
As it turns out the terrorist training organization ‘Nerf’ had it right all along. Foam missiles are the ideal weapon.
- Ever hear of an evil genius getting caught while trying to sneak foam missiles onto an airplane? There’s a reason for that…
- Since they are sponges after all, why not soak them in poison to make them more lethal?
In order to get your hands on a foam-missile-WMD you could enslave a few engineers to build you one, but if you are more in a time crunch you can pick one up at Amazon for about $40.
Dream Cheeky USB Missile Launcher
However, this piece of specialized military hardware isn’t quite ideal for terror operations. The software it comes with is a little clunky and doesn’t offer the degree of precision required for world domination. What would any evil genius do in this case? Haxor, of course!
A great magician never reveals his tricks, however that rule didn’t stop Pedram Amini from detailing the process of reverse engineering how the USB Rocket Launcher works.
https://dvlabs.tippingpoint.com/blog/2009/02/12/python-interfacing-a-usb-missile-launcher
Once you know the protocol for communicating with the missile launcher, it’s just a matter of reading and writing to the underlying USB device. Which you can do using P/Invoke in .NET. The code provided is in C# and was written by level-9 ninja master Matt Ellis.
[DllImport("kernel32.dll", SetLastError = true)]
[return: MarshalAs(UnmanagedType.Bool)]
public static extern unsafe bool ReadFile(SafeFileHandle hHandle, byte* lpBuffer, uint nNumberOfBytesToRead, ref uint lpNumberOfBytesRead, IntPtr lpOverlapped);
[DllImport("kernel32.dll", SetLastError = true)]
[return: MarshalAs(UnmanagedType.Bool)]
public static extern unsafe bool WriteFile(SafeFileHandle hHandle, byte* lpBuffer, uint nNumberOfBytesToWrite, ref uint lpNumberOfBytesWritten, IntPtr lpOverlapped);
/// <summary>
/// That is, call directly into the drivers for the Rocket Launcher
/// </summary>
/// On a 64-bit OS: [DllImport(@"C:\Program Files (x86)\USb Missile Launcher\USBHID.dll")]
[DllImport(@"C:\Program Files\USb Missile Launcher\USBHID.dll")]
public static extern SafeFileHandle OpenHID(ushort VendorId, ushort ProductId, ushort ReveisionId);
Once you have a wrapper on top of the rocket launcher, then it’s just a matter of writing some code to interact with it. For RocketLauncherOfDoom v1.0 we’ll just have a console interface. Something like this:
let ckInfo = Console.ReadKey(true)
let t =
if (ckInfo.Modifiers = ConsoleModifiers.Shift) then
15
else
5
let action =
match ckInfo.Key with
| ConsoleKey.UpArrow -> MoveUp(t)
| ConsoleKey.DownArrow -> MoveDown(t)
| ConsoleKey.LeftArrow -> MoveLeft(t)
| ConsoleKey.RightArrow -> MoveRight(t)
| ConsoleKey.Spacebar -> Fire
| ConsoleKey.Q -> (loop <- false)
NoOp
| _ -> NoOp
performAction rocketLauncher action |> ignore
However, we are far from done.
Once you have control over your USB rocket launcher, the problem is that you spend the rest of your time either making demands for ransom or listening to people beg for their lives. Both are kind of annoying and tedious. Fortunately we can automate this.
Build into Windows are the Microsoft Speech APIs (SAPI) which can do both speech recognition – transcribing feeble cries for help – and speech synthesis – giving your robo-army a verbal presence.
Getting setup is simple. First, you need a quality microphone. I’ve been very pleased with my purchase of:
Plantronics Audio 995 Wireless Stereo Headset
Next, you can dramatically improve your computer’s accuracy at recognizing your voice by training your computer. Simply open up Control Panel and click ‘Speech Recognition’ and then ‘Train your computer to better understand you’. Essentially you will dictate to your computer for 10 minutes but the results are worth it.
Once you dictate Windows into submission, it’s time for code! The System.Speech APIs come with .NET 3.0 and are quite straight forward. The following snippet prints all installed voices on your system and says “Hello, World!”
#r "System.Speech.dll"
open System
open System.Speech.Synthesis
// Say "Hello, World!"
let prompt = new PromptBuilder()
prompt.StartParagraph()
prompt.AppendText("Hello", PromptEmphasis.Strong)
prompt.AppendText("World", PromptEmphasis.Reduced)
prompt.EndParagraph()
// List installed voices
let synthesizer = new SpeechSynthesizer()
let installedVoices = synthesizer.GetInstalledVoices()
for installedVoice in installedVoices do
printfn
"Enabled:%b, Name:%s, Age:%A, Description:%s"
installedVoice.Enabled
installedVoice.VoiceInfo.Name
installedVoice.VoiceInfo.Age
installedVoice.VoiceInfo.Description
synthesizer.Speak(prompt)
Speech recognition is even easier. Once you instantiate the recognizer object, simply add an event handler to the ‘SpeechRecognized’ event. For more information about the inner workings of the System.Speech namespace, check out the documentation on MSDN.
#r "System.Speech.dll"
open System
open System.Speech.Recognition
let sp = new SpeechRecognitionEngine()
let defaultDictationGrammar = new DictationGrammar()
defaultDictationGrammar.Name <- "Default Dictation"
defaultDictationGrammar.Enabled <- true
sp.LoadGrammar(defaultDictationGrammar)
sp.SetInputToDefaultAudioDevice()
sp.RecognizeAsync(RecognizeMode.Multiple)
//sp.SpeechHypothesized.Add(fun args -> printfn "Hypothesized [%f] '%s'" args.Result.Confidence args.Result.Text)
sp.SpeechRecognized.Add(fun args -> printfn "Recognized [%f] '%s'" args.Result.Confidence args.Result.Text)
With a mastery of speech and sound, we can now upgrade our missile launcher software to be controlled via voice activation. The following snippet shows F#’s first-class events in action. Rather than interacting with the System.Speech APIs directly, the getRecognizer function returns an event object, to which you can add event handlers later. The result is a pretty clean implementation IMHO.
// Returns an event handler which fires when spoken text is
// recognized
let getWordRecognizer() =
let sp = new SpeechRecognitionEngine()
let defaultDictationGrammar = new DictationGrammar()
defaultDictationGrammar.Name <- "Default Dictation"
defaultDictationGrammar.Enabled <- true
sp.LoadGrammar(defaultDictationGrammar)
sp.SetInputToDefaultAudioDevice()
sp.RecognizeAsync(RecognizeMode.Multiple)
// Return an instance of an event which fires when words are recognized.
// Callers can then subscribe to that event.
sp.SpeechRecognized
|> Event.map(fun args -> args.Result.Text.ToLower())
With the speech recognized event, all we need to do is write two event handlers. One to simply interact with the rocket and another to listen for when the user wants to exit the program.
let recognizerEvent = getWordRecognizer()
// Main handler - convert spoken text into RL commands
let handleWord spokenText =
printfn "Recognized Word: %s" spokenText
let action =
match spokenText with
| "up" // Has a hard time recognizing this :(
| "north" -> MoveUp(20)
| "down" -> MoveDown(20)
| "left" -> MoveLeft(20)
| "right" -> MoveRight(20)
| "fire" -> Fire
| _ -> NoOp
performAction rocketLauncher action |> ignore
// Exit handler - specifically look for exit/quit
let terminateLoop = ref false
let terminateLoopHandler = function | "exit"
| "quit" -> terminateLoop := true
| _ -> ()
// Hook up event handlers
recognizerEvent.Add(handleWord)
recognizerEvent.Add(terminateLoopHandler)
while terminateLoop.Value = false do
System.Threading.Thread.Yield() |> ignore
()
Of course, the biggest challenge isn’t actually controlling your rocket launcher, but determining what to shoot.
Next we’ll try to add in some computer vision to identify faces. This is where things get serious, computer vision truly is the rocket science of software. Think about it, I give you an image, which is a matrix of pixels, which are a tuple of color intensities, which are just values, and you tell me if there is a ‘face’ somewhere in that sea of numbers? The sheer idea is ludicrous, but somehow people much smarter than me have figured out how to do it.
Currently the best computer vision library is Open CV, which continues to evolve with active research. If you are interested in learning more, check out:
Learning OpenCV, O’Reilly
However, OpenCV is written in C/C++ and we all know that true evil geniuses use .NET. While you could just write a million P/Invoke calls and use the OpenCV library as-is, fortunately the Egmu CV is a .NET wrapper on top of OpenCV.
For the purposes of this blog post I’ll avoid the specifics of how face detection works in OpenCV. In general, OpenCV uses a technique developed by Paul Viola and Michael Jones to quickly identify ‘features’ of an image. If a region shares many ‘features’ of a human face, then a face is detected. However, the Viola-Jones method can be used for detecting more than just faces. You can actually train the computer to identify any custom ‘features’ as long as you have an ample set of example images.
For more information check out the following resources:
https://www.cognotics.com/opencv/servo_2007_series/index.html
https://www.cognotics.com/opencv/servo_2007_series/part_2/sidebar.html
https://en.wikipedia.org/wiki/Viola-Jones_object_detection_framework
Now let’s get back to sewing evil!
To get setup detecting faces you’ll need to install OpenCV, EmguCV, and optionally get a webcam. For a cheap and viable option, try:
Using OpenCV to detect faces is very straightforward. The following code opens up a HaarCascade (or set of features to identify a specific thing in an image) and puts a square around the provided image file.
let detectFaces (filePath : string) =
// Load the image file
let img = new Image<Bgr, byte>(filePath)
// Resize to something resonable
let img = img.Resize(float 300 / float img.Width, CvEnum.INTER.CV_INTER_CUBIC)
// Load the object detector
let haarCascades = @"C:\OpenCV2.0\data\haarcascades\"
let objectToDetect = new HaarCascade(haarCascades + "haarcascade_frontalface_default.xml")
// Convert to grayscale
let imgGray : Image<Bgr, byte> = img.Convert()
let faces : MCvAvgComp[] = imgGray.DetectHaarCascade(objectToDetect).[0]
for face in faces do
img.Draw(face.rect, new Bgr(Color.White), 1)
// Display the image
let viewer = new ImageViewer()
viewer.Image <- img
viewer.Show() |> ignore
OpenCV comes with a slew of these HaarCascades for detecting faces from different angles. I' wrote a simple WinForms application where you can open all the images in a directory and see what the HaarCascades identify.
The results are hit and miss, but overall I’m very impressed with the results.
While automating USB missile launchers and striking terror into the heart of men is fun, there are other worthwhile uses of these libraries. Using the System.Speech APIs you can write software to go hands free. For example, have a program dictate new F# questions on StackOverflow as they are posted. Or, training OpenCV to detect image features in real-time, such as defects in industrial manufacturing.
If you are interested in learning more about F#, you should consider buying my book. Have fun conquering!
Disclaimer #1
All code samples are provided "AS IS" without warranty of any kind, either express or implied, including but not limited to the implied warranties of merchantability and/or fitness for a particular purpose. So in other words, if the attached source files cause you any grief, up to and including destroying your USB rocket launcher, you can’t blame me or Microsoft.
Disclaimer #2
Theme of evil genius-ery aside, consider the following. If you have above average programming skills, such as knowledge of reverse engineering debuggers or decompilers, then just like Spiderman you have amazing powers. Please consider the legal and ethical ramifications of your actions. Pedram Amini showing how to reverse engineer a USB rocket launcher is innocent enough. But you can apply the same techniques to cause real evil in the world. (For example spying on ordinary citizens.)
The point is, like Uncle Ben said, “with great power comes great responsibility.” This applies to super heroes and evil geniuses alike.
Comments
Anonymous
January 24, 2010
Chris, What a great post. It was fun, engaging and I bet the attendees had a blast! Thanks for sharing. CoryAnonymous
January 25, 2010
Awesome presentation Chris and thanks for the F# discussion afterwords.Anonymous
January 25, 2010
Good Job....... Nice Style to discussion. ... . . . :)Anonymous
January 26, 2010
Thanks everyone for your kind words! I'm glad you found the talk entertaining; hopefully it will make F# seem a bit more approachable. Cheers, -ChrisAnonymous
February 12, 2010
Most Excellent talk, but I apologize, Chris, if you noticed me during the talk, in and out, in and out. Just far too many things going on at that time that I had to pay attention to. So glad that I finally got around to looking up the talk online, and found the parts that I missed (very little,thank goodness, just some details). Off to buy your book! :)Anonymous
February 12, 2010
Great presentation! Just brilliant! Although you might want to add a third disclaimer. Disclaimer #3 I do not intend to conquer the world nor am I intending to teach others to do so. On these days you never know. You might get flagged by a bot as a terrorist. Cheers! Nacho