XNA from Silverlight on Windows Phone 7 – The Microphone

One of the things I showed at the WPUG meeting was accessing XNA APIs from Silverlight. In some cases because it’s the only way to achieve what you need (eg access to the microphone) and in others because it makes your life easier (eg gestures). In this post I’ll cover microphone access from Silverlight. Peter Foot’s blog entry helped me a lot in getting this up and running.

Access to the microphone is provided through the Microsoft.Xna.Framework.Audio namespace in the Microsoft.Xna.Framework assembly. You’ll need to add a reference to this XNA assembly from your WP7 Silverlight app if you need microphone access.

The class we’re interested in is Microphone. This class provides access to all the available microphones on the system and exposes a static property – Default – that returns the Microphone instance for the current default recording device. Once we Start the microphone, it begins buffering data and, at some point, will fire the BufferReady event. At this point it’s our responsibility to empty the buffer. This goes on until we want to stop recording. To do that we simply call Stop().

Once we have this data we can use the SoundEffect class to play it back and even mess with it a little. I’ve wired the pitch parameter up to a slider in my app so I can sound like either Pinky and Perky or Don LaFontaine depending on my mood. Here’s video of the app in action:

 

As you can see, the UI is ultra-simple:

Start and Stop are just buttons to start and stop recording (for ease, stop also initiates playback). The pitch slider is on the right.

 using System;
using System.IO;
using System.Windows;
using Microsoft.Phone.Controls;
using Microsoft.Xna.Framework.Audio;

namespace HelloSoundWorld
{
    public partial class RecordSound : PhoneApplicationPage
    {
        MemoryStream ms;
        Microphone mic = Microphone.Default;

        // Wire up an event handler so we can empty the buffer when full
        // Crank up the volume to max
        public RecordSound()
        {
            InitializeComponent();
            mic.BufferReady += Default_BufferReady;
            SoundEffect.MasterVolume = 1.0f;
        }

        // When the buffer's ready we need to empty it
        // We'll copy to a MemoryStream
        // We could push into IsolatedStorage etc
        void Default_BufferReady(object sender, EventArgs e)
        {
            byte[] buffer = new byte[1024];
            int bytesRead = 0;

            while ((bytesRead = mic.GetData(buffer, 0, buffer.Length)) > 0)
                ms.Write(buffer, 0, bytesRead);
        }

        // The user wants to start recording. If we've already made 
        // a recording, close that MemoryStream and create a new one.
        // Start recording on the default device.
        private void start_Click(object sender, RoutedEventArgs e)
        {
            if (ms != null)
                ms.Close();

            ms = new MemoryStream();

            mic.Start();
        }

        // The user wants to stop recording. Checks the microphone
        // is stopped. Reset the MemoryStream position.
        // Play back the recording. Pitch is based on slider value
        private void stop_Click(object sender, RoutedEventArgs e)
        {
            if (mic.State != MicrophoneState.Stopped)
                mic.Stop();

            ms.Position = 0;

            SoundEffect se = 
                new SoundEffect(
                    ms.ToArray(), mic.SampleRate, AudioChannels.Mono);
            se.Play(1.0f, (float)slider1.Value, 0.0f);
        }
    }
}

Run this and you’ll hit an InvalidOperationException. There’s something else we need to do:

We need to add some boilerplate code to make sure the XNA Framework is happy. You can find more information here.

Add a new class (in your App.xaml.cs will do):

 public class XNAAsyncDispatcher : IApplicationService
{
    private DispatcherTimer frameworkDispatcherTimer;

    public XNAAsyncDispatcher(TimeSpan dispatchInterval)
    {
        this.frameworkDispatcherTimer = new DispatcherTimer();
        this.frameworkDispatcherTimer.Tick +=
            new EventHandler(frameworkDispatcherTimer_Tick);
        this.frameworkDispatcherTimer.Interval = dispatchInterval;
    }

    void IApplicationService.StartService(ApplicationServiceContext context)
    {
        this.frameworkDispatcherTimer.Start();
    }

    void IApplicationService.StopService()
    {
        this.frameworkDispatcherTimer.Stop();
    }

    void frameworkDispatcherTimer_Tick(object sender, EventArgs e)
    {
        FrameworkDispatcher.Update();
    }
}

And hook this up in the constructor for the App (Application) class:

 this.ApplicationLifetimeObjects.Add(
    new XNAAsyncDispatcher(TimeSpan.FromMilliseconds(50)));

Which adds it as a service to the Silverlight application. XNA should now be happy. Record away!

Comments

• Anonymous
  August 30, 2010
  I would be great if the microphone minimum specs were specified for at least, the sample rate. The current sample rate in the emulator is 16000Hz, which is far from what's standard on other phone (48000Hz on iPhone 3G/+/4G). This spec is important when you are relying on some spectral analysis of the audio signal (or quality audio recording). Having no minimum spec requires to handle various different sample rate, meaning also that the microphone experience is not consistent across different hardware... Do you know what's the status concerning this - small - issue?