# Monday, June 09, 2008

A Simple Sound Effects Engine for Silverlight 2

Audio support is all too often one of the last things a developer thinks of when it comes to building a user interface. Fortunately, Silverlight offers some basic building blocks that make it fairly trivial to add event-driven sounds to our applications. I have taken the built-in functionality one step further, and created a simple run-time audio engine that makes it even easier.

First, I will discuss the requirements that I had in mind when I created this engine. Second, I will discuss the ideal API for a complete solution. Third, I submit the simple effects engine that I developed in order to meet those requirements and API.

Sound Effects Engine Requirements

  • Must be able to play looping “background music” tracks.
  • Must be able to play smaller sound effects on demand.
  • Must support concurrent (and overlapping) sounds effects without interfering with already-playing audio.
  • Must adequately handle the possibility of download delays when pulling down external audio files.
  • Should cache audio files between playbacks.
  • Should allow serving of audio from high-bandwidth CDN such as Silverlight Streaming.

An Ideal Sound Effects API

  • Should be very approachable. If possible, only a single line of code to play any audio (similar to PlaySound() API).
  • Should support any formats supported by Silverlight.
  • Should be efficient.
  • Should offer download and playback completion callbacks (for chaining and synchronization).
  • Should support aggressive background pre-caching of audio files.

The SilverlightToolbox Solution

To address the simplicity requirements, this sound effects engine is implemented as a single static C# class that can be easily included into any Silverlight project. It can exist in a referenced class library, or can be linked directly into the main project.

To address looping background music, a single method SetBackgroundLoop() is supplied. This will begin playback of a single audio clip, and will automatically repeat it. Only one track can be played as the background loop – calling this method again will terminate the running track and start the new one. You can also pass String.Empty to stop all background music.

To address typical sound effect clips. two overloads of the PlaySoundEffect() method are supplied. The simpler of the two methods will simply play a clip once it has been downloaded. The extended version allows you to specify a callback for when the clip has been downloaded and started playing, and a second callback that can be used for notification that the clip has ended.

To address the scenario where multiple sound effects might be played at the same time, the engine internally maintains a queue of MediaElements. When a new sound needs to played, it pulls an unused element and puts it into service. Once the sound is finished, the element is returned to the queue. This allows Silverlight itself to handle media mixing, and by caching those MediaElements, it does so without placing undue stress on the environment.

To address the issue of download delays, the startedCallback and completedCallback parameters of PlaySoundEffect() were introduced.

To address caching of audio files, a WebClient is used (which in turn leverages the browser’s cache). Furthermore, the engine will reuse any downloaded file streams (and is smart enough to not re-queue the same file if it already queued for download).

By using a MediaElement for playback, audio tracks are automatically supported from streaming sources and Content Delivery Networks, and can be of any type supported by Silverlight.

To support chaining of audio effects, and to support synchronization of UI with audio, the PlaySoundEffect() method accepts callback events that are fired when the clip has been downloaded, and again when it has completed playback. To further help with UI synchronization, a second utility class is provided (DelayedAction) which provides a simple wrapper for BackgroundWorker that can be used to easily delay execution of a block of code after a specified amount of time.

To support pre-caching of audio files, the PreloadMedia() method can be used.

Typical usage:

In order for the sound effects engine to function properly, it must be provided with a top-level XAML container (MediaElement currently will not perform playback without a parent object). This is done by calling the Initialize() method, typically from your program’s startup code:

SoundEffects.Initialize(this.LayoutRoot);

 

After this has been done, background music can be played, and we can also queue up some sound effect clips for later:

SoundEffects.SetBackgroundLoop("cautious-path.wma");
SoundEffects.PreloadMedia("pop1.wma");

 

Then, at various points throughout the application, we can play the sound effect (for example, in response to a control event):

SoundEffects.PlaySoundEffect("pop1.wma");

 

If at any point we need to delay the sound effect slightly, we can control this be introducing a short delay:

DelayedAction.Execute(2.0, () => SoundEffects.PlaySoundEffect("pop1.wma"));

 

Complete Source Code for DelayedAction.cs:

   1: using System;
   2: using System.ComponentModel;
   3: using System.Threading;
   4:  
   5: namespace Wintellect.SilverlightToolbox
   6: {
   7:     public class DelayedAction
   8:     {
   9:         class DelayedCallback
  10:         {
  11:             public TimeSpan Delay { get; set; }
  12:             public Action Callback { get; set; }
  13:         }
  14:  
  15:         public static void Execute(double seconds, Action callback)
  16:         {
  17:             BackgroundWorker Delay = new BackgroundWorker();
  18:             Delay.DoWork += (s, e) =>
  19:             {
  20:                 DelayedCallback DelayCallback = (DelayedCallback)e.Argument;
  21:                 Thread.Sleep(DelayCallback.Delay);
  22:                 e.Result = DelayCallback.Callback;
  23:             };
  24:             Delay.RunWorkerCompleted += (s, e) =>
  25:             {
  26:                 Action Callback = e.Result as Action;
  27:                 Callback();
  28:             };
  29:             Delay.RunWorkerAsync(new DelayedCallback
  30:             {
  31:                 Delay = TimeSpan.FromSeconds(seconds),
  32:                 Callback = callback
  33:             });
  34:         }
  35:     }
  36: }

 

Complete Source Code for SoundEffects.cs:

   1: using System;
   2: using System.Collections.Generic;
   3: using System.IO;
   4: using System.Net;
   5: using System.Windows;
   6: using System.Windows.Controls;
   7: using System.Windows.Media;
   8:  
   9: namespace Wintellect.SilverlightToolbox
  10: {
  11:     public static class SoundEffects
  12:     {
  13:         static Panel Root;
  14:         static MediaElement BackgroundLoop = new MediaElement();
  15:         static WebClient EffectDownloader = new WebClient();
  16:  
  17:         static Queue<MediaElement> AvailableSoundEffectGenerators = new Queue<MediaElement>();
  18:         static Dictionary<string, Stream> DownloadedEffects = new Dictionary<string, Stream>();
  19:         static Queue<string> PendingDownloads = new Queue<string>();
  20:         static Queue<QueuedEffect> PendingEffects = new Queue<QueuedEffect>();
  21:         static Dictionary<MediaElement, Action> PendingStartupCallbacks = new Dictionary<MediaElement, Action>();
  22:         static Dictionary<MediaElement, Action> PendingCompletionCallbacks = new Dictionary<MediaElement, Action>();
  23:  
  24:         enum TargetType
  25:         {
  26:             BackgroundMusic,
  27:             SoundEffect,
  28:         }
  29:  
  30:         class QueuedEffect
  31:         {
  32:             public string MediaName { get; set; }
  33:             public TargetType Target { get; set; }
  34:             public Action StartedCallback { get; set; }
  35:             public Action CompletedCallback { get; set; }
  36:         }
  37:  
  38:         public static void Initialize(Panel root)
  39:         {
  40:             Root = root;
  41:             InitializeTarget(root, BackgroundLoop);
  42:             EffectDownloader.OpenReadCompleted += (s, e) =>
  43:             {
  44:                 DownloadedEffects[(string)e.UserState] = e.Result;
  45:                 DownloadEffects();
  46:                 PlayEffect();
  47:             };
  48:         }
  49:  
  50:         static void DownloadEffects()
  51:         {
  52:             if (PendingDownloads.Count == 0)
  53:                 return;
  54:             string MediaName = PendingDownloads.Dequeue();
  55:             EffectDownloader.OpenReadAsync(new Uri(MediaName, UriKind.Relative), MediaName);
  56:         }
  57:  
  58:         static void InitializeTarget(Panel root, MediaElement target)
  59:         {
  60:             target.Width = 0;
  61:             target.Height = 0;
  62:             target.Visibility = Visibility.Collapsed;
  63:             root.Children.Add(target);
  64:             target.AutoPlay = false;
  65:             target.MediaOpened += (s, e) =>
  66:             {
  67:                 MediaElement Target = s as MediaElement;
  68:                 Target.Volume = 0.35;
  69:                 Target.Play();
  70:                 if (PendingStartupCallbacks.ContainsKey(Target))
  71:                 {
  72:                     Target.Dispatcher.BeginInvoke(PendingStartupCallbacks[Target]);
  73:                     PendingStartupCallbacks.Remove(Target);
  74:                 }
  75:             };
  76:             target.MediaEnded += (s, e) =>
  77:             {
  78:                 MediaElement Target = s as MediaElement;
  79:                 Target.Stop();
  80:                 if (s == BackgroundLoop)
  81:                     Target.Play();
  82:                 else
  83:                     AvailableSoundEffectGenerators.Enqueue(Target);
  84:  
  85:                 if (PendingCompletionCallbacks.ContainsKey(Target))
  86:                 {
  87:                     Target.Dispatcher.BeginInvoke(PendingCompletionCallbacks[Target]);
  88:                     PendingCompletionCallbacks.Remove(Target);
  89:                 }
  90:             };
  91:         }
  92:  
  93:         static MediaElement GetUnusedEffectGenerator()
  94:         {
  95:             if (AvailableSoundEffectGenerators.Count > 0)
  96:                 return AvailableSoundEffectGenerators.Dequeue();
  97:             else
  98:             {
  99:                 MediaElement Result = new MediaElement();
 100:                 InitializeTarget(Root, Result);
 101:                 return Result;
 102:             }
 103:         }
 104:  
 105:         static void PlayEffect()
 106:         {
 107:             lock (PendingEffects)
 108:             {
 109:                 if (PendingEffects.Count == 0)
 110:                     return;
 111:                 QueuedEffect Effect = PendingEffects.Dequeue();
 112:                 if (DownloadedEffects.ContainsKey(Effect.MediaName))
 113:                 {
 114:                     MediaElement TargetElement = null;
 115:                     switch (Effect.Target)
 116:                     {
 117:                         case TargetType.BackgroundMusic:
 118:                             { TargetElement = BackgroundLoop; break; }
 119:                         case TargetType.SoundEffect:
 120:                             { TargetElement = GetUnusedEffectGenerator(); break; }
 121:                     }
 122:                     if (Effect.StartedCallback != null)
 123:                         PendingStartupCallbacks.Add(TargetElement, Effect.StartedCallback);
 124:                     if (Effect.CompletedCallback != null)
 125:                         PendingCompletionCallbacks.Add(TargetElement, Effect.CompletedCallback);
 126:                     TargetElement.SetSource(DownloadedEffects[Effect.MediaName]);
 127:                 }
 128:                 else
 129:                 {
 130:                     PendingEffects.Enqueue(Effect);
 131:                 }
 132:             }
 133:         }
 134:  
 135:         static void PlaySound(TargetType target, string mediaName, Action startedCallback, Action completedCallback)
 136:         {
 137:             if (target == TargetType.BackgroundMusic)
 138:                 if (BackgroundLoop.CurrentState != MediaElementState.Stopped)
 139:                     BackgroundLoop.Stop();
 140:  
 141:             if (mediaName != String.Empty)
 142:             {
 143:                 lock (PendingEffects)
 144:                 {
 145:                     if (!DownloadedEffects.ContainsKey(mediaName))
 146:                     {
 147:                         PendingDownloads.Enqueue(mediaName);
 148:                     }
 149:                     PendingEffects.Enqueue(new QueuedEffect
 150:                     {
 151:                         MediaName = mediaName,
 152:                         Target = target,
 153:                         StartedCallback = startedCallback,
 154:                         CompletedCallback = completedCallback
 155:                     });
 156:                 }
 157:                 DownloadEffects();
 158:                 PlayEffect();
 159:             }
 160:         }
 161:  
 162:         public static void SetBackgroundLoop(string mediaName)
 163:         {
 164:             PlaySound(TargetType.BackgroundMusic, mediaName, null, null);
 165:         }
 166:  
 167:         public static void PlaySoundEffect(string effectName)
 168:         {
 169:             PlaySound(TargetType.SoundEffect, effectName, null, null);
 170:         }
 171:  
 172:         public static void PlaySoundEffect(string effectName, Action startedCallback, Action completedCallback)
 173:         {
 174:             PlaySound(TargetType.SoundEffect, effectName, startedCallback, completedCallback);
 175:         }
 176:  
 177:         public static void PreloadMedia(string mediaName)
 178:         {
 179:             if (!DownloadedEffects.ContainsKey(mediaName))
 180:             {
 181:                 PendingDownloads.Enqueue(mediaName);
 182:                 DownloadEffects();
 183:             }
 184:         }
 185:     }
 186: }
.NET | Games | General | Media | Silverlight | Tips and Tricks
Monday, June 09, 2008 5:59:22 PM (Eastern Daylight Time, UTC-04:00) #  Disclaimer | Comments [7] | Related posts:
Asynchronous UI in WPF and Silverlight (and Windows Forms too)
Excellent Free Sound Effects Resource
Gem Blaster entered into "RIA Run" contest
Gem Blaster updated and enhanced for Silverlight 2
Awesome tool for Media Center - Lifextender
Atlanta Code Camp 2008

Wednesday, June 11, 2008 10:43:03 AM (Eastern Daylight Time, UTC-04:00)
Thanks for sharing this code, it looks great in principle.

However, unfortunately, on trying to implement it, it throws a System.NotSupportedException on the following line:

static void DownloadEffects()
{
if (PendingDownloads.Count == 0)
return;
string MediaName = PendingDownloads.Dequeue();
>>>> EffectDownloader.OpenReadAsync(new Uri(MediaName, UriKind.Relative), MediaName);
}


"Specified Method is not supported."

I've tried this in Firefox 2.0 and also IE6. Running on Vista x64.

best wishes


Carl
Carl
Wednesday, June 11, 2008 12:30:31 PM (Eastern Daylight Time, UTC-04:00)
It should be working for you. This code worked on the beta1 bits and also the beta2 bits. What kind of media file are you trying to play, and is it from a streaming source?
Keith
Wednesday, June 11, 2008 1:15:27 PM (Eastern Daylight Time, UTC-04:00)
Hi Keith

Thanks for your prompt reply. I'm working with the latest Silverlight 2 Beta 2 SDK with Vis Studio 2008. The sound file is in a subdirectory 'Sounds/swish.wma' and is set to build as Content' -> Copy If Newer.

I'm using a canvas as my top layout item, is this OK? It doesn't seem to be the bit that's causing the exception, that seems to be that the OpenReadAsync method of the WebClient. Here's the code to reproduce:


MAINMENU.XAML:

<UserControl x:Class="TheGame.mainMenu"
xmlns="http://schemas.microsoft.com/client/2007"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
Width="320" Height="240">
<Canvas x:Name="topCanvas" Width="320" Height="240" Background="DarkBlue">
...


MAINMENU.XAML.CS

public mainMenu()
{
InitializeComponent();

SoundEffects.Initialize(topCanvas);
SoundEffects.PreloadMedia("Sounds/swish.wma");
...



I've also tried, unsuccessfully:
- Not preloading the media and calling SoundEffects.PlaySoundEffect("Sounds/swish.wma");
- Putting swish.wma in the project root.
- Compiling swish.wma as a resource, rather than content.



Any ideas? Thanks again for what is a much-needed code example.


Carl
Carl
Wednesday, June 11, 2008 1:18:10 PM (Eastern Daylight Time, UTC-04:00)
Keith

EDIT: Some Googling and I believe this may be because I am running the project in a test html page
(http://silverlight.net/forums/p/11045/52294.aspx#52294) - I will try uploading to my host and see if this helps!

Carl

Carl
Wednesday, June 11, 2008 1:30:38 PM (Eastern Daylight Time, UTC-04:00)


EDIT: ...and now it's working, hosted. Hopefully this experience may help somebody else. And thanks again for the great sample code, Keith!
Carl
Monday, June 23, 2008 12:05:48 PM (Eastern Daylight Time, UTC-04:00)
Hi. I built a silverlight sound effect engine that works similiarly. I had some serious problems with it though in SL2 b1 and b2. To summarize, sounds would eventually stop working. I noticed that the State of a mediaelement would be Playing even though it shouldn't. Does your class have the same problem?
DDtMM
Monday, June 23, 2008 1:44:44 PM (Eastern Daylight Time, UTC-04:00)
I have not seen that problem with this code as of yet (not saying it is immune - but I haven't seen that happen).
Keith
Comments are closed.
View Keith Rome's profile on LinkedIn

On this page....

Archives

Navigation

Categories

About

Disclaimer
The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.

Sign In
Certification Logo Certification Logo Certification Logo Certification Logo Certification Logo

Powered by: newtelligence dasBlog 2.3.9074.18820