Monday, June 09, 2008

A Simple Sound Effects Engine for Silverlight 2

Audio support is all too often one of the last things a developer thinks of when it comes to building a user interface. Fortunately, Silverlight offers some basic building blocks that make it fairly trivial to add event-driven sounds to our applications. I have taken the built-in functionality one step further, and created a simple run-time audio engine that makes it even easier.

First, I will discuss the requirements that I had in mind when I created this engine. Second, I will discuss the ideal API for a complete solution. Third, I submit the simple effects engine that I developed in order to meet those requirements and API.

Sound Effects Engine Requirements

  • Must be able to play looping “background music” tracks.
  • Must be able to play smaller sound effects on demand.
  • Must support concurrent (and overlapping) sounds effects without interfering with already-playing audio.
  • Must adequately handle the possibility of download delays when pulling down external audio files.
  • Should cache audio files between playbacks.
  • Should allow serving of audio from high-bandwidth CDN such as Silverlight Streaming.

An Ideal Sound Effects API

  • Should be very approachable. If possible, only a single line of code to play any audio (similar to PlaySound() API).
  • Should support any formats supported by Silverlight.
  • Should be efficient.
  • Should offer download and playback completion callbacks (for chaining and synchronization).
  • Should support aggressive background pre-caching of audio files.

The SilverlightToolbox Solution

To address the simplicity requirements, this sound effects engine is implemented as a single static C# class that can be easily included into any Silverlight project. It can exist in a referenced class library, or can be linked directly into the main project.

To address looping background music, a single method SetBackgroundLoop() is supplied. This will begin playback of a single audio clip, and will automatically repeat it. Only one track can be played as the background loop – calling this method again will terminate the running track and start the new one. You can also pass String.Empty to stop all background music.

To address typical sound effect clips. two overloads of the PlaySoundEffect() method are supplied. The simpler of the two methods will simply play a clip once it has been downloaded. The extended version allows you to specify a callback for when the clip has been downloaded and started playing, and a second callback that can be used for notification that the clip has ended.

To address the scenario where multiple sound effects might be played at the same time, the engine internally maintains a queue of MediaElements. When a new sound needs to played, it pulls an unused element and puts it into service. Once the sound is finished, the element is returned to the queue. This allows Silverlight itself to handle media mixing, and by caching those MediaElements, it does so without placing undue stress on the environment.

To address the issue of download delays, the startedCallback and completedCallback parameters of PlaySoundEffect() were introduced.

To address caching of audio files, a WebClient is used (which in turn leverages the browser’s cache). Furthermore, the engine will reuse any downloaded file streams (and is smart enough to not re-queue the same file if it already queued for download).

By using a MediaElement for playback, audio tracks are automatically supported from streaming sources and Content Delivery Networks, and can be of any type supported by Silverlight.

To support chaining of audio effects, and to support synchronization of UI with audio, the PlaySoundEffect() method accepts callback events that are fired when the clip has been downloaded, and again when it has completed playback. To further help with UI synchronization, a second utility class is provided (DelayedAction) which provides a simple wrapper for BackgroundWorker that can be used to easily delay execution of a block of code after a specified amount of time.

To support pre-caching of audio files, the PreloadMedia() method can be used.

Typical usage:

In order for the sound effects engine to function properly, it must be provided with a top-level XAML container (MediaElement currently will not perform playback without a parent object). This is done by calling the Initialize() method, typically from your program’s startup code:

SoundEffects.Initialize(this.LayoutRoot);

 

After this has been done, background music can be played, and we can also queue up some sound effect clips for later:

SoundEffects.SetBackgroundLoop("cautious-path.wma");
SoundEffects.PreloadMedia("pop1.wma");

 

Then, at various points throughout the application, we can play the sound effect (for example, in response to a control event):

SoundEffects.PlaySoundEffect("pop1.wma");

 

If at any point we need to delay the sound effect slightly, we can control this be introducing a short delay:

DelayedAction.Execute(2.0, () => SoundEffects.PlaySoundEffect("pop1.wma"));

 

Complete Source Code for DelayedAction.cs:

   1: using System;
   2: using System.ComponentModel;
   3: using System.Threading;
   4:  
   5: namespace Wintellect.SilverlightToolbox
   6: {
   7:     public class DelayedAction
   8:     {
   9:         class DelayedCallback
  10:         {
  11:             public TimeSpan Delay { get; set; }
  12:             public Action Callback { get; set; }
  13:         }
  14:  
  15:         public static void Execute(double seconds, Action callback)
  16:         {
  17:             BackgroundWorker Delay = new BackgroundWorker();
  18:             Delay.DoWork += (s, e) =>
  19:             {
  20:                 DelayedCallback DelayCallback = (DelayedCallback)e.Argument;
  21:                 Thread.Sleep(DelayCallback.Delay);
  22:                 e.Result = DelayCallback.Callback;
  23:             };
  24:             Delay.RunWorkerCompleted += (s, e) =>
  25:             {
  26:                 Action Callback = e.Result as Action;
  27:                 Callback();
  28:             };
  29:             Delay.RunWorkerAsync(new DelayedCallback
  30:             {
  31:                 Delay = TimeSpan.FromSeconds(seconds),
  32:                 Callback = callback
  33:             });
  34:         }
  35:     }
  36: }

 

Complete Source Code for SoundEffects.cs:

   1: using System;
   2: using System.Collections.Generic;
   3: using System.IO;
   4: using System.Net;
   5: using System.Windows;
   6: using System.Windows.Controls;
   7: using System.Windows.Media;
   8:  
   9: namespace Wintellect.SilverlightToolbox
  10: {
  11:     public static class SoundEffects
  12:     {
  13:         static Panel Root;
  14:         static MediaElement BackgroundLoop = new MediaElement();
  15:         static WebClient EffectDownloader = new WebClient();
  16:  
  17:         static Queue<MediaElement> AvailableSoundEffectGenerators = new Queue<MediaElement>();
  18:         static Dictionary<string, Stream> DownloadedEffects = new Dictionary<string, Stream>();
  19:         static Queue<string> PendingDownloads = new Queue<string>();
  20:         static Queue<QueuedEffect> PendingEffects = new Queue<QueuedEffect>();
  21:         static Dictionary<MediaElement, Action> PendingStartupCallbacks = new Dictionary<MediaElement, Action>();
  22:         static Dictionary<MediaElement, Action> PendingCompletionCallbacks = new Dictionary<MediaElement, Action>();
  23:  
  24:         enum TargetType
  25:         {
  26:             BackgroundMusic,
  27:             SoundEffect,
  28:         }
  29:  
  30:         class QueuedEffect
  31:         {
  32:             public string MediaName { get; set; }
  33:             public TargetType Target { get; set; }
  34:             public Action StartedCallback { get; set; }
  35:             public Action CompletedCallback { get; set; }
  36:         }
  37:  
  38:         public static void Initialize(Panel root)
  39:         {
  40:             Root = root;
  41:             InitializeTarget(root, BackgroundLoop);
  42:             EffectDownloader.OpenReadCompleted += (s, e) =>
  43:             {
  44:                 DownloadedEffects[(string)e.UserState] = e.Result;
  45:                 DownloadEffects();
  46:                 PlayEffect();
  47:             };
  48:         }
  49:  
  50:         static void DownloadEffects()
  51:         {
  52:             if (PendingDownloads.Count == 0)
  53:                 return;
  54:             string MediaName = PendingDownloads.Dequeue();
  55:             EffectDownloader.OpenReadAsync(new Uri(MediaName, UriKind.Relative), MediaName);
  56:         }
  57:  
  58:         static void InitializeTarget(Panel root, MediaElement target)
  59:         {
  60:             target.Width = 0;
  61:             target.Height = 0;
  62:             target.Visibility = Visibility.Collapsed;
  63:             root.Children.Add(target);
  64:             target.AutoPlay = false;
  65:             target.MediaOpened += (s, e) =>
  66:             {
  67:                 MediaElement Target = s as MediaElement;
  68:                 Target.Volume = 0.35;
  69:                 Target.Play();
  70:                 if (PendingStartupCallbacks.ContainsKey(Target))
  71:                 {
  72:                     Target.Dispatcher.BeginInvoke(PendingStartupCallbacks[Target]);
  73:                     PendingStartupCallbacks.Remove(Target);
  74:                 }
  75:             };
  76:             target.MediaEnded += (s, e) =>
  77:             {
  78:                 MediaElement Target = s as MediaElement;
  79:                 Target.Stop();
  80:                 if (s == BackgroundLoop)
  81:                     Target.Play();
  82:                 else
  83:                     AvailableSoundEffectGenerators.Enqueue(Target);
  84:  
  85:                 if (PendingCompletionCallbacks.ContainsKey(Target))
  86:                 {
  87:                     Target.Dispatcher.BeginInvoke(PendingCompletionCallbacks[Target]);
  88:                     PendingCompletionCallbacks.Remove(Target);
  89:                 }
  90:             };
  91:         }
  92:  
  93:         static MediaElement GetUnusedEffectGenerator()
  94:         {
  95:             if (AvailableSoundEffectGenerators.Count > 0)
  96:                 return AvailableSoundEffectGenerators.Dequeue();
  97:             else
  98:             {
  99:                 MediaElement Result = new MediaElement();
 100:                 InitializeTarget(Root, Result);
 101:                 return Result;
 102:             }
 103:         }
 104:  
 105:         static void PlayEffect()
 106:         {
 107:             lock (PendingEffects)
 108:             {
 109:                 if (PendingEffects.Count == 0)
 110:                     return;
 111:                 QueuedEffect Effect = PendingEffects.Dequeue();
 112:                 if (DownloadedEffects.ContainsKey(Effect.MediaName))
 113:                 {
 114:                     MediaElement TargetElement = null;
 115:                     switch (Effect.Target)
 116:                     {
 117:                         case TargetType.BackgroundMusic:
 118:                             { TargetElement = BackgroundLoop; break; }
 119:                         case TargetType.SoundEffect:
 120:                             { TargetElement = GetUnusedEffectGenerator(); break; }
 121:                     }
 122:                     if (Effect.StartedCallback != null)
 123:                         PendingStartupCallbacks.Add(TargetElement, Effect.StartedCallback);
 124:                     if (Effect.CompletedCallback != null)
 125:                         PendingCompletionCallbacks.Add(TargetElement, Effect.CompletedCallback);
 126:                     TargetElement.SetSource(DownloadedEffects[Effect.MediaName]);
 127:                 }
 128:                 else
 129:                 {
 130:                     PendingEffects.Enqueue(Effect);
 131:                 }
 132:             }
 133:         }
 134:  
 135:         static void PlaySound(TargetType target, string mediaName, Action startedCallback, Action completedCallback)
 136:         {
 137:             if (target == TargetType.BackgroundMusic)
 138:                 if (BackgroundLoop.CurrentState != MediaElementState.Stopped)
 139:                     BackgroundLoop.Stop();
 140:  
 141:             if (mediaName != String.Empty)
 142:             {
 143:                 lock (PendingEffects)
 144:                 {
 145:                     if (!DownloadedEffects.ContainsKey(mediaName))
 146:                     {
 147:                         PendingDownloads.Enqueue(mediaName);
 148:                     }
 149:                     PendingEffects.Enqueue(new QueuedEffect
 150:                     {
 151:                         MediaName = mediaName,
 152:                         Target = target,
 153:                         StartedCallback = startedCallback,
 154:                         CompletedCallback = completedCallback
 155:                     });
 156:                 }
 157:                 DownloadEffects();
 158:                 PlayEffect();
 159:             }
 160:         }
 161:  
 162:         public static void SetBackgroundLoop(string mediaName)
 163:         {
 164:             PlaySound(TargetType.BackgroundMusic, mediaName, null, null);
 165:         }
 166:  
 167:         public static void PlaySoundEffect(string effectName)
 168:         {
 169:             PlaySound(TargetType.SoundEffect, effectName, null, null);
 170:         }
 171:  
 172:         public static void PlaySoundEffect(string effectName, Action startedCallback, Action completedCallback)
 173:         {
 174:             PlaySound(TargetType.SoundEffect, effectName, startedCallback, completedCallback);
 175:         }
 176:  
 177:         public static void PreloadMedia(string mediaName)
 178:         {
 179:             if (!DownloadedEffects.ContainsKey(mediaName))
 180:             {
 181:                 PendingDownloads.Enqueue(mediaName);
 182:                 DownloadEffects();
 183:             }
 184:         }
 185:     }
 186: }
.NET | Games | General | Media | Silverlight | Tips and Tricks
Monday, June 09, 2008 4:59:22 PM (Eastern Standard Time, UTC-05:00) #  Disclaimer | Comments [7] | 

 Sunday, June 01, 2008

Gem Blaster entered into "RIA Run" contest

As a follow-up to the previous post, I have entered Gem Blaster into a game development contest held by the