Building a Better User Experience http://www.mindfusioncorp.com/weblog/ A weblog authored by Keith Rome, with a focus on the User Experience. http://www.mindfusioncorp.com/images/MainLogo.jpg Building a Better User Experience http://www.mindfusioncorp.com/weblog/ en-us Keith Rome Thu, 19 Jun 2008 18:17:48 GMT newtelligence dasBlog 2.0.7226.0 krome@wintellect.com krome@wintellect.com http://www.mindfusioncorp.com/weblog/Trackback.aspx?guid=a207c534-3e58-4daf-8de0-a203f31ad191 http://www.mindfusioncorp.com/weblog/pingback.aspx http://www.mindfusioncorp.com/weblog/PermaLink,guid,a207c534-3e58-4daf-8de0-a203f31ad191.aspx Keith Rome http://www.mindfusioncorp.com/weblog/CommentView,guid,a207c534-3e58-4daf-8de0-a203f31ad191.aspx http://www.mindfusioncorp.com/weblog/SyndicationService.asmx/GetEntryCommentsRss?guid=a207c534-3e58-4daf-8de0-a203f31ad191

I ran across SoundSnap today – what a great resource for finding free sound effects and music loops. For those participating in the GGMUG Silverlight Game Development Contest, this might come in handy for you, so please check it out!

http://www.soundsnap.com/

By the way, the GGMUG is a great venue to meet new people who are passionate about .NET development. If you live on the East side of town, and just can’t make it to those Alpharetta meetings held by the other user groups, then please check this new group out if you haven’t already.

Excellent Free Sound Effects Resource http://www.mindfusioncorp.com/weblog/PermaLink,guid,a207c534-3e58-4daf-8de0-a203f31ad191.aspx http://www.mindfusioncorp.com/weblog/2008/06/19/Excellent+Free+Sound+Effects+Resource.aspx Thu, 19 Jun 2008 18:17:48 GMT <p> I ran across <a href="http://www.soundsnap.com/">SoundSnap</a> today – what a great resource for finding free sound effects and music loops. For those participating in the <a href="http://www.ggmug.com/">GGMUG</a> Silverlight Game Development Contest, this might come in handy for you, so please check it out! </p> <p> <a title="http://www.soundsnap.com/" href="http://www.soundsnap.com/">http://www.soundsnap.com/</a> </p> <p> By the way, the GGMUG is a great venue to meet new people who are passionate about .NET development. If you live on the East side of town, and just can’t make it to those Alpharetta meetings held by the other user groups, then please check this new group out if you haven’t already. </p> <img width="0" height="0" src="http://www.mindfusioncorp.com/weblog/aggbug.ashx?id=a207c534-3e58-4daf-8de0-a203f31ad191" /> http://www.mindfusioncorp.com/weblog/CommentView,guid,a207c534-3e58-4daf-8de0-a203f31ad191.aspx Games General Media Silverlight http://www.mindfusioncorp.com/weblog/Trackback.aspx?guid=ce802397-894c-4772-994b-5d5ba4df3fda http://www.mindfusioncorp.com/weblog/pingback.aspx http://www.mindfusioncorp.com/weblog/PermaLink,guid,ce802397-894c-4772-994b-5d5ba4df3fda.aspx Keith Rome http://www.mindfusioncorp.com/weblog/CommentView,guid,ce802397-894c-4772-994b-5d5ba4df3fda.aspx http://www.mindfusioncorp.com/weblog/SyndicationService.asmx/GetEntryCommentsRss?guid=ce802397-894c-4772-994b-5d5ba4df3fda 7

Audio support is all too often one of the last things a developer thinks of when it comes to building a user interface. Fortunately, Silverlight offers some basic building blocks that make it fairly trivial to add event-driven sounds to our applications. I have taken the built-in functionality one step further, and created a simple run-time audio engine that makes it even easier.

First, I will discuss the requirements that I had in mind when I created this engine. Second, I will discuss the ideal API for a complete solution. Third, I submit the simple effects engine that I developed in order to meet those requirements and API.

Sound Effects Engine Requirements

  • Must be able to play looping “background music” tracks.
  • Must be able to play smaller sound effects on demand.
  • Must support concurrent (and overlapping) sounds effects without interfering with already-playing audio.
  • Must adequately handle the possibility of download delays when pulling down external audio files.
  • Should cache audio files between playbacks.
  • Should allow serving of audio from high-bandwidth CDN such as Silverlight Streaming.

An Ideal Sound Effects API

  • Should be very approachable. If possible, only a single line of code to play any audio (similar to PlaySound() API).
  • Should support any formats supported by Silverlight.
  • Should be efficient.
  • Should offer download and playback completion callbacks (for chaining and synchronization).
  • Should support aggressive background pre-caching of audio files.

The SilverlightToolbox Solution

To address the simplicity requirements, this sound effects engine is implemented as a single static C# class that can be easily included into any Silverlight project. It can exist in a referenced class library, or can be linked directly into the main project.

To address looping background music, a single method SetBackgroundLoop() is supplied. This will begin playback of a single audio clip, and will automatically repeat it. Only one track can be played as the background loop – calling this method again will terminate the running track and start the new one. You can also pass String.Empty to stop all background music.

To address typical sound effect clips. two overloads of the PlaySoundEffect() method are supplied. The simpler of the two methods will simply play a clip once it has been downloaded. The extended version allows you to specify a callback for when the clip has been downloaded and started playing, and a second callback that can be used for notification that the clip has ended.

To address the scenario where multiple sound effects might be played at the same time, the engine internally maintains a queue of MediaElements. When a new sound needs to played, it pulls an unused element and puts it into service. Once the sound is finished, the element is returned to the queue. This allows Silverlight itself to handle media mixing, and by caching those MediaElements, it does so without placing undue stress on the environment.

To address the issue of download delays, the startedCallback and completedCallback parameters of PlaySoundEffect() were introduced.

To address caching of audio files, a WebClient is used (which in turn leverages the browser’s cache). Furthermore, the engine will reuse any downloaded file streams (and is smart enough to not re-queue the same file if it already queued for download).

By using a MediaElement for playback, audio tracks are automatically supported from streaming sources and Content Delivery Networks, and can be of any type supported by Silverlight.

To support chaining of audio effects, and to support synchronization of UI with audio, the PlaySoundEffect() method accepts callback events that are fired when the clip has been downloaded, and again when it has completed playback. To further help with UI synchronization, a second utility class is provided (DelayedAction) which provides a simple wrapper for BackgroundWorker that can be used to easily delay execution of a block of code after a specified amount of time.

To support pre-caching of audio files, the PreloadMedia() method can be used.

Typical usage:

In order for the sound effects engine to function properly, it must be provided with a top-level XAML container (MediaElement currently will not perform playback without a parent object). This is done by calling the Initialize() method, typically from your program’s startup code: 

SoundEffects.Initialize(this.LayoutRoot);

 

After this has been done, background music can be played, and we can also queue up some sound effect clips for later: 

SoundEffects.SetBackgroundLoop("cautious-path.wma");
SoundEffects.PreloadMedia("pop1.wma");

 

Then, at various points throughout the application, we can play the sound effect (for example, in response to a control event): 

SoundEffects.PlaySoundEffect("pop1.wma");

 

If at any point we need to delay the sound effect slightly, we can control this be introducing a short delay: 

DelayedAction.Execute(2.0, () => SoundEffects.PlaySoundEffect("pop1.wma"));

 

Complete Source Code for DelayedAction.cs: 

               1:
              using System;
               2:
              using System.ComponentModel;
               3:
              using System.Threading;
               4:  
               5:
              namespace Wintellect.SilverlightToolbox
               6: {
               7:
              public
              class DelayedAction
               8: {
               9:
              class DelayedCallback
               10: {
               11:
              public TimeSpan
Delay { get; set; }
               12:
              public Action
Callback { get; set; }
               13: }
               14:  
               15:
              public
              static
              void Execute(double seconds,
Action callback)
               16: {
               17: BackgroundWorker
Delay = new BackgroundWorker();
               18: Delay.DoWork
+= (s, e) =>
               19: {
               20: DelayedCallback
DelayCallback = (DelayedCallback)e.Argument;
               21: Thread.Sleep(DelayCallback.Delay);
               22: e.Result
= DelayCallback.Callback;
               23: };
               24: Delay.RunWorkerCompleted
+= (s, e) =>
               25: {
               26: Action
Callback = e.Result as Action;
               27: Callback();
               28: };
               29: Delay.RunWorkerAsync(new DelayedCallback
               30: {
               31: Delay
= TimeSpan.FromSeconds(seconds),
               32: Callback
= callback
               33: });
               34: }
               35: }
               36: }

 

Complete Source Code for SoundEffects.cs: 

               1:
              using System;
               2:
              using System.Collections.Generic;
               3:
              using System.IO;
               4:
              using System.Net;
               5:
              using System.Windows;
               6:
              using System.Windows.Controls;
               7:
              using System.Windows.Media;
               8:  
               9:
              namespace Wintellect.SilverlightToolbox
               10: {
               11:
              public
              static
              class SoundEffects
               12: {
               13:
              static Panel
Root;
               14:
              static MediaElement
BackgroundLoop = new MediaElement();
               15:
              static WebClient
EffectDownloader = new WebClient();
               16:  
               17:
              static Queue<MediaElement>
AvailableSoundEffectGenerators = new Queue<MediaElement>();
               18:
              static Dictionary<string,
Stream> DownloadedEffects = new Dictionary<string,
Stream>();
               19:
              static Queue<string>
PendingDownloads = new Queue<string>();
               20:
              static Queue<QueuedEffect>
PendingEffects = new Queue<QueuedEffect>();
               21:
              static Dictionary<MediaElement,
Action> PendingStartupCallbacks = new Dictionary<MediaElement,
Action>();
               22:
              static Dictionary<MediaElement,
Action> PendingCompletionCallbacks = new Dictionary<MediaElement,
Action>();
               23:  
               24:
              enum TargetType
               25: {
               26: BackgroundMusic,
               27: SoundEffect,
               28: }
               29:  
               30:
              class QueuedEffect
               31: {
               32:
              public
              string MediaName
{ get; set; }
               33:
              public TargetType
Target { get; set; }
               34:
              public Action
StartedCallback { get; set; }
               35:
              public Action
CompletedCallback { get; set; }
               36: }
               37:  
               38:
              public
              static
              void Initialize(Panel
root)
               39: {
               40: Root
= root;
               41: InitializeTarget(root,
BackgroundLoop);
               42: EffectDownloader.OpenReadCompleted
+= (s, e) =>
               43: {
               44: DownloadedEffects[(string)e.UserState]
= e.Result;
               45: DownloadEffects();
               46: PlayEffect();
               47: };
               48: }
               49:  
               50:
              static
              void DownloadEffects()
               51: {
               52:
              if (PendingDownloads.Count
== 0)
               53:
              return;
               54:
              string MediaName
= PendingDownloads.Dequeue();
               55: EffectDownloader.OpenReadAsync(new Uri(MediaName,
UriKind.Relative), MediaName);
               56: }
               57:  
               58:
              static
              void InitializeTarget(Panel
root, MediaElement target)
               59: {
               60: target.Width
= 0;
               61: target.Height
= 0;
               62: target.Visibility
= Visibility.Collapsed;
               63: root.Children.Add(target);
               64: target.AutoPlay
= false;
               65: target.MediaOpened
+= (s, e) =>
               66: {
               67: MediaElement
Target = s as MediaElement;
               68: Target.Volume
= 0.35;
               69: Target.Play();
               70:
              if (PendingStartupCallbacks.ContainsKey(Target))
               71: {
               72: Target.Dispatcher.BeginInvoke(PendingStartupCallbacks[Target]);
               73: PendingStartupCallbacks.Remove(Target);
               74: }
               75: };
               76: target.MediaEnded
+= (s, e) =>
               77: {
               78: MediaElement
Target = s as MediaElement;
               79: Target.Stop();
               80:
              if (s
== BackgroundLoop)
               81: Target.Play();
               82:
              else
            
               83: AvailableSoundEffectGenerators.Enqueue(Target);
               84:  
               85:
              if (PendingCompletionCallbacks.ContainsKey(Target))
               86: {
               87: Target.Dispatcher.BeginInvoke(PendingCompletionCallbacks[Target]);
               88: PendingCompletionCallbacks.Remove(Target);
               89: }
               90: };
               91: }
               92:  
               93:
              static MediaElement
GetUnusedEffectGenerator()
               94: {
               95:
              if (AvailableSoundEffectGenerators.Count
> 0)
               96:
              return AvailableSoundEffectGenerators.Dequeue();
               97:
              else
            
               98: {
               99: MediaElement
Result = new MediaElement();
               100: InitializeTarget(Root,
Result);
               101:
              return Result;
               102: }
               103: }
               104:  
               105:
              static
              void PlayEffect()
               106: {
               107:
              lock (PendingEffects)
               108: {
               109:
              if (PendingEffects.Count
== 0)
               110:
              return;
               111: QueuedEffect
Effect = PendingEffects.Dequeue();
               112:
              if (DownloadedEffects.ContainsKey(Effect.MediaName))
               113: {
               114: MediaElement
TargetElement = null;
               115:
              switch (Effect.Target)
               116: {
               117:
              case TargetType.BackgroundMusic:
               118: {
TargetElement = BackgroundLoop; break; }
               119:
              case TargetType.SoundEffect:
               120: {
TargetElement = GetUnusedEffectGenerator(); break;
}
               121: }
               122:
              if (Effect.StartedCallback
!= null)
               123: PendingStartupCallbacks.Add(TargetElement,
Effect.StartedCallback);
               124:
              if (Effect.CompletedCallback
!= null)
               125: PendingCompletionCallbacks.Add(TargetElement,
Effect.CompletedCallback);
               126: TargetElement.SetSource(DownloadedEffects[Effect.MediaName]);
               127: }
               128:
              else
            
               129: {
               130: PendingEffects.Enqueue(Effect);
               131: }
               132: }
               133: }
               134:  
               135:
              static
              void PlaySound(TargetType
target, string mediaName, Action startedCallback,
Action completedCallback)
               136: {
               137:
              if (target
== TargetType.BackgroundMusic)
               138:
              if (BackgroundLoop.CurrentState
!= MediaElementState.Stopped)
               139: BackgroundLoop.Stop();
               140:  
               141:
              if (mediaName
!= String.Empty)
               142: {
               143:
              lock (PendingEffects)
               144: {
               145:
              if (!DownloadedEffects.ContainsKey(mediaName))
               146: {
               147: PendingDownloads.Enqueue(mediaName);
               148: }
               149: PendingEffects.Enqueue(new QueuedEffect
               150: {
               151: MediaName
= mediaName,
               152: Target
= target,
               153: StartedCallback
= startedCallback,
               154: CompletedCallback
= completedCallback
               155: });
               156: }
               157: DownloadEffects();
               158: PlayEffect();
               159: }
               160: }
               161:  
               162:
              public
              static
              void SetBackgroundLoop(string mediaName)
               163: {
               164: PlaySound(TargetType.BackgroundMusic,
mediaName, null, null);
               165: }
               166:  
               167:
              public
              static
              void PlaySoundEffect(string effectName)
               168: {
               169: PlaySound(TargetType.SoundEffect,
effectName, null, null);
               170: }
               171:  
               172:
              public
              static
              void PlaySoundEffect(string effectName,
Action startedCallback, Action completedCallback)
               173: {
               174: PlaySound(TargetType.SoundEffect,
effectName, startedCallback, completedCallback);
               175: }
               176:  
               177:
              public
              static
              void PreloadMedia(string mediaName)
               178: {
               179:
              if (!DownloadedEffects.ContainsKey(mediaName))
               180: {
               181: PendingDownloads.Enqueue(mediaName);
               182: DownloadEffects();
               183: }
               184: }
               185: }
               186: }
A Simple Sound Effects Engine for Silverlight 2 http://www.mindfusioncorp.com/weblog/PermaLink,guid,ce802397-894c-4772-994b-5d5ba4df3fda.aspx http://www.mindfusioncorp.com/weblog/2008/06/09/A+Simple+Sound+Effects+Engine+For+Silverlight+2.aspx Mon, 09 Jun 2008 21:59:22 GMT <p> Audio support is all too often one of the last things a developer thinks of when it comes to building a user interface. Fortunately, Silverlight offers some basic building blocks that make it fairly trivial to add event-driven sounds to our applications. I have taken the built-in functionality one step further, and created a simple run-time audio engine that makes it even easier. </p> <p> First, I will discuss the requirements that I had in mind when I created this engine. Second, I will discuss the ideal API for a complete solution. Third, I submit the simple effects engine that I developed in order to meet those requirements and API. </p> <p> <strong>Sound Effects Engine Requirements</strong> </p> <ul> <li> Must be able to play looping “background music” tracks. </li> <li> Must be able to play smaller sound effects on demand. </li> <li> Must support concurrent (and overlapping) sounds effects without interfering with already-playing audio. </li> <li> Must adequately handle the possibility of download delays when pulling down external audio files. </li> <li> Should cache audio files between playbacks. </li> <li> Should allow serving of audio from high-bandwidth CDN such as Silverlight Streaming. </li> </ul> <p> <strong>An Ideal Sound Effects API</strong> </p> <ul> <li> Should be very approachable. If possible, only a single line of code to play any audio (similar to PlaySound() API). </li> <li> Should support any formats supported by Silverlight. </li> <li> Should be efficient. </li> <li> Should offer download and playback completion callbacks (for chaining and synchronization). </li> <li> Should support aggressive background pre-caching of audio files. </li> </ul> <p> <strong>The SilverlightToolbox Solution</strong> </p> <p> To address the simplicity requirements, this sound effects engine is implemented as a single static C# class that can be easily included into any Silverlight project. It can exist in a referenced class library, or can be linked directly into the main project. </p> <p> To address looping background music, a single method SetBackgroundLoop() is supplied. This will begin playback of a single audio clip, and will automatically repeat it. Only one track can be played as the background loop – calling this method again will terminate the running track and start the new one. You can also pass String.Empty to stop all background music. </p> <p> To address typical sound effect clips. two overloads of the PlaySoundEffect() method are supplied. The simpler of the two methods will simply play a clip once it has been downloaded. The extended version allows you to specify a callback for when the clip has been downloaded and started playing, and a second callback that can be used for notification that the clip has ended. </p> <p> To address the scenario where multiple sound effects might be played at the same time, the engine internally maintains a queue of MediaElements. When a new sound needs to played, it pulls an unused element and puts it into service. Once the sound is finished, the element is returned to the queue. This allows Silverlight itself to handle media mixing, and by caching those MediaElements, it does so without placing undue stress on the environment. </p> <p> To address the issue of download delays, the startedCallback and completedCallback parameters of PlaySoundEffect() were introduced. </p> <p> To address caching of audio files, a WebClient is used (which in turn leverages the browser’s cache). Furthermore, the engine will reuse any downloaded file streams (and is smart enough to not re-queue the same file if it already queued for download). </p> <p> By using a MediaElement for playback, audio tracks are automatically supported from streaming sources and Content Delivery Networks, and can be of any type supported by Silverlight. </p> <p> To support chaining of audio effects, and to support synchronization of UI with audio, the PlaySoundEffect() method accepts callback events that are fired when the clip has been downloaded, and again when it has completed playback. To further help with UI synchronization, a second utility class is provided (DelayedAction) which provides a simple wrapper for BackgroundWorker that can be used to easily delay execution of a block of code after a specified amount of time. </p> <p> To support p