Gotcha: MarkerReached event of MediaElement returns new Markers

I just checked in the implementation code for a new feature for ClipFlair Studio’s Captions/Revoicing component:

When playing back recorded (or loaded from a WAV or MP3 file) audio for a caption/revoicing entry, the playback is now limited to the duration of the respective caption, (End-Start) time that is (btw that component has a duration column too that is hidden by default and can be shown by flipping it with the gear button on its titlebar and selecting the respective option to show the column).

The original audio is not affected and is stored in whole inside the saved state of the component/activity, so that you can adjust the caption entries timerange at any time to fit all or part of that recorded audio entry if you wish.

Will see into adding an “Limit playback” option to the backpanel of that component (it will default to true/checked) for any ClipFlair activities that don’t use the Start/End/Duration columns (e.g. if some activity just wants a grid of Captions and Audio entries for practicing and maybe for comparing to audio samples provided by the activity author or teacher at the optional “Comments (Audio)” column).

While implementing this feature, there were some “gotchas” that caused me some headache to spot:

1) When MediaOpened event is called by the MediaElement control, the Markers collection has just been reset and you need at that point to add your TimelineMediaMarker that will notify you when the playback limit point has been reached to stop the playback. In the case above this event is called after recording some audio or loading some WAV or MP3 audio file at an AudioRecorderControl (one is used at each row of the captions grid)

2) One shouldn’t remove an added marker at MediaEnded or MediaFailed events. This is since those will fire at each revoicing entry playback, whereas the MediaOpened while only occur once when the Audio property is populated at the AudioRecorderControl. As I mention above, MediaElement clears the Markers collection every time new content is loaded to it, so we need not worry about removing the marker we had added before.

3) Maybe the least obvious issue and the one that caused me most of headache to spot was that the MediaElement’s MarkerReached event gets back a MediaMarker that isn’t the same Marker instance as the one you had added to the Markers collection. So you have to use the Text property of the marker when you create it and then compare with the text from the one you got in the event to see if they are equal strings (btw, when C# compares strings it does it by content even if you use == instead of Equals method, unlike Java, where you shouldn’t use == to compare strings)

Gotcha: MediaElement Source=null releases stream, SetSource(null) fails

This is my contribution to:

http://stackoverflow.com/questions/19294258/forcing-mediaelement-to-release-stream-after-playback/27436323

If you use MediaElement, make sure you don’t get bitten by this one: http://msdn.microsoft.com/en-us/library/cc626563(v=vs.95).aspx

ArgumentNullException – The mediaStreamSource is null.
…

After calling this method, MediaElement.Source returns null. If this is called and MediaElement.Source is set, the last operation wins.

If a MediaElement is removed from the UI tree while is has an opened MediaStreamSource, subsequent calls to SetSource may be ignored. To ensure featureSetSource calls will work, set the Source property to null before detaching the MediaElement from the UI tree.

naturally one would expect, if they only use SetSource(somestream) to use SetSource(null) to release the resources. Nope, they thought "better", you have to use Source=null instead to release resources and SetSource(null) throws ArgumentNullException

that is what I call a design bug (breaks the rule of "least expected" behavior and causes bugs that bite you at runtime only [unless somebody has made a static analysis rule to catch such a thing – would need metadata of course that some argument can’t be null, like in Code Contracts])

I managed to introduce this bug while refactoring some code in ClipFlair Studio‘s AudioRecorder control the other day 😦

Note that you can’t use at MediaElement something like Source = stream to open a Stream, since that is a Uri property (not an Object property to also accept Stream) and you have to use SetSource(stream) instead, so you’d also expect to be able to use SetSource(null) to release the resources.

Update: Fixed this in AudioRecorderView class (uses MVVM pattern) of AudioRecorderControl, at Audio property’s "set" accessor it needed the following null-guarding pattern:

if (mediaStreamSource != null) 

  player.SetSource(mediaStreamSource); 
      //must set the source once, not every time we play the same audio, 
      //else with Mp3MediaSource it will throw DRM error

else

   player.Source = null;

Fix: Allow rewind of Mp3MediaStreamSource back to start

My comment at:

https://github.com/loarabia/ManagedMediaHelpers/issues/16

While trying to use Mp3MediaSource at ClipFlair Studio’s AudioRecorder control (http://clipflair.codeplex.com), I noticed that when I was doing Stop() at MediaElement and then Play() it kept on playing from where it was before at Mp3MediaStreamSource

So I did the following fix:

 

1) added these fields

    /// <summary>
    /// The first known frame (for rewind)
    /// </summary>     
    private MpegFrame firstFrame;

    /// <summary>
    /// The first audio stream position (for rewind)
    /// </summary>
    private long firstAudioStreamPosition;

 

2) added to the end of ReadPastId3v2TagsCallback:

this.firstFrame = mpegLayer3Frame; //keeping the 1st frame position for rewinding
this.firstAudioStreamPosition = audioStream.Position;

 

3) changed SeekAsync:

    /// <summary>
    /// <para>
    /// Only supporting seeking back to start of the stream 
    ///                              (e.g. when doing MediaElement.Stop()).
    /// </para>
    /// <para>
    /// In a fuller MediaStreamSource, the logic here would be to actually seek to
    /// the correct mpeg frame matching the seekToTime passed in.
    /// </para>
    /// </summary>
    /// <param name="seekToTime">
    /// The time to seek to (in 100-nanosecond units [hns])
    /// </param>
    protected override void SeekAsync(long seekToTime)
    {
        /*
        if (seekToTime > this.trackDuration.TotalMilliseconds * 10)
        {
          throw new InvalidOperationException(
                      "The seek position is beyond the length of the stream");
        }
        */
        if (seekToTime != 0) //only supporting rewinding back to start
            throw new InvalidOperationException(
                       "Only supporting seeking back to start of the stream");
        else
        {
            this.currentFrame = firstFrame;
            this.currentFrameStartPosition = MpegFrame.FrameHeaderSize;
            this.audioStream.Position = firstAudioStreamPosition;
        }

        this.ReportSeekCompleted(seekToTime);
    }

 

note that I changed the documentation for that method to say that time passed to it is in 100 ns units, not in ns units (according to WaveMediaStreamSource code that I found, unless that one has it wrong)

Gotcha: MediaElement must be in visual tree for MediaOpened, MediaEnded to be fired

At ClipFlair’s AudioRecorderControl (used in Captions/Revoicing component of ClipFlair Studio), I use the following code to initialize a MediaElement to use for playback.

After a long time a found out that if the MediaElement is not in the visual tree (for example defined in XAML, or defined in code and then added to the visual tree), then it will not always fire MediaOpened and MediaEnded events, which are crucial if you want to have a two-state Play/Stop button (a ToggleButton).

 

public MediaElement Player
{   get { return player; }   
    set {     
      if (player != null) {       
        player.MediaOpened -= MediaElement_MediaOpened;
        player.MediaEnded -= MediaElement_MediaEnded;     
      }     
      
      player = value;


     
      if (player != null)  {       
        player.MediaOpened += MediaElement_MediaOpened;
        player.MediaEnded += MediaElement_MediaEnded;
 
       player.AutoPlay = false;
       player.PlaybackRate = 1.0;
       player.Balance = 0;
       Volume = DEFAULT_VOLUME;   
  } 
  }
}

 

protected void MediaElement_MediaOpened(object sender, RoutedEventArgs e)
{   
  try   {     
    //player.Position = TimeSpan.Zero;      
    player.Stop(); //this stops current playback (if any) and rewinds 
    player.Play();   
  }   catch (Exception ex)   {     
    PlayCommandUncheck(); //depress playback toggle button 
                          //don't talk to ToggleButton directly     
     Status = MSG_PLAY_FAILED + ex.Message;   
  }
}
 

protected void MediaElement_MediaEnded(object sender, RoutedEventArgs e)
{ 
  PlayCommandUncheck(); //depress play button to stop playback 
                        //don't talk to ToggleButton directly
}

Gotcha: MediaElement AutoPlay faster than doing Play at MediaOpened

Just added the following comment to: https://github.com/loarabia/ManagedMediaHelpers/issues/15

Managed Media Helpers contains the very useful Mp3MediaSource class for .NET / Silverlight / Windows Phone.

Added compile-time SWITCHES and respective code to Silverlight demo code for PRELOAD (into memory stream) and AUTOPLAY (this was a bit tricky, need to call Play at MediaOpened event, not right after setting the source to the MediaElement).

Also seems to be 2-3 sec slower than AutoPlay (!) irrespective of using PRELOAD. Maybe it is faster to set the source again every time you want to play and just keep AutoPlay=true

Update: had a bug at the following code, obviously you first must set the MediaOpened event handler, then set the Source to the MediaPlayer. Also, one could set that handler once at the InitializeComponent method.

 

//———————————————————————–
// <copyright file="Page.xaml.cs" company="Larry Olson">
// (c) Copyright Larry Olson.
// This source is subject to the Microsoft Public License (Ms-PL)
// See http://code.msdn.microsoft.com/ManagedMediaHelpers/Project/License.aspx
// All other rights reserved.
// </copyright>
//
// Edited by George Birbilis (http://zoomicon.com)
//———————————————————————–

#define AUTOPLAY
//#define PRELOAD

namespace Mp3MediaStreamSourceDemo
{
  using Media;
  using System;
  using System.IO;
  using System.Windows;
  using System.Windows.Controls;

    /// <summary>
    /// A Page of a Silvelight Application.
    /// </summary>
    public partial class Page : UserControl
    {
        /// <summary>
        /// Initializes a new instance of the Page class.
        /// </summary>
        public Page()
        {
            InitializeComponent();

            me.Volume = 1.0; //set max volume
           
            #if !AUTOPLAY
            me.AutoPlay = false;

             me.MediaOpened += (s, e) =>
            {
                //me.Position = TimeSpan.Zero;
                me.Stop(); //this stops current playback (if any) and rewinds back to start
                //me.PlaybackRate = 1.0;
                try
                {
                    me.Play();
                }
                catch (Exception ex)
                {
                    MessageBox.Show(ex.Message);
                }
            };  
            #endif
        }

        /// <summary>
        /// Event handler for the Button on the Page.
        /// </summary>
        /// <param name="sender">
        /// The button which was clicked.
        /// </param>
        /// <param name="e">
        /// The state when this event was generated.
        /// </param>
        private void OpenMedia(object sender, RoutedEventArgs e)
        {
            OpenFileDialog ofd = new OpenFileDialog();
            ofd.ShowDialog();

            Stream data = ofd.File.OpenRead();

            #if PRELOAD
            Stream m = new MemoryStream((int)ofd.File.Length);
            data.CopyTo(m);
            m.Position = 0;
            data = m;
            #endif

 

             Mp3MediaStreamSource mediaSource = new Mp3MediaStreamSource(data);     
            me.SetSource(mediaSource);
        }
    }
}