Archive

Posts Tagged ‘ClipFlair’

Gotcha: MarkerReached event of MediaElement returns new Markers

I just checked in the implementation code for a new feature for ClipFlair Studio’s Captions/Revoicing component:

When playing back recorded (or loaded from a WAV or MP3 file) audio for a caption/revoicing entry, the playback is now limited to the duration of the respective caption, (End-Start) time that is (btw that component has a duration column too that is hidden by default and can be shown by flipping it with the gear button on its titlebar and selecting the respective option to show the column).

The original audio is not affected and is stored in whole inside the saved state of the component/activity, so that you can adjust the caption entries timerange at any time to fit all or part of that recorded audio entry if you wish.

Will see into adding an “Limit playback” option to the backpanel of that component (it will default to true/checked) for any ClipFlair activities that don’t use the Start/End/Duration columns (e.g. if some activity just wants a grid of Captions and Audio entries for practicing and maybe for comparing to audio samples provided by the activity author or teacher at the optional “Comments (Audio)” column).

While implementing this feature, there were some “gotchas” that caused me some headache to spot:

1) When MediaOpened event is called by the MediaElement control, the Markers collection has just been reset and you need at that point to add your TimelineMediaMarker that will notify you when the playback limit point has been reached to stop the playback. In the case above this event is called after recording some audio or loading some WAV or MP3 audio file at an AudioRecorderControl (one is used at each row of the captions grid)

2) One shouldn’t remove an added marker at MediaEnded or MediaFailed events. This is since those will fire at each revoicing entry playback, whereas the MediaOpened while only occur once when the Audio property is populated at the AudioRecorderControl. As I mention above, MediaElement clears the Markers collection every time new content is loaded to it, so we need not worry about removing the marker we had added before.

3) Maybe the least obvious issue and the one that caused me most of headache to spot was that the MediaElement’s MarkerReached event gets back a MediaMarker that isn’t the same Marker instance as the one you had added to the Markers collection. So you have to use the Text property of the marker when you create it and then compare with the text from the one you got in the event to see if they are equal strings (btw, when C# compares strings it does it by content even if you use == instead of Equals method, unlike Java, where you shouldn’t use == to compare strings)

Fix: TortoiseHg not commiting files

It seems that sometimes TortoiseHg Mercurial (Hg) client on Windows fails to remove a 0-byte-sized file named “wlock” (a write lock to avoid simultaneous write to a local repository) in the “.hg” subfolder that holds a local repository.

This results in the Commit dialog never commiting changed/removed/added files and you have to press Close there and confirm cancelation of the commit action. After trying multiple times and even rebooting I realized that deleting that “wlock” file fixed the issue and could then open up TortoiseHg commit dialog again to retry the commit action (luckily it remembers the commit message from the last effort, you just have to select which files to commit again.

The default seems to be to commit only modified files, thus if files have been removed or added you have to select them. Note that selecting the 3-state checkbox at the top selects/deselects all files quickly.

Then you can proceed from the “Hg Workbench” dialog (available when you right click in Windows Explorer the folder name that contains the “.hg” subfolder) to sync the local changes with any external repository you’re using. Hg is a distributed version control, unlike other centralized ones, so you can commit even without a network connection available, since it is commiting to the local repository.

Suggestion: Making Intellisense in Visual Studio more Intelligent

Just sent this to the Visual Studio team via the “send a frown” button:

pressing CTRL+SPACE after "buttonDropDown." (see screenshot) seems to give me suggestions of members of buttonDropDown that aren’t a fit to the Rectangle object that I’m trying to assign to

I suppose you do so since I could eventually drill down into more properties of those objects

you could think of it as a tree though and from the current object’s (buttonDropDown) members give me as suggestions only those that satisfy condition X, where condition X would be defined recursively as "is a Rectangle or has members that satisfy X". Could maybe have some max depth in that case down to which you check and have an option at the end of the popup to show all members to pick from

another idea (easier to implement) is to show on top of the popup the members that are Rectangle and then the rest of the members separated visually, however I think the former suggestion is possible to implement and more powerful, plus could be combined with this later suggestion

 

image

Gotcha: MediaElement Source=null releases stream, SetSource(null) fails

This is my contribution to:

http://stackoverflow.com/questions/19294258/forcing-mediaelement-to-release-stream-after-playback/27436323

If you use MediaElement, make sure you don’t get bitten by this one: http://msdn.microsoft.com/en-us/library/cc626563(v=vs.95).aspx

ArgumentNullException – The mediaStreamSource is null.

After calling this method, MediaElement.Source returns null. If this is called and MediaElement.Source is set, the last operation wins.

If a MediaElement is removed from the UI tree while is has an opened MediaStreamSource, subsequent calls to SetSource may be ignored. To ensure featureSetSource calls will work, set the Source property to null before detaching the MediaElement from the UI tree.

naturally one would expect, if they only use SetSource(somestream) to use SetSource(null) to release the resources. Nope, they thought "better", you have to use Source=null instead to release resources and SetSource(null) throws ArgumentNullException

that is what I call a design bug (breaks the rule of "least expected" behavior and causes bugs that bite you at runtime only [unless somebody has made a static analysis rule to catch such a thing – would need metadata of course that some argument can’t be null, like in Code Contracts])

I managed to introduce this bug while refactoring some code in ClipFlair Studio‘s AudioRecorder control the other day 😦

Note that you can’t use at MediaElement something like Source = stream to open a Stream, since that is a Uri property (not an Object property to also accept Stream) and you have to use SetSource(stream) instead, so you’d also expect to be able to use SetSource(null) to release the resources.

Update: Fixed this in AudioRecorderView class (uses MVVM pattern) of AudioRecorderControl, at Audio property’s "set" accessor it needed the following null-guarding pattern:

if (mediaStreamSource != null) 

  player.SetSource(mediaStreamSource); 
      //must set the source once, not every time we play the same audio, 
      //else with Mp3MediaSource it will throw DRM error

else

   player.Source = null;

Fix: Allow rewind of Mp3MediaStreamSource back to start

My comment at:

https://github.com/loarabia/ManagedMediaHelpers/issues/16

While trying to use Mp3MediaSource at ClipFlair Studio’s AudioRecorder control (http://clipflair.codeplex.com), I noticed that when I was doing Stop() at MediaElement and then Play() it kept on playing from where it was before at Mp3MediaStreamSource

So I did the following fix:

 

1) added these fields

    /// <summary>
    /// The first known frame (for rewind)
    /// </summary>     
    private MpegFrame firstFrame;

    /// <summary>
    /// The first audio stream position (for rewind)
    /// </summary>
    private long firstAudioStreamPosition;

 

2) added to the end of ReadPastId3v2TagsCallback:

this.firstFrame = mpegLayer3Frame; //keeping the 1st frame position for rewinding
this.firstAudioStreamPosition = audioStream.Position;

 

3) changed SeekAsync:

    /// <summary>
    /// <para>
    /// Only supporting seeking back to start of the stream 
/// (e.g. when doing MediaElement.Stop()). /// </para> /// <para> /// In a fuller MediaStreamSource, the logic here would be to actually seek to /// the correct mpeg frame matching the seekToTime passed in. /// </para> /// </summary> /// <param name="seekToTime"> /// The time to seek to (in 100-nanosecond units [hns]) /// </param> protected override void SeekAsync(long seekToTime) { /* if (seekToTime > this.trackDuration.TotalMilliseconds * 10) { throw new InvalidOperationException(
"The seek position is beyond the length of the stream"); } */ if (seekToTime != 0) //only supporting rewinding back to start throw new InvalidOperationException(
"Only supporting seeking back to start of the stream"); else { this.currentFrame = firstFrame; this.currentFrameStartPosition = MpegFrame.FrameHeaderSize; this.audioStream.Position = firstAudioStreamPosition; } this.ReportSeekCompleted(seekToTime); }

 

note that I changed the documentation for that method to say that time passed to it is in 100 ns units, not in ns units (according to WaveMediaStreamSource code that I found, unless that one has it wrong)

Gotcha: MediaElement must be in visual tree for MediaOpened, MediaEnded to be fired

At ClipFlair’s AudioRecorderControl (used in Captions/Revoicing component of ClipFlair Studio), I use the following code to initialize a MediaElement to use for playback.

After a long time a found out that if the MediaElement is not in the visual tree (for example defined in XAML, or defined in code and then added to the visual tree), then it will not always fire MediaOpened and MediaEnded events, which are crucial if you want to have a two-state Play/Stop button (a ToggleButton).

 

public MediaElement Player
{   get { return player; }   
set {    
if (player != null) {      
player.MediaOpened -= MediaElement_MediaOpened;
player.MediaEnded -= MediaElement_MediaEnded;    
}    

player = value;


    
if (player != null)  {      
player.MediaOpened += MediaElement_MediaOpened;
        player.MediaEnded += MediaElement_MediaEnded;

       player.AutoPlay = false;
       player.PlaybackRate = 1.0;
       player.Balance = 0;
       Volume = DEFAULT_VOLUME;  
  }
  } }

 

protected void MediaElement_MediaOpened(object sender, RoutedEventArgs e)
{   
try   {    
//player.Position = TimeSpan.Zero;     
player.Stop(); //this stops current playback (if any) and rewinds
    player.Play();  
}   catch (Exception ex)   {    
PlayCommandUncheck(); //depress playback toggle button
//don't talk to ToggleButton directly    
Status = MSG_PLAY_FAILED + ex.Message;  
} }
 

protected void MediaElement_MediaEnded(object sender, RoutedEventArgs e)
{
  PlayCommandUncheck(); //depress play button to stop playback
//don't talk to ToggleButton directly }

Gotcha: MediaElement AutoPlay faster than doing Play at MediaOpened

Just added the following comment to: https://github.com/loarabia/ManagedMediaHelpers/issues/15

Managed Media Helpers contains the very useful Mp3MediaSource class for .NET / Silverlight / Windows Phone.

Added compile-time SWITCHES and respective code to Silverlight demo code for PRELOAD (into memory stream) and AUTOPLAY (this was a bit tricky, need to call Play at MediaOpened event, not right after setting the source to the MediaElement).

Also seems to be 2-3 sec slower than AutoPlay (!) irrespective of using PRELOAD. Maybe it is faster to set the source again every time you want to play and just keep AutoPlay=true

Update: had a bug at the following code, obviously you first must set the MediaOpened event handler, then set the Source to the MediaPlayer. Also, one could set that handler once at the InitializeComponent method.

 

//———————————————————————–
// <copyright file="Page.xaml.cs" company="Larry Olson">
// (c) Copyright Larry Olson.
// This source is subject to the Microsoft Public License (Ms-PL)
// See http://code.msdn.microsoft.com/ManagedMediaHelpers/Project/License.aspx
// All other rights reserved.
// </copyright>
//
// Edited by George Birbilis (http://zoomicon.com)
//———————————————————————–

#define AUTOPLAY
//#define PRELOAD

namespace Mp3MediaStreamSourceDemo
{
  using Media;
  using System;
  using System.IO;
  using System.Windows;
  using System.Windows.Controls;

    /// <summary>
    /// A Page of a Silvelight Application.
    /// </summary>
    public partial class Page : UserControl
    {
        /// <summary>
        /// Initializes a new instance of the Page class.
        /// </summary>
        public Page()
        {
            InitializeComponent();

            me.Volume = 1.0; //set max volume
           
            #if !AUTOPLAY
            me.AutoPlay = false;

             me.MediaOpened += (s, e) =>
            {
                //me.Position = TimeSpan.Zero;
                me.Stop(); //this stops current playback (if any) and rewinds back to start
                //me.PlaybackRate = 1.0;
                try
                {
                    me.Play();
                }
                catch (Exception ex)
                {
                    MessageBox.Show(ex.Message);
                }
            };  
            #endif
        }

        /// <summary>
        /// Event handler for the Button on the Page.
        /// </summary>
        /// <param name="sender">
        /// The button which was clicked.
        /// </param>
        /// <param name="e">
        /// The state when this event was generated.
        /// </param>
        private void OpenMedia(object sender, RoutedEventArgs e)
        {
            OpenFileDialog ofd = new OpenFileDialog();
            ofd.ShowDialog();

            Stream data = ofd.File.OpenRead();

            #if PRELOAD
            Stream m = new MemoryStream((int)ofd.File.Length);
            data.CopyTo(m);
            m.Position = 0;
            data = m;
            #endif

 

             Mp3MediaStreamSource mediaSource = new Mp3MediaStreamSource(data);     
            me.SetSource(mediaSource);
        }
    }
}

%d bloggers like this: