Fix: Silverlight Media Framework Player VolumeElement out of sync

I was just adding a storable (persistent) Volume property to the MediaPlayerView class used at ClipFlair’s MediaPlayerWindow (connected to the underlying SMF player’s VolumeLevel property), when I realized that after reloading saved state, the SMF player’s Volume control would show a different value than the value set to it (which I could confirm by flipping the MediaPlayer control and looking at its properties on the backpanel that the ClipFlair app features for each component and for the activity container itself).

It seems to be some bug in the SMF logic or in its default template, since the following fix makes the issue disappear. The trick is that at a descendent class from SMFPlayer (like the MediaPlayer class that ClipFlair’s MediaPlayerWindow uses), one can override the OnApplyTemplate method (a standard method in templated XAML-based controls at Silverlight and WPF) and call the UpdateVolumeElement method shown below.

That method temporarily sets the volume to another value than its current one (this is important, just setting again to the same value would be ignored), then set again the current value to force the VolumeElement control’s UI to update.

 

//Project: ClipFlair (http://ClipFlair.codeplex.com)
//Filename: MediaPlayer.cs
//Version: 20130211

using System;
using System.Linq;
using System.Windows;
using System.Windows.Media;

using Microsoft.SilverlightMediaFramework.Core;
using Microsoft.SilverlightMediaFramework.Core.Media;
using Microsoft.SilverlightMediaFramework.Core.Accessibility.Captions;
using Microsoft.SilverlightMediaFramework.Plugins.Primitives;

namespace ClipFlair.MediaPlayer
{

  public class MediaPlayer : SMFPlayer
  {

    public override void OnApplyTemplate()
    {
      base.OnApplyTemplate();

      //...
      UpdateVolumeElement(); //patch for SMF bug
    }

    protected void UpdateVolumeElement()
    {
      //patch for SMF to update VolumeElement UI with any already set VolumeLevel
      double volume = VolumeLevel;
      VolumeLevel = (volume == 1) ? 0.9 : 1;
      VolumeLevel = volume;
    }

//...

 

Update: While submitting this as a bug to SMF source site on Codeplex it came to me that I could try setting the VolumeElement’s VolumeLevel value directly at ApplyTemplate and indeed it works:

protected void UpdateVolumeElement() 
{
  //patch for SMF to update VolumeElement UI with any already set VolumeLevel 
  VolumeElement.VolumeLevel = VolumeLevel;
}

HowTo: Multi-row captions and wrapping at Silverlight Media Framework

One of the main goals of the ClipFlair project is to explore the use of Video Captioning (together with Revoicing) for Foreign Language Learning (FLL). There both are important enhancements compared to the simpler two-line subtitling that was put in use in its ancestor project LeViS.

Captioning means that multi-row text is needed while text wrapping is also useful, especially if cartoon-like balloons over the video area are to also be examined as an option. Implementing multi-row captions and caption wrapping was a bit tricky in the SMF based player, due to the lack of relevant documentation, so adding some relevant notes below.

As I discuss in more detail near the end of the discussiion at  http://smf.codeplex.com/discussions/245424, the way I finally managed to make SMF player show multi-row captions is by setting the caption region bounds (via Origin and Extend properties of the CaptionRegion object) to take up nearly all the video area and combined it with DisplayAlign = DisplayAlign.After (to bottom align the caption rows group) and Overflow = Overflow.Dynamic (to extend the visible caption area dynamically inside the defined bounds).

Regarding caption wrapping there seems to be a bug in SMF, it wraps caption text at the video boundary instead of at the caption region max boundary (as defined by Origin/Extend properties of the CaptionRegion object). Have reported it at:

http://smf.codeplex.com/workitem/23634

A workarround for caption wrapping to work perfectly is to set top to 0 for the origin and set extend width to 100% for the caption region, but this won’t be useful if you want to show cartoon-like bubbles on the video for some captions. If Microsoft don’t fix that in the near future I guess I’ll have to fix it myself at the source-code of SMF (luckicly it’s an open-source project).

The relevant code snippet  from http://ClipFlair.codeplex.com for the MediaPlayer component is:

private const double CAPTION_REGION_LEFT = 0; //0.05;
private const double CAPTION_REGION_TOP = 0.05;
private const double CAPTION_REGION_WIDTH = 1; //0.9; //SMF 2.7 has a bug here,
               //it wraps caption text at the video boundary instead of at the 
               //caption region max boundary (as defined by Origin and Extend)
private const double CAPTION_REGION_HEIGHT = 0.9;

public static void StyleCaptions(CaptionRegion theCaptions)
{
  if (theCaptions == null) return;

    theCaptions.Style.ShowBackground = ShowBackground.WhenActive; 
               //doesn't seem to work if other than transparent color is used
    theCaptions.Style.BackgroundColor = Colors.Transparent;

    //set caption region (max) bounds
    theCaptions.Style.Origin = new Origin() { 
      Left = new Length() { 
        Unit = LengthUnit.Percent, Value = CAPTION_REGION_LEFT }, 
      Top = new Length() {
        Unit = LengthUnit.Percent, Value = CAPTION_REGION_TOP } };
    theCaptions.Style.Extent = new Extent() { 
      Width = new Length() { 
        Unit = LengthUnit.Percent, Value = CAPTION_REGION_WIDTH }, 
      Height = new Length() { 
        Unit = LengthUnit.Percent, Value = CAPTION_REGION_HEIGHT } };

    //theCaptions.Style.Direction = Direction.LeftToRight;
  
    theCaptions.Style.DisplayAlign = DisplayAlign.After; 
                                 //align multirow catpions to bottom of region
    theCaptions.Style.TextAlign = TextAlignment.Justify; 
                                 //horizontally center captions

    theCaptions.Style.WrapOption = TextWrapping.Wrap; 
                                 //wrap too long captions to next row
    theCaptions.Style.Overflow = Overflow.Dynamic; 
         //extends the area for the captions as needed, up to the given Extent

    foreach (CaptionElement caption in theCaptions.Children)
      StyleCaption(caption);
}

public static void StyleCaption(TimedTextElement theCaption)
{
  if (theCaption == null) return;

  theCaption.CaptionElementType = TimedTextElementType.Text;
  theCaption.Style.ShowBackground = ShowBackground.WhenActive;
  theCaption.Style.BackgroundColor = Color.FromArgb(100, 0, 0, 0); 
                                     //use a semi-transparent background
  theCaption.Style.Color = Colors.White;

  Length length = new Length
 {
   Unit = LengthUnit.Pixel, //must use this, since the default LengthUnit.Cell
                           //used at TimedTextStyle constructor is not supported
   Value = 20
 };
  theCaption.Style.FontSize = length;
}

I also tried setting Padding property at the code above in case it would help with the correct wrapping, but it somehow results in crashing Silverlight (may be related to the fact that I have hardware acceleration enabled for our Silverlight app).

HowTo: Hide Video area of Silverlight Media Framework Player

Since our onoing project ClipFlair is a follow-up of the succesful project LeViS, during its development I’m also looking at covering possible use-cases I infer from feedback entries at LvS application issue tracker on Codeplex (note that ClipFlair is also opensourced on Codeplex at http://ClipFlair.codeplex.com).

One such issue (http://lvs.codeplex.com/workitem/11511) was titled “Allow video hiding while controller is visible for audio only”, so since Silverlight Media Framework (SMF) is used for the media player in ClipFlair, I looked into how to tell it to hide the video area.

Well, it seems that to hide the video and hear just the audio in SMFPlayer one needs to set MediaPresenterElement‘s (a property of SMFPlayer class) MaxWidth and MaxHeight to 0. One can do that at an overriden OnApplyTemplate method in a class descding from SMFPlayer, or could also do it at a custom SMF player template.

Note that setting MediaPresenterElement’s Visibility property to Visibility.Collapsed makes the player not load the video at all, so that one was not an option.

One could define a VideoVisible DependencyProperty at a class descending from SMFPlayer:

    #region VideoVisible

    /// <summary>
    /// VideoVisible Dependency Property
    /// </summary>
    public static readonly DependencyProperty VideoVisibleProperty =
        DependencyProperty.Register("VideoVisible", typeof(bool), 
            typeof(MediaPlayer),
            new FrameworkPropertyMetadata(false,
                FrameworkPropertyMetadataOptions.None,
                new PropertyChangedCallback(OnVideoVisibleChanged)));

    /// <summary>
    /// Gets or sets the VideoVisible property.
    /// </summary>
    public bool VideoVisible
    {
      get { return (bool)GetValue(VideoVisibleProperty); }
      set { SetValue(VideoVisibleProperty, value); }
    }

    /// <summary>
    /// Handles changes to the VideoVisible property.
    /// </summary>
    private static void OnVideoVisibleChanged(DependencyObject d, 
                                         DependencyPropertyChangedEventArgs e)
    {
      MediaPlayer target = (MediaPlayer)d;
      bool oldVideoVisible = (bool)e.OldValue;
      bool newVideoVisible = target.VideoVisible;
      target.OnVideoVisibleChanged(oldVideoVisible, newVideoVisible);
    }

    /// <summary>
    /// Provides derived classes an opportunity to handle changes to the 
    /// VideoVisible property.
    /// </summary>
    protected virtual void OnVideoVisibleChanged(bool oldVideoVisible, 
                                                 bool newVideoVisible)
    {
      MediaPresenterElement.MaxWidth = (newVideoVisible)? 
                                             double.PositiveInfinity : 0;
      MediaPresenterElement.MaxHeight = (newVideoVisible) ? 
                                             double.PositiveInfinity : 0;
    }

    #endregion

 

At first, to restore the MaxWidth/MaxHeight to show the video area again I tried to set them to double.NaN which didn’t work. Then I checked MaxWidth documentation which was saying:

The maximum width of the element, in device-independent units (1/96th inch per unit). The default value is PositiveInfinity. This value can be any value equal to or greater than 0.0. PositiveInfinity is also valid.

and

String representation of a Double value equal to or greater than 0.0. This is interpreted as a device-independent unit (1/96th inch) measurement. Strings need not explicitly include decimal points. For instance a value of 1 is acceptable.

The same Double range restrictions as mentioned in the Property Value section apply, except that you must use x:Static Markup Extension to set the value to be PositiveInfinity.

Per the 1st quote, one just needs to set MaxWidth or MaxHeight to double.PositiveInfinity (and not double.NaN that I originally expected) to reset it to its default value, that is behave as if a maximum width or height respectively has never been set.

As for the second quote, it basically says that XAML double fields need special treatment in order to specify a positive infinity value (if you ever need to, since you’d usually just skip the MaxWidth/MaxHeight field, unless you need to override some non-default inherited value). You would need to define a XAML namespace to point to the System namespace at mscorlib assembly (library), either at the root control in the XAML or directly where you need it:

<MyNameSpace:MyControl

      xmlns:sys="clr-namespace:System;assembly=mscorlib"

      MaxWidth=”{x:Static sys:Double.PositiveInfinity}”

      MaxHeight=”{x:Static sys:Double.PositiveInfinity}”

      …

/>

BTW, to hide the controller bar (called ControlStrip in SMF) you just set IsControlStripVisible property to false.

Collection of Smooth Streaming Video URLs

Here are some Smooth Streaming URLs I found on the Internet that you could use during development to test your smooth streaming players (like those based on SMF).

If you haven’t yet started building your SMF-based player you can try the following URLs here: http://www.smoothhd.com/livetestplayer/

Update: you can also try Smooth Stream URLs at ClipFlair Studio Silverlight-based application, turning over its Clip component (using the “Wrench” button on its titlebar) and pasting the Media URL there. Alternatively, you can launch it directly using a URL of the form http://studio.clipflair.net?video=http://smoothstreamer.doit.wisc.edu/doit-nms/BBB_carbon/BBB.ism/Manifest

http://smoothstreamer.doit.wisc.edu/doit-nms/BBB_carbon/BBB.ism/Manifest

http://streams.smooth.vertigo.com/BigBuckBunny_30sec/bigbuck.ism/manifest

http://streams.smooth.vertigo.com/elephantsdream/Elephants_Dream_1024-h264-st-aac.ism/manifest (this one also features multiple audio streams, director’s comments in english and also english and spanish dialogs)

http://playready.directtaps.net/smoothstreaming/TTLSS720VC1/To_The_Limit_720.ism/Manifest

http://playready.directtaps.net/smoothstreaming/SSWSS720H264/ SuperSpeedway_720.ism/Manifest

At the same test server one can also find two audio-only smooth streams:

http://playready.directtaps.net/smoothstreaming/ISMAAACLC/Taxi3_AACLC.ism/Manifest

http://playready.directtaps.net/smoothstreaming/ISMAAACHE/Taxi3_AACHE.ism/Manifest

http://mediadl.microsoft.com/mediadl/iisnet/smoothmedia/Experience/ BigBuckBunny_720p.ism/Manifest

http://ecn.channel9.msdn.com/o9/content/smf/smoothcontent/bbbwp7/big%20buck%20bunny.ism/manifest

http://ecn.channel9.msdn.com/o9/content/smf/smoothcontent/elephantsdream/ Elephants_Dream_1024-h264-st-aac.ism/manifest

http://video3.smoothhd.com.edgesuite.net/ondemand/Big%20Buck%20Bunny%20Adaptive.ism/Manifest

http://video3.smoothhd.com/ondemand/Turner_Sports_PGA.ism/Manifest

http://video3.smoothhd.com/ondemand/Turner_Sports_NASCAR.ism/Manifest

http://video3.smoothhd.com/ondemand/Turner_Sports_MLB.ism/Manifest

http://video3.smoothhd.com/ondemand/Akamai_ASP_Cutdown.ism/Manifest

http://video3.smoothhd.com/ondemand/mix1/mix1.ism/Manifest

http://video3.smoothhd.com/ondemand/mix2/mix2.ism/Manifest

http://video3.smoothhd.com/ondemand/Big%20Buck%20Bunny%20Adaptive.ism/Manifest

http://video3.smoothhd.com/ondemand/ElephantsDream.ism/Manifest

http://video3.smoothhd.com/ondemand/Got_Imagination_(Indo).ism/Manifest

http://video3.smoothhd.com/ondemand/Got_Imagination_(California).ism/Manifest

http://video3.smoothhd.com/ondemand/Coral_Reefs.ism/Manifest

http://video3.smoothhd.com/ondemand/eHow_Wakeboard.ism/Manifest

http://video3.smoothhd.com/ondemand/eHow_Baseball.ism/Manifest

http://video3.smoothhd.com/ondemand/eHow_Alligator.ism/Manifest

http://video3.smoothhd.com/ondemand/NBA.ism/Manifest

http://video3.smoothhd.com/ondemand/Changeling.ism/Manifest

http://video3.smoothhd.com/ondemand/Livestrong_ThyroidCancer.ism/Manifest

http://video3.smoothhd.com/ondemand/Livestrong_BeginnerGuideExercising.ism/Manifest

http://video3.smoothhd.com/ondemand/Livestrong_Autism.ism/Manifest

Also found some (probably) Bollywood movie:

http://az280594.vo.msecnd.net/athadu/athadu480.ism/Manifest

You can use the following search query to search for “.ism/Manifest”:
http://www.google.com/?q=%22http%22+%22.ism%2FManifest%22&oq=%22http%22+%22.ism%2FManifest%22

A good source found is http://www.smoothhd.com/content/smoothhd/smoothhd.xml

which seems to be from http://wwwns.akamai.com/hdnetwork/demo/silverlight/default.html

Some more URLs (use the unencrypted ones that write CLEAR) are found at:

http://playready.directtaps.net/smoothstreaming/

HowTo: load CaptionElements into Silverlight Media Framework player

Trying to make CaptionsGridWindow of ClipFlair serve captions editing on-the-fly to SMF (Silverlight Media Framework [now called MMPPF]) player component, I had a real hard time, plagued by a bug at TimedTextElementStyle. It seems to be setting default FontSize for captions using a “Cell” unit instead of using a “Pixel” unit. Currently SMF only supports “Pixel” units at FontSize of TimedTextElements according to the codedoc notes.

Another issue I had was that CaptionRegion constructor sets it to be active all the time (specifying a Begin value of a min possible TimeExtent and an End value of the max possible TimeExtent), so it renders ShowBackground.WhenActive setting useless for it. That is if you want the captions background to show up only when there are captions showing (active), then you have to set the CaptionRegion’s background to Colors.Transparent and set each CaptionElement’s background to some non-transparent color (note that both CaptionRegion and CaptionElement are TimedTextElements, with the later added to the former’s Children property, forming a TimedTree that is).

 

public void UpdateMarkers(MediaMarkerCollection<TimedTextElement> newMarkers)
 {
   if (newMarkers == null) return;

   CaptionRegion  region = new CaptionRegion();
   region.Style.ShowBackground = ShowBackground.WhenActive; 
             //doesn't seem to work if other than transparent color is used
   region.Style.BackgroundColor = Colors.Transparent;

   foreach (CaptionElement marker in newMarkers)
   {
     region.Children.Add(marker);
     marker.CaptionElementType = TimedTextElementType.Text;
     marker.Style.ShowBackground = ShowBackground.WhenActive;
     marker.Style.BackgroundColor = Color.FromArgb(100, 0, 0, 0); 
                                       //use a semi-transparent background
     marker.Style.Color = Colors.White;
     //marker.Style.TextAlign = TextAlignment.Center;
     Length length = new Length
     {
       Unit = LengthUnit.Pixel, 
       Value = 20
     }; //must use this, since the default LengthUnit.Cell used
        //at TimedTextStyle constructor is not supported
     marker.Style.FontSize = length;
   }

   Captions.Add(region);
}