HowTo: Multi-row captions and wrapping at Silverlight Media Framework

One of the main goals of the ClipFlair project is to explore the use of Video Captioning (together with Revoicing) for Foreign Language Learning (FLL). There both are important enhancements compared to the simpler two-line subtitling that was put in use in its ancestor project LeViS.

Captioning means that multi-row text is needed while text wrapping is also useful, especially if cartoon-like balloons over the video area are to also be examined as an option. Implementing multi-row captions and caption wrapping was a bit tricky in the SMF based player, due to the lack of relevant documentation, so adding some relevant notes below.

As I discuss in more detail near the end of the discussiion at  http://smf.codeplex.com/discussions/245424, the way I finally managed to make SMF player show multi-row captions is by setting the caption region bounds (via Origin and Extend properties of the CaptionRegion object) to take up nearly all the video area and combined it with DisplayAlign = DisplayAlign.After (to bottom align the caption rows group) and Overflow = Overflow.Dynamic (to extend the visible caption area dynamically inside the defined bounds).

Regarding caption wrapping there seems to be a bug in SMF, it wraps caption text at the video boundary instead of at the caption region max boundary (as defined by Origin/Extend properties of the CaptionRegion object). Have reported it at:

http://smf.codeplex.com/workitem/23634

A workarround for caption wrapping to work perfectly is to set top to 0 for the origin and set extend width to 100% for the caption region, but this won’t be useful if you want to show cartoon-like bubbles on the video for some captions. If Microsoft don’t fix that in the near future I guess I’ll have to fix it myself at the source-code of SMF (luckicly it’s an open-source project).

The relevant code snippet  from http://ClipFlair.codeplex.com for the MediaPlayer component is:

private const double CAPTION_REGION_LEFT = 0; //0.05;
private const double CAPTION_REGION_TOP = 0.05;
private const double CAPTION_REGION_WIDTH = 1; //0.9; //SMF 2.7 has a bug here,
               //it wraps caption text at the video boundary instead of at the 
               //caption region max boundary (as defined by Origin and Extend)
private const double CAPTION_REGION_HEIGHT = 0.9;

public static void StyleCaptions(CaptionRegion theCaptions)
{
  if (theCaptions == null) return;

    theCaptions.Style.ShowBackground = ShowBackground.WhenActive; 
               //doesn't seem to work if other than transparent color is used
    theCaptions.Style.BackgroundColor = Colors.Transparent;

    //set caption region (max) bounds
    theCaptions.Style.Origin = new Origin() { 
      Left = new Length() { 
        Unit = LengthUnit.Percent, Value = CAPTION_REGION_LEFT }, 
      Top = new Length() {
        Unit = LengthUnit.Percent, Value = CAPTION_REGION_TOP } };
    theCaptions.Style.Extent = new Extent() { 
      Width = new Length() { 
        Unit = LengthUnit.Percent, Value = CAPTION_REGION_WIDTH }, 
      Height = new Length() { 
        Unit = LengthUnit.Percent, Value = CAPTION_REGION_HEIGHT } };

    //theCaptions.Style.Direction = Direction.LeftToRight;
  
    theCaptions.Style.DisplayAlign = DisplayAlign.After; 
                                 //align multirow catpions to bottom of region
    theCaptions.Style.TextAlign = TextAlignment.Justify; 
                                 //horizontally center captions

    theCaptions.Style.WrapOption = TextWrapping.Wrap; 
                                 //wrap too long captions to next row
    theCaptions.Style.Overflow = Overflow.Dynamic; 
         //extends the area for the captions as needed, up to the given Extent

    foreach (CaptionElement caption in theCaptions.Children)
      StyleCaption(caption);
}

public static void StyleCaption(TimedTextElement theCaption)
{
  if (theCaption == null) return;

  theCaption.CaptionElementType = TimedTextElementType.Text;
  theCaption.Style.ShowBackground = ShowBackground.WhenActive;
  theCaption.Style.BackgroundColor = Color.FromArgb(100, 0, 0, 0); 
                                     //use a semi-transparent background
  theCaption.Style.Color = Colors.White;

  Length length = new Length
 {
   Unit = LengthUnit.Pixel, //must use this, since the default LengthUnit.Cell
                           //used at TimedTextStyle constructor is not supported
   Value = 20
 };
  theCaption.Style.FontSize = length;
}

I also tried setting Padding property at the code above in case it would help with the correct wrapping, but it somehow results in crashing Silverlight (may be related to the fact that I have hardware acceleration enabled for our Silverlight app).

HowTo: load CaptionElements into Silverlight Media Framework player

Trying to make CaptionsGridWindow of ClipFlair serve captions editing on-the-fly to SMF (Silverlight Media Framework [now called MMPPF]) player component, I had a real hard time, plagued by a bug at TimedTextElementStyle. It seems to be setting default FontSize for captions using a “Cell” unit instead of using a “Pixel” unit. Currently SMF only supports “Pixel” units at FontSize of TimedTextElements according to the codedoc notes.

Another issue I had was that CaptionRegion constructor sets it to be active all the time (specifying a Begin value of a min possible TimeExtent and an End value of the max possible TimeExtent), so it renders ShowBackground.WhenActive setting useless for it. That is if you want the captions background to show up only when there are captions showing (active), then you have to set the CaptionRegion’s background to Colors.Transparent and set each CaptionElement’s background to some non-transparent color (note that both CaptionRegion and CaptionElement are TimedTextElements, with the later added to the former’s Children property, forming a TimedTree that is).

 

public void UpdateMarkers(MediaMarkerCollection<TimedTextElement> newMarkers)
 {
   if (newMarkers == null) return;

   CaptionRegion  region = new CaptionRegion();
   region.Style.ShowBackground = ShowBackground.WhenActive; 
             //doesn't seem to work if other than transparent color is used
   region.Style.BackgroundColor = Colors.Transparent;

   foreach (CaptionElement marker in newMarkers)
   {
     region.Children.Add(marker);
     marker.CaptionElementType = TimedTextElementType.Text;
     marker.Style.ShowBackground = ShowBackground.WhenActive;
     marker.Style.BackgroundColor = Color.FromArgb(100, 0, 0, 0); 
                                       //use a semi-transparent background
     marker.Style.Color = Colors.White;
     //marker.Style.TextAlign = TextAlignment.Center;
     Length length = new Length
     {
       Unit = LengthUnit.Pixel, 
       Value = 20
     }; //must use this, since the default LengthUnit.Cell used
        //at TimedTextStyle constructor is not supported
     marker.Style.FontSize = length;
   }

   Captions.Add(region);
}