Live Streaming from a continuous MemoryRandomAccessStream

Topics: Windows 8 Xaml
Mar 1, 2013 at 12:05 AM
We are writing an app that live streams mp4 or h.264 video and audio to the MediaPlayer. What is the appropriate way to feed the buffered bytes to the player? When we initially use SetSource(), the player works. However, the player seems to be caching the video and its length, so if we add new bytes to the stream the player does not play them. Can we refresh that information in the player so that it continues to play? Are there settings in the media player itself we need to set so that it does not cache the length? Is there a method we can call?

The closest we've gotten so far to getting the player to work is refreshing the source by calling SetSource() everytime we add new bytes to the Stream. The problem with this method is that the player's audio and video skips during the refresh.
Coordinator
Mar 4, 2013 at 5:48 PM
Is the issue that playback stops before the end or that the timeline is not updated to reflect the full duration? I'm assuming the former based on your description. In this case, this is something the Windows 8 MediaElement is responsible for and is not something the player framework controls. Note: the player framework internally uses a MediaElement to facilitate playback.

I would suggest trying this technique out with just a plain MediaElement to see if you can find a way to make it playback the entire stream. If you have difficulties, you might need to post a question on the MSDN dev forums to get assistance.

Once/if you get something working with a MediaElement, you swap in the player framework and it should just work. Regarding updating the UI during playback, you can set EndTime in your code and the timeline will automatically be updated (but this will not affect playback).

Note: Two other routes are 1) create your own Media Foundation Extension. This will give you complete control over what the MediaElement is rendering. 2) Use smooth streaming which supports live playback out of the box.

Hope this helps.
Mar 4, 2013 at 11:44 PM
Thanks for your response!

We are looking into the Media Foundation Extension.
Mar 5, 2013 at 7:31 PM
For anyone else who stumbles across this question or has a similar problem, here's additional information from another resource we contacted:
It is not possible to use InMemoryRandomAccessStream for "live" content. The size of the MPEG-2 TS file needs to be specified up front. Once you specify the size of the file, it must never change, and you would need to support random access to any location in the file.

There are a few other options that might work.

1) You might be able to use an IInputStream instead. The file size does not need to be specified in that case, and seeking will not be possible. But this only works if your are using the HTML5 video tag. The XAML MediaElement cannot take an InputStream as input.

2) Create your own implementation of IMFByteStream and then wrap it in an IRandomAcessStream. In that case, the media pipeline will access the wrapped IMFByteStream, which does support live content.

To wrap the IMFByteStream, use the MFCreateStreamOnMFByteStream API:
http://msdn.microsoft.com/en-us/library/windows/desktop/hh162759(v=VS.85).aspx

There is no API to create an IMFByteStream on top of a growing memory buffer, so you probably would have to create your own implementation of IMFByteStream. http://msdn.microsoft.com/en-us/library/windows/desktop/ms698720(v=vs.85).aspx

You would make your IMFByteStream::GetLength() method return -1, to indicate that the size of the stream is unknown (and growing).
Mar 22, 2013 at 12:51 AM
Edited Mar 23, 2013 at 12:52 AM
I have exactly the same problem. I try to find a solution but without success for the moment. I tried with the HTML5 video tag and an IInputStream, but it doesn't seems to work.
Is someone succeeded in wrapping the IMFByteStream ? Or is there another solution ?
Mar 24, 2013 at 4:30 PM
Finally, I found a simple workaround that seems to work : my LiveStream class implements IRandomAccessStream interface. In the Seek method, I do nothing and the Size property throws a new NotImplementedException();
The length of the video is well 0 seconds in the player and it plays my video !
Mar 25, 2013 at 11:20 PM
Kakone, thanks for the followup! We have also been trying to implement the IRandomAccessStream and have a few questions, if you have time:

Are you using the C# MediaElement or the Playerframework MediaPlayer?
How did you implement Size in your RandomAccessStream wrapper?
We tried to just throw an exception like you described, and it causes the App to crash:
public ulong Size
{
    get {
        throw new NotImplementedException();
    }
    set {
         throw new NotImplementedException();
    }
}
Mar 26, 2013 at 9:48 AM
Edited Mar 26, 2013 at 10:52 AM
It works with the C# MediaElement and also the Playerframework MediaPlayer. The only problem I have is that buffering is very long (10 seconds - probably due to many unnecessary seeks). But, perhaps it works only with my video format : H.264 Elementary Stream in MPEG-TS container. I didn't try with other video formats.
I use the BlockingStream class where I implements the IRandomAccessStream interface like this :
private IInputStream _inputStream;

public BlockingStream()
{
   _inputStream = this.AsInputStream();
   ...
}

public IRandomAccessStream CloneStream()
{
   throw new NotImplementedException();
}

public IInputStream GetInputStreamAt(ulong position)
{
   return _inputStream;
}

public IOutputStream GetOutputStreamAt(ulong position)
{
   throw new NotImplementedException();
}

ulong IRandomAccessStream.Position
{
  get { throw new NotImplementedException(); }
}

public void Seek(ulong position)
{
}

public ulong Size
{
  get { throw new NotImplementedException(); }
  set { throw new NotImplementedException(); }
}
Apr 21, 2013 at 9:20 PM
How did you implement ReadBytesAsync method.
My stream keeps playing the first block.
        public IAsyncOperationWithProgress<IBuffer, uint> ReadAsync(IBuffer buffer, uint count, InputStreamOptions options)
        {
            var result = await _inputStream.ReadAsync(buffer, count, InputStreamOptions.ReadAhead);
            return result;
        }
_inputStream is WebResponseStream, with "response.GetResponseStream()"
Dec 18, 2013 at 10:30 PM
kakone wrote:
Finally, I found a simple workaround that seems to work : my LiveStream class implements IRandomAccessStream interface. In the Seek method, I do nothing and the Size property throws a new NotImplementedException();
The length of the video is well 0 seconds in the player and it plays my video !
kakone, could you share the code for playing an MPEG-TS source? Thank you so much.
Apr 17, 2014 at 2:48 PM
kakone wrote:
Finally, I found a simple workaround that seems to work : my LiveStream class implements IRandomAccessStream interface. In the Seek method, I do nothing and the Size property throws a new NotImplementedException();
The length of the video is well 0 seconds in the player and it plays my video !
I'm also very interested, could you share your code please ?

Best regards