I have Media Services 4.0 beta on a server set up to deliver two streams on one publishing point originating from an Expression Encoder 4 Pro Live session: a Smooth Streaming one for PC's and Macs, and a transmuxed one for iPhone/iPad. (It's based on the
Widescreen.H.264.Smooth Streaming iPhone WiFi preset.) It's working well, but we have a few users who reported that they sometimes randomly hear old audio from previous sessions in 2 second chunks. It comes and goes, and apparently this is only with the
Smooth Streaming Client (latest version), not the iPhone. Video does not appear to be affected, and some people say they never have this problem.
At first I suspected they may have been behind a caching proxy that was occasionally delivering up stale audio chunks, but now I just had a report of someone watching from home where I know there wasn't any cache inline. They thought it may have cleared
up when they flushed IE's temporary Internet files, so maybe it's strictly local? (But does the Smooth Streaming Client really have its fetches cached in the Temp files dir?)
Does anyone else have any other ideas, or know what I might be able to do to prevent this? Might I have to create a new publishing point for every live session so there can never be any stale manifests or audio chunks anywhere by the same name? Currently
I'm re-using the same publishing point for each live session but shutting it down and re-starting it each time it's time for a new session.
The stale cache may be the issue here. Smooth Streaming Client leverages the browser's HTTP channel and caches for streaming and by default all chunks from the server are cache-able. So if the chunk timestamps overlap
between live sessions which would result in duplicated URLs, an old cached chunk may get returned from the cache. IIS Media Services supports a notion called "event ID" which allows the encoder to pick an ID for each live session. With an unique ID for each
live session, the client will get a different URL template each time which can solve the stale caching problem even if the same publishing point is reused. Expression Encoder 4 does not support "event ID" today but it should come in future versions.
Another thing you can do, if you don't care about HTTP caching, is to tweak the caching parameter on the media server. At the server (root) level of the IIS Manager, go to "Live Smooth Streaming", you can see a "Client
Cache Settings" section. You could either disable client caching all together or set a smaller max-age value (in seconds). Server will send the proper caching directives to the client and it should be hornored by all HTTP caches.
I like the latter suggestion for now. Though it'd be great if it could be set on a more granular level, like at a virtual directory level or publishing point level.
I've set it to 60 seconds for now. Since it's a live stream there shouldn't be much need for caching unless the viewers use trick mode, which most probably will not.
I am a little surprised that the Smooth Streaming Client uses the browser's HTTP channel, caching and all. I realize it simplifies certain things and has its advantages, but for lengthy live streams it also seems like it'd mean a lot of disk thrashing and
cache index lookups. Are there any guidelines for optimizing this scenario? Would it be a "bad" idea to set the max-age to zero?
BTW, when EEPro supports EventID, I'm hopeful there'll be an "auto-increment" feature so it won't require manual incrementing each time? ;)
I'll know in a couple days how this works, but I suspect it will fix the problem. Thank you again for your reply.
HTTP caching is typically very valuable in smooth streaming scenarios due to its boost in server network scalability and end-user experience. For example, caching data in browser's cache store can help a lot with live-DVR scenarios. That is why we enable
HTTP caching by default.
Setting max-age to zero or disabling caching is ok as long as you can make sure you have enough network bandwidth and server capacity for all the data delivery.
Well, I'm not sure what to think now. I changed the default cache expiration of 2 days down to 1 minute. The packets timestamped to expire under the old 2 days setting went out Sunday night, so they should have expired by Tuesday night. Wednesday night
we broadcast a new live stream with the 1-minute timeout, and this same person reported they were still receiving random audio chunks from 3 days before! I had them flush their browser cache, and they said that even that did not fix it. (??!)
They are at a university, and at this point I don't know what else to think the cause could be other than a rather "unlikely" scenario. The university would: a) have to have a caching proxy with a lot of storage - enough for 3 days, and b) it would have
to disobey the timeout rule. Can you think of anything else that could be causing this? It's just odd that the video chunks are never stale, and the audio is only on occasion. (I don't know how much. I'd guess less than 25% from the way they described
it.)
Any suggestions you can offer would be greatly appreciated.
Galan Bridgm...
3 Posts
Media Services 4.0 and stale audio chunks
Sep 27, 2010 02:52 AM|LINK
I have Media Services 4.0 beta on a server set up to deliver two streams on one publishing point originating from an Expression Encoder 4 Pro Live session: a Smooth Streaming one for PC's and Macs, and a transmuxed one for iPhone/iPad. (It's based on the Widescreen.H.264.Smooth Streaming iPhone WiFi preset.) It's working well, but we have a few users who reported that they sometimes randomly hear old audio from previous sessions in 2 second chunks. It comes and goes, and apparently this is only with the Smooth Streaming Client (latest version), not the iPhone. Video does not appear to be affected, and some people say they never have this problem.
At first I suspected they may have been behind a caching proxy that was occasionally delivering up stale audio chunks, but now I just had a report of someone watching from home where I know there wasn't any cache inline. They thought it may have cleared up when they flushed IE's temporary Internet files, so maybe it's strictly local? (But does the Smooth Streaming Client really have its fetches cached in the Temp files dir?)
Does anyone else have any other ideas, or know what I might be able to do to prevent this? Might I have to create a new publishing point for every live session so there can never be any stale manifests or audio chunks anywhere by the same name? Currently I'm re-using the same publishing point for each live session but shutting it down and re-starting it each time it's time for a new session.
Thanks,
Galan
media Audio chunks services 4.0 stale beta
samzhang
249 Posts
Microsoft
Re: Media Services 4.0 and stale audio chunks
Sep 27, 2010 09:13 PM|LINK
Hello Galan,
The stale cache may be the issue here. Smooth Streaming Client leverages the browser's HTTP channel and caches for streaming and by default all chunks from the server are cache-able. So if the chunk timestamps overlap between live sessions which would result in duplicated URLs, an old cached chunk may get returned from the cache. IIS Media Services supports a notion called "event ID" which allows the encoder to pick an ID for each live session. With an unique ID for each live session, the client will get a different URL template each time which can solve the stale caching problem even if the same publishing point is reused. Expression Encoder 4 does not support "event ID" today but it should come in future versions.
Another thing you can do, if you don't care about HTTP caching, is to tweak the caching parameter on the media server. At the server (root) level of the IIS Manager, go to "Live Smooth Streaming", you can see a "Client Cache Settings" section. You could either disable client caching all together or set a smaller max-age value (in seconds). Server will send the proper caching directives to the client and it should be hornored by all HTTP caches.
Galan Bridgm...
3 Posts
Re: Media Services 4.0 and stale audio chunks
Sep 27, 2010 10:00 PM|LINK
Thank you, Sam.
I like the latter suggestion for now. Though it'd be great if it could be set on a more granular level, like at a virtual directory level or publishing point level.
I've set it to 60 seconds for now. Since it's a live stream there shouldn't be much need for caching unless the viewers use trick mode, which most probably will not.
I am a little surprised that the Smooth Streaming Client uses the browser's HTTP channel, caching and all. I realize it simplifies certain things and has its advantages, but for lengthy live streams it also seems like it'd mean a lot of disk thrashing and cache index lookups. Are there any guidelines for optimizing this scenario? Would it be a "bad" idea to set the max-age to zero?
BTW, when EEPro supports EventID, I'm hopeful there'll be an "auto-increment" feature so it won't require manual incrementing each time? ;)
I'll know in a couple days how this works, but I suspect it will fix the problem. Thank you again for your reply.
--Galan
samzhang
249 Posts
Microsoft
Re: Media Services 4.0 and stale audio chunks
Sep 27, 2010 10:28 PM|LINK
Hello Galan,
HTTP caching is typically very valuable in smooth streaming scenarios due to its boost in server network scalability and end-user experience. For example, caching data in browser's cache store can help a lot with live-DVR scenarios. That is why we enable HTTP caching by default.
Setting max-age to zero or disabling caching is ok as long as you can make sure you have enough network bandwidth and server capacity for all the data delivery.
Galan Bridgm...
3 Posts
Re: Media Services 4.0 and stale audio chunks
Oct 01, 2010 07:21 PM|LINK
Well, I'm not sure what to think now. I changed the default cache expiration of 2 days down to 1 minute. The packets timestamped to expire under the old 2 days setting went out Sunday night, so they should have expired by Tuesday night. Wednesday night we broadcast a new live stream with the 1-minute timeout, and this same person reported they were still receiving random audio chunks from 3 days before! I had them flush their browser cache, and they said that even that did not fix it. (??!)
They are at a university, and at this point I don't know what else to think the cause could be other than a rather "unlikely" scenario. The university would: a) have to have a caching proxy with a lot of storage - enough for 3 days, and b) it would have to disobey the timeout rule. Can you think of anything else that could be causing this? It's just odd that the video chunks are never stale, and the audio is only on occasion. (I don't know how much. I'd guess less than 25% from the way they described it.)
Any suggestions you can offer would be greatly appreciated.
Thanks,
Galan
samzhang
249 Posts
Microsoft
Re: Media Services 4.0 and stale audio chunks
Oct 01, 2010 08:30 PM|LINK