How Does H.264 SVC Work?

H.264 SVC provides the scalability described previously by building video streams out of individual, complementary layers. This all starts with the base layer, which has the lowest resolution and frame rate that can be displayed. Enhancement layers are then provided as needed, which will provide higher quality to endpoints that request it. Again, these enhancement layers are complementary, so when combined with the base layer, the video resolution and frame rate are increased. This is best described in the form of a diagram, as shown in Figure 19.1.

Figure 19.1. H.264 SVC layering example.

In this example, a PC endpoint is able to send up to 1080p HD video, and is connected to a conference. That conference is run by an MCU (Lync A/V Conferencing Server), and there are three other participants in the call. In previous versions of Lync, the video resolution would be dropped to the lowest common denominator. For example, if the phone that supported only 360p video joined the video conference, everyone would send and receive 360p video. With Lync 2013 and H.264 SVC that limitation no longer exists.

In this scenario, the base layer is built on 360p video, which is the lowest resolution needed for participants in the conference. The endpoint knows how to build its layers based on what each endpoint asks the MCU for when joining the conference. The sending endpoint then adds additional layers, for additional resolutions all the way up to 1080p, because another participant has asked for that resolution.

The sending endpoint sends all layers to the MCU, and the MCU then sends each layer that is requested to the other participants. Since the mobile phone is limited to 360p, it is sent only the base layer. The tablet, on the other hand, can support 720p video, so it is sent two layers to make up the 720p video stream. The third participant can also support 1080p video, so it is sent all three layers. It is important to remember that these layers are additive, so there is no duplicate data sent, only the delta. Because only the delta data is sent in the enhancement layers, the bandwidth used by H.264 SVC calls is similar to that of other codecs. Another great feature introduced with this codec is that everything is dynamic. Participants in the conference notify the MCU of any changes to its capabilities. A great example of this happening would be if the second PC that is capable of 1080p video did not have the video in full-screen mode. If that user is not requesting to view a 1080p video stream, there is no reason for the other party to encode and transmit a 1080p stream. If that user then decided to expand to full-screen, however, the MCU would dynamically change the stream resolutions. This allows for efficient processing and bandwidth utilization on the MCU and the endpoints.

The largest benefit to this technology is seen by the A/V Conferencing Server. Although this role has never encoded or transcoded media, only with the introduction of H.264 SVC is Lync Server now able to introduce more functionality, without increasing processing requirements. This does, however, increase the processing load on endpoints. Microsoft has always leveraged endpoints for encoding and decoding media streams, for both audio and video. In Lync 2013, this processing load is increased for HD video scenarios, and the details of those requirements are covered in the “Peer-to-Peer Video Endpoint Requirements” section.