Measuring Quality of Service and Experience

Pixelmetrix' Electronic Couch Potato monitors and reports Quality of Experience from the end user point of view.
JOHNSTON, IOWA— There has been much written about the difficulties of delivering video services over an IP network. Packet loss, latency, timing, jitter and bandwidth all present challenges even in a well-designed private network. When the consumer's connection is via the public internet, the challenges are magnified by all of the other stuff that is sharing the road. There are some people who believe that ultimately all media consumption will be via a data network (wired or wireless) and that real-time steams of content sent via traditional broadcast mechanisms will cease to be. I have to admit that I am not one of them as I believe that each method is a tool and at times one tool is better for the job than the other.


Be that as it may, the fact remains that the content that originates at our stations is even now being delivered as video over IP network connections. Therefore understanding the measurement criteria for the services throughout the entire system is crucial. Quality of Service (QoS) is a term that most broadcast engineers are familiar with but they may not fully understand what it actually means. Quality of service is a term from the telecommunications engineering business and is essentially a service reservation. I liken it to booking tickets on a plane. It allows the person making the reservation to select which cabin they want to sit in and sometimes even the seat they want. Another term that we are now becoming more familiar with is Quality of Experience (QoE). Quality of Experience is the subjective measure of the end users experience. It is essentially the overall perception of how good our service is to the viewer.

Quality of Service is an important metric to fully understand. Overtime, even a well-engineered network will experience changes that may affect the performance of the service. In the old days of point to point systems, the paths between send and receive devices were dedicated and the traffic was tightly controlled and well regulated. Bandwidth was a factor of inputs and outputs on the routing switcher rather than the amount of information traveling on the wires. IT- based networks use a shared switching fabric and infrastructure. They are typically designed using load statistics rather than counting inputs and outputs. It is common for even a well-designed IP network to have some QoS issues like packet loss timing variations caused by congestion. The designs are done at a point in time, taking into account current network loading and making a best guess at future loads. However, new ideas spring up all the time that may not have been considered when the system was designed which add to the load on the network.

A more subtle issue is the fact that as many of the high speed components in the system age, their performance tends to roll off. In the old days of analog video, as components aged it was not uncommon to see high frequency roll-off which showed up as a loss of detail in video. This was pretty easy to diagnose and track because everything traveled independently on dedicated paths. When devices start to exhibit age-related performance issues in an IP network, my experiences have been that the indicators tend to be more intermittent because of the statistical nature of the loading and the fact that they're harder to track down. Of course if I was following Moore's Law, I would be replacing my infrastructure every 18 months and this wouldn't be a problem. Unfortunately, my budget doesn't adhere to Moore's Law and prevents me from wholesale replacement of systems. This is why it is vital to monitor closely the systems and record each and every modification, addition and subtraction so that QoS issues may be determined as quickly as possible.


The other side of this coin is the Quality of Experience. Within a facility, there isn't a whole lot of difference between the two. The service provider and the customer typically all work for the same organization and the communication while not always perfect is pretty much immediate. Engineers in the station can work closely with their colleagues to diagnose and correct issues or at the very least perform a work around. But what happens if the customers are the viewers in their homes? Now the content is leaving the tightly controlled and theoretically well designed internal network and traveling through the public network? One of the Quality of Service metrics that will have a direct relationship to Quality of Experience in the home is bandwidth. Bandwidth to the home is typically specified as "up to" some number. However, the theoretical minimum may be far below that number and will vary based on time and the amount of other consumers sharing the fabric. As more and more consumers migrate to these types of systems to consume content, including high definition, the Quality of Experience they have may be far worse than they will tolerate and yet no Quality of Service metrics will be violated.

I have recently started acquiring devices such as the Logitech Revue, the Boxee Box and AppleTV because I want to experience what many of the consumers of the content Iowa Public Television produces. In order to improve the quality of service though these devices, I want to make sure that I share the quality of experience.

Bill Hayes is the director of engineering for Iowa Public TV. He can be reached via TV Technology.

Bill Hayes

Bill Hayes, director of engineering and technology for Iowa PBS, has been at the forefront of broadcast TV technology for 40 years, 23 of them at Iowa PBS. He’s served as president of IEEE’s Broadcast Technology Society, is a Partnership Board Member of the International Broadcasting Convention (IBC) and has contributed extensively to SMPTE and ATSC.  He is a recipient of Future's 2021 Tech Leadership Award.