Measuring video quality

Quality control (QC) is an important part of bringing content into broadcasters' workflows and archives. Industry-wide, broadcasters spend a large proportion of their revenue on acquiring content, but this content cannot be monetized by a broadcaster until it has successfully made it into the business's workflow. Making QC a part of the ingest and transcoding process reduces the risk of poor quality content being included in the business' content archive and consequently decreasing the value of that archive. Automated QC also lowers the cost of bringing content into the business, enabling smaller markets to be addressed profitably. When ingesting content from tape, QC allows many problems to be fixed before the operator takes the tape out of the VTR, saving time and money. (See Figure 1.)

Ingest: The business process

In the broadcast industry, ingest is the “goods-in” department of the manufacturing process. Material from a number of content providers is supplied to the business, traditionally either on tape or as a contribution feed. This material is then ingested into the business's workflow. During ingest, the business systems are updated with data about the content, and then once ingest is completed, the rest of the business process is kicked off.

As businesses look for ways to reduce costs, many are moving to use digital file formats instead of tapes and the Internet instead of video contribution feeds. While interchanging media files removes the conversion from tape to file, the other parts of the ingest process still need to be undertaken. These include metadata entry and conversion of the file into a format supported by the rest of the workflow. (See Figure 2.)

Defining ingest as “the business process of accepting tapes or files into the workflow” gives a common approach to the process of bringing content into the broadcaster's systems. This way the broadcaster can be sure that all of the media assets in its workflow have gone through the same carefully defined processes. If you set up your business to treat ingest and QC as a single business process at the front end of your workflow, then the move from tape to file can be managed at the operational level and will involve minimal changes to the rest of the business.

Why QC is important in any workflow

In any business, QC plays an important part in ensuring that the output meets customers' expectations. QC is often viewed as an output-side process, checking the final product. However, it should also be applied as an input process in order to save the wasted effort of processing already damaged goods and prevent “garbage in, garbage out” situations.

Early QC can significantly reduce the cost of fixing errors, as detecting and fixing problems upstream is always cheaper than letting them infiltrate into the downstream workflows. QC can be used at the boundaries of any business process to check that the process is operating correctly. At each of these points, the tests undertaken as part of the QC process will be different and should be tuned to detect the appropriate errors for that point in the workflow. If the QC process can be automated, then its reliability is improved and its cost reduced. An automated QC system can pick up on errors that are undetectable by a human operator, but that still affect the downstream workflow.

In the broadcast industry, the input QC checks to make sure that incoming material is not damaged and meets the broadcaster's specifications. Material rejected at the input is returned to the supplier for repair. QC at a broadcaster's output is used to check that the broadcaster's workflow is operating correctly. Any failures point to a process or piece of equipment that needs to be fixed.

Verification vs. measurement

In an ideal world, all content purchased would be undamaged and would meet all of the delivery specifications. The ingest process, whether tape-to-file or file-to-file, would always have a perfect input and thus always create the desired output. The world, however, is not ideal, and things do go wrong. In an ingest process for playout or for library and archive use, the important question is: “What sort of QC do I need?”

A test undertaken as part of the QC process involves measuring media parameters and then making a decision as to whether the results of that test are acceptable to the business. The test results are combined into a QC report to allow the appropriate workflow action to be undertaken. The test results must be agreed beforehand to suit individual case requirements, because while too little information prevents useful actions from taking place, huge quantities of measurement results may overwhelm the business and hinder the QC operator from deciding what path the content should take in the business's workflow.

QC as a part of ingest

When creating a broadcaster's business processes, the QC process will often be considered a separate step in the workflow. This is mainly because QC is often done by a separate operator, after the tape-to-file conversion has occurred. Operationally, incorporating the QC process alongside ingest allows an operator to be responsible for all of the steps involved in bringing content into the business. It also allows the operator to address many errors before the tape is even taken out of the VTR, thereby improving operational efficiency. Considering ingest and QC operations as one business process simplifies the business workflow and makes it resilient to changes in the operational aspects of ingest and QC tasks. This improvement can be used to reduce the overall cost of the ingest process to the business.

When choosing what tests need to be undertaken as a part of the ingest process, there is a large selection available. It is worth taking into account that when the tests fail, the operator should be able to act on the results. For example, when a test that checks for clipped audio fails, the operator can address the error by reducing the audio gain on the VTR and retrying the ingest. A test for letterboxing in the video could have the operator modify the ingest parameters or send the content to an editor for letterbox removal. However, there are many tests with no obvious solution; these are not a useful part of the ingest process. One example might be MPEG transport stream structural checks which, while important for transmission, may not be useful checks to undertake during ingest, when the content is being repackaged before transmission. (See Figure 3.)

The QC process for ingest should be designed to detect errors that can be introduced by the incorrect operation or configuration of equipment, either at the ingest point or upstream of it, not errors in the design of the equipment. One advantage of integrating QC with transcoding and ingest is that any design errors in upstream equipment (e.g. poor metadata structures or low-level MPEG issues) can be identified, logged and often corrected as part of the integrated QC process.

As human error plays a role in errors introduced into any content, increasing the level of automation involved not only decreases running costs, but also improves the reliability and quality of the output of the workflow. Introducing integrated, automated QC means that operators can inspect and review content by exception, rather than being required to view every frame. For a highly automated workflow to be achievable, the inputs have to meet a tight specification, so QC during ingest can enable a highly automated media factory.

However, automated QC cannot detect all of the important errors in an asset (such as lip sync errors). This makes integrating a human operator with the automated tools an important part of any solution.

In closing

QC is an important part of the ingest process, and a tightly integrated solution allows a business to rely on its ingested content as a valuable revenue-generating asset. In order to make the process cost-effective, broadcasters must carefully choose the tests required to prevent paying for equipment with unnecessary features. However, if broadcasters take the right steps toward treating ingest and QC tasks as one business process, their business will be better equipped to cope with the industry's migration from tape to file delivery.

Bruce Devlin is CTO and Tim Harris is systems design engineer at AmberFin.