Guides & Articles

Fundamentals of AV over IP

What Is AV over IP?

AV over IP stands for "Audio-Visual over Internet Protocol". Essentially, it is the transmission of audio-visual data over a network such as a LAN, WAN, or the internet. As opposed to traditional AV environments, AV over IP (also known as AV/IP) refers to the use of standard network equipment to transmit and switch video and audio.
 
The concept of "Video over IP" has existed a long time. It encompasses everything from internet-based live or on-demand video streaming, to professional video distribution infrastructures in production and broadcast studios. What is being discussed in the professional AV (facilities AV) space over the last few years is the gradual replacement of traditional AV infrastructures with IP-based infrastructures—hence the term AV over IP. For some people, the idea of using IP in the facilities AV space is quite new, while for others, it is familiar territory.
 

For more information on Multi-Channel Encoding for Live and On-Demand Streaming, please visit this link.

Similarities Between Traditional AV and AV over IP Setups

At the base, the elements in AV environments stay the same. Traditional AV infrastructures are mainly concerned with the extension and switching of audio and video assets. The goal of an AV facility is to provide users the ability to see and/or hear their video and sound sources on their viewing stations and on their sound systems or speakers. To do this, sources need to be captured, moved, switched, and displayed. The user interface to change sources can be push buttons on hardware equipment, all the way to digital interfaces on computer-based technologies.

Similarities between traditional AV and AV over IP setups
 
In both AV setups, the elements are very similar. In AV over IP, the "A/V Transmitters" become "Encoders", "A/V Receivers" become "Decoders", and the “A/V Switcher" (also known as video matrix switcher) becomes a standard "IP Switch" just like what your computers connect to at the office.

Current AV Technologies

Many AV products and services are designed to preserve maximum image and/or sound quality while the sources are being moved and switched. Other performance elements include fast switching and low latency throughout the pipeline. Additionally, some AV equipment is capable of performing processing operations such as multiplying the sources and making them simultaneously available in more than one place for consumption.
 
More advanced processing involves performing modifications to the sources in real time. This can include changing the video signal from one type to another (like DisplayPort to HDMI), cropping, up- or down-scaling (for example changing from HD to 4K or 4K to HD), compositing (text overlays or combining multiple videos), and more.
 
Everything described above for traditional AV is preserved when implementing AV over IP. The only difference is that the video and audio moving through the series of boxes and cables changes from circuit-based to packet-based —like computer data networks and telephone over IP. Basically, Internet Protocol (IP) is a set of rules governing the format of data sent over the internet or other video and audio networks. AV-over-IP technology organizes the audio-visual data so that it conforms to those rules. Data transmitted over IP is subdivided into packets. Each packet contains part of the original file as well as additional control information such as source, destination, and sequence.
 

Differences Between Traditional AV and AV over IP

AV over IP differs from traditional AV by evolving the following key aspects: scalable switching (many more ports and easier to add just what you need), breaking the barriers of distance, improved ratio of inputs to outputs, video standards that extend beyond the local facility, convergence with data and communications, and new options in video processing.
 

Switching

Perhaps the cornerstone of why organizations are gradually replacing traditional AV infrastructures with IP-based AV infrastructures has the switch as the focal point.
 
Hardwired, circuit-based switching is basically a point-to-point technology. Video matrix switchers are a « destination» (relative to an AV transmitter box) and a "source" (relative to an AV receiver box) simultaneously. All the combinations of transmitters to receivers are resolved inside the matrix switch and it is possible to use any source at any destination according to the number of ports available on the video matrix switch. For example, an 8x8 matrix switch allows eight sources to be used at any of eight destinations.
 
More advanced products can perform processing operations. Instead of just making any input available on any output, for example, it might be possible to show any input on any—as well as many—outputs. Hence, a PowerPoint presentation from a PC can be a source that gets routed from an AV transmitter box to a video matrix switcher and then the switcher can be wired to multiple AV receiver boxes that can simultaneously be showing the PowerPoint presentation in real time.
 
What’s different about IP (and packet-based switching) is the number of sources attached to the IP switch is no longer as limited. When physical ports run out, multiple IP switches can be connected to expand. What this means is that you can scale the number of ports to your needs much more conveniently. It is possible to keep adding sources and destinations without a substantial overhaul of the video matrix switcher centerpiece being a major limiting factor.
 
The ratio of inputs to outputs can also be far more tailored in AV over IP compared to the traditional hardwired video matrix switcher. It is possible to have MANY inputs but only a few outputs, or only a few inputs but MANY outputs. Or you can have MANY of both and in widely different quantities.
 

Distance

Another limit of traditional AV is the distance between boxes. All hardwired digital transmissions have a practical limit to distance. Short distances of only a few feet can be wired cheaply. Once you’re at several meters, the cost of extension cables goes up. And when wires are being run throughout facilities over hundreds of meters, the cost of installation and extension becomes higher still.
 
IP-based AV can be transmitted over copper (Category) cable and over fiber optic quite conveniently. The Category (CAT-5, CAT-6, etc.) cable has a maximum of 100 meters. But it is possible to switch and repeat in series. The video you watch in your home on Netflix or YouTube has travelled quite far using this exact packet-based switching technology.
 
Thus, AV over IP significantly increases flexibility by overcoming limits to number of sources and destinations as well as by conquering distance limits. There is no penalty to capabilities for migrating from traditional AV to AV over IP as all elements of performance are retained with IP-based solutions.
 
The illustration below highlights this with the help of three main architectures. The first is based on traditional AV switching followed by the next two on AV over IP.
Diagram of AV over IP vs. traditional AV switcher

Standards

Some AV-over-IP products use standards-based packetization for transmission on IP networks and compatibility with IP switches, and some use proprietary packetization schemes which also work on IP networks and standard IP switches but which do not work with other products in the market.
 
In general, standards-based schemes provide the potential for interoperability between products from different vendors. Far more importantly however, standards arise from the work of many stakeholders balancing the needs from many different technological perspectives. Standards-based products tend to have a road map and evolution that provides much greater infrastructure migration benefits than anything based on a single supplier.
 

Interoperability

Whether the packetization scheme is standards-based or proprietary does not alone determine interoperability and also does not establish whether a product is more, or less, secure. It only refers to the potential for the streams to work with a broad base of technology, or their inability to do so.
 
Some vendors provide AV-over-IP encoders and decoders that are tightly coupled. In other words, the encoders and decoders need to be from the same vendor. The number one reason for this tight coupling is usually because vendors try to provide their customers with guaranteed specifications and performance. This also allows for a very controlled out-of-the-box ease of set up and ease-of-use experience.
 
Other vendors provide completely wide-open compatibility. In other words, they either make encoders that work with decoders from other vendors, or they make decoders that work with encoders from other vendors, or they make both. These products put a strong emphasis on interoperability and the ability to leverage features and capabilities from an enormous range of hardware and software providers.
 
Some other vendors provide products that support both tight coupling and interoperability. They deliver all the features and benefits of tight coupling when you get encoders and decoders from the same vendor. However, they also offer modes of operation that allow you to greatly leverage features and capabilities from a wide range of third-party hardware and software providers.
 

Traditional AV Examples

AV over IP Examples

1. Digital video Tx and Rx boxes and/or cables 

Digital video matrix switcher

  • HDMI
  • DisplayPort
  • SDI
  • HDBaseT                                                                                                                                                                                                                                                                                       

1. Standards-based AV-over-IP products

  • SMPTE 2110 specification for uncompressed video over IP
  • JPEG-2000 lightly compressed video over IP
  • H.264 high-efficiency compression over IP

2. Analog video Tx and Rx boxes and/or cables 

Analog video matrix switcher

  • Composite video
  • S-Video
  • Component video                                                                                                                                                                                                                                                                                                                                                                                                                                                      

2. Proprietary AV-over-IP products

  • The packets between encoders and decoders respect Internet Protocol and the streams can be switched on standard IP switches, but the encoding scheme (packetization of video) is proprietary and incompatible with any devices using standards-based codecs.

Example: SDVoE

Focusing on the leading AV over IP products in the market – which are all founded on standards-based packetization (including well-established codecs in many cases) – the next topic is encryption and security.

How Secure Is AV over IP?

A common misconception about migrating from hardwired AV to AV over IP is that the latter introduces moresecurity risks than traditional AV. With more flexibility and new deployment options, education is the best ally to AV administrators. Encryption technologies exist for several aspects of AV-over-IP products, and they address multiple components of AV system design.
 
Some products provide encryption on the command-and-control signaling to encoder and decoder devices. This offers security against hacking the actions of the boxes—including turning streaming on or off, or switching what source is being displayed. Another security aspect is the ability to encrypt the video streams themselves. This ensures that if the video stream is intercepted, it cannot be simply decoded and viewed.
 

Digital content protection

When dealing with interoperability, some products provide support for third-party devices using digital key exchanges or encryption. For example, the overwhelming majority of AV customers are concerned with the most straightforward case—which is HDCP. The purpose of High-bandwidth Digital Content Protection (HDCP) is to protect digital copyrighted content as it travels between devices. For instance, a cable or satellite receiver box, or a media player with HDMI outputs, might play content in HD or 4K that is protected content. Such content is locked and can only be viewed by HDCP-compatible products once they are properly authenticated. There are restrictions on how protected content can be extended, multiplied, altered, or viewed.
 

Encryption technologies

While some encryption technologies are purpose-built to offer security options to AV professionals, sometimes they are also used to simply reduce openness. Vendors aiming to provide an assured experience to customers might choose to lock down the compatibility of their encoders and decoders to just their own brand as a measure to limit the scope of their quality assurance testing, technical support, and customer care options. This also allows to provide good user experiences to customers for set up and product use.
The blue lines represent the audio and video content flow. The yellow lines represent the command-and-control signaling
The blue lines represent the audio and video content flow. The yellow lines represent the command-and-control signaling
 
In this illustration, the main elements of AV-over-IP encoders and decoders are visible.
 
  • An audio/video port (like HDMI in the example)
  • A network jack to stream the packetized video in or out
  • A command-and-control port
 
(Although an RS-232 connection is shown here, this is often combined with the network port used for sending and receiving packetized video. It is also possible to use a separate RJ-45 port to connect to a separate network for command and control.)

Top Security Concerns for the AV Industry

Command and control

There are push-button-only products in traditional AV where behaviors such as switching a source can only occur manually on the hardware itself. This is of course secure from remote hacking, but undeniably restricts the convenience and functionality as well.
 
Most useful AV products, whether they are traditional AV or based on AV over IP, have command-and-control ports that allow for remote control of the behaviors of the boxes, including turning on and off or switching the sources. For example, when a touch panel or AV processor is involved, some form of remote command and control is in use.
 
The command-and-control layer can be protected with permissions, passwords, and encryption. There is an equivalent responsibility for the command-and-control layer on vendors selling traditional AV as well as those selling AV over IP.
 

Video and audio content security

Can private assets like streaming camera feeds be intercepted by wrong doers? Of course. But this is true of ANY video or audio feed. Whether the video or audio is in circuit-based form on analog or digital wiring, or whether the video or audio is in packet-based form on IP networks, in both cases, it is possible to hack and access the video and audio feeds.
 
There is no substitute for knowledge and responsible deployment efforts when it comes to securing video and audio. In fact, some feel that the ability to encrypt packetized video and audio is superior to traditional baseband video if there is a concern that someone might try to hack and snoop the feeds. Keep in mind that IP security has been around for a long time. Both data and telephone over IP have already gone through multiple generations of constant iterative improvements on this. In addition to information about IP network security and content encryption, there are also many well-established consultants and experts in the security field to assist you—no matter how sophisticated or basic your requirements are.
 

Network security

How to deploy AV over IP is a question of fit. AV over IP can be deployed on entirely segregated networks that never co-exist with packets of data from an organization’s data network or communications network. Alternatively, existing infrastructures of network cabling and switching are capable of, and already frequently used, for AV-over-IP applications. AV-over-IP implementations, whether on separate or existing infrastructure, can be done without compromising the IT network security. In many organizations, the ability to use data, communications, and AV together (also known as “convergence”) is a driving force and key benefit for how AV is being re-fitted or newly deployed.

Maximum Value for Video and Audio Assets

There is no question that the IP network expertise from the computer networking world has value in the AV space. Old walls that once existed between AV and IT are melting away. Nevertheless, a strong understanding of AV-over-IP technologies and network requirements is still field-of-expertise focused on one thing only: providing maximum value to customers for their audio and video assets.
 
Moving away from centralized A/V switching to distributed encoding and decoding, AV over IP puts the emphasis back on the value of the audio-video assets themselves. This is possible because IP allows better distribution of the AV processing capabilities and the ability to easily scale with user needs.
 
By allowing users more flexible deployments, providing options for using AV assets over greater distances, and letting users to pursue powerful new capabilities that better reflect evolving technology and changing worker habits, the migration towards AV over IP is well under way.