Let’s review some of the important feature sets typically found in a cloud solutions provider.

First, the cloud provider should almost always store or process your data in multiple locations, aka data centers. These data centers provide the physical elements for connecting all of your data, anywhere. Data access will generally include cloud apps, databases and hundreds to thousands of both on-prem and off-prem systems, using “prebuilt” connectors that integrate the solutions handling your data and allow it to be processed through established services.

A cloud provider should be able to effectively leverage your existing infrastructure with an ability to query or analyze your data with features including replication, movement/migration and “rework.”

‘AI-Ready’ Data
Given the global emphasis on artificial intelligence, one would almost expect this service-level statement—“all our data is AI-ready”—given the levels of artificial intelligence that the marketplace continually promotes, irrespective of the reference or workplace. Fig. 1 depicts a workflow inside a cloud that could aid in preparing data for AI-ready states or actions—ideas shown in Fig. 2 generally feed back into systems, as shown in Fig. 1.

AI-ready data means that your information has been systematically prepared, evaluated, managed and governed to meet the needs of AI projects. With financial-related data, expectations are that transaction records are properly prepared before that data is fed into an AI model.

Assume certain checks that your (cloud) services provider can include or package can identify patterns (or repetitive series of characters that could flag harmful routines that might represent fraudulent transactions, loops or means to generate a code sequence that would alter, falsify or get a back door to an unwanted action).

In retail applications, your cloud provider might offer “AI prep” capabilities and readiness for applications such as “demand forecasting,” which uses historical data on sales volumes and costs, as well as comparative product details that can be shared across hybrid and multiple cloud providers located regionally, globally or both.

For organizations (like original equipment manufacturers) storing preventative or predictive maintenance for industrial purposes such as aircraft maintenance, the cloud services provider should be capable of tracking and cataloging short-term and long-term historical data, plus real-time data derived from sensors and performance variables. Applications for the cloud-storage systems would leverage and train AI models to accurately predict equipment repair times, schedules and relative downtime.

Sometimes referred to as a “digital vault,” immutable storage is a paradigm where information, once written, cannot be modified, overwritten or deleted for a specified retention period. It is also referred to as WORM (write-once, read-many) storage or object-locked storage. The opposite term is “mutable storage,” which can be edited, replaced, modified or destroyed at any time.

Unlike mutable values, an immutable value or content cannot be changed without creating an entirely new value. For example, in JavaScript, primitive values are immutable—once a primitive value is created, it cannot be changed, although the variable it holds may be reassigned to another value.

Supply-Chain Security
In an “open source” age, malicious activities are common and almost expected in nearly every software and data system—and especially in cloud services. In e-commerce services (such as eBay or Etsy), users place assurance expectations on their vendor’s services, who in turn rely on the respective e-commerce company to “pre-protect” the data and services of their customers and clients, using industry best practices and some of the services listed in the following:

• For an in-depth understanding of how certain software is protected, Software Composition Analysis (SCA) emphasizes control over inventory, dependency mapping via Common Vulnerabilities and Exposure (CVE) and license tracking, as well as enforcement policies in pull requests (PR) and continuous integration (CI) before release.

Note that SCA also stands for Strong Customer Authentication, a regulatory requirement under the European Union’s Revised Payment Services Directive (PSD2), designed to reduce fraud in online payments. Strong Customer Authentication requires at least two of three elements—knowledge (password), possession (phone) or inherence (fingerprint)—for payment validation.

• CVE is a standardized, international dictionary of publicly known cybersecurity vulnerabilities in software and hardware. Managed by the MITRE Corp. with U.S. government support, it provides a unique ID (e.g., CVE-2024-1234) for tracking flaws. It facilitates fast, secure communication about threats and feeds the National Vulnerability Database. There are currently over 330,000 CVE Records accessible via download or keyword search.

Securing Against Ransomware
A “zero-trust” architecture does not implicitly mean “don’t trust anything,” but it does signify an architecture that is harder to breach and is an upgrade to your access control and much more. Zero-trust often demands multifactor authentication at all access points and insists that all connected devices are regularly updated and well-maintained.

Hybrid Cloud Vulnerability
In today’s hybrid cloud world, enterprises struggle to keep track of the slew of certificates managed by different siloed teams and tools. The lack of a centralized view of health increases the risk of application disruptions due to expired certificates. In the AI and open-source era, vulnerabilities in open-source dependencies expose applications leading to unwanted attacks.

Ignoring production usage of open-source packages can lead to breaches and disruptions. Malicious bad guys often weaponize disclosed vulnerabilities quickly, shrinking your remediation window at each cloud source transition (e.g., in hybrid or multi­cloud). You’ll need regular, thorough monitoring to be sure your access control is tight. And you must improve management by limiting access to individual components in the network.

A Flexible and Forward-Thinking Approach
There’s segmentation, and then there’s ZTS (“Zero Trust Segmentation”). You can be certain of some things—the big ones include:

• Cyberattacks are unavoidable: Statistics show this to be true, yet for many organizations there’s a surprising lack of preparedness.
• Cybersecurity mindsets are often outdated: Even with continued investment in perimeter controls, organizations still get breached. When you recognize and accept that breaches are inevitable and start to assume breach, you can focus on isolating them and stopping their spread. ZTS is by far the fastest and easiest way to do that.

ZTS is a flexible and forward-thinking approach that is “AE strengthened” by default. “AE strengthened” refers to key applications, including structural health monitoring using Acoustic Emission (AE) monitoring or, contextually, the bolstering of organizational or technical capabilities (e.g., AE engineer, Advanced Energy—refer to Fig. 2 for example details).

Who’s Responsible?
Essentially, it is the duty of the cloud service provider and end user management to ensure appropriate safety factors are in place and routinely updated before opening the door to widespread public use of cloud-service capabilities. In a future discussion, we’ll look at cloud egress fees and egress payments, an area that’s becoming a bigger part of modern cloud operations.

What can we begin to expect in terms of types of cloud services, including the trends for today and tomorrow? To best understand the kinds of services available in the cloud, readers should have fundamental perceptions of how various services function and articulate performance as driven by the user or subscriber to those particular services.

To review from the many previous articles in this column and TV Tech, these are the key “ready-to-use” applications found in many cloud computing platforms:

• “Infrastructure as a Service (IaaS),” see Fig. 1, for fundamental resources like servers.
• “Platform as a Service (PaaS),” see Fig. 2, for cloud-development environments. 
• “Software as a Service (SaaS),” for ready-to-use applications.
• Serverless computing or “Function as a Service (FaaS),” see Fig. 3, for event-
  driven functions, which focuses primarily on pure code execution.

That said, most of the various XaaS applications are pretty much solid, running in fixed data centers or as cloud-compute services. So, where indeed does “cloud computing” head next?

Again, we move back to those topics covered over the past 18 months or so in my TV Tech columns, which detail the four main types of cloud computing deployment models: public, private, hybrid and multi- and/or community clouds. In the not-too-distant future, the next generation of cloud-compute platforms will need to offer different levels of control, security and cost, especially when looking at operations and utilization from shared public resources (i.e., “you” the end user).

Another division of overall “cloud” models, infrastructures or architecture must further address different types of cloud computing, including deployment and service models. Defining cloud computing models further describes which computing resources are appropriate for which applications, such as server vs. serverless, shared or distributed storage, databases, software and specialized applications, which are, generally, delivered over the internet. This implies companies can utilize these (and other emerging resources) without possessing or maintaining a huge or costly physical infrastructure.

What’s Coming Next?
Trends to expect in 2026 for cloud computing will likely involve the harmonization of techniques for deep AI/ML (artificial intelligence/machine learning) integration. We also anticipate significant shifts due in part to changes in data-center development and related infrastructures, i.e., the expansion of edge computing and the provision of sufficient bandwidth to deliver solutions “to the edge” needed to address expected demands from a variety of devices, mobile and otherwise.

As competition for services increases and the adaptation of existing cloud-centric data centers yields more choices for users, we can certainly expect widespread adoption of multicloud and hybrid cloud strategies.

New developments that will leverage a focus on cloud-native technologies, including serverless and containers, will change the level of I/O requirements. Internet egress (i.e., the on-ramps and off-ramps) will continue to expand as more players enter the cloud and/or AI marketplace.

Quantum Growth
Furthermore, we can anticipate a growing interest in quantum computing—i.e., quantum mechanics (superposition and entanglement)—with qubits to process information, allowing them to be 0, 1, or both simultaneously, unlike classical bits (0 or 1 only).

This level of structure aimed at supporting quantum computing through the cloud will surely require a strong emphasis on sustainability while managing costs via FinOps, or “finance” and “DevOps” defined as a collaborative, cultural practice. Financial management discipline must help organizations reach high business value from cloud spending by bringing engineering, finance and business teams together to make data-driven decisions for optimizing costs, improving efficiency and aligning cloud usage with business goals.

In addition to more depth on each of the cited trends, data litigation and protection are also expected to be important global trends that may be categorized as Data Sovereignty and Compliance. Each of these trends plays on new or expanded capabilities in data-systems design and engineering including—if not especially—enhanced security (including DevSecOps and/or Zero Trust). The data industry will soon need to embrace Intelligent Security (otherwise known as DevSecOps) beginning at the initial software-development process.

Driven in part by the emphasis on AI, expect that an increasing focus on meeting strict data regulations and ensuring data privacy could dramatically change the landscape, if it is not carefully orchestrated on an international basis that bypasses politics or attempts at “global dominance” by any governing body.

Developers and service providers will be expected to provide still “yet-to-be-fully-defined” levels of embedded security into the development pipeline (i.e., DevSecOps) and to adopt zero-trust models for automated, reliable cloud security.

An Intelligent Edge
Artificial intelligence is evolving deeply into embedded systems built on or in cloud platforms, optimizing every facet of cloud operations and security. Of importance is this integration of machine learning and AI at all levels of the computer chain. Among the features we can expect to see are real-time resource allocation, automated scaling (or the resizing of compute resources based upon the load), predictive maintenance and advanced security-threat detection. Promoters believe such changes are crucial to realizing the true needs and value of AI in any cloud computing environment.

Edge computing is a distributed IT approach that processes data nearer to its source (the “edge” of the network) instead of in distant, centralized cloud data centers. This is exemplified by some of the functionality of mobile devices that make choices or provide answers without necessarily being specifically connected (wired or wirelessly) to “the network”—akin to how IDP systems will cache certain sets of predicted replies to the local server rather than rely on every communication sourcing back through the network to a mainstream data center.

AI will be used to make edge devices more intelligent, improving speed, accessibility and endurance for select mobile devices.

By balancing source vs. edge computing capabilities, the provider extends the reach of the cloud to the edge of the network—enabling faster data processing from internet of things (IoT) devices, autonomous vehicles and other edge devices. AI in edge computing through on-device AI inference, on-the-edge AI model training and thin-edge AI was a key trend in 2024. Today and going forward, bringing compute and storage closer to devices, in turn, cuts latency and improves efficiency for time-sensitive tasks. In essence, the edge device needs only to send back certain essential data (information) back to the core “source” data center.

Moving to the Cloud?
Some 76% of businesses moving to the cloud use a hybrid or multicloud approach, according to a May 2025 blog post from managed services provider All Covered. In a broad sense, the primary trends in cloud computing include a rise in platform engineering that aims to manage multicloud complexity, as well as AI adoption as the main driver. Such improvements are properly coupled with FinOps (i.e., cloud-cost optimization trends and tools) and loud sustainability or “Green Cloud” computing trends.

Threat protection is a serious part of any organization’s network, storage or active workspace—especially if you’re invested in the cloud. While the basics are somewhat easy to follow or prescribe, the real-time monitoring and any aggressive corrective actions can be challenging for the individual(s) whose tasks in the organization include both IT administration and security, plus cloud-services management.

That is, it can or will become overwhelming to control without some secondary support or a built-in solution integrated by the cloud provider. Simply expecting an ISP to handle this is not a good answer.

Visibility Resilience and Rapid Response
Being prepared is essential in the current cloud environment. Fig. 1 gives some of the suggestions for this preparedness from a Reactive, Proactive and Adaptive perspective.

Breaches in security can and likely will become unmanageable without appropriate planning and active, real-time monitoring alongside aggressive enforcement of IT policies and practices.

In computer programming, a runtime system or runtime environment is a subsystem that exists in the computer where a program is created, as well as in the computers where the program is intended to be run.

A cloud runtime is the environment where cloud applications and services execute—including the operating system—language support and other necessary software. Furthermore, it is a commercial term (such as Google’s Cloud Run) that ranges from a platform of managed services to the specific software stack required to run serverless functions.

Noting that, a serverless platform is a cloud-based service that automatically manages the underlying infrastructure, allowing developers to deploy and run code without provisioning, scaling or maintaining servers and otherwise is a cloud-computing execution model that allocates machine resources on an as-used basis. Examples of serverless applications include chatbots, task schedulers and Internet of Things (IoT) applications.

Typically, in the serverless environment, any server management, configuration, scaling and billing activities are abstracted from the end user—i.e., the complex, underlying details are hidden and simplified, so the user need only interact with a higher-level, easier-to-understand interface.

Set Up a Unified Platform
One core solution to ensure adequate security precautions is to establish a unified platform that spans across development, operations and (real-time) runtime to provide continuous visibility, protection, simple and continuous detection and proactive response.

Some of the groundwork for building a modern cloud-security program is provided in the following, as outlined by a familiar cloud-support trendsetter who specializes in providing cybersecurity in an era of change and movement toward the cloud, regardless of business model.

One option is to take a proactive approach to cybersecurity (Fig. 2), which is critical to reducing risks and being prepared to address or prevent breaches. Examples follow:

Container Runtime Security and Modeling
One part of understanding and preparing for cloud security involves some of the actual code-centric and application development, such as understanding Container Runtime Security, a key aspect of Kubernetes (as an open-source container orchestration platform, with cloud computing services) and security. This prospect focuses on safeguarding containers (e.g., Docker) during their execution.

When active, operational containers are most vulnerable to malicious activity since traditional security tools weren’t designed to monitor running containers. Runtime security is the only way to secure cloud-native applications at scale and in today’s environments, the use of AI and machine learning will necessitate runtime automating the processes for modeling of a healthy activity built against a model.

“Modeling” is the process of creating a representation of normal, safe behavior—for applications and services, especially when running in a cloud-native environment. Such a representation serves as a baseline to identify and detect deviations or anomalies that might indicate security threats.

When the system has an established model, and then uses continuous monitoring and lets software make comparisons of the runtime activities of applications and services against the established model, teams can then identify and respond to unauthorized actions, privilege escalations and other potential incidents. Major cloud-service providers offer managed Kubernetes services that simplify infrastructure management, allowing organizations to build and run modern applications on a hybrid or multicloud environment.

Build a Strategy for Risk Prioritization
Most security teams lack the time, experience or context to effectively handle and react to a deluge of security alerts that could come from various security tools or practices. As DevOps and engineering teams rely more on cloud technologies to achieve faster development cycle times, they may often prioritize development deadlines over security issues.

While they fundamentally understand the needs and values of security, they often don’t deem it highly important or may end up wasting their time and resources remediating vulnerabilities that seem impactful but don’t present an active risk in production. The organization’s administration should consider utilizing a tool set that correlates threat intelligence, business impact and data sensitivity to provide an accurate representation of active risk.This approach can then help in enabling alignment between DevOps and security teams as to which security issues to prioritize and when.

Avoid centralizing into one workstream that is sandwiched between DevOps, engineering (implementation/operations) and security teams. Avoid manifesting risks and mitigate the risk of a breach, since the functions of each workgroup then isolate (silo) the interdependency of each job function or task.

In the current age of cloud, one where all of your teams’ efforts are likely focused on innovation, it is crucial to leverage a solution (team, outside party or vendor) that can manage every stage of cloud detection and response for you.

To maximize the investments in a security solution, those teams (internal or external) must have the expertise to leverage activities fully. If you don’t already have those teams in place (and properly structured), then look for a solution provider that can function as a partner and who you can rely on to guard your cloud assets 24 hours a day, 365 days a year.

Cloud service providers are expanding and broadening their business—and especially their service offerings—as everything seems to be moving toward AI. To help accomplish this, without rebuilding or replacing existing infrastructures, the new term “AI cloud” becomes a service offering that leverages some of the new and advanced telecom technologies the cable industry has been utilizing for a few years now.

Wondering how new service providers, including the cable industry, are effectively linking AI into cloud computing? This article will explain some of those new advances and how they are being melded into the cloud.

Don’t be surprised if, in the next five years or sooner, we start to see AI and/or cloud computing services being added to our cable systems for consumers, home and business. You’ve likely seen TV ads that exemplify the foundation of such services and may not even realize it is “already” happening.

For the future, one approach that will be needed and become of value to users is the integration of AI tool sets and their models into cloud infrastructures for tasks such as (home or business) automation, model training and data analysis. Such prospects will involve cloud platforms that leverage AI-specific resources, and in turn, can help streamline the AI development life cycle and deployment. Cloud becomes a very significant part of this AI movement.

Several new steps and approaches will provide unique opportunities for the more traditional segments of the media and technology industries and include some of the following issues, changes and conclusions.

Access to Expertise
Cloud service providers are beginning to change their physical data and transmission centers. They are investing heavily in AI topologies for end users and those with business-related requirements in such areas as operations, research and development. As the major cable providers continue to add new service segments (e.g., business connectivity and cellular communications), there becomes a great new need for these media providers to edge towards becoming cloud vendors that will, in turn, provide user access to emerging AI tools and technologies.

These providers will need to expand and provide sufficient compute power and resources, leading to the building of newer and larger data centers with the specifics and needs for AI applications. This further need for more power to support cooling and servers will force these providers to new locations, such as more rural areas where power is affordable and space is not at a premium. Connectivity to end users (customers) will further require additional high-speed (fiber optic) services which, in turn, means a change in physical distribution platforms.

Virtualization
Regardless of today’s core functions, data centers will soon need to be capable of providing new insights employing systems such as virtual cable modem termination systems (vCMTS) and Distributed Access Architecture (DAA). vCMTS is a software-based CMTS, while DAA decentralizes the cable network by moving functions from the headend to the edge of the network, closer to the customer. The vCMTS replaces the traditional, hardware-based CMTS with a software platform running on servers and includes more centralized management and control of the network (Fig. 1).

Some of the early advances made by cable operators and vendors in virtualizing the industry’s access networks and expanding their capacity could become a hybridized means for multipurpose cloud providers who are increasingly moving to virtualize hybrid fiber-coax (HFC) access networks as these next-gen technologies emerge.

Such a change is critical to meeting the challenges of emerging GenAI solutions to deploy and manage the mass data-distribution industry’s long-term health and competitive prospects. Cable service providers are always using new systems technology, such as AI, to mitigate failures. For example, one methodology is limiting no more than just a single fiber deep node issue per 40 customers. A legacy cable hub population of 20,000 to 30,000 customers using a traditional CMTS was the expectation less than a decade ago.

Moving to the Edge
In another way, the models being used by the cable industry are taking key access network functions out of the traditional cable headend and placing them in software running “at the edge” on much simpler and less costly commercial off-the-shelf (COTS) servers. Despite a slow start, cable operators are increasingly moving to virtualize their hybrid fiber-coax (HFC) access networks as the next-gen AI technology emerges as critical to the industry’s long-term health and competitive prospects (Fig. 2).

Benefits for operators include addressing the future needs to meet the capacity and connectivity challenges of a new decade. Whether the systems are to be used for traditional media distribution or as an augmentation to offerings that include “smart features” driven under the “AI umbrella,” speed, efficiency and continual access will be essential to the overall success of the mainstream media and cable solutions providers.

Cloud Platform Changes
Additional related practical requirements and capabilities for cloud platforms headed toward AI services include these core requirements:

Scalability: To rapidly and efficiently—based on fluctuating workloads—expand services without overspending on resources.

Speed and Agility: Cloud-managed abilities to provide rapid deployment and development capacities, allowing for accelerated time-to-market especially for AI applications.

Data Management Enhancement: Cloud platforms must offer robust and stabilized data storage and compute management capabilities. For AI applications, this means the cloud provider must be able to securely store, process and deliver data to customers in large volumes, especially when offering AI model training and insight.

AI Models With Prebuilt Solution Sets: Cloud AI platforms must further be able to offer “pretraining” AI models for concepts such as image recognition, LLM (large language models) and speech-to-text/text-to-speech translation.

Outsourcing Automation/Security
Process automation and optimization is a normal and routine process for AI solution systems; thus, the AI cloud platform providers are tailoring their systems to help streamline tasks that have semi-consistent applications (such as delivery of services or products and order entry prediction requirements). Optimization of these tasks, in turn, will improve accuracy and reduce costs across myriad common business functions.

AI-cloud platforms may also help or be used to improve personalized customer interactions, enable better, more efficient customer service and streamline workflows. We’ve already recognized that the “bot” is the new customer service representative!

Last but not least, AI-cloud platforms should offer robust security features.
Real-time threat detection and (automatic) compliance monitoring will aid businesses in protecting not only their sensitive information, but also in ensuring customers are clear of potential bad actors—both of which may be essential to meet regulatory requirements.

Public Interaction
AI-cloud platforms are becoming increasingly important, and we now see them expanding focus on their core competencies. Businesses—including media-centric enterprises—that must interact publicly can now outsource AI infrastructures and management to their cloud providers as part of a regular service offering. This dramatically changes the core capabilities and pushes new strategic initiatives without a need to retool and invest in additional secondary services outside their core directives.

Many organizations use the cloud to augment traditional data center capabilities. Such complementary uses often include storage, data processing and records management, such as HR or taxes.

According to World Wide Technology, 90% of business organizations will leverage the cloud at some level. Half or more of businesses surveyed said they are shifting IT budgets to cloud, and nearly two-thirds are already at some level of cloud deployment.

Cloud Stall
All these efforts mean workflows are rapidly changing, despite the fact that cloud migration can be complicated. When shifting to the cloud or an adoption initiative slows down (or, in some cases, grinds to a halt), this is referred to as “cloud stall,” which can be debilitating and embarrassing across the board. Therefore, adopting a “cloud strategy” is essential to mitigating the potential impact of cloud stall.

Missing the planning portion of cloud implementation often results in misconfigurations and usually leads to suboptimal cloud deployments. Eventually, the organization will find itself needing to backtrack and correct various errors—sometimes with lengthy downtimes or costly reworks.

Cloud Maturity Model
The modern digital transformation, for business or private purposes, has almost completely shifted to the cloud. Successfully reaching this transformation—whether you’re new to cloud or you’ve already invested in a public, private or hybrid deployment model—could very well depend on your ability to systematically reach and mature your cloud capabilities.

A “cloud maturity model” (CMM) is a framework that helps organizations assess their current cloud capabilities and identify areas for improvement as they adopt and utilize cloud services. This model (Fig. 1) outlines distinct stages of progress, each with increasing levels of cloud usage and optimization; you start out small and add capabilities until the model develops into a mature, functional solution that meets the needs of the business and its activities.

A cloud solution is not an overnight activity; it can and often will require many iterative processes or stages. Technologies available today may become inefficient or even obsolete during workflow development and utilization.

Given the broad reach and capabilities of the varying technology stacks, it can be challenging to properly and efficiently harness the cloud’s potential. Many who “go at it on their own” find themselves making mistakes such as:

1. Lack of Adequate Cloud Security: As the organization rushes to migrate its applications to the cloud, it may find the oversight of robust security measures ends up exposing sensitive data to potential threats, including data breaches, unauthorized access or even service disruptions.

2. No Proper Strategy for Monitoring and Maintenance: Failing to establish proper protocols, including monitoring tools and maintenance procedures, can result in performance bottlenecks, outages and a compromised user experience.

3. Inadequate Backup and Disaster Recovery Plans: Envision an engineer who assumes that data stored in the cloud is inherently secure. The team, therefore, doesn’t implement robust backup procedures. Actions, including accidental deletions, cyberattacks or system failures, could expose the absence of a backup plan, which could result in irretrievable data loss.

4. Insufficient Employee Training: Lack of sufficient and proper training is a common cloud deployment mistake that cloud startup managers often overlook. Administrators and managers might assume a team can rapidly adapt to new cloud tools and services without comprehensive training. Such misconceptions often lead to a significant gap in skillsets in various segments of the business units and staff.

5. Not Understanding Cloud Costs: This is a common mistake and will eventually result in a costly impact. Teams often don’t recognize costs and may overlook the nuances of cloud pricing models, underestimating the financial implications of their deployments. This can result in a cycle of reactive cost optimization, i.e., scrambling to rein in expenses after they’ve spiraled out of control.

6. Rushing Cloud Migration: The excitement around the “cloud promise,” such as improved scalability or flexible agility, could lead teams to not thoroughly assess their readiness for the cloud environment and, in turn, hastily migrate applications with improper parameters or useless end results.

By rushing into cloud migrations, poor results with cost overruns may occur. To mitigate this possibility, engineers and startup founders should prioritize cloud implementation processes using a comprehensive planning procedure, sometimes provided by a third party skilled in the steps and stages of cloud implementation for your business type.

7. Improper Resource Allocation: A common stumbling block for engineers, administrators and startup founders is not providing adequate resources (people, materials, testing, etc.). This could also include the overprovisioning of server resources in anticipation of much higher demand than materializes in reality. These startup errors could risk losing customers and damaging your reputation as users experience a subpar application performance.

Mitigating
“On average cloud deployments don’t go well,” as was quoted in a CIO article in 2015. Back in those days, “70% of the implementations required changes,” “43% of the cloud projects failed or stalled” and “close to half required an increase in budget within six months,” according to THINKstrategies and INetU surveys on enterprise migration to the cloud.

Things have changed dramatically in the past decade and success rates are much higher now. Nonetheless, mitigating common cloud deployment mishaps is essential for ensuring the success, security and efficiency of any cloud-based implementation. Fig. 2 shows some examples for consideration that might make sense when looking to move your organization to the cloud or “re-architecting” an existing platform into a more mature solution set.

Whatever your situation, look thoroughly and plan appropriately for both the immediate and long-term activities your organization must address and seek advice accordingly to alleviate risk and failures.

There are a number of important steps to be taken before your organization moves full speed into the cloud. For example, senior leadership should aggressively focus on the company’s data-management principles, including analytics and security, in concert with how they approach their digital transformation goals.

Modernizing and optimizing your organization’s data management requires going beyond the basics of analytics and security. Criteria for selecting a cloud service provider should be created much as a project leader or project manager would do when structuring a series of steps to achieve success in any new project or endeavor.

First on the agenda should be setting and understanding the business needs of the organization. Are you a small business with 50 or fewer employees? If the organization is at a level of 250 workers or more, it would be considered an “enterprise” business and by now likely already has a serious infrastructure level as it relates to networking, internetworking, financial planning and structure, legal, technology and other mainstream solutions consistent with businesses of this scale.

Services and Solutions
Depending on the scale of the enterprise (or small business), you should thoroughly itemize and lay out how your current or future approach to IT support will be structured, as this is one area that will shift into cloud harmony as a solution provider is selected. Services should include managed and/or co-managed support and a help-desk solution to support employees’ desktops and remote workstation (or mobile) environments. On-site, automated or remote support levels should be defined. An ongoing IT “health check” and cost management/containment platform should be outlined for presentation to a cloud provider or solution’s management resource.

Solutions that should be evaluated when looking at a cloud support and services platform involve the requirements (or not) for a security operations center, how to routinely use and apply concepts such as “penetration testing,” cyber awareness training, managed detection and response (MDR) services, extended detection and response (XDR) services and compliance as a service (CaaS), and include a “cyber risk assessment.”

Other deeper practices that may be available through a cloud service provider could include vulnerability management; identity and access management; mobile-device management; monitoring of dark web and credentials security (Fig. 1); password protection and malware download protection; advanced email protection and monitoring; cloud application (APP) security and firewall as a service (FaaS); SaaS protections; and cyber essentials service through a certification program.

Protection Practices
Cyber essentials certification procedures vary depending on the country, region or locale in which those practices are governed or offered. The scheme is a set of fundamental security measures designed to produce a robust foundation for protecting your organization’s sensitive data and systems.

Key areas of cybersecurity include the implementation of (a) boundary firewalls and internet gateways; (b) security practices and configuration; (c) access control (e.g., “ACLs”); (d) malware protection; and (e) software and application patch management.

When looking at cloud service providers, inquire how or if they offer extensions for any of the above practices (in items a-e) and how they address them.

Broadcast Essentials in Connectivity
Fast and secure access to cloud and data center applications are essential to the organization, regardless of user location. For broadcasters, this became most relevant during the initial stages of the pandemic, when central equipment operations shifted to remote functionality.

Today, an ongoing common challenge is achieving centralized management of multiple locations, including branch offices. For media systems (broadcast news and production), high-speed, reliable bandwidth is necessary for voice, video, data and control/management of data centers and production centers. Support for unified communications has become essential, unlike what news broadcast requirements needed even long before the pandemic.

SD-WAN (Fig. 2) offers a relevant solution that optimizes WAN connectivity with centralized manageability at a fraction of the cost of WAN-only dedicated connectivity. Check that a potential cloud solution provider can support systems such as SD-WAN and where the limitations are.

Service Level Agreements
Check if your candidate cloud provider has service level agreements (SLAs) in place before signing up.

A good cloud service provider will have considered such critical SLA components of a contract and will already have them in place. The contract should define (a) the level of service you can expect; (b) the uptime guarantees; and (c) compensation for service disruptions.

The contract can be refined to include response times and compensation policies in case something goes wrong. An SLA should outline measurable performance benchmarks and should detail response times for support requests and service restoration timelines.

How a provider manages downtime is a metric of how it manages planned maintenance and unplanned outages. Communication protocols and informant timelines related to planned or unplanned service disruptions should be included in any cloud service provider contract.

Administrative Support
Get a checklist that states how the provider deals with (a) performance reporting; (b) resource monitoring; (c) configuration management procedures; (d) and billing or accounting management—each is an important item to check and understand. A provider who cannot address these items to your satisfaction is not the provider for you.

Decision Keys
In summary, the following are key takeaways that should be considered to make an informed decision:

(a) Security and Compliance: Itemize, review and understand that the provider will offer and support a set of robust security measures and that the cloud services will comply with relevant regulations. If you are expecting to work across clouds or international borders, be sure your provider will meet those regulations and report accordingly.

(b) Service Offerings: Evaluate each potential provider and its supported solutions. Ensure the selected provider will offer the services and technical capabilities that support your business needs. If using live A/V solutions, test and be certain the codecs you need are supported and that the data rates meet the needs of your services under heavy loads.

(c) Flexibility and Scalability: Suggest, assess and have the provider demonstrate how they allow you to scale resources as needed. Do they offer customizable solutions? How do they work? How do you assess feasibility and fluidity?

(d) Evaluate the Provider’s Pricing Models: Look at current costs, review expansion costs and look at any penalties and/or costs you will pay to make changes. Model any expected adjustments in advance of signing any contract to best understand the current and future pricing and cost models to avoid unexpected expenses.

(e) Preview SLAs and Reliability: Determine if the provider can offer reliable service with clear SLAs that you understand. Check the extreme ends of your usages (loads, data rates, peak demand) and understand how “overages” impact your cost model—where applicable.

Using Consultants
Cloud consulting services are often available from various companies throughout the industry in which you engage. Most will start with a comprehensive cloud assessment that will present a thorough evaluation of your current needs, available technologies and suggest (or provide) the best cloud-based solution for your business.

Often these are fee-based services that can offer additional services—where desired—such as end-user monitoring and integration, and that will be there on a contract basis to support operations from installation through deployment and turn-up on your system.

Look for a third-party consulting or provisioning organization that specializes in your business’ operational needs (e.g., media operations, streaming, news, production).

Cloud computing administrators and their superiors probably have been asked at least once in their life, “What keeps you up at night?” An AI-overview answer (courtesy Google AI) came back with this answer: “Essentially, the responsibility of ensuring critical data and applications remain accessible and secure within a dynamic cloud environment.” But this “overview” didn’t really answer the question, so this article may help open your eyes to what’s coming.

Fig. 1 summarizes the major points discovered in the research for this article and, not surprisingly, most of the findings from several sources round out to about the same bullet points from 2024 and before. We’ll take some of these points and practices apart in the following segments and see if you agree on the importance of their implementation.

The Security Elephant
The elephant in the room is most likely security. Like many other branches of technology, security is a pressing concern in cloud-based computing—no surprise here, and certainly not contained to just cloud. In general, security has become the biggest issue for all organizations—irrespective of size, revenues or locations (ground, cloud or home).

Refining the security topic a bit brings into play “sensitive” data, which is another major concern. Sensitive data is defined as information that, if disclosed, misused or accessed without authorization, could result in harm, discrimination or adverse consequences for the individual to whom the data pertains (per a simple Google search).

Fig. 2 briefly summarizes the categories of sensitive data for any organization. However, if your organization associates at a government (e.g., intelligence community) level, your people may require a top-secret (TS) clearance, the most restrictive level. TS-data is that information which can cause grave damage to national security if disclosed without authorization.

In this example, performance of your systems (ground or cloud) must meet TS-level criteria, and your administrative people must be reinvestigated for continued eligibility every five years.

Complex Challenges
Security risks for cloud replicate those found in everyday activities with mobile devices, home computing, work compute environments and daily life. Included in the risk-list and prevention methodologies are avoiding phishing emails, forged messages (e.g., fictitious representations) and ensuring strict user access control policies are in place.

Securing data (including its accessibility) is becoming increasingly complex and challenging. Those organizations with poor data management practices, weak network security, little, poor or no encryption methods, and/or a lack of endpoint protection may face significant challenges in 2025 and beyond.

Primary Methods of Protection
Data is one of the most valuable assets to the organization or individual. Learning basic, primary methods for protecting sensitive or confidential data is critical to the organization to avoid potential data breaches or data loss. Data loss can be devastating, often resulting in identity theft, loss of business or exposure of classified or confidential information.

Data classification is a good first step in managing your organization’s information—whether in the cloud or on the ground. Data classification are those processes whereby the user organizes its data into multiple categories within a system to make it easier to access and secure.

Ranking the data by sensitivity to reduce storage and backup costs is just one step. Such policies can greatly reduce inefficiencies and create better safeguards whether for personal or company data. This further helps assess how sensitive data is used, which permissions or accessibility can be administered, all of which increase data privacy and security for third and fourth parties.

Data Protection Impact Assessments
Any time data storage or data processing is involved, it’s important to assess and identify each potential risk before they occur. Data protection impact assessments (DPIA) are active (live) tools designed to help organizations secure their data if they involve significant risk to exposure of personal information.

For the United States, this makes cloud data management more complex if your organization deals with any international (i.e., EU) people or companies. DPIA defines data processing roles within the company, data flow between systems and individuals, and the security policy in the event of a cyberattack.

Encryption and Masking
Organizations operating with highly sensitive data (in the cloud or on the ground) should consider encryption to prevent unauthorized parties from accessing it. Using complex algorithms and ciphers, data can then be protected from being stolen or exposed during a cyber-event. Blockchain is a methodology used to protect authenticity and identification of data alteration. It is often used to protect sensitive media (motion pictures, especially those in production) and in the exchange or execution of contracts.

U.S. military and government entities have used data encryption to transmit and receive any classified communications for decades. For businesses, some cloud service providers may also provide those capabilities for additional fees.

Data masking, similar in practice to data encryption, replaces the original data with fictional data to protect its security. Masking processes are generally for internal use to prevent developers, testers or researchers from accessing sensitive data—thus mitigating potential leaks or breaches by disgruntled employees or human errors (the biggest cause of security violations).

Data masking is sometimes used in the testing process to evaluate patching services/systems, various security protocols, or to involve building new features without using real user data.

Password Protection and Authentication
Most of us are, by now, aware of two-factor authentication (2FA) or multifactor authentication (MFA), which is used by banks, credit card companies and such to protect user accounts and mitigate enterprise breaches.

2FAs or MFAs are the easiest—and most protective of the ata security practices—and while a bit more complicated in a cloud environment, are not beyond implementation with a couple extra steps to access the cloud or the network.

According to ZDNet, Microsoft revealed 99.9% of compromised accounts did not use MFA, and that only 11% of enterprise accounts had MFA in place.

Strong Network Security
This is not a single-solution-only practice. Protecting the network often employs many different security solutions to best protect data from being compromised (i.e., stolen or simply accessed). IT managers must create and manage a secure environment on-prem before leveraging the advantages of the cloud for additional services.

Since the cloud essentially “sits at the edge” of the network, administrators must feel confident that the cloud side of the enterprise meets a minimum set of criteria to facilitate the needs of the organization.

Solutions include antivirus and antimalware software applications, data loss protection (DLP) practices, intrusion detection systems (IDS) and prevention (IPS), proper firewalls, VPNs and endpoint response and detection (EDR). Additional suggestions include network segmentation and secure data removal tools.

Additional Concerns in 2025 for Cloud
Briefly, in addition to a well-defined and properly managed security solution set, the other suggestions to consider when looking into the cloud for enterprise operations include:

• Internet dependency: Given that cloud computing generally relies on internet connectivity, should there be an outage, businesses might not be able to access their data or applications. Providing a hardened alternative (dedicated lines) to/from the access points can be valuable, but costly, too.
• Vendor lock-in: Businesses using cloud computing are ill-advised to become dependent on a specific single vendor, making it difficult to switch providers should something unforeseen occur. 
• Complexity: For businesses that are new to the technology, cloud computing can be complex. Be prepared for a lengthy startup period and engage a solutions architect who is experienced in what you do and with various cloud provider solutions.
• Cost management: Spending can be a significant challenge. Be certain your financial officers are prepared for the sometimes rapidly changing cost structures and “bring them along” in the process.
• Skills: The skills gap is one of the biggest challenges for cloud computing technologies. These individuals are hard to find, sometimes hard to engage and can cost a pretty penny to employ.  

This is by no means a “complete” list of the 2025 and beyond” cloud-computing concerns, but they hit the high points of the topics. If you’re even slightly concerned about how to address these issues, hire a professional to get you through the planning and implementation. You’ll be glad you did.

Many organizations still operate without a defined cloud strategy or even a cloud skills plan to follow. This, among others, poses challenges to the users who wish to take the best advantages of the newest emerging trends of cloud and multi-cloud services.

According to Pluralsight, as much as 70% of organizations struggle to drive “customer value” in the cloud. Of those 70%, more than 50% say they have half or more of their infrastructure in the cloud. And ironically, another 49% say they are “actively moving more of their data” into the cloud.

The question becomes, without the needed resources that are essential to keeping the wheels on the bus going round and round, how long will it be before a potential head on collision with cloud and business be expected? And what can, should or must we be doing to cut the potential risk down?

This isn’t any longer about “migration” (to the cloud), it’s about the “rapid acceleration of and the adoption of many new services” that depend on and leverage the cloud. But a surprising percentage of these leaders don’t have a strategy, plan or approach on how to achieve these targets.

Moving From Tactical to Strategic
Cloud maturity is a hypothesis that takes the steps of implementing the “mechanics” of cloud and moves it to a more focused, lean and effective methodology of making the cloud work without overtaxing the team or the budget. Using an “ad hoc” tactical approach may resolve short-term issues, but it is by no means a methodology for the long term.

In some circumstances, leaders have become so focused on the technology of the tactical approach that they forgot to see the forest for the trees. Prioritization was lost, structure was fragile, and the outcomes were meniscal if any.

Culturalism
Your organization is likely very good at what it does, but sees and hears about the cloud, about AI, about machine learning (ML) and yet can’t see how or why it might be advantageous for the team—likely because no one on the team knows about those terms or how they might fit into their own structure or day-to-day life.

Well, my friends, get ready for a change because those previous three terms (AI, ML and cloud) are well-integrated into this new world and are making huge inroads into the next generation of tactics, strategies and operations right now (not next year or the next quarter, but right now!).

One of the first steps in approaching cloud maturity is to change/shift the culture in the organization. Many (around two-thirds) of these organizations put an emphasis on technical training of their teams, ignoring the elements associated around “cloud native” learning company-wide.

Cloud is a culture with its own language, its own “speak,” its own management, and its own sales approach. Perhaps this is why the whole concept of artificial intelligence is so challenging. The public is asking our government to place limits on things that even they (the rule makers) don’t live, breath or speak about daily. This has got to change, starting on the ground floor.

Cloud Literacy
According to findings used in the research of this article, only about 15% of most organizations’ team members understand or know much about cloud terminology. However, more than 40+% of the IT team members of those organizations surveyed have a sufficient technical understanding of cloud to aid in supporting a transition to the “new world.” A surprising amount of the IT staff and leadership staff possess a high degree of cloud understanding, with almost 75% of the IT staff being either cloud-certified or are learning cloud.

This puts the burden clearly on the educator’s communities and in getting everyone up to speed, just like learning reading, writing and arithmetic. This is priority one!

Policies, Tactics and Issues
Last year (2023) security was described as the one area that continually needs to be improved upon. It was listed as “the number one challenge” and the area with “the largest skills gap.” Most searches on cloud needs will start with security being the top issue. The other positions cloud users and administrators should look to include (see Fig. 1):

• Compliance: ISO 27001 is the international gold standard for information security management. It proves the strength of your security posture to prospects and customers in global markets. Achieving ISO 27001 certification can be time-consuming and expensive.
  
• Performance:
  Cloud performance tuning is the process of optimizing the speed, efficiency and reliability of cloud-based applications and services. It requires a combination of technical skills, analytical tools and best practices to identify and resolve bottlenecks, errors and resource wastage.
  
• Reliability and Availability:
  Availability in cloud computing is a crucial aspect of reliability, focusing on the accessibility of services to users. It indicates the percentage of time that a cloud service is operational and reachable.
  
• Data Breaches:
  Are numerous, avoidable and can be mitigated by following suggested practices in this and other documents related to security and best practices. The largest (top) in the amount of exposed data was the Indian Council of Medical Research (ICMR) in October 2023, where a threat actor using the alias “pwn0001” posted a thread on Breach Forums brokering access to identification and passport details (including names, addresses and phone numbers) of 81.5 million citizens of India.
  
• Lack of Expertise:
  Covered in part in this article, the cloud and IT skills shortage is apparently damaging organizational landscapes and is expected to impact businesses in critical areas such as not meeting financial targets, increased exposure to security risks and digital transformation delays.
  
• Security:
  Four top areas of concern include (1) Unmanaged Attack Surface; (2) Human Error; (3) Misconfiguration; (4) and Data Breach.
  
• Multiple Cloud Management:
  A set of tools and procedures that allows a business to monitor and secure applications and workloads across multiple public clouds. Ideally, a multi-cloud management solution allows IT teams to manage multiple clouds from a single interface and supports multiple cloud platforms (such as AWS and Azure) as well as new tools like Kubernetes.
  
• Portability:
  Portability and interoperability relate to the ability to build systems from re-usable components that will work together “out of the box.” A particular concern for cloud computing is cloud onboarding—the deployment or migration of systems to a cloud service or set of cloud services.
  
• Control and Governance:
  Is a set of rules and policies adopted by companies that run services in the cloud. The goal of cloud governance is to enhance data security, manage risk and enable the smooth operation of cloud systems.

As one can tell, cloud management tools for success are not unlike those found in many IT administration practices. The topics are broad, and the tool sets available from service providers are vast. The best advice this can provide will be to ensure the organization has a structure and an aligned forum inside the leadership to start today on a foundation and a plan to be ready for the “now” in cloud. Seek a consultant or a professional who understands the prospects, needs and has the capabilities to drive forward the cloud maturity needs for the entire organization. l

Karl Paulsen is the retired CTO from One Diversified LLC. He has 50 some years of experience in broadcast media technologies and is a regular contributor to TV Tech in the fields of cloud, media, IP and AI. Look for an upcoming series on AI over the coming months. He can be reached at karl@ivideoserver.tv.

In looking for the most intriguing topics to address related to cloud for 2023, I began by exploring what were the most significant challenges for cloud technologies and found that one of the most repetitively stated challenges related to “security” and maintaining “data integrity” (Fig. 1).  Data breaches remain one of the most significant threats facing cloud computing today. 

What did I find in my search? Most reports predicted that cybercriminals would continue to target the cloud as a means of gaining access to sensitive information. Summarily, the kinds of sensitive information included customer data, financial records and proprietary business intelligence.

Figuratively, most organizations today operate to some degree in the cloud. While employing the cloud simplifies operations in many ways, this comes with its own set of risks that can significantly impact the bottom line for enterprise and similar scaled organizations. From a report published by Lookout, an IBM Security Cost of a Data Breach Report (prepared in 2021), found that “the average cost of a public cloud breach was $4.8 million.” 

The Challenges 
A significant grouping of priorities related to IT initiatives now involve cloud services and tools, automation and DevOps—which continually evolve as leaders seek to unlock new efficiencies from the front office to the back and every space between. The findings of a CyberArk report recently issued suggested that this technology adoption rate will see a 2.4x growth in human and machine resources, which is coupled with a 68% increase in the deployment of SaaS tools for such services.

This surely means that utilizing the cloud for operational activities is essential, as when trying to build out the scale of similar services on-prem (including construction, supporting and managing) is found to be many times more costly. Furthermore, of the many elements incorporated into developing a SaaS environment or their application is that of creating a set of “secure” authentication steps.

Creating identities that can authenticate the human user(s) and/or the machine(s) involved, can be automated in the cloud, which will significantly reduce the hands-on requirements otherwise required for upkeep, deployment and maintenance. The growing number of SaaS activities businesses must address in the digital future will be highly dependent upon these evolving cloud services.

What Can We Expect?
Trending technologies in 2023 included the Internet of Things (IoT), blockchain, artificial intelligence, machine learning, Kubernetes and docker. With many of those technologies already in place and in full use, we can expect other new technologies such as quantum computing, cloud gaming, augmented and virtual reality coming forth in the near term/upcoming years.

What will cloud computing be like in 2024? Expect a nonstop evolution of new capabilities enhanced by consumer growth, automation, virtuality and more.

Despite these advances, the top challenges expected in cloud computing seem to remain almost the same as they were in previous near-term years (i.e., that last three to five years). We distinguish cloud computing as characterized by those processes and components associated with “deploying computing services,” such as servers, storage, software, analytics, databases, networking and intelligence. Such services rely upon deployment, and of operations over the internet, which characteristically offers flexible resources, faster innovation and economies of scale.

Data Security and Privacy 
At the top of the challenges chart (Fig. 2) continues to be that of data security and privacy (including customer trust). 

Not unexpected in this group is the challenge of password security and protection. Try as we might with multifactor authentication (MFA), people still don’t fully understand or recognize the importance of having a secure, unique and protected password. A 14-character, mixed alpha+numeric+special-character password is essential when working within any compute environment, including the cloud. Continually changing your password—while time-consuming—is an effective (and essential) part of maintaining that security.

We note that not all cloud providers can assure 100% data privacy, so users should understand the values in privacy and security protection (see Fig. 3 for validation). Another methodology to protect your data privacy is to routinely install and implement the latest software updates, especially on the network hardware and configure those components properly and fully.

Cybersecurity compliance includes certain compliance processes and ensures that the provider(s) meet industry standards, regulations, legislation—including international policies and procedures. The NIST Cybersecurity Framework and ISO 27001 are both excellent guidelines for the prevention of cyberattacks and compliance. Even if you don’t believe you’ll be “working” internationally, you should still follow such guidelines as data may indeed cross over to those parts of the world without you knowing it.

Multicloud Environments
Given the growing number of cloud service providers, users will be expecting to work amongst more than one cloud platform, even sometimes to support the same applications or activity. A “multiple public cloud services environment” includes services provided from different vendors within one architecture at the same time. For instance, a business might use AWS for data storage, Google Cloud Platform for development and testing, and then Microsoft Azure for disaster recovery.

We also hear the term “multicloud computing.” Fundamentally, there are three main types of cloud computing: public cloud, private cloud and hybrid cloud. Today, using one or more of these is not uncommon. (I discussed multimedia cloud and hybrid cloud uses and values in my October 2021 column, “Evolution of Multimedia Cloud;” my February 2022 column, “Cloud Production for Media,” and December 2023 column on “Hybrid Cloud Choices”.)  

A private cloud is one built, usually by the owner, for its own independent uses and it most likely would be built on-prem. Public clouds have the most familiar and recognizable cloud service naming with provisions from Google Cloud, Microsoft Azure, Amazon Web Services (AWS), IBM Cloud, Oracle Cloud Infrastructure (OCI) and others. 

Each of these public offerings differs in varying ways and can offer hybrid cloud services and migration paths from one platform to another. Be sure to crosscheck the capabilities from each vendor’s offerings when developing a cloud architecture for your uses.

Performance, Reliability and Availability
Interoperability, flexibility and performance are another set of challenges but possibly less expected are the performance and reliability/availability of the services to, from and within the cloud. Transferring large data sets (volumes) between cloud data servers depends upon sufficient internet bandwidth, which is a common problem. 

On the topic of availability—as with any internet service provider—getting to (or from) the host is usually a core “bottleneck” concern that is essentially out of the user’s control. And of course, once “in the (public) cloud” a user is now in a somewhat “hands off” world where the internal architecture of the cloud is something that you can only minimally affect—and are often determined by the SLAs written into the cloud agreement.

What are the Drawbacks?
A more serious and probably obvious challenge will be the lack of knowledge. Finding the appropriate cloud talent is another common challenge when maneuvering the cloud computing environment. 

As workloads increase through cloud dependencies, so do the number of tools available to global users. Enterprises, regardless of size, need strong expertise in order to properly utilize a growing set of tools and capabilities in the cloud. The solution here is to use/hire cloud professionals who have DevOps and automation specializations and experience.

Karl Paulsen is a frequent TV Tech contributor who has been writing about storage, the cloud and media solution technologies for the past three decades. He can be reached at karl@ivideoserver.tv.

Cloud and IT go hand in hand, so as IT leaders, you likely need a comprehensive, clear insight into the technologies that feed both the enterprise and the cloud. Of paramount importance to IT leaders and cloud architects is keeping your technology assets secure, well-governed and cost effective. This is a far cry from where IT was a decade or more ago, whereby IT was pictured more as support for the back office and keeping “the network” functional—as well as supporting the users and workplace.

IT leaders need comprehensive, clear insights into their technology to fuel this evasive and evolving data-driven, decision-making processes, which lead to the best of results. The cloud, while convenient and less involved (compared to an enterprise-size data center) is not without its concerns. This is not to be “negative” about the cloud. Quite the contrary, knowing pitfalls of the cloud can only make your implementation(s) better and less risky.

By looking at known issues, we hope to broaden perspective and set the tone for understanding how, why or why not “in the cloud.”

Lifecycle 
Most products sold to users have some concept of how long it will last. This “lifecyle” is no different for automobiles, appliances and certainly electronics. Like hardware, software also has its own version of how long it will last—sometimes based on the hardware it lives on and sometimes just how long the original equipment manufacturer (OEM) wishes to support it or feels it is worthy of maintaining due to technology changes (usually advancements) or their cost of keeping it alive. 

In IT (and cloud) , there is also a software lifecycle. In this case, the software asset lifecycle is about having your IT resources accounted for, cost-effective and properly employed. Vulnerabilities of the products lead the list of concerns about IT assets according to Flexera’s 2021 “State of IT Visibility Report.” This issue is a chief concern of the enterprise information management and staff. The same issues can be and are extended into cloud practices and, like the “ground-based” enterprise, must be carefully understood, watched and protected. The sidebar provides simple definitions as they apply to software systems and hardware components.

In considering cloud, many look at using a combination of ground-based (on-prem) services and cloud services. This is generally referred to as a “hybrid” cloud service and some feel this is not only important, but it may also be essential to its operation. In this model there is a dual role—that is, here the enterprise must manage its own services (on-prem) and in turn manage its cloud services as well.

Is a Hybrid Cloud the Right Answer?
According to Morpheus’ “Gartner Market Guide for Cloud Management Tooling,” “The requirement to support hybrid and/or multicloud deployments is stressing current enterprise operational processes and tooling that had been designed for their on-premises environment. The main use cases continue to be around cloud governance and resource management as enterprises try to avoid overspending or falling prey to security breaches.”

By definition, a hybrid cloud is a mixed computing environment where applications are run using a combination of computing, storage and services in different environments—public clouds and private clouds, including on-premises data centers or “edge” locations. 

On a broader perspective, hybrid cloud architectures are widespread primarily because almost no one today relies entirely on a single public cloud. Figs. 1 and 2 show the values and benefits of employing a hybrid cloud to your solutions.

The value is that in hybrid cloud solutions you need to migrate and manage workloads between various cloud (and ground) environments. In turn, this allows users to create more versatile setups based on specific business needs. Many organizations choose to adopt hybrid cloud platforms to reduce costs, minimize risk and extend their existing capabilities to support digital transformation efforts. 

A hybrid cloud approach is one of the most common infrastructure architectures of modern computing applications for IT, media, healthcare, and the list goes on. Today, most cloud migrations often lead to hybrid cloud implementations as organizations often have to transition applications and data slowly and systematically. Hybrid cloud environments allow you to continue using on-premises services while taking advantage of the flexible options for storing and accessing data and applications offered by public cloud providers, such as Google Cloud.

The Good and the Bad
On the other side of the coin, there are many reasons why there are plenty of workloads that will never go to public cloud—some of which include: regulatory response, life/safety reasons, subscription vs. permanent cost models, less control over your data security, and of course, you must have good Internet. 

The “if it’s not broke don’t fix it,” and the “one throat to choke (your own),” along with long-term costs of cloud computing being higher, all stack up against the “all in the cloud” model. Users also say some applications actually run better on a local server along with not knowing “where” your data really is (risk of regulations that could impact data retention or recovery, and how much does it take of your time plus the inability to control reliability.

Another not-so-pleasant concern is that every action leaves a trail (i.e., Where’s your privacy?). Many of us grew up thinking that almost everything online was anonymous. Wrong. Connecting to the web generates an IP address. Every website we visit can see that IP address, and others can “see” that information, everything from what operating system we use to the size of our screen resolution.

Nothing you do is private any longer, especially when your interaction requires or expects one to “create an account.” The main point in creating an account is to retain certain data in order to display it again later. (Privacy is lost, even if the website “says” differently. If it weren’t important, why does that site need it?)

Everyone Has a Data Profile
Every detail about us can, and often is, regularly bought and sold; cookies track us routinely. Even if you don’t accept the cookies, there are means to track and trace you. Artificial intelligence now “fills in the gaps” using other resources, such as Facebook, one of the largest repositories of personal information on the planet.

This issue isn’t limited to services with public data (as in Facebook and Twitter/X). Amazon, Google and Dropbox each store different but very intimate details about each of us. Personally Identifiable Information (“PII”) is valuable to you, to businesses and to hackers. Personable data generates a “profile” that now positions you into “classes” or groupings, which now link to other connections and end up being “mined” by organizations that profit from knowing what they know and, at times, exploiting that information for less than personal reasons.

And it’s not just cybercriminals who want your data; countless services and governmental agencies want and collect your personal details too.

HR Concerns
So, think further about your own organization or enterprise data, much of which may indeed include “your” personal data. For good reasons, HR departments take worthy concerns over their employees’ personal information and that you, the employee, trust that data to your employer. 

The same might go for corporate records, contracts and such. Hence the interest and concern over security whereby they may employ technologies like blockchain to protect transactions, contracts and other confidential information.

Types of PPI
Direct identifiers, also called “sensitive PII,” are data sets that can be used to pinpoint you and only you. Quasi-identifiers, also called “non-sensitive,” are those details that can be combined with other quasi-identifiers to “label” you—classify or group you for geographic, domestic or other reasons. Quasi-identifier designations are often used for statistical analysis placing you into group(s) that describe which “you belong to.”

Direct identifiers, as mentioned, are about you and only you. Nothing you share with another person is a direct identifier—that data such as your full name, medical history, credit card details, personal identifiers such as social security or insurance numbers, or your passport number. 

Europe took a hard stand less than a decade ago with its General Data Protection Regulation (GDRP), a European law enacted in about mid-2018). The GDPR concept aimed to protect you (the user) and provided a legal means to have to prove that any collector of said data had actually scrubbed and/or expunged your data from their files, should you request such action. 

Initially, the 1995 EU Data Protection Directive set goals and requirements, which the EU member states were free to interpret its general goals and requirements as they see fit when complying with their national (EU) laws. Essentially, this says that organizations must have a lawful reason for collecting personal data; the amount of personal data collected must be limited to the minimum necessary to complete the lawful purpose; and that data must be deleted once the lawful purpose has been completed. 

The U.S. (country-wide) has been struggling to develop a similar protection, but has essentially gotten no-where; yet California has implemented similar policies.

Where is my Data, Really?
The depth of these kinds of actions apply not only to local servers and services, but extend into the cloud, which leads to an interesting caveat. What happens when the data actually resides in a cloud that is in a non-EU country?

Hence, you can see the concerns for using “the cloud” when you don’t know where the data is or under which jurisdiction that data might be controlled at any given moment. Furthermore, these concerns become more complicated as the cloud providers expand and the resiliency models (data duplication and distribution) grow for faster, more improved services.

So, know your solution set thoroughly before jumping on the “all in the cloud” bandwagon. Keep informed or use a knowledgeable entity to help support and maintain your investments.

When an organization has the ability to bounce back from a disruption, e.g., a power outage, natural disaster, or a cyberattack or ransomware attack, that ability to recover is referred to as being “data resilient.” Typically, data resiliency is framed within a disaster recovery (DR) plan and is best protected when its data is backed up regularly and stored in multiple locations.

Examples of data center (or private cloud) resiliency might include having server power supply redundancy, where each server’s power supply is duplicated to protect from failure of its primary supply; or at the extreme where there is a duplicated server (i.e., a “secondary” server) that is active or in a hot-standby mode and automatically takes over should the primary server fail.  

On a larger scale, data center redundancy is another resiliency option, with the same redundancy concept holding true at the level of the data center facility itself—that is, a portion (or all) of the data center is replicated on site or at an alternative location including the cloud.

Co-Lo Options
Colocation (also called “co-lo”), whereby organizations that support “hot sites” (readily primed to take over in the event the primary site is compromised), is another methodology for digital data resiliency.  

Data center colocation, aka “future-proof” allows your digital footprint to be replicated. In turn, this concept serves multiple purposes but most importantly, it allows the organization to scale faster and easier while providing flexibility and the ability to adapt quickly. It further enables you to take advantage of new opportunities without depending upon other “non-controlled” means (as in a short-term application using the cloud). 

Critically Safe
Your IT infrastructure is everything. Keeping it safe from natural disasters, threats and bad actors falls into the category of “critical services.” Modernizing a hybrid IT infrastructure not only improves performance (as in applications and services) but it allows the organization to unlock innovation, efficiency and cost savings. When critical services are finetuned through practices associated with resiliency, then connectivity is improved, reliability increases and faster responsiveness is achieved.

No matter where the data center is located, where you are or where you’re going, colocation can enable you to adapt to an ever-changing digital landscape. Flexibility constraints are reduced while resiliency is enhanced. Services become instantly scalable, enabling a flexible infrastructure with high-speed connectivity and proximity to partners and carriers.

International Data Corp. has identified some rationale as to why organizations should store more of the data it creates.

First, data is essential to any organization’s efforts to establish digital resiliency. Defined as “the ability for an organization to rapidly adapt to business disruptions by leveraging digital capabilities,” this applies not only to the restoration of operations, but it allows the organization to capitalize on changing conditions. Sometimes framed in part as a “digital operating model”—having an efficient environment enables digital resiliency because businesses are dependent on their data.

Second, digitally transformed companies (those who have adopted a digital operating model) use their data to develop innovative solutions for their future. Companies are quickly discovering that having more data not only helps affirm the direction they are heading, but also creates opportunities to launch new revenue streams.

Today, there are needs in the organization to monitor the pulse of their employees, customers and partners in order to retain a high level of trust and empathy that ensure customer satisfaction and loyalty. Data is the source for this pulse and those entities believe there is latent, potentially unmined value from analyzing both current and older data. 

However, the flip side is that the cost to store more (or all) data holds organizations back from modifying their data retention policies. Media is certainly a part of that “video hording” model, especially news organizations that never know when that one lone, exclusive story content would propel them forward. A deep “glacial” archive that could take hours to days to access is not the answer as the news mandate demands accessibility, given that no one knows when that “special clip” might need to be recovered, fast!

Never Enough Storage
There is a cost to pay for these capabilities, as IDC reported in a May 2022 forecast, which stated: “worldwide, the (circa 2021) global StorageSphere forecast for 2022–2026 would produce a base of 7.9 zettabytes (1 zettabyte=1 trillion gigabytes) of storage capacity at a base install cost of $370 billion.” However you read this, IDC sees total storage in excess of 175 ZB by 2025. Predictors speculated in 2021 that having this much storage “may still not be enough.” Furthermore, there may never be a known ending point for storage to stop growing, and no one knows what those numbers may really be.

Organizations are just now beginning to show a positive ROI on data analytics initiatives, especially with older data—lending to the need for a well-protected and resilient data management agenda. In support of this, a proven ROI (on analytics initiatives) would only amplify the need for storing more data or retaining data longer. Leveraging AI for active search and recovery of older information will certainly depend on the ability to access this older data—the question looming being: “How does this get paid for?” But that is a topic for another time…

The summation of all this data, whether it is created, captured or replicated, is called the “Global Datasphere,” and it is experiencing tremendous growth. IDC predicts that the Global Datasphere will grow from 33 ZB in 2018 to 175 ZB by 2025. Ironically, it is estimated that >30% of the 175 ZB of global data in 2025 will be generated in real time. In 2017, about 15% of that DataSphere was already real-time data. 

To keep up with the storage demands stemming from all this data creation, IDC forecasts that over 22 ZB of storage capacity must ship across all media types from 2018 to 2025, with nearly 59% of that capacity supplied from the hard disk drive (HDD) industry.

Equivalency is mathematically stated as 1,024 ZB = 1 Yottabyte (YB) with a yottabyte of storage taking up a data center the size of the states of Delaware and Rhode Island. A yottabyte is the largest unit approved as a standard size by the International System of Units (SI).

Enterprise Renaissance
The enterprise is fast becoming the world’s data steward… again. In the recent past, consumers were responsible for much of their own data, but their reliance on and trust of today’s cloud services, especially from connectivity, performance and convenience perspectives, continues to increase while the need to store and manage data locally continues to decrease. 

Moreover, businesses are looking to centralize data management and delivery (e.g., online video streaming, data analytics, data security and privacy) as well as to leverage data to control their businesses and the user experience (e.g., machine-learning, machine-to-machine only communication, IoT, persistent personalization profiling). The responsibility to maintain and manage all this consumer and business data supports the growth in provider cloud datacenters. 

As a result, the enterprise’s role as a data steward continues to grow, and consumers are not just allowing this, but expecting it. 

As recent as 2019, more data would be stored in the enterprise core than in all the world’s existing endpoints. (Reference: “The Digitization of the World – from edge to core”: an IDC white paper in November 2018). The diagram above shows the Data Hierarchy, adapted from the SNIA Dictionary 2023.

M&E Jump Start
According to IDC, “mankind is on a quest to digitize the world,” and one of the key drivers of growth in the core is the shift to the cloud from traditional datacenters. As companies continue to pursue the cloud (both public and private) for data processing needs, cloud datacenters are becoming the new enterprise data repository. In essence, the cloud is becoming the new core. In 2025 IDC predicts that 49% of the world’s stored data will reside in public cloud environments.

Not all industries are prepared for their digitally transformed future. So, to help companies understand their level of data readiness, IDC developed a DATCON (DATa readiness CONdition) index, designed to analyze various industries regarding their own Datasphere, level of data management, usage, leadership and monetization capabilities.

It examined four industries as part of its DATCON analysis: financial services, manufacturing, healthcare and media and entertainment.

Manufacturing’s Datasphere is by far the largest—given its maturity, investment in IoT and 24×7 operations, manufacturing and financial services are the leading industries in terms of maturity, with media and entertainment most in need of a jump start.

A Data Explosion
Every geographic region has its own Datasphere size and trajectories that are impacted by population, digital transformation progress, IT spend and maturity, and many other metrics. China’s Datasphere is on pace to becoming the largest in the world and is expected to grow 30% on average over the next seven years; by 2025, it will be the largest Datasphere of all regions (compared to EMEA, APJxC, U.S., and the rest of world) as its connected population grows and its video surveillance infrastructure proliferates. (APJxC includes AsiaPac countries, including Japan, but not China.)

Consumers are addicted to data, and more of it in real time, i.e., active or stored video from entities such as TikTok, Facebook, Instagram and more.  

You cannot hide from the data. More than 5 billion consumers interact with data every day; and by 2025, that number may be 6 billion or more. That equates to 75% of the world’s population. So, by 2025, each connected person will have at least one data interaction every 18 seconds. Many of these interactions are because of the billions of IoT devices connected across the globe, which by themselves, are expected to create over 90 ZB of data in 2025.

The magic just gets more involved. What “magic” you say? And one answers “all the magic”... which infers that the “magic” is still being defined, developed, and experimented with.  

As the title of this article infers, this “magic” consists of elements in Web 3.0 (which claims to empower creatives and users—both—to share in the value they create). The values, summarized in the introduction to Shelly Palmer’s Metacademy, provide businesses (whether new or existing) the knowledge and linkage into communities associated with these new entities including blockchain, NFT, and the metaverse. 

So how does this fit into your needs or interests in cloud-based activities?  A good example is blockchain, which is like “information carved into stone.” Such information (e.g., a contract or copy protected media) is configured as permanent, unchangeable, and ambiguous as to what you’ve written that data into.  Cloud-centric services can properly handle such configurations because the data lives on a distributed network across a large number of computer-based systems (servers) as in a mesh or a peer-to-peer (P2P) environment. 

Because the information is essentially “permanent” one can easily tell if anything associated with that information has been altered. Although you can “add” to the blockchain, you cannot modify the existing data set—thus rendering a protected value that can be recognized for its authenticity for eternity.  

Immutable Cloud via Distributed Data
Blockchain uses a cryptographic hash algorithm as a one-way function which is derived from the original data. Altering that data (even a date-code or GPS coordinate) generates a new unique functional representation, which can be compared against the original authenticated data and signifies that the original data and the current data are different.  

This original data is “hard-and-fast”—otherwise called “immutable”—which means it is unchanged over time and is unable to be changed. Thus, blockchain, in this described context, is protected and cannot be modified. Blockchain is extensible across any cloud service as an inalterable contract that is safe and valid irrespective of where the data is stored or sent to/from.

Sometimes called a distributed ledger, blockchain uses a digital file which is continuously growing via an encrypted transaction (the “blocks”) which are copied to a P2P network of distributed computers. Here, each computer (node) accesses information on a computer that serves information to its clients (i.e., other servers). Nodes on this P2P network create a consensus, i.e., an algorithm which is used to resolve conflicts and to ensure the accuracy of the blockchain. 

Distributed and decentralized ledgers are what power the system. P2P networks are very stable because the information is replicated in a multitude of places, irrespective of the scale of the system. This makes such applications perfect for a cloud-based environment, since the cloud (in theory) can function at any scale.

Hashing as Sequential Records
As the name implies, blockchain is a set of records that are linked—or chained—together sequentially using an index, a series of timestamps, a listing of each of the transactions, a “proof” and the hash (algorithmic set of mathematically derived data) from the previous block.  Alteration of any one block changes the hash of all downstream successive blocks which are unrecoverable or reversable.

A hashing function (Fig. 1) simply takes a variable number of characters (the ”message”) and converts it into a string with a fixed number of characters—referred to as the “hash value.”

Cloud Computing and Blockchain
The blockchain is a set of unchangeable and decentralized data sets (the blocks) which utilize a shared database. In cloud computing the system is orchestrated for delivering computing services.  

It is comprised of servers, databases, storage, and associated activities. To increase data security, cloud computing will use blockchain peculiarities. Due to its scaling capabilities, the cloud can provide on-demand computing resources for blockchain operations.

Secure, Unique, Adaptive and Semantic
Security is often thought of as “the elephant in the room” with most people having only limited familiarity with it. The goal of the new “Prime Web 3.0” is to change that perspective with examples which include ubiquity, decentralization, artificial intelligence, blockchain-security, and connectivity.

Web 3.0 is meant to be a bottom-up design with decentralization, open to everyone, and built on top of blockchain technologies and developments in the Semantic Web (Fig. 2), which describes the web as a network of “meaningfully linked data.” 

The World Wide Web Consortium’s (W3C’s) vision of the Web 3.0 is conceptually about linked data. The Semantic Web employs technologies that enable the creation of data stores on the Web, to build vocabularies and to manage or write rules for handling such data. 

Web 3.0 is empowered by technologies such as RDF, SPARQL, OWL, and SKOS, which has linked data within an internet framework that employs understandable machine-readable data. Web 3.0 is machine-interpretable, structured and interlinked, with open access sets of data repositories utilizing globally edited adaptive information resources composed of unique web resource identifiers for every bit of information. The concept is a bit like a media-based GUID (Globally Unique IDentifier) whereby each table data cell of a table has a unique identifier.

The current World Wide Web is more than sufficient, but as a tool, many feel they need much more from “the Web” (i.e., the original Web 1.0 and today as Web 2.0). The term “Semantic Web” is an extension to the current Web proposed a decade ago by the “father of the web” Tim Berners Lee, known as TimBL, who conceived and developed it in the late ‘80’s while working at CERN.  

Why is the Semantic Web Worth Having?
The current Web of linked documents can present a tremendous amount of human-understandable Web pages from a single search through lexical marching of the search keywords with similar or exact words found in the documents available on the web. 

This information is vast and mostly irrelevant to the user (especially after the first pages of the search result). Moreover, users have to scan through individual search results to make meaning out of its content. Such machines store and present the web documents knowing nothing about their content.  

On the other hand, a Web that can store data in documents that are both human as well as machine readable opens up a whole new experience of human-machine interaction. Allowing computers to understand Web 3.0 content allows one to only submit basic data or answer a single query before an entire transaction is carried out efficiently by a machine.  

Shared Digital Reality
Contrary to the decentralized Web 3.0 and blockchain concepts (the technology behind bitcoin transactions), the rather new and still somewhat confusing term, “the metaverse” is a shared digital reality enabling users to connect with each other, build economies and interact in real time. The metaverse doesn’t care who owns the data or its information.

Web 3.0 brings the internet into the future—noting that we’re still in Web 2.0, which was developed two decades ago and emphasizes user-generated content, collaboration, and social networking. Web 3.0 further allows users to “manage and claim ownership of their works, online material, online personas and digital assets.” 

Currently, many businesses only concentrate on the development and delivery of their goods and services. So the evolving Web 3.0 evolution tends to overcome some of the drawbacks or flaws in the previous Web 2.0 internet era by addressing and administering crucial concerns including data ownership, control and management.  

The “metaverse is a global 3D network of virtual reality worlds” according to some and further to that “the metaverse is a hypothetical version of the internet” which is confined to a uniform, global virtual world using VR and AR headsets in a near science fiction, futuristic working environment. 

Examples include the use of avatars as personal representatives, interchange costs or sales through digital currency exchanges, and complete purchases or acquisitions in its own in-world currency. Users thereby travel aimlessly through the “metaverse” in a means not heretofore used or found practical.

Browsing (the internet) now becomes navigation of a virtual world which mirrors parts of the actual (“real”) world now known affectionately as “the metaverse.” Thus, users now create and enable a new virtual economy, without the transactional limitations of (previously) modern living conditions.

Not to be Confused
Here is where the actuality vs. practicality and the goals of the Web 3.0 experience seem to collide. Gamers understand this new reality—they embed themselves in these interactions which value immersive, interactive, and social platforms to extend their VR/AR/AI experiences without boundaries.

Web 3.0 users and its community have decentralized (i.e., unconnected) ownership and control over the Web. The metaverse, on the contrary, is a shared, digital environment focused on the ability for people to communicate, create their own economy (i.e. the use of bitcoin) and engage in real-time interaction without ownership concerns or knowledge of the “real” owner.

The cloud is essential for this to occur since it knows no boundaries and serves no limitations. It may indeed take decades to see the many wonderful values of the metaverse and its intrinsic differentiations in the evolving Web 3.0. Advancing, aggressive technologies are critical to these foundations… so watch this evolution carefully—they are not easy to fully comprehend, but certainly command an unexpected eye-opening set of changes in our world. 

Cloud architectures may include a few definitions that surround the overall concept of what the “cloud” is as well as “when” or “how” it began. Some have conveyed the expression that “the cloud is really only somebody else’s computer,” which may be a bit shy of reality and more of a tongue in cheek colloquialism. According to the Journal of Accountancy and IBM, the concept of “cloud computing” first arose in the 1950s in the form of dumb terminals connecting to a mainframe computer.

We hear more of the term “cloud computing,” which is mostly related to cloud as a service, and not an architecture. The term “cloud computing” was coined within a 1996 Compaq Computer Corp.’s internal document (by George Favaloro), with the term “cloud” originally linked to the concept of “distributed computing.” In May 1997, Sean O’Sullivan of NetCentric, attempted to trademark the phrase “cloud computing,” but abandoned that effort in April 1999. The term went mainstream at Apple-spawned software and electronics company General Magic in the early 1990s, with even earlier mentions in academic work before that.

The ”where” of the term is purported to be when the British computer scientist Christopher Strachey (National Research Development Corp., London) published an academic paper at the International Conference on Information Processing, which was also about the shared use of mainframes, called “Time Sharing in Large Fast Computers” (1959-06-15) available for download at archive.org. The paper covered several of the initial issues that had to be addressed in the very early stages of compute processing and operations.

When it comes to the origin of cloud computing, the public generally believes that Eric Schmidt, the former CEO of Google, was the first proponent of the concept when on August 9, 2006, at the Search Engine Conference (SES San Jose 2006), he proposed the concept of “Cloud Computing.” In its early stages, circa late 1990s, the cloud was used to express “the empty space between the end user and the provider,” per a brief “History of Cloud Computing” by Keith D. (Foote 2021-12-17). Additionally, Professor Ramnath Chellapa of Emory University defined cloud computing in 1997 as the new “computing paradigm, where the boundaries of computing will be determined by economic rationale, rather than technical limits alone.”

Not Fully Defined
As mentioned earlier, in its simplest of terms, the cloud (operations-wise) is truly the “use of someone else’s computer.” As the evolution evolved into the early 2000s, five key cloud characteristics emerged: on-demand self-service, broad network access, resource pooling, rapid elasticity and measured service. Industry-recognized companies and other authors have also concluded that “a [cloud] solution must exhibit these five characteristics to be considered a true cloud solution.” 

This doesn’t mean it must possess all five characteristics, however, some or more of these topics are anticipated in some structure to represent cloud and cloud computing. A graphic depiction of services in a cloud computing environment is shown in Fig. 1.

Cloud Deployment Models 
We have further established that there are four familiar types of cloud deployment models: public, private, community and hybrid; each quite similar but separated by who or what entity provides those services. NIST also defines these four cloud deployment models according to where the infrastructure for the deployment resides and who has control over that infrastructure. 

Deciding which deployment model you will engage is one of the most important cloud deployment decisions you will make. Cloud deployment models satisfy different organizational needs, so it’s important that you choose the right model to satisfy the needs of your organization. 

Most important is that each cloud deployment model has a different value proposition and different costs associated with it. In many cases, the choice of a cloud deployment model may simply come down to economics. 

Private Popularity 
Although initiated in the 2008-timeframe, “private clouds” were still undeveloped and not very popular—but public opinion began to change once companies like AWS and Microsoft solved many of the public’s security concerns. 

Hybrid clouds began to pop up around 2011—a concept that required an interoperability model be established, which allowed private and public clouds to shift workloads between two (or more) clouds. Business systems lacked the ability to accomplish this until the now ubiquitous three structures—created in part and recognized by Oracle Cloud in 2012—created Software as a Service (Saas), Platform as a Service (PaaS) and Infrastructure as a Service (IaaS) foundations that nearly all clouds employ for their functionality in part or in whole. 

Multi-clouds arose once organizations began as SaaS-developed services such as HR and CRMs, and supply chain management became optimized for cloud services around mid-2010–2020. Multi-cloud let users select the best-of-breed cloud to perform specific services while allowing those independent clouds the ability to interchange data and operational solutions among each other.

Developer-Driven 
Cloud began to shift from developer-friendly to developer-driven practices around the 2016-time frame. Application developers started to take realistic advantage of the cloud because of the tools the cloud had available. 

Services strive to be developer-friendly to draw more customers. Realizing the need, and the potential for profit, cloud vendors developed (and continue to develop) the tools apps developers want and need. Practices that include containers (e.g., Docker, Kubernetes and other open-source capabilities) are driving the resource-needs capabilities, which all clouds—irrespective of the structures—are moving toward.

Applications and Resources
Many are familiar with cloud-computing resources, yet it sometimes comes with the uncertain knowledge of “what” those resources are. Essentially, cloud computing is “a method of delivering computing resources.” First, those evolving cloud-computing services supported the needs for data storage and processing, and eventually moved on to software. Cloud-computing business systems, such as Customer Relationship Management (CRM), have become available instantly and on demand—changing the dimensions of those services traditionally dedicated to mainframe-like computing or application-specific packages provided by Oracle and others in their heyday. 

These low-cost-of-ownership models for cloud computing have gotten lots of attention and are seeing increasing global investment. In times of financial and economic hardship, this approach becomes more affordable on a OpEx (pay-as-you-go) model instead of up-front investing in hardware (CapEx) and fixed-priced applications that strain budgets and become valueless as technology capabilities increase.

From a generalized perspective, cloud computing provides users with implementation agility, lower capital expenditure, location independence, resource pooling, broad network access, reliability, scalability, elasticity and ease of maintenance. Each of these topics will play differently into your business needs. And similar perspectives can be applied to entertainment-based media from an operations (or OpEx) model, but will generally be applied with differing practices and payloads (i.e., media content and not monetary hardware-centric considerations). 

Legacy IT infrastructures can no longer provide those needed services that will allow the organization to remain competitive. Continuing to add expensive hardware in order to maintain massive IT-centric data centers are gradually, if not assuredly, coming to an end. As evidence of this trend, according to IDC, worldwide spending on public cloud services grew 26% in 2019 to a total of $233.4 billion, up from $185.2 billion in 2018 (Oracle’s “IaaS for Dummies,” 5th edition).

Those users who are ready to migrate services to the cloud need to understand and even directly ask their potential cloud service providers some important questions.

Know Your Needs
Fundamentally one needs to understand which types of applications your potential provider would be set up to run. Know and itemize what your (the user’s) needs will be. Are those needs enterprise applications (such as ERP, supply chain management or human resources)? 

Or is the potential cloud provider a technical compute-centric solution provider (such as high-performance computing—HPC or data and/or database analytics). Perhaps your organization’s needs are focused on marketing or web-scaled applications such as ecommerce or mobile? Do you have an explicit need to support social networking or video streaming and delivery services?

Cloud-to-Cloud or Single Source
Be sure you know the nature and best approaches on how the cloud providers’ services match your needs. You may find that more than a single service provider will be required, thus you need to know if there are means to interact with more than one cloud provider—that is “cloud-to-cloud” services.

If you’re in the media and entertainment industry and you plan to provide live cloud production services, can the actual applications you need or will enable be usable in that cloud? What are those costs to implement the ground-to-cloud solutions initially? And what will be the expected usage periods for cost of operations?

Not Always Equal
Noting further that some providers may indeed suggest that their cloud is engineered to support every application, which will, in turn, require a bit more investigation on your part to assure that your needs are best suited in their cloud vs. another’s cloud. Remember, not all clouds are created equal. Fig. 1 provides examples found today in many of the cloud provider services, some of which may be useful in live M&E applications; and other services may need to be created to support the specifics of certain use cases—including live “production in the cloud.”

The next major decision is to investigate the effort, costs and requirements to migrate your applications to the cloud. Again, looking at the M&E model for live production, know what is required to move the live studio or field content into the cloud, including minimum and maximum bandwidth needed, the content compression expectations, and especially the overall set of latencies that will be expected. A live music video concert will have completely different requirements compared to a remote sporting venue or a talking head interview with guests that are half a globe away from the primary cloud’s site. 

Scale, Performance and Control
Another consideration is the scale of the environment. Users or service providers should not be expected to rewrite your (or their) applications in order to fit their cloud agenda. Performance and control are equally important compared with or against the other objectives such as compression and latency. 

Specialized Adaptation
As cloud adoption increases, operating expenses for managing an organization’s IT components also will grow in complexity. This is similar, to a degree, for cloud implementation. Users should engage a cloud provider that is simple and straightforward to manage; minor changes, typical in live production applications, should be easy to alter or augment without having to engage a consultant or third party to make those adjustments. 

Do scripts and updates require a knowledgeable human who knows the cloud’s operating environment in detail or are changes allowed using a simplified set of menus? Furthermore, changes should be expressed in billable dollars—that is, what will it cost to make those changes and for how long will those costs be expected in terms of length of run time or upticks in computer processing costs?

Change is Expected
How automated are those changes going to be and how long does it take to “spin up” those changes? How will those changes impact the databases? Are these system-wide, autonomous or some other factor to functionality going to “upset the applecart” or run the risk of collapse and/or create a reduction in performance? How much “manual labor” will be needed to install, update, adjust or monitor the applications you expect or need to run?

Given the topics expressed earlier, does the primary cloud service provider offer a multi-cloud solution? Can and how are loads distributed should there be a crash or a need to connect across global regions? Is your cloud provider in a competitive mode or just a simple service provider mode? If multi-cloud operations appear to yield better performance capabilities, do both sections of the two cloud providers play nice together or will there be hurdles (costs, egress, performance) that must be overcome that might not yield a net-net benefit?

Cost, Security and Trust
Security is always of concern, whether on prem or in the cloud. Do the expected cloud provider’s security practices allow for harmonization, or do they conflict with each other? Can your potential provider be configured to mesh with corporate security practices? Is zero-trust a mandated practice within your organization and will the cloud provider’s practices align with those policies? Will the security practices be easy to accomplish, or will there need to be overlapping tools employed that could reduce performance, increase complexity or add latency to the live applications?

Costs are always inherently a part of the delivery equation. Pricing models are often confusing and, in some cases, will put additional layers of costs especially when moving data into the cloud and then extracting. Users should insist they fully understand and can predict the costs for all the services they need. 

For example, be sure that at the conclusion of a particular service experience that unnecessary services are terminated and shut off to preclude the continual “meter is still running” impact. But also understand the balance of any “startup” costs that would be incurred as the session starts up or is being configured. In other words, costs can creep in from any corner of the equation or operation—be certain these costs are identified, are manageable and can be mitigated during times when the services are not in active duty.

Contract Considerations
Extensible, contractual costs may help mitigate the larger expenditures, but understand those contract expectations before engaging. If your service does not pan out from a benefit perspective, be certain you can back out of the contract easily and without additional penalties. 

Also, since services are continually adjusted or added to for each cloud provider, you may find that, after six months of services, another new provider now offers similar services at less cost—and you should be ready or prepared to move providers if the overall savings are worth the effort.

Code and Efficiencies
Applications require code and developing the efficiency of that code is paramount to improving the performance of the cloud services employed. Having the right tools and design for the services are keys to ensuring sufficient profitability and use of cloud services. Leveraging the time, cost and resources are the pinnacle to success in a cloud service implementation. 

Like any three-legged triangle, each leg of this “stool” must support other remaining legs. Traditional practices used in the development of these new cloud-based applications will typically be too slow or too cumbersome when deployed as a “cloud-native” solution, so expect to spend time, money and resources in finding new means to deploy your applications and needs. 

Monitoring and continual analysis of the complete solution will become a new factor in the implementation of any enterprise-grade cloud solution. If your new potential service provider offers these tools, learn how and when to use them—they will be necessary to ensure that peak performance for least cost can be achieved and monitored. 

Cloud services (as shown in Fig. 2) mean users need to rethink the business and technology approaches to their business. When reaching out beyond the cloud provider’s traditional services, be certain you ask all the questions of each potential provider. Get the best answers you can and then be sure the contract(s) you engage in meet those expectations. 

Under the cloud umbrella are the familiar variations—public, private and hybrid clouds. The latter, hybrid, is an environment featuring a combination of both public cloud (e.g., AWS, Azure, Google, others) and private cloud. 

By “private cloud,” we mean a “physical location” such as an on-prem data center, a “co-lo” type center, or a managed service dealing directly with users on a contractual basis. Many variations in private clouds exist. Facebook, for example, created its own private cloud for its infrastructure and does not utilize a public cloud to store its data. When scale, security or functional need dictate something outside the boundaries of a public, commercial cloud, users may turn to this privately controlled model.

With those basics defined, an emerging user trend is now developing. Certain specialties now command specifics that demand sets of capabilities that span more than a single service whether public or private cloud. While such a model might be classified as a hybrid cloud environment, there are places and applications where that traditional hybrid cloud implementation expands outside those classic dimensions.

Hybrid Prevails
To place this into perspective—almost no one, today, relies exclusively on the public cloud for computing. Today, a hybrid cloud is categorized as a solution or environment in which applications are running in a combination of differing environments. Reasons include capabilities that cannot be supported exclusively in any public cloud—for example, the production of an entertainment program that demands services at a venue that can’t be done in a virtual cloud platform (live sports clearly needs cameras, sound and human interaction).

Hybrid models are here for the near term, yielding new opportunities to assemble other elements of the production using tools that heretofore were relegated mostly to large frame, custom hardware. Such development is going full speed, but it’s not clear when this will become mainstream. Thus, there are cloud service providers who are focusing on supporting such migrations using specialized capabilities (their own “secret sauce”) that set them apart from their competition. 

More than One Cloud
Enter a relatively newer approach to methodologies that promote multiple methods to reach the users’ goals. This relatively recent trend is referred to as “multi-cloud,” (Fig. 1)

Multi-cloud and hybrid cloud models refer to cloud deployments that integrate more than one cloud. Each differ in the kinds of cloud infrastructure they include (Fig. 2). For example, a hybrid cloud infrastructure blends two or more different types of clouds, while multi-cloud blends different clouds of similar type.

Multi-cloud, according to IBM, is “the use of cloud services from two or more vendors,” inferring two or more different public cloud vendors, yet that doesn’t necessarily mean only “public cloud” providers. The multi-cloud approach gives organizations increased flexibility to optimize performance, control costs, and in turn, leverage the “best of breed” cloud technologies available. 

Services Today 
To set the framework, we need to examine where the technologies lie today. The three better-recognized “cloud services” include Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and Software as a Service (SaaS). Each of these three services utilize some degree of “cloud computing,” usually contained within that cloud under specialties and services that the specific provider has optimized for their particular cloud infrastructure. 

Each of the three services’ platforms employ what we refer to as “cloud computing.” By fundamental definition, cloud computing are those technologies that “make the cloud work.” More expanded descriptions include “a remote data center, which contains the (computing) resources that support those applications” and include physical (and virtual) servers, development tools and networking. Each of these services are provided in a cloud-like environment, i.e., “native cloud.”

Some definitions will include “via the internet” and others will describe cloud computing using a “network of remote servers” that are “hosted on the internet” for functionality, which includes “the management, processing and storing of data.” These remote data centers may be managed by a third-party cloud service provider (CSP) who then charges as a subscription-based service with on-demand billing. To the end user/subscriber, they are likely unaware of which services are provided on which cloud model, i.e., private or public.

Compute Offerings 
Leading cloud providers, as well as cloud-solution providers (e.g., VMware), may participate in multi-cloud solutions for compute. Wrapped in the cloud compute structure is “infrastructure, development, data warehousing, cloud storage, disaster recovery/business continuity” and more. 

Cross cloud utilization is where this multi-cloud “best of breed” concept plays best. For example, artificial intelligence and machine learning may best be practiced in a cloud service environment specially tailored for these operations, given that the scale of servers and storage necessary for a large AI/ML deployment can be intensive. In this example model, deploying a multi-cloud model might involve data acquisition from one cloud provider and compute analytics that might best be served in another.

Benefits gained in this strategy may be realized in different ways by various organizations. Live event or studio production might best be served by vendor A while post production (transcoding, formatting, versioning) might be better served by vendor B. Allowing the user to choose cloud services from varying providers based on performance, function, costs, etc., is now a marketplace-driven agenda compared to past models where you went to a single provider for all those services wrapped into a package—and where you might find certain service capability limitations.

Risk, Vulnerability & Loss 
Other important rationale for the multi-cloud model include risk reduction, reduced vulnerability to losses in data, compromise, unplanned downtime or outages, and more. Additional operation strategies that can leverage multi-cloud include licensing, security, service-compatibility and alleviation of signing up with a single provider who can’t offer all the services under one umbrella. 

Unfortunately, as all these cloud models grow, user/administrators have been facing what is referred to as “shadow-IT,” i.e., “you can’t protect what you can’t see.” The explosion of unsanctioned applications, software and devices is weighing heavily on the CIOs who must develop policies based upon the knowns. By limiting the services to a single solution, you could risk pushing the limits, which you can’t see, touch or know about until you arrive there and find “you just can’t do that here.”

Multi-cloud uses are still evolving. Full feature sets will require additional support and conduits to or from the variations. Inside the media and entertainment technology domains, efforts to establish best current practices for elements like multi-cloud—which requires ground-to-cloud and cloud-to-ground (GCCG) parameters—are moving forward. Harmonization of how well one cloud provider’s services (e.g., software-defined networking for compressed signal production) versus another’s capabilities (e.g., in compilation and distribution) will allow users to experience opportunities in cloud-to-cloud services that can more easily react to the needs of the entire ecosystem.

Navigating our traditional production ecosystems at scale can be complicated enough. Moving some or all that environment to the cloud—well, that’s a horse of a different color. While steady progress continues in moving portions of the production world into the cloud, many elements and workflows need to be solidified before the production world—as we know it today—can be fully satisfied.

Spawned in part from the COVID-19 protocol changes to the entire television and media working environment, “production in the cloud” seemed to move forward rapidly. According to findings from Devoncroft surveys conducted during the pandemic, the importance of “cloud production” has accelerated dramatically to the top of the importance food chain. 

Until COVID-19, the top leader had clearly been identified as the “transition to IP,” which had become the most important industry focus on technology since around 2019.

Depending on IP
The terminology “production in the cloud” has become quite broad and possibly even misleading. To set the stage, we need to set some perspective on the meaning of “IP.” 

Networks, whether on-prem or in the cloud rely upon IP. The anomaly attached to the term “internet protocol” or simply IP, becomes a slightly less contentious topic once the marketing term is removed. 

IP is a definition that must be properly applied to actual applications and in the appropriate space for which the usage applies. In the technology stack, “IP in the studio” (i.e., full bandwidth, uncompressed signal systems) is not even close to “IP in the cloud” except by its namesake, internet protocol. Cloud and ground are not interchangeable. Collectively, there is considerable differentiation in structure, layers and applications—a topic well outside the space available here.

How IP is tailored in a cloud vs. an on-prem (or ground) activity is a governing factor in what can be accomplished in which environment. Users should not assume that the IP applications in a live studio (i.e., SMPTE ST 2110) are the same as IP in other production environments. Take just the latency and timing aspects for live, switched studio environments on the ground—that’s simply not deployable in a cloud-only application or on a widespread commercial offering, at this time. 

Functional Awareness 
These statements are not intended to degrade the value, potential or range of capabilities for the cloud. On the contrary, the perspective is about "functional awareness." 

For example, consistent, broadcast-quality, real-time production in the cloud has opportunistic capabilities when appropriately confined using a finite set of parameters under the umbrella of a select set of usually proprietary products from a limited array of providers. Cloud playout is a perfectly mature example where systems address needed capabilities within the confines of the application they are employing.

Understand the perspective. In-the-cloud live production is simply not the same IP animal when compared to a studio-grade, well-engineered, ground-based managed network fabric using a system such as in SMPTE’s ST 2110 and their associated protocols. Furthermore, one would not expect weekend NFL live football games to be produced in the cloud with the same level of production value we’ve seen accomplished when using two or more mobile units, 12+ cameras, 10–15 slomos, and a set of five M/E production switchers. 

Over time, this analogy is expected to change as evidenced by what we’ve already witnessed during the pandemic. Everything changed and some of those changes have indeed remained.

Cloud Layers
Structurally, elements of a cloud-enabled production environment consist of one to multiple layers supporting various segments of the production ecosystem. Much of this rudimentary cloud workflow is accomplished through software-enabled sets of applications relegated to pools of servers and storage elements. For complete integration, all the other necessary components are a more complicated matter, in similar ways that ground-based (on-prem) implementation use SDI or ST 2110.

One initial requirement to getting content into and through this chain is the preparation necessary to get content from its source (the ground) to the cloud and back again to the intended destination(s). This hurdle is gradually being resolved as owner/operators take new steps to reaching those goals.

None of these capabilities happened overnight. Years of development continue today as cloud service providers ramp up with experienced, technically savvy professionals whose goal is to move the entire media ecosystem into the cloud. 

The technology needed to enable these capabilities include compression rate and transport improvements, synchronization of multicamera images, audio-manipulation in real time, communications and non-AV signaling such as tally or foldback, and a serious level of effects capabilities that model today’s ground-based production switchers in the studio and from the field.

Offline to Online
Emerging development for cloud-service-based production and editorial purposes follow a similar agenda to when linear to nonlinear (ground-based) productions advanced to digital some 25–30 years ago. 

Those who remember online linear tape editing during the 1980–’90s certainly recall the enthusiasm (and complications) of initializing the online editing world into offline for preproduction purposes. Timeline-wise, offline (the predecessor to nonlinear editing) was a multiyear effort, which occurred well before the desktop-editing revolution. Offline, developed as a tool for rough-cutting a conceptual product that then generated an edit-decision-list (EDL), was the goal, which was then exported to the online (usually videotape-based) cutting environment.

Several iterations of this workflow followed, taking the offline editorial version (compressed, usually motion-JPEG video) up a notch, which used better image compression, faster computer processors and high-volume disk storage. Within a few years, that evolution essentially drove the demise of the online linear edit suite and through the harmonization of high-performance disk-based storage, moved into the world of desktop editorial production.

Cloud-production is now following similar steps, but at a much faster time frame.

Parallel Demise
Think of cloud vs. studio production in a similar framework to how the demise of linear videotape for editorial production progressed. None of this happened instantaneously. For a parallel perspective of where the cloud is headed for media production, take another look at how the industry has evolved comparatively.

Today, users first need to get the source content to the cloud. Ingest, at data rates comparable to studio-quality IP is impractical, unnecessary, extremely expensive and not very cost-conscious. Access point costs for high-bandwidth, uncompressed, signal transport in turn limits profitability for any commercial entrepreneur.

By previous comparison, high bandwidth, standard-definition linear video (270 Mbps) in the late 1980s was relatively complex to apply to any network. A T1 (1.544 Mbps) line (popular at that time for transport) was insufficient for real-time SD video. A DS3 (44.7 Mbps) line was cost-prohibitive, even on a part-time basis and commanded at a minimum 6:1 compression ratio just to reach 270 Mbps. Some used an ASI-transport over another carrier, but again, this was costly, harder to find, and required high-priced terminal equipment to convert the SDI signals for ASI transport.

Fast Forward
Today’s HD video (1.5 or 3 Gbps) transports have a similar level of complication when moving uncompressed HD (ST 292M/424 SDI) or even ST 2110-20 IP video into the cloud. Uncompressed doesn’t make economic sense as alternative solutions for smaller-scaled production using proprietary implementations in a compressed signal domain (e.g., NDI or GV AMPP) are appreciable options.

Once in the cloud, software solutions for what were once silicon-centric hardware-based devices must take over. Unless that cloud is a private co-lo and can facilitate placing physical hardware there that can be managed remotely, everything shifts to code-based software systems. 

To make real-time, live, cloud production work efficiently and effectively, each of those previously exampled dimensions need to change the signal structure to accommodate the cost and signal transport infrastructure and to address the needed production elements and services. (See Fig. 1 for selected relative comparisons.) Such changes demand that users compress the video signal to a format (e.g., JPEG-XS), which can be affordably transported from the studio (ground) to the nonlinear compressed dimension and eventually up into the cloud world. 

This new work is ongoing, making very good progress, and is a key topic in the eyes of the media-production entities within the major cloud services providers. We fully expect that given demand by the enterprise user, such capabilities will be forthcoming, cost-effective and viable for many. 

The advice for today is that content production technical professionals need to immediately start exploring what it will take to stay in this loop—as this train has left the station and will reach full speed in a very short period of time. 

Migrating from on-prem to cloud storage can drive the inexperienced user to new sets of knowledge well outside those they would encounter when managing in-house storage. The most apparent differentiators are that in-cloud storage users pay on a per-consumption basis—usually monthly or at some incremental time-based period—and by the type of store they send their data into. Both conditions are set by the agreement established with the cloud service provider (Fig. 1). Scrutinized or scrutiny 

Definition-wise, cloud storage is a service model whereby data in one or more locations is transmitted to and stored in a remote storage system. Cloud storage components are managed by the cloud storage provider. Responsibilities of the storage provider’s solution include maintenance, backup and availability to the user. The latter, availability, is often stipulated by agreement, sometimes called a contract or service level agreement (SLA).

Storage availability has two components: a time-to-retrieve (recover) data factor and a cost-to-store factor. Usually, these two are tied together. For example, if the user puts its data into deep storage and does not depend on that data for routine (daily) applications, then the cost is much lower per unit gigabyte than for sustained, readily accessible, low-latency/fast-recovery storage applications—as in editing, rendering or compositing. 

VIRTUALIZING INFRASTRUCTURE

When storage is based on an infrastructure—which includes accessible interfaces, near-instant elasticity and scalability, multitenancy and metered resources—the storage is usually cloud-based and “virtualized.” Meter resources, also known as pay-per-use, are those offered with potentially unlimited storage capacity resources. Commonly found in enterprise-grade IT environments, this application has moved from a flat-fee (cost-per-month) world to a pay-for-the-use fee structure.

A familiar structure for pay-per-use is Apple’s iTunes model, the “sample for free and pay for what you want to ‘own’” (so to speak). Obviously in cloud storage you will not “own” the physical platform that holds your data, but you will pay for what you use based upon its structure, endurance/availability, its accessibility and the length of time you utilize the storage space.

LOGICAL STORAGE POOLS

A cloud service provider will manage and maintain the data that was transferred to the cloud by the user. Data is usually distributed across disparate, commodity storage systems. Data storage topologies may be on-premises, in a third-party managed data center or in a public or private cloud.

In an on-prem environment, for various reasons, data may be structured in logical pools. In such a shared environment, storage pools are capacity aggregated and formed from various physical storage resources. Pools may vary in size, yield variable but conglomerate performance (IOPS), and provide collective improvements like cohesive management and aggregate data protection. Logical pools are usually provisioned by administrators via management interfaces. A cloud infrastructure generally utilizes this logical form of storage.

RAW AND COOKED—LAKES AND PUDDLES

Storage pools may be equated to a data lake, although there are differences between these two derivatives. The data lake is a storage repository holding a large and vast quantity of unprocessed raw data known as source or atomic data. A giant bit-bucket where data is pushed without specific organization is a form of data lake. Processed data is, although less often, referred to as “cooked” data.

Raw data may not necessarily be called “information” since there needs to be an abstraction or applicational use, accomplished through processing, elevating its worthiness and value from raw to informational purpose. A data puddle is a single-purpose data mart built on big data technology, which is essentially an extract from a data lake.

ON-DEMAND STORAGE

No one likes to pay for things that they don’t need or use. In the cloud, storage services are provided on demand with elastic (increasing and decreasing) capacity as needed, when needed. Opting for cloud storage eliminates the requirements to buy, manage, depreciate and maintain storage infrastructures that reside on-prem, (Fig. 2).

Cloud models have radically driven down the cost-per-gigabyte of storage. However, cloud storage providers, also known as a managed service provider (MSP), may add different sets of operating expenses vs. those in owned, on-prem storage. Added OpEx costs could make certain cloud-based technologies more expensive, depending upon whether or not the equation considers how or when the storage is used. Look for options when selecting and architecting your cloud storage solution. 

THOUGHTFUL ACCESS SHUFFLING

Cloud providers concoct many useful and appropriate names for their various services, utilities and architectures. As of 2021, Azure and AWS each offer 200+ different products and services. Without naming anything specific, the concept of shuffling data storage sets to appropriately price and utilize varying locations in the cloud is a topic growing in popularity.

Optimizing storage costs, without burdening the user, employs automated methodologies that might be termed “thoughtful.” Otherwise called tiered storage, such architectures are not new, especially for ground-based (on-prem) facilities. In non-cloud datacenters, the practice of using high-performance storage for editing, compositing or rendering, where accessibility and speed is essential, is commonplace. This is known as Tier 1 storage.

A large SAN or Fibre Channel data store can be a costly proposition. Not all data workflows need this capability, so less-needed data is typically pushed to a Tier 2 (mid-level) environment. Archive data, which is seldom used or set aside for protection or redundancy (longterm) is known as Tier 3, where it may live on tape or object storage or both. Some may also use Tier 3 as that single occurrence where data is pushed to the cloud (known as deep archive) knowing it won’t require fast retrieval anytime soon.

CUTTING COSTS BY INTELLIGENCE

Early in cloud storage history, moving data around to different data containers (buckets) was accomplished manually and by specific direction of the user. While there may have been cost advantages to deeper storage, the labor effort was not advantageous to those early cloud storage users.

Things have changed over time as varying new services, increasing volumes of data and acceptance (i.e., trust) of the cloud provider by the user continually improve.

Accomplished by learning elements of data usage patterns, unattended cloud storage migration between cloud storage tiers was introduced in the 2017–18 timeframe. By automatically moving data between cloud stores, users and the cloud providers gained new advantages. With simple recognition of access periods for data, cloud providers would shift dormant storage to a “deeper” level without ever having to contact the user/owner of that data.

Users typically authorize automatic migration when signing up or establishing a particular SLA. Actions might be via user interface or APIs associated to work orders or activities. Adding intelligence to this practice has evolved over time. Intelligent tiering gives cloud service providers opportunities to expand archive tiers to other levels, which in turn improves the cost structures accordingly.

APPLICABILITY

When engaging cloud storage services, users should look into automated options and evaluate, via cloud-provided models, the value of tiered and automated storage electives. For large projects extending over months to years, one set of guidelines might be appropriate. Smaller projects utilizing shared data sets across multiple production activities may yield different answers.

Experiment with the numbers, frequencies, volumes and applicability to a particular workflow before going down any cloud storage path. Don’t let the “I’m not paying attention to this” excuse be the reason for higher storage costs, which then provide no improved results.

Karl Paulsen is chief technology officer at Diversified and a frequent contributor to TV Tech in storage, IP and cloud technologies. Contact him at kpaulsen@diversifiedus.com. 

Users of cloud services access a huge volume of capabilities, processes and opportunities when they open an account and start moving data through the system. These users—whether as businesses, individuals or government—are actively engaging in the continually changing digital transformation. This installment examines updates and concepts in what is referred to as a “composable” infrastructure.

Agility is just one of the core reasons for utilizing the cloud and its infrastructure. Other objectives include services that are “friction free,” with control systems that maximize available resources and systems that provide a peak ROI over an infrastructure organized using automated provisioning and intelligent managed resources. Each of these signify basic requirements and rationale for choosing a cloud solution, however, some still wish to have similar functionality in their own managed private datacenter.

Infrastructures composed of compute and storage are built upon a foundational network with onramps and offramps between “ground-based” users and a cloud-provider that interleaves on a fabric that glues together those systems, which vacillate between compute and storage.

Such an environment is being referred to as a “composable infrastructure,” that is, a fabric of Ethernet-based hardware and innovative software layered in a distributed network that claims no limitations (Fig. 1).

Fully integrated datacenter solutions consist of controllers, switches and other hardware steeped in a managed set of software subsystems that respond to the needs and objectives of the consumers. Missing are the traditionally inherent constructs found mainly in on-premises implementations, which serve only the single sets of solutions for which they were conceived to provide.

Conventional architectures for datacenters incur specific limitations based on how the topology of the switches and configurations are architected. For example, while it is possible to build a non-blocking fabric, such as in spine-and-leaf, it usually will require dedicating half the bandwidth in the top of rack (TOR) switches to that fabric, which in many designs and applications, is expensive.

PROTOCOLS, BOUNDARIES AND DISTRIBUTION

System solutions are generally designed to maximize the usage of the available fabric connections (or links). Those designs then must further model for failure possibilities while mitigating the probability of network loops. Protocol boundaries (L2/L3 boundaries) are used as considerations whereby L2 (Layer 2) traffic is confined for communications to leaf nodes, while using L3 (Layer 3) addressing for inter-rack traffic.

Evenly distributing bandwidth across the fabric is a target objective of the L2/L3 boundary. When the datacenter bandwidth cannot be evenly distributed or cannot consume bandwidths uniformly, load balancing will be necessary. Distribution models will be random, driving the network design to some form of equal cost multipath technology (ECMP). Hot spots become the result of uneven traffic in the network, which is managed using techniques such as OSPF (open shortest path first).

LATENCY VARIATIONS AND WORKLOAD SEGMENTATION

Latency is a direct consequence to variations in bandwidth. Latency should be deterministic and will generally be based on a specific workload. In leaf-and-spine, however, these are (generally) not the case.

Multistage traffic transport can take on varying paths when considered end-to-end yielding to latency variances of great proportions and unpredictability. Boundaries, which are often determined by the networking topology’s wiring, can be harmful to workloads.

A strategy to combat these variations is to use workload segmentation—something that the cloud-solution provider has baked into its own architectures given the variability and the multitude of applications expected (and managed for) in the cloud architecture.

CONTROL PLANE/DATA PLANE

Besides the above descriptions of latency, cloud-based bandwidths should be distributed based on workload needs. Bandwidth will be dynamic. Much like the loading required when the compute demand is high, and I/O is low, bandwidth among the compute-serving bare metal devices will be allocated according to the needs of the system. Conceptually this is better managed in a cloud-solution environment than in a firmly structured on-premises datacenter, simply because of cost-to-value unpredictability.

Datacenter topologies will typically utilize a control plane and data plane as their foundations. Often the datacenter’s control plane is tightly controlled. Only a limited set of protocols may be available and are usually not designed to facilitate external, user-defined and dynamic demands—ones that change upon need and are more likely designed for specific operational threads and models.

SDN solutions, for “software-defined networking,” are used to address known limitations by decoupling the control plane from the network itself. Implementations of SDN made in early network architectures used fixed assumptions about how and where traffic would flow in that network. Cloud solutions, however, continually manipulate the flows and adjust (using principles of SDN) to mitigate the usually tightly controlled architectures, thus mitigating the fixed “single-lane/road-like” architectures in a hardwired datacenter environment.

COMPOSABLE FABRIC

A relatively new approach to resolving several of the issues described in these previous discussions looks at addressing data plane, control plane and integration plane (i.e., automation-centric) issues holistically and individually—and based upon dynamic workloads. In a composable architecture, the planes will independently evolve and leverage each other dynamically. Here, the data plane takes on the needs of physical connectivity across the network via its topology model. The data plane, through the application of distributed software, handles the functions of packet (data) forwarding—collapsing and routing the entire system to a single building block.

In large cloud environments, intelligent routing distributes the pathing among the various connectivity components using composable—that is, capable of being assembled, alterable and then disassembled—networking, that found in fabric management components distributed throughout the cloud.

High-performance, advanced fabrics can be found in closed systems to enable high-performance, cluster-based compute architectures. Ethernet-and IP-based networks previously suffered from these capabilities, but no longer. Such new approaches are evolving as cloud-based datacenters take on new foundations to serve the needs of an evolving set of clienteles, workloads and demands.

PERFORMANCE UPSURGES

End-user customers, for the most part, are unaware of these accelerating background systems—they just see performance increases once their data gets into the cloud. Cloud providers can charge for the faster, more powerful services.

Alternatively, building a new on-prem datacenter is further afforded through similar interconnection solutions that leverage logic-based systems for TOR and rack-to-rack fabric topologies. Logical fabrics can be selectively placed throughout the system, thus utilizing composable software-centric control plane solutions (that is, an automated, self-managing set of software subsystems).

New opportunities continue to grow outside the basic SDN controller’s ability to define workloads through server-based APIs or other associate parameters. By employing embedded protocols, equal-cost algorithms let workloads be autonomously managed without requiring the end user to manually manipulate the data services or integration plane. Results are transparent, no workload-awareness capabilities.

Keeping the nuts-and-bolts necessities away from the user is a plus. Intelligent systems can now make monumental improvements in capabilities, driving the fluidity and flexibility of the cloud even higher. 

Karl Paulsen is chief technology officer at Diversified and a frequent contributor to TV Technology in storage, IP and cloud technologies. Contact Karl at diversifiedus.com.

If you’re thinking about shifting storage to the cloud or if expanding the local storage area network (SAN) makes better sense, knowing the impacts of both options is an important study in capabilities and cost management. Even if you think you know what cloud storage might cost, perhaps because you’ve done it before, be sure you’re updated on all the recent facts, variables and combinations of services before making a move in either direction. Cloud services, like SAN storage, is an evolving and frequently changing environment. 

ALWAYS A CHALLENGE

Expanding storage localized at the facility or in a cooperative datacenter has always been a challenge—add in the cloud options and you have much to understand. Selecting the type of storage has a direct reflection on its cost—either way. The storage performance desired is directly related to the volume of work activities and processing speed (the I/O) that you will need for the selected storage architecture and workflow.

For example, if you need to render a large set of animation clips, you will want fast access cache-like storage that can handle the throughput from the render engines without delay or latency. However, the life of that content, once rendered on that storage type, is relatively short compared to the volume size and length of retention for the entire set of finished, rendered files. Workflows will mandate migration of in-process render farm storage to secondary, longer-term storage to keep flows consistently moving from render engine to holding storage.

Costs for local storage are fairly predictable. For the cloud it becomes harder to cost and difficult to plan for. While cloud is fast to create, the total costs can go through the roof if not well-analyzed and controlled from the start.

CLOUD STORAGE COST BREAKDOWN

Cloud storage is a bit like going to the smorgasbord; many items are à la carte. Fees include monthly access, retention time, storage volume and/or the use of inherent capabilities of the store itself. Fig.1 depicts storage services and on-ramps to cloud services (via software-defined WAN) whose services may include storage access or may stand as separate items. 

Use it once and quickly—a short term “put it there and take it back out”—and you’ll have one price. “Put it there and leave it there” for a lengthy period of time—another price. Need rapid access to something you placed into a long-term holding pen (an “archive”)? You can watch previously expected low-budget costs to take off like SpaceX launching multiple satellites one at a time.

Storage types and the services available are common cloud cost factors. Each services menu may have different parameters. Simple, elastic, deep (e.g., cold or “glacier”) vs. high-performance, regional or multiregional physical locations each have different cost factors. Choices become decisions that the user needs to make, each having a differential base on near- or long-term requirements, and, in turn, are reflected in each cost.

Costs vary based on where the storage is located; that is, where the physical data centers are geographically. Global replication and access to the storage, which gives lower latency and another degree of safety, is great if needed. If offices are scattered all over the planet with needs for shared file access, the added costs may not be as painful compared to the work lost in having to wait hours to days for file recovery.

Capacity, based on monthly consumption, definitely affects cloud storage budgets. Should your storage needs be layered (tiered), how much is placed into which bucket and how many copies of each bucket are needed for protection and/or accelerated access will impact costs accordingly. Local, hot or redundant storage will be priced at “so many pennies per gigabyte” for a set amount of storage (e.g., the first 100 TB) on a “per month” basis. Incremental increases won’t see much change, but 10x increases will see a suitable cost decrease measuring around a few points (100ths of cents) per 100 TB.

DATA DELETION—IT ISN’T FREE

Just because you paid to put storage in the cloud, and/or to keep it in the cloud for some period, doesn’t mean you can just “get rid of it” without incurring a fee. Surprise! Sometimes the cloud vendor’s “hook” is to let you use their buckets for a few cents per 100 TB per month. The shocker comes when you either want it back—now; or if you don’t need it any longer and you want to dump it. Expect a bill to dump the storage based on your agreement.

Data at rest fetches one price, but dead data for deletion brings another. If you’ve paid to put the data into deep, extended storage (usually at a much lower cost than rapidly available recoverable storage), the exit strategy will likely be different than if you paid a higher price for rapidly accessible storage, took the data out and never put anything back in its “formerly” empty space.

Contract terms are key to storage costs from time-zero to time-end. Variables are based on volume, accessibility, minimum retention policy and more.

POLICIES, PROVISIONING—AUDIT AND MOVEMENT

Thinly provisioned policies or confusingly complicated requirements should raise red flags. Before signing, verify the service level agreement (SLA); e.g., how it predefines maximum capacities for specific storage instances. There’s no standardization for cloud SLAs, but there can be large variances based upon multiple factors including short-term and long-term associations (Fig. 2). 

Audit your cloud service billing to be certain the policies are followed and that your own needs don’t over or under exceed those requirements. Use available monitoring tools included in the service offering.

Be observant of cloud data movement and minimize it wherever practical. Public cloud storage should not be your primary backup, unless of course all your activities are 100% cloud-based. If using a hybrid cloud/local storage solution, take advantage of data minimization features. Deduplication can save costs by eliminating duplicate, redundant files before the data moves from on-prem to the cloud.

COMPLIANCE AND SECURITY ASSURANCE

Cloud computing widens attack surfaces in multiple dimensions. Know which side is responsible for security controls and assurances related to operations, carriage, storage and protection of the data entrusted to those services. Cloud providers usually take responsibility for physical security, business continuity, network security and disaster recovery (DR). Other security controls are likely the responsibility of the consumer.

What can or cannot be moved to the cloud is dictated by security and regulatory requirements guided by a raft of state, federal and international regulations. GDRP, the SOX Act, HIPPA (in medical) and other rights and acts can be complicated. If you’re not sure or willing to take the steps to figure them out, your better choice might be to keep your data stored locally. Don’t second-guess the value of your assets; be informed.

FACTORS GALORE

Add into the cloud vs. SAN (local) storage stew things like backup, recovery, automation, support, vendor prescriptions and lock-in, plus manageability and reliability, and you have a set of concoctions that just might make you think again about where, how and into “what” you place your most precious data.

You should also evaluate multicloud storage agendas. In this scenario, it’s critical to your business operations to determine precisely what data will be managed and where; how it will be stored; and how that data is transported from ground-to-cloud, cloud-to-cloud and back to ground again. Egress or access points may actually be bundled in the contract or the services might be discounted in order to offset third-party costs for access.

Don’t take strategies for storage, whether on the ground or in the cloud, lightly. Organizations expect consistency, uninterrupted performance and unprecedented reliability at a manageable cost. In an owner-provided on-prem SAN storage solution or an all-in-the-cloud alternative, be sure you know the expectations. Hire an expert outside entity to guide you through the marshlands ahead of making that final choice.

Karl Paulsen is CTO for Diversified. He can be reached at kpaulsen@diversifiedus.com.

If you are a media creation entity, you want to leverage as many potential opportunities as possible to progress through the stages of content creation. One of those is the development of a repeatable set of requirements, focused on specific workflow needs, such that operating in a “routine” mode is more easily achieved.

The ability to customize or replicate those functioning modes is advantageous when running multiple sets of processes simultaneously or independently. Cloud services or on-premise datacenters can provide effective conduits for such opportunities; however, having to reconfigure based upon systemic changes in the infrastructure can be time consuming, complex and require specialized resources especially for routine processes and simple updates.

Provisioning and managing datacenters through machine-readable definition files is the premise of what is known as infrastructure as code (IaC). Rather than supporting direct physical hardware configurations or solutions built on interactive configuration tools, IaC uses computecentric, machine language-based “files” to manage those compute processes.

In a cloud-based solution set, IaC deploys resources using templates, i.e., files that are both human-readable and machine-consumable and that instruct the systems to autonomously configure their functionality virtually automatically. Cloud service providers offer such IaC solution sets as a “built-in choice”—one that a user may use or ignore.

Fundamentally, once a code template is created, the cloud system then takes those code instructions and administers them to the cloud’s resources without any further direct user intervention. Any needs for the updating of called-out resources or for replacing any of the processor chains to achieve goals is handled as a background function and essentially become a “hands off” operation. Fig. 1 depicts the workflow basics from the user through the services, whether in the cloud or in an on-prem datacenter.

BENEFITS TO IAC

Benefits to the applications and uses of IaC include visibility, stability and scalability. Others include security, verification, repeatability and extensibility.

Repeatability, with security, is achieved when the same settings are utilized in each instance of the template. Verification that a given provisioning is stable and ready to run assures that if there is a failure, the infrastructure can be rolled back to a known state without a catastrophic collapse of the components. Operations can continue or be temporarily suspended depending upon the prescribed workflows.

Visibility lets the user obtain a clear reference point to what resources are being used on the account. Should something inadvertently change—such as a wrong setting or an accidentally deleted resource—the stability mechanism utilized in an IaC deployment can help resolve that change using a combination of a current or a previous control management version.

Scalability is equally important. Building a library around reusable code sets lends to the templated model, which can be easily and readily distributed to multiple services globally. Should a particular region need to ramp up for an unexpected deliverable, the closest cloud port could rapidly spin up the services and the infrastructure, based on the templates likely in use at another geographically distanced site. Users would not necessarily need to transport data to an alternate site if the repository can be brought into service in another region.

Fig. 2 diagrams where templates, scripts and policies are held in a common repository, which can be appropriately relegated to each global point-of-presence, i.e., a cloud zone or datacenter. Each of the practices can then be pushed into (or pulled from a repository) to the associated locations and functions.

EVERYTHING AS CODE

A similar approach is the practice of treating all the components of the solution as code. By storing configurations along with source code, in a repository and as a virtual environment, code sets can be cycled or recreated whenever needed. Even system designs would be stored as code in this model.

The everything as code (EaC) model mitigates the need for physical hardware and connections to be installed for each functional activity or task. This obviously would be impractical—and impossible—in a cloud-centric atmosphere. Thus, the previously required specialized physical skill sets and designer practices are transformed into a code-ready environment.

Native cloud applications once relegated to physical modifications have changed the entire cost model, making it easy to spin up a “virtual” infrastructure foundation regardless of location.

FAMILIAR STATEMENTS

Like IaC, an EaC model has similar beneficial statements. Repeatability, including the ability to move from one cloud provider to another, allows for the precise recreation of the environment that can further leverage new feature sets (such as faster performance or less cost per cycle). Tested infrastructure code can be developed, validated at scale (through compute modeling), and then directly promoted into production with the expectations, confidence and assurance it will function quickly and as designed.

The fear, uncertainty and doubt factor (FUD) with respect to server configuration drift is all but eliminated. These new models can literally self-heal themselves to almost any level—including a complete redeployment should a server die or need patching for continued operability. Since the entire infrastructure is developed in code, a mirror image of the system with no crossover dependencies can be spun up the moment an anomaly is detected. Operations just keep running.

INFRASTRUCTURE TOOLS

For cloud solutions to be practical, they need to be dynamic. Infrastructure resources fall into that category. It is akin to having infinite patching and shuffling capabilities without having a human actively manipulating the functionality. Each cloud provider is likely to have their own “flavor” of either IaC or EaC depending upon their feature sets.

Such tool sets allow cloud customers to specify their needed infrastructure resources without having to actually understand (logically or physically) how they are interfaced to one or another. The tools further let the users allocate which resources are needed, the parameter limits (how much for how long), and how those resources should be configured to perform selected tasks and activities.

In platform as a service (PaaS) architectures, users could use a particular platform’s user interface to assign or create resource sets and then manage those resources throughout its operations. In similar fashion, third-party solutions providers would make graphical user interface (GUI) products to manage both cloud and virtual infrastructures and sell those products to consumers. The drawback, however, was these were essentially “constrained” (specific) services that required substantial investment in initial specifications, design and testing before they could be rolled out into service.

While arguably the PaaS practice is practical once configured—and could be likely transported to various other cloud providers—the model was not as flexible. Apps required maintenance and upkeep when a systemic change in the cloud’s internal models were updated. Sometimes the changes impacted the PaaS applications and sometimes the PaaS would “self-adapt.” It was all about the type, use and applications, which were deployed at that time for that particular service.

CODE EXPERTISE EVOLVES

With open access to the virtual “moving parts” of the cloud, IaaS and PaaS models are changing. Where once codebased development was limited to a set of code-level experts, the new era is evolving to integrate machine learning and human-readable practices to become more prevalent and more productive.

Early adopters of cloud services recognized the needs for dynamic infrastructure platforms and are now changing their internal applications to implement their own self-provisioning and configuration capabilities. For those systems housed in private (non-public-cloud) datacenters, once the user/operators learn about processes, patterns, practices and accessibility, they can eventually orchestrate their own server structures, build their own server templates and promote the ability to update running servers without disrupting operations.

Karl Paulsen is the chief technology officer at Diversified, a SMPTE Fellow, and a regular contributor to TV Technology. You may reach Karl at kpaulsen@diversifiedus.com. 

What is the cloud, really? “Special Publication 800-145 (September 2011)” from the National Institute for Standards and Technology says to be considered a “cloud” there are essentially five characteristics that must be included. Cloudspotter’s Journal will explore these five characteristics according to the descriptions provided in the NIST document and including other applicable definitions, applications and terminologies from the consumer/user perspectives.

ON-DEMAND SERVICES

For users to validate a cloud provider’s services there must be the ability to unilaterally provision certain computing capabilities, e.g., server cycles and time, applications or network storage. Services must be deployed in an automatic fashion and without human interaction. The automatic, on-demand services are run under certain scripting commands, using abstraction principles administered under “orchestration” components that are aware of the system resources, locations, as well as the demands by other users or services throughout the cloud environment.

Cloud services may be either thick or thin client platforms. The thick, sometimes called “fat” clients, are those clients that will perform the bulk of the processing—as described in general client/server applications. The “thin” client often refers to the software, which describes the networked computer itself. The thin client is software designed to enable communications amongst the servers.

NIST described the “broad network access” as capabilities available over the network and accessed through standard mechanisms that promote use by heterogeneous thin or thick client platforms. Cloud clients include computers, e.g., workstations, tablets, laptops, smartphones and other mobile devices. Users access the cloud services by connecting to these networked cloud client devices.

The third qualifier enables the doling out of compute resources in a pooled fashion that can serve multiple consumers using a “multitenant” model. Each tenant’s data is insulated from any other tenant’s data—essentially remaining invisible to other operations or tenants.

NIST further explains that the model pools different physical and virtual resources, which are dynamically assigned and reassigned according to consumer demands. Customers are completely unaware of the location of the resources and have no control or knowledge of the exact locations of the provided (or available) resources. Resources, in these cases, include storage, processing, memory and network bandwidth.

RAPID ELASTICITY

Elasticity is the degree to which a system can change and adapt. In cloud applications, this applies to workloads and includes the ability to move from one use configuration to another and then back, or to yet another, depending upon demand. The term “rapid” is added in the NIST profile to include the system’s capability to be provisioned and released, to scale rapidly outward and inward commensurate with demand.

To the consumer, this provisioning appears to be unlimited, allowing services to be appropriated in any quantity at any time. The degree of elasticity then becomes transparent and is governed by the consumer’s (user’s) expectations for a deliverable service (infrastructure or platform) based upon cost, quantity and quality.

MEASURED SERVICE

The last of the five essential characteristics is that cloud systems will automatically control and optimize resource use by leveraging a metering capability at some level of abstraction appropriate to the type of service. Again, these services could be storage, processing, bandwidth and the activity of the specific user accounts.

Resource usage can be monitored, controlled, and reported to the user and the provider. This continues the transparency (of the services) for both the provider and the consumer of the utilized service.

In computer science and software engineering, the principles of abstraction are designed to reduce complexity and ensure efficiency in complex software—as in cloud-based—systems. A common theme of cloud services is that of “abstraction through the processes of virtualization.” Abstraction is described as “existing in thought or as an idea but not having a physical or concrete existence.”

In a cloud-based environment there is a “physical” existence for the compute (GPU, CPU, servers), the network, storage and all the associated software components. All these components are distributed so that in the event any one location should fail or be short of resources for any reason (i.e., for maintenance, failure or updates), the general topology of the entire cloud “network” will pick up those services and continue transparently and without interruption.

In creating shared pools of resources, there is an abstraction mechanism that maps a logical address to a physical resource. Cloud computing networks utilize various techniques, which they develop to create virtual servers, virtual storage and virtual networks. Depending on your definition, virtual applications are also available to any device and provided by the cloud providers.

Abstraction enables the key cloud computing benefits of shared, ubiquitous access irrespective of location or demand by others.

CLOUD SERVICE AND DEPLOYMENT MODELS

NIST describes the service models and the deployment models for cloud computing (Fig. 1). Principally, the three most familiar service models are Software as a Service (SaaS); Platform as a Service (PaaS); and Infrastructure as a Service (IaaS).

SaaS allows the consumer to use the cloud provider’s applications and services, which run in a cloud infrastructure, but prohibits consumers from managing or controlling the underlying infrastructure or its individual applications, with the possible limited exception of users-pecific confituration settings.

PaaS allows the consumer to deploy consumer-created or acquired applications, which are created using the programming resources (tools, languages, libraries and services) and are supported by the cloud provider. In the PaaS environment, consumers do not manage or control the cloud infrastructure (i.e., the network, servers, operating systems, or storage), but the consumer does have control over the deployed applications possibly including the configuration settings for the application-hosting environment.

Consumers, in IaaS, may provision processing, storage, networks and other “fundamental” computing resources where the consumer is able to deploy and run arbitrary software. Again, the consumer does not control or manage the provider’s underlying cloud infrastructure, but they may have control over the operating systems, storage and the deployed applications. Select networking components, such as host firewalls may be controlled on a limited basis by the consumer.

Cloud computing is an evolving paradigm with an unknown actual starting point and an open door into how compute-centric business will prevail.

Irrespective of where or when the term or the practice was created, there remains no doubt that “cloud” is here now and will remain the future for an untold number of users and applications.

Karl Paulsen is CTO at Diversified and a SMPTE Fellow. He is a frequent contributor to TV Technology, focusing on emerging technologies and workflows for the industry. Contact Karl at kpaulsen@diversifiedus.com.

The phrase “the future is now” can be no more reflective than when you consider the merging of the cloud with IP and virtualization. This Cloudspotter’s Journal looks at changes that are reachable and already on the horizon. The merging of IP (in various flavors) with virtualization, and their machines, are creating a new environment that yields suitability for cloud production, continuity playout or consolidation of operational resources.

With the continuing maturity of the latest IP standards for high bit rate—professional media on a managed network—comes the question “Could this be applicable to ‘the cloud’?” If so, just how would you sandwich 1.5 to 12 gigabits per second onto a public highway in an efficient and productive way? Today, this is likely an unsubstantiated perception; yet, there is certainly an assumption given that everything else is headed in that direction, so why not IP?

Such an assumption might be valid if you had an unlimited budget and you were physically parked next to Azure in Redmond with direct-to-cloud fiber connectivity that didn’t need to go any further. Even with that “pipe dream” the security issues and egress management alone would likely put a halt to that nonsense in a heartbeat. Pipe dreams aside, alternatives for high bit rate data movement and manipulation still need to be realized.

CLOUD ORIGINS

What if the cloud went back to what it originally started as? The actual cloud symbol—that puffy squashed circle-like icon—was just a representation of “the network” on a diagram (Fig. 1). The network could, at the time, be described as anything from a short-connected Ethernet segment with files moving on it to a full datacenter or anything in between. It might have been a campus-like topology infrastructure where resources were easily shared; that is, doled out, when or as needed and then reconfigured to serve other purposes once that “compute cycle” effort was completed.

Then consider how a “shared resource” model, as in a datacenter, would be analogous to the cloud. A shared resource model, when appropriately configured, is a cloud; with on-premise cloud versions functioning like a datacenter. Pools of resources are allocated as needed with a central management and monitoring platform overseeing operational functionality (sometimes called “orchestration”). How the services are efficiently and effectively abstracted from the resource pool becomes the challenge.

The “co-lo” (co-located) facility is similar, but they are not generally “on-prem located.” In this model, a datacenter-like structure houses several screened-off areas (i.e., private “cages”) where customers place their own equipment and connect it to the internet or a similar network topology (Fig. 2). Sometimes the co-lo equipment cages are managed independently and sometimes not. The co-lo is usually connected at a main internet point, whose point-of-presence (POP) is usually very close or actually co-located in the facility.

Nonetheless, the co-lo concept appears a bit like a cloud, without the customary sharing of resources found in a dedicated pooled resource environment.

Co-lo datacenters usually service many customers. However, they only become a true “cloud” if the individual sets of components (from the individual customers in their own cages) can leverage one another, pooling their collective resources into a managed, coherent system of compute resources. This doesn’t happen very often, unless it is a campus research center where the cages are interconnected to/from a central processing core that can allocate and then reallocate their functionality to serve varying purposes.

Co-lo models have been applied to broadcast operations, such as in Jacksonville, Fla., where a datacenter approach first served Florida Public Broadcasters and then later added customers from outside the Florida region. This was known in its early days as “centralcasting” and was adapted to yield the “hub-and-spoke” concept for consolidation of television station playout and content ingest. The model was extended to differing formats, but it is essentially a cloud-like central facility, which sometimes shares its resources in a virtual environment and sometimes houses discrete components assigned to specific distant station markets but managed (i.e., controlled) from a single location.

SHARING THE RESOURCES

Shared-resource private-cloud functionality, as found in current and emerging environments, seems better served when the physical architecture of the supporting components (switches, servers and compute topologies) is virtualized to support multiple functions depending upon the day-part needs for that playout region, city or operations center. Such capabilities, promoted by broadcast equipment manufacturers as products that were once built into application-specific and dedicated functionality (e.g., an up/down/cross converter, graphics generating device, render engine) are moved away from purpose-built hardware to software-based products landed on universal “pizza-box” servers.

As FPGAs and other compute devices (such as GPUs and multi-core CPUs) evolve technically, manufacturers are likely to move further away from single purpose black boxes to a shared services model on COTS equipment. Manufacturers already recognize the capability to port functionality into a series of blade processor servers, or such. This model is becoming more viable to end user’s applications (Fig. #3), where a server ‘pool’ is repurposed depending upon facility needs).

For traditional facilities, where the hardware gets only about a 25-35 percent average utilization factor, the concept is to move from a ‘dedicated’ (per device) model to a ‘shared’ (i.e., pooled) environment. The model is now considered virtualized, with the devices (the bare metal COTS servers) becoming multifunctional through a software defined architecture. In the ‘cloud-like’ model, the functionality of a pool of shared compute engines, for example, can be spun up to transcode for a portion of the day and then spundown, reconfigured (in software) and spun-back-up to do another function perhaps as a format converter or MAM processing engine at another time. As more content is generated in a single cloud, the model then expands to provide services to multiple channels (distribution points) and thus, the cost-per-channel goes down considerably.

FULL FUNCTION FLEXIBILITY

Media organizations are now starting to head in the direction of cloud-flexibility and in mixed locations such as on-prem, hybrid and private or public cloud. While in the past, functions were optimized for compressed workflows (e.g., DNxHD, XDCam, AVI formats), the capabilities of high-speed NICs and faster processors have enabled workflows in full resolution, uncompressed formats. Some systems now coming online are utilizing the recent SMPTE ST 2110 IP studio production standards for formats from HD (1.5 Gbps) to UHD (4K) at upwards of 12 Gbps.

Returning to an earlier discussion, a cloud utilization model now makes practical sense whereby owners can create an on-premises, cloud-like architecture that supports full bandwidth (HD/3G or UHD) and can serve many entities locally and afar. Here processing is done in full bandwidth and then distributed (as completed work) in a compressed format suitable for air or CDN/distribution. This spares each local facility from building up services that take up expensive real estate, yet only get used a fraction of the time.

Extending these capabilities to full-spectrum, high-quality production capabilities that support end-to-end services is gaining momentum on a global scale. Remote (REMI or “at home”) productions are now carrying out field production needs that once required a fully equipped mobile (outside) broadcast unit and a crew of dozens to produce sports and entertainment. Outfitting the datacenter to share large sets of servers in a network topology can now raise the utilization factor manyfold, while reducing the overall capital expenditures (and labor costs) that were once required at every station or venue location.

Virtualized environments, whether on-prem or in the cloud, make a lot of sense as facilities consolidate locations and share their resources. Deploying in a virtualized world requires a few more additional learning curves: applications in SMPTE ST 2110 as networked IP, virtual machines (VM), and not to be left out, security. That will be the broadcast engineer/network technician’s future, and that future is now!

Karl Paulsen is CTO at Diversified and a SMPTE Fellow. He is a frequent contributor to TV Technology, focusing on emerging technologies and workflows for the industry. Contact Karl atkpaulsen@diversifiedus.com.

Depending upon who you talk to or what you read, the topic of cloud seems to be one of the most prevalent of all the various technology opportunities—irrespective of whether the application is for science, research, media or data management. Cloud computing is listed by technology experts as “one of the topmost influential [IT] trends of this century;” lending credit to having fundamentally altered the way business and industry reacts to solving its technology needs.

The earliest known use of the term “cloud computing” (in print) is in a Powerpoint document for a business plan presentation at Compaq (founded in 1982) with the title “Internet Solutions Division Strategy for Cloud Computing.” That internal 1996 Compaq document stated that “Internet Cloud will have substantial impact on Compaq customers… across enterprises, small/medium businesses, and consumers and SOHO.” By October 2011, the term “cloud computing” had appeared 48 million times on the internet according to the MIT Technology Review.

Today, the cloud (a metaphor for the internet) and cloud computing impact a great deal more than just business and industry. With the emergence of the Internet of Things (IoT)—which is strongly connected to the cloud—use of the technology now affects aspects of both business and our personal lives in growing ways.

CLOUD, AI AND IOT

So, naturally, one of the higher-level topics of concern is the interaction of the cloud and the IoT, and how artificial intelligence (AI) will play into those activities.

Cloud computing, often simplified to “the cloud” involves the delivery of data, applications, media content (photos, videos, sound) and more to data centers utilizing the internet. The IoT, however, references the connection of devices to the internet and purposely segregates the computer or other straight communications devices—such as smartphones or computers (Fig. 1).

The IoT is generating an enormous amount of data, most of it in the cloud, and as such depends more and more on secondary intelligence (such as AI and ML) for its analysis, categorizing, storing or distributing. According to a two-year-old report from International Data Corp., “By 2019… 100% of IoT initiatives will be supported by AI capabilities.” This further infers that IoT and cloud will be directly connected, and intelligently managed, in order to sustain its growth and purpose long-term.

GROWTH OF CLOUD COMPUTING VS. THE DATA CENTER

Cloud generally provides multiple services; among those are the familiar offerings of compute, storage and disaster recovery—but the other offerings (SaaS, PaaS, etc.) are the “other side of the coin” with dynamics that are continually changing. For example, Infrastructure as a Service (IaaS) is, according to Gartner, forecasted to be the leader in the growth of cloud services, expected to increase in excess of 27% for 2019.

However, cloud application services, such as SaaS (software as a service) remains the largest segment of the cloud markets, revenue-wise, with growth around 18% expected between 2018 and 2019. Those providers who offer both IaaS and PaaS (platform as a service) will lead the pack when considering cloud-computing services for business and industry.

Observers and prognosticators alike seem to be aligning the perspective that, as the cloud matures and consumes market share, the typical data center will, in some circles, “die a slow death.” Perhaps the reality is that the makeup and functionality of the data center will adjust to balance the kinds of cloud services based upon applications, scale or purpose. With that, Gartner predicts that by 2025, 80% of companies will cease operations of the “traditional” data center and move those environments to that of a legacy “holding area.”

PRIVATE OR PULIC?

The argument for public vs. private cloud implementation also continues to ripple throughout the industry. Some believe and promote the private cloud, stating it can “decrease costs, increase efficiencies and offer more security” given that the private cloud lives within the firewalls of the organization. Datacenter operators who place computing resources and its hardware in a unified, software-defined and virtualized unit that is privately managed (for security or compliance reasons) are defined as “the private cloud.”

Effectively, the private cloud can provide similar services to those found in the public cloud, except that the private cloud’s abilities to scale are limited by physical space and capital availability; harmonized with the complexities of obtaining rapid change outside the boundaries of the physical datacenter’s walls.

The public cloud generally offers services that can scale to most any level, can provide flexibility of services “on demand,” but, like private cloud, offer certain challenges that counter the value of the on-premise capabilities of a private cloud.

MANAGED CLOUDS

Private clouds may be “in-house” (i.e., on-premises) or they may be provided by a private, third-party hosting service referred to as “externally hosted” or a “managed private cloud.” In-house clouds are run by the owner/organization, provide a greater sense of security and pose less risk to the organization in most cases. An externally managed private cloud, as opposed to the “public cloud” (such as AWS, Google, etc.), changes the security equation, issues of ownership and risk/worries over what happens should the provider be sold or go out of business.

A rapidly growing segment of cloud is that of the remote hosted private cloud. Here the organization has an ownership-interest in the physical hardware; it is just managed remotely by a third party. This practice was common during the mid-1990s when large co-located (“colo”) centers provided caged sections of the datacenter where customers would land their physical equipment and either care for it themselves or pay others to “keep it operational.” Such spaces were often placed directly adjacent to or in the same building where a major internet POP (point of presence) was located, thus providing quick access and less costly “last mile” services.

CONCERNS FOR CLOUD IMPLEMENTATION

One need not go far, on any search engine, to find the top five to 10 issues and concerns about this continually evolving cloud space. The top three-to-four concerns seem in near total alignment, regardless of which report or article you read.

The number one concern is that of security, often coined “the elephant in the room.” Ironically, until only recently (circa 2018), the number one challenge for cloud was the “lack of resources and expertise” needed to design, deploy and manage the services. Today, that has shifted, moving security to the top and the latter down a notch or two in the concerns stack (Fig. 2).

Depending upon where you put the expertise/resources availability concern, the next in the group of issues for cloud is that of cost containment and management.

It is generally agreed that cloud computing can save businesses money, but the long-term verdict remains open. Organizations don’t need to put huge investments into hardware that then must be recycled every three-to-five years as the obsolescence factor raises its ugly head. The ability, in a cloud environment, to easily ramp up processing capabilities without additional fixed cost investments in hardware continues to attract the customer to this environment. Pay-as-you-go models are further promoted by most public cloud providers; however, the predictability of user need (i.e., on-demand) together with scalability (i.e., growth balanced with requirements for services) sometimes makes it difficult to pre-define costs or predict actual total costs of operations.

Other risks to the cloud services environment include governance, control, compliance, performance and the rapidly changing capabilities of the various cloud service offerings that put a risk to engaging in any long-term contracts that may become stagnant compared to alternatives.

Some feel further that full scale applications should be spread across multiple clouds, driving the cost and deployment equations even higher. So is the answer still “build you own private cloud,” or do you go all in with the cloud and cross your fingers that, long term, you’ll fulfill your needs objectives and come out ahead in the overall race for success at the lowest cost? Time will tell.

Karl Paulsen is CTO at Diversified and a SMPTE Fellow. He is a frequent contributor to TV Technology, focusing on emerging technologies and workflows for the industry. Contact Karl atkpaulsen@diversifiedus.com.

A prologue and a handover: It is with great pleasure that I am taking the reins of the Cloudspotter’s Journal from longtime friend and fellow TV Technology contributor Al Kovalick. With the help and support of the TV Technology editorial staff, it is my hope and goal to continue Al’s vision of this technology sector. I will continue, with alternate issues, to cover topics in my long-running Storage Technology column.

Cloud service-provider offerings continue to grow rapidly, among which are those for disaster recovery (DR) services. Disaster recovery practices began long before the founding of the term “cloud”—so this is nothing new to industry and business. Many methodologies have evolved over the years, beginning with making copies of data to floppy disks or cassette tape, which were then stored in an offsite location. As data grew, DR practices migrated to massive robotic-managed tape libraries that also copied the data to magnetic media for storage at alternative, protected locations.

The state and practices of DR are still evolving as our industry finds that the total costs and upkeep for on-premises tape—optical or other mediums, which are still quite prevalent—keep growing in sync with storage systems and the data sets themselves. Steadily these practices are moving to the cloud; with that being any of the cloud-flavors from private onsite systems to third-party managed offsite solutions, or any of the various public cloud solutions.

The driving forces moving DR to the cloud are many, and the cloud services available to support DR are just as vast. Historically, DR meant finding an alternative location to store data or operate from when or if disaster struck. Our industry realized that the planning and budgeting for a second site to either operate from or store and maintain critical operational components had, until recent times, been addressed by putting those assets (both physical and soft) in another building owned and operated by the enterprise.

Alternatives to the “owned and operated” method include third-party resources (e.g., Iron Mountain), a “co-lo” (co-located) data center also operated by a third party, or by completely replicating operations at another purpose-built site, which might be shared with others or kept in the dark until needed. Those, and variations on those solutions, now have yet another equally effective solution with a pay-as-you-go approach that is essentially a hands-off model once up and running. That model is today’s cloud.

ENTERPRISE RISK MANAGEMENT

Risk to the enterprise is one of the highest priority business continuity issues an organization can face. Data has become the heart of most organizations, whether in the trade and commerce world or in the media and entertainment space. Beyond direct security issues (i.e., the compromise of the enterprise’s data), the loss of its assets—as data—ranks a close second.

To protect these business assets, most organizations will, at the very least, adopt relatively simple and often inconsistent methodologies for media asset backups. The simpler approach is no longer wise or practical in today’s business environment. Scenarios implemented for data protection and business continuity are usually left to the IT department. In the past these methods may have consisted of linear data tapes managed by a third-party software-based backup solution which replicates, in a compressed format, the data added to the storage drives since the last full backup.

Today, given the huge volumes of data being accumulated, irrespective of the business model, this is a cumbersome solution steeped in manual steps that can easily overwhelm IT administration roles.

Automated, unattended methods are becoming the norm and those solutions are best implemented using cloud services technologies.

PROTECTING THE ASSETS

For any IT department charged with protecting the company’s data assets there should be some fundamental common practices employed to provide sufficient levels of data protection enterprise-wide.

Disruptions caused by an inability to get at its data, wherever it is, can cost thousands of dollars per day, per hour or per minute. If you’re a content distributor, the loss of some materials could cause irreparable harm, especially if it is a highly promoted program with a high dollar revenue return expectation.

To prevent such disruptions, IT departments should maintain an updated, tested and monitored DR solution with verifiable backups that can be executed rapidly—regardless of how or where the backups are executed or located. If the organization prefers to use conventional “digital tape” methodologies, at least one, and preferably two, regular sets of “full backups,” augmented by incremental backups, should be maintained.

DR CANNOT BE IGNORED

When it comes to a serious data compromise that causes a near total loss of assets, the disaster may be irrecoverable. Statistics indicate that 43 percent of businesses that experience a disaster will never reopen (per VMware, “Disaster Recovery: Any way you want it,” McGladrey and Pullen).

DR requires a prescribed level of labor and services, which is often incomplete or sometimes ignored. Besides having the data sets replicated, all your applications, configurations and operating systems should be included in the overall DR processes. Traditional DR comprised of setting up and maintaining a separate site, is expensive, complicated and time consuming. And the best test of whether your DR solution is adequate is based on how quickly the organization can fully recover when or if the requirements present itself.

Questions to fully understand include: When disaster strikes, who handles the reconstruction of the network and the re-enablement of the protected data? Do the administrators have the capability to accurately estimate the time to partial and then execute a full recovery?

Does the organization have a simulation or test drill of process to identify gaps, holes or unexpected issues? How often is this simulation tested and does it have a means to grow from (i.e., expand as services expand)?

These previously described practices are fraught with complicated and often impractical implementation issues. In the case where the facility was heavily compromised because of an extended power loss, fire, forced evacuation, physical vandalism or even ransomware, then the DR process—no matter how complete or well thought out—is valueless if there is not a secondary data center or technical site available from which to operate from.

Today’s business environments mandate that alternatives be developed. Looking to a cloud solution can be a game saver for any organization.

Cloud-based disaster recovery practices emerged well over a decade ago. As the sophistication of cloud technologies expanded, so did the viability of DRaaS or “disaster recovery as a service.”

DRaaS, in the cloud, can be a standalone provision and may be configured for near-instant recovery. In similar fashion to the ACO switch (automatic change over) on a broadcast master sync generator—a fully engaged, standby-ready DR system can essentially transition to an operational environment nearly seamlessly.

DRaaS typically will run in your own private-cloud or in a third-party private-cloud. DRaaS may be a part of a number of other cloud-based services already utilized by your organization at a local or enterprise level. If the organization is a playout or content prep center that routinely depends upon the cloud for other services (e.g., ingest collections, transcoding or preparation for CDN or OTT), then linear playout from a master control could be switched to provide the DRaaS option simply by a manual or automated command.

FULL-TIME MONITORING

For third-party DRaaS operations, monitoring of the services should be provided on a 24/7/365 basis. Under a managed service contract, DR execution can be manual or automatic and is usually based on rules governed as a BPM (business process management) solution set. Pricing is based on the platform count, what is protected, if there is long-term deep asset archival included, and on the amount of storage needed and accessibility to the storage (data) itself.

Services can include straightforward IP backup, tape alternative data protection, on-site and offsite backup, and replication of data to a public cloud if part of the business plan.

DEVIL IN THE DETAILS

Understanding the details of a complete implementation requires levels of analysis outside what many organizations typically do—and the analysis methodology is outside the scope of this column. Employing an outside entity that offers the bridging technologies to address DRaaS are well worth the added incremental costs.

IDC Research recommends that DRaaS suppliers be ready and capable to help potential DRaaS clients with their other “DR and business continuity requirements.” Such assistance includes helping with risk analysis, migration planning and process development. Other aids and benefits include those summarized in Fig. 1.

Nonetheless, the possibilities embedded into cloud services for DRaaS are endless, making it an intelligent choice for your data assets’ future.

Karl Paulsen is CTO at Diversified (www.diversifiedus.com) and a SMPTE Fellow. Contact Karl atkpaulsen@diversifiedus.com.

A few years ago, the thought of running a TV channel’s infrastructure on a public cloud was considered abhorrent to many broadcasters. The cry was, “not reliable, not secure, no direct control.”

Brinton Miller

In 2017, things have changed with the likes of Discovery, Disney, Fox, Hearst Television, PBS and others making big cloud moves. Discovery has recently announced they are moving their worldwide channel signal chains to the public cloud. This is big news for our industry and is a proof point of cloud acceptance.

Below is an interview with Brinton Miller, senior vice president of technology strategy and architecture for Discovery Communications. I have known Brinton for many years and have followed Discovery’s move to the cloud.

Al Kovalick:Please tell us about Discovery’s channel empire.
Brinton Miller: Discovery Communications offers a portfolio of premium nonfiction, sports and kids programming brands, reaching 3 billion viewers across pay-TV and free-to-air platforms in more than 220 countries and territories. Our programming [like “Deadliest Catch”] is supplied across hundreds of linear channels worldwide.

Kovalick:What are some of your motivations for moving Discovery’s broadcast workflow chains, including playout, to the cloud?
Miller: Eighteen months ago, we started looking at our global media infrastructure and started thinking about what’s next. We had multiple facilities around the world all running aging playout infrastructures. So, planning for a refresh we needed to ask: “Where do we want to be in a few years? Will any infrastructure we build today meet our future business needs, some of which are unknown? With the speed of technology change, does it make sense to spend the next three years building data centers around the world?”

It quickly became clear that we needed to move to a software-based environment and we wanted to build it in a public cloud. We needed to normalize our technology stack so the tools and systems we use to launch linear products were the same as for our nonlinear products.

“Deadliest Catch” crew members of the Cape Caution emptying a crab pot.

Kovalick:How do you put a price on cloud agility when you do an ROI analysis for new infrastructure?
Miller: Agility is a bonus and not a cornerstone of our financial analysis. That said, a cloud-based channel can be built from scratch and be on air in 20 minutes. Sure, content needs to be prepared, but building the on-air chain is fast.

This compares to about four months using current on-premise methods. Agility allows Discovery to quickly deploy pop-up or digital channels to meet new business needs.

Kovalick:What is your legacy equipment utilization across ingest processing, asset management, file delivery, QA, playout and other workflow components?
Miller: Let’s just say that it is nowhere near 100-percent utilization as measured across a 24-hour/day week. Cloud provides the ability to pay for what we use versus building to a ceiling that will ultimately be a bottle neck for the business.

Kovalick:Did you encounter an internal army of naysayers when your technical staff made the cloud transition proposal?
Miller: Not at the management layer. If a cloud-native competitor is doing this, why can’t Discovery? We have a great culture at Discovery that embraces change. There will always be people that take a bit longer than others. However, for the most part the team embraced the idea.

Most of the negative comments came from incumbent equipment providers. They want to sell us boxes and we were not interested in that discussion. I do think that the vendor space is a bit more open to discussing now 18 months later.

Kovalick:What are some of the challenges to the cloud move?
Miller: Licensing of software can be problematic. Most vendors are in the perpetual license business and this model is aging fast. We wanted “per-widget” consumption models whenever possible.

That said, we needed to accept it for some cases. It was also a challenge to duplicate the real-time signal flow portions of video workflows on a cloud platform not specifically designed for video, but we did it and with the same reliability our business requires.

Kovalick:Did you consider starting with only a few cloud-based channels then migrating others later?
Miller: If you are considering migrating to the cloud, don’t dabble in it. Go all in.

Kovalick:Where did you find savings?
Miller: Oh, many places. Equipment maintenance, support, real estate costs, power, cooling, building services, the list goes on. Our legacy air chain had about 130 racks of on-premise equipment just for our 32 domestic feeds. The cloud-based system has five racks on premise and all the remaining in the cloud. All this amounts to tangible savings for the long-term. Of course, we do channel stat-muxing and satellite up-linking outside of the cloud.

To check out our recent webinar, “Project On-Ramp: Migrating Discovery’s Media Supply Chain to the Cloud,” visit www.tvtechnology.com, click on Webinars under “Resources.”

Kovalick:Let’s talk channel reliability, a huge concern for broadcasters. Can you share some of the architectural principles you followed?
Miller: Our initial rollout is based on Amazon Web Services (AWS). We use S3 for object storage and hundreds of EC2 compute instances for media processing and other functions.

The end-to-end workflow is a mix of file-based processing and real-time streams. We worked with selected vendors to implement a world-class broadcast software architecture.

For reliability, each channel’s signal chain is duplicated in two different Regions (AWS U.S. East Region and Dublin, Ireland Region, for example) and each Region’s signal output is fed to our Sterling, Virginia Broadcast Center where they connect to a simple 2x1 switch. It is there we decide which Region’s feed goes to air. If one faults, we switch to the other. We have created a very reliable system that can withstand both geographical disruptions and multiple equipment failures.

Kovalick:Why did you choose AWS?
Miller: Amazon did a good job reaching out to our industry and this built our confidence and trust in their ability to support our workflows and business needs. We built our foundational flows on Linux and virtualization so that we can migrate to other cloud providers if and when it makes business sense. We deliberately applied design principles to not impede any future migrations.

Kovalick:What new vistas has the cloud opened to you?
Miller: There are many, but I like programmatic workflows and supply chain efficiencies. Quickly setting up signal chains and apps as required by a business and rapidly changing these to meet our needs is very cool.

Al Kovalick is the founder of Media Systems consulting in Silicon Valley. He is the author of “Video Systems in an IT Environment (2nd ed).” He is a frequent speaker at industry events and a SMPTE Fellow. For a complete bio and contact information, visitwww.theAVITbook.com.

Not too long ago, the words “public cloud” and “media enterprise” had almost nothing in common. Well, at the 2017 NAB Show the public cloud was being praised at nearly every turn by big and small equipment, services and software vendors. The public cloud is only about 11 years old, starting when Amazon first offered IT infrastructure services in 2006. It’s amazing to see the progress this once controversial service has made.

According to International Data Corp., worldwide spending on public cloud resources is expected to increase from $67 billion in 2015 to $162 billion in 2020, attaining a 19-percent CAGR. This is amazing growth and the media enterprise is not immune to this trend.

PUBLIC IS HOT, PRIVATE IS COOLING
Where do we stand today in terms of IT adoption? Let’s turn to a recent survey report from cloud management vendor RightScale, the “2017 State of the Cloud Report.” The report summarizes the results from a survey of 1,002 IT technical professionals across a broad cross-section of organizations about their adoption of the cloud. The report has been compiled annually since 2012, so RightScale has a good memory for how trends have changed since their first report. Here are some of the salient points that are relevant to building media infrastructure and related applications.

Fig. 1: Ninety-five percent of respondents are using the cloud in one form or another.

The survey shows (Fig. 1) that the hybrid cloud is the preferred enterprise strategy with 67-percent adoption while private-only is at 5 percent. Importantly, 95 percent of respondents are using the cloud in one form or another. The percentage of respondents adopting private cloud is 72 percent, down from 77 percent in 2016. This is an important trend and means more companies are trusting the public cloud.

Hybrid cloud is perfect for the media facility. Not all workflows are ready for public-only, whereas some intelligent sharing of local/private and public is practical and leads to more confidence in public-only. All media services organizations (production, post, broadcast, corporate, theatrical, etc.) use apps spanning from editing to media asset management to scheduling to project management and beyond. There are major benefits to the public cloud-based apps model and according to Gartner this segment had a growth rate of 20 percent in 2016 and a value of $38 billion (http://tinyurl.com/Gartner-saas-2015). Many vendors at the NAB Show offered either subscription apps or apps that you can own and run in a cloud of your choice.

One reason for the cooling of the private cloud is the trust in public has surged over recent years. Early on, private was the go-to choice because the public providers lacked trust and overall security was suspect. Well, the pendulum has changed sides and public (at least the major providers) has gained noteworthy respect. One reason for this is that the providers are showing substantial profits (sustainability metric) coupled with long-term growth prospects. It is accepted that private offers key advantages for massive data rate workflows (e.g. live, uncompressed HD/UHD).

Fig. 2: Usage patterns when running apps or experimenting with public and private clouds

USAGE PATTERNS
Cloud users are running apps in 1.8 different public clouds and 2.3 different private clouds, (the values in the table are averages across all responders). It’s true that fewer companies are using private clouds, but those that do are using them more often. The so-called multicloud strategy is prevailing across all responders. End users don’t want to be locked into one cloud vendor, so they spread the risk across about two, (Fig. 2.) Plus, each cloud provider offers a variety of services with differing performance levels. So, using the multicloud strategy allows end users to cherry pick what they need from the clouds they use.

For a real-world example, consider Walmart. Their strategy uses OpenStack for its private cloud and relies on Azure, Rackspace and possibly other public clouds.

Not too long ago, the main reason to use the public cloud was to leverage the pay-as-you-go (CapEx/OpEx) model to save money. In 2017 the priorities are completely different. According to the survey, the top four reasons for cloud use are:

• Faster access to infrastructure
• Greater scalability
• Higher availability
• Faster time to market

Interestingly, the ninth reason for the same question was “cost savings.” It’s amazing to see this has fallen near the bottom of the list. Why? Cloud users have learned that the other benefits outweigh cost savings. This explains why using a trusty spreadsheet to calculate cloud ROI is very difficult. How do you put a price on the ability to scale or faster time to market? Most savvy executives understand this, so don’t require a bulletproof ROI to sign off on cloud-based projects.

Another survey question asked, “What are your top cloud challenges?” For the cloud beginners the first choice was “security” and third was “managing costs.” This is not surprising since beginners are not confident in their cloud skills and don’t want to get slammed with a security breach. For the cloud-focused (mature) architect the first choice was “managing costs” and the fourth was “security.” The general trend is, more experienced users worry less about security.

This does not mean that security is not important. To the contrary, public cloud providers focus on this since it is a major differentiator to on-premise systems. Who has more security experts working for them, Amazon or your application development group? Security is a joint responsibility between you and the cloud provider.

KINGS OF THE HILL
Finally, respondents were asked what cloud providers they are using to run apps. The order was: Amazon AWS (57 percent); Microsoft Azure (34 percent) up from 20 percent in 2016; Google Cloud (15 percent) up from 10 percent in 2016 and IBM (8 percent). Oracle and Digital Ocean had a combined 5-percent usage. The multicloud approach is creating opportunity for Azure and Google. While it’s true that AWS has the market-leading position, there is potential for the others to continue gaining on AWS.

This survey shows the pace of the public cloud and its acceptance by all corners of business. As a media professional, keep both eyes trained on it and always think “cloud first.” That is, ask, “Why won’t the cloud work for me?”

Al Kovalick is the founder of Media Systems consulting in Silicon Valley. He is the author of “Video Systems in an IT Environment (2nd ed.).” He is a frequent speaker at industry events and a SMPTE Fellow. For a complete bio and contact information, visitwww.theAVITbook.com.

It’s starting to happen… media companies are beginning to move some or all of their high-valued content to the public cloud. One driving point is that major public cloud vendors are gaining the respect and trust of media and other companies big and small. Of course, there are reasons to keep some content local including uncompressed video editing, critical real-time operations and others. However, some media workflows can leverage the cloud including long-term archive, high-volume processing, media asset management and consumer streaming.

Let’s consider an example of leveraging the cloud for storage. Say a media enterprise has 100 TB of bulk data that it wants to transfer to a public cloud for archive, with occasional access. There are at least three ways to realize the transfer of data.

One method is to use the public internet; a second is to use a private network; and third, load the content into physical drives and “mail” these to the cloud vendor for import. Each method is in use today and has a time/cost tradeoff.

For the first method, using a 1 Gbps (used at 80 percent capacity) upload rate, it would take about 12 days to upload the 100 TB. This is optimistic and assumes an ideal continuous upload rate not throttled by inevitable network congestion. For the case of a private “internet bypass” network with a 10 Gbps link the upload time is reduced to about 1.2 days. Private networks have low latency and low packet loss.

Fig. 1: Amazon’s 100 Petabyte Snowmobile truck

However, private transports can be costly and usually require a two-stage link. The first stage link is from the customer datacenter to a “cloud access location” (e.g. 60 Hudson St. in New York City, One Wilshire in Los Angeles). This link is normally some form of private Ethernet from vendors such as AT&T, Comcast, Verizon or Level 3 and may require a long-term contract. The second stage link is from the access location to the cloud. An example of this link type is Amazon’s Direct Connect, Microsoft Azure’s ExpressRoute and Google’s Direct Peering. Each has a different fee structure, but are mostly “pay as you go” at some level up to 10 Gbps.

Once private cloud connectivity is established it will be used for more than bulk data uploads. It is becoming the trusted method to link the datacenter with a cloud and avoid the performance and reliability issues of the internet.

A SNOWBALL’S CHANCE

The third method for bulk data transfer to a cloud is physical drive transport (flash and hard disk). The major cloud vendors all support sending actual drives with data to their cloud facility for import, however, this can be troublesome. There is a need to specify, purchase, maintain, format, package, ship and track the individual drives.

In October 2015 Amazon introduced “Snowball,” designed to simplify bulk transfer. Snowball is a portable purpose-built appliance owned by Amazon, has 80 TB of internal encrypted storage, is rugged enough to withstand a 6G jolt, tamper proof, includes a 10Gigabit Ethernet port and is weather resistant.

The Snowball model is faster, more secure and an efficient way to transport bulk data. Here are the steps to perform a transfer:

• Use the Amazon self-service page to order the appliance(s). These are shipped to the customer facility.
• Connect the Snowball to your network and log onto it with provided credentials.
• Copy your data into Snowball and ship the unit(s) back to Amazon. The unit is ready to ship without any packaging. A shipping label will automatically appear on the “E Ink” display.
• When Amazon receives the unit(s) they will load the data into your S3 storage account.

There is a usage charge of only $250 per job, plus any shipping charges (e.g. FedEx). Users have up to 10 days (starting the day after delivery) to copy data to the appliance and ship it out. There is no cost to load the data from a Snowball into S3 or Glacier storage. Of course, once the data is loaded into storage there are charges to keep it persistent.

Depending on your bulk data size, Snowball may be the right choice compared to using internet or private networks to upload. However, what if you have petabytes of data to move? In this case even using many Snowballs may not be practical. Enter the Amazon Snowmobile.

Snowmobile is a new way to move massive volumes of data to the cloud, including video libraries, image repositories or even complete data center migrations. Transferring data with Snowmobile is secure, fast and cost-effective. Users can transfer 100 PB in a few weeks compared to over 20 years using a 1 Gbps connection or using about 1,250 Snowballs. Fig. 1 shows a Snowmobile at its product release announcement in Las Vegas, November 2016. Internally it is stuffed with servers, storage drives, IP switching gear and power distribution.

This is major league data transfer. The 45-foot long Snowmobile truck is driven to your site and Amazon personnel assist in setting up the transfer. The Snowmobile website states: “A fully powered Snowmobile requires ~350 KW. Snowmobile can be connected to available utility power sources at your location if sufficient capacity is available. Otherwise, AWS can dispatch a separate generator set along with the Snowmobile if your site permits such generator use.”

So, there are many options for transferring bulk data to the cloud; from low data rate internet connections to the massive Snowmobile. If you face a data migration problem, factor these methods into your decision logic.

The adage “Compute where the data is” makes even more sense in a cloud environment. The more data stored in the cloud, the more related compute and distribution is likely to occur; it’s a beneficial synergy.

If you want to leverage the cloud for business operations, but your data is “stuck at home,” consider doing a bulk transfer. One added benefit of cloud storage is its long-term durability. The cloud vendor is responsible for managing the headache of replacing aging storage hardware. It’s not your worry.

Al Kovalick is the founder of Media Systems consulting in Silicon Valley and author of “Video Systems in an IT Environment (2nd ed).” He is a frequent speaker at industry events and a SMPTE Fellow. For a complete bio and contact information, visit www.theAVITbook.com.

There is little doubt that compute virtualization has changed the data center forever. It has led the way for software-defined resources: compute, storage and networking. The cloud is the biggest beneficiary of software-defined concepts and the trickle-down effect is influencing how media systems are designed and implemented.

This article will review three trends important to media system designers related to the “global data center.” This term describes private/local, hybrid and public clouds. The themes are: cloud progress, virtualization maturity and service workloads. This article highlights some conclusions from the “Cisco Global Cloud Index: Forecast and Methodology 2914–2019” report and from other research sources.

CLOUD PROGRESS
The five basic value propositions of the public cloud (and private to an extent) are:

• On-demand services
• Network access
• Resource pooling
• Agility of usage
• Pay by use

Each provides a value not provided by the traditional data center. Infrastructure-as-a-Service (IaaS) is the king of cloud offerings. IaaS adoption has been slow in some quarters, especially for media operations. There are reasons for this, not the least is user “trust” in the cloud service provider (CSP). Will they lose my data? Will the data be secure? Will they be in business in 10 years? In 20?

One factor in CSP trust is the ability to achieve growth with continued profit. Profitable companies have “staying power” and this creates trust and confidence by end users. Consider, Amazon Web Services (AWS) the largest of the CSPs, generated $2.56 billion in revenue during the first quarter of 2016 (~$10.24 billion/year run rate), up 64 percent compared to a year earlier with operating income of $604 million. Morgan Stanley predicts Microsoft cloud will be 30 percent of its revenue by 2018. In 2016, spending on public IaaS is forecast to reach $38 billion, growing to $173 billion in 2026 (Statista 2016). The cloud is here to stay and so are the most successful CSPs.

It is interesting that Netflix, Hertz, Intuit, Juniper and many other companies have their business operations “all in” the cloud. Much of General Electric does as well. This is a strong confirmation that these companies put long-term trust in their selected CSPs.

Media companies have announced projects, too, such as Disney’s announcement last year to move its ABC network control room and playout operations to the cloud. Bottom line, the cloud is ready for the media business even if only in portions. Filebased operations are more cloud-friendly than real-time studio operations in 2016. Proxy video resolutions, compared to uncompressed, are more aligned with cloud-based workflows. There are many factors, of course, but some designers are finding value in moving relevant media operations to the cloud.

Important, too, is vendor product support. If vendors don’t test and support their software in a cloud environment, end users will be reluctant to purchase. If you attend the NAB Show or IBC or whenever you speak to vendors, ask about their cloud support for products. Fortunately, more and more vendors appreciate the need. If not, demand it.

VIRTUALIZATION MATURITY
By 2019, more than 86 percent of all workloads will be processed in cloud data centers. A workload is a unit of measure describing applications that range from simple Software-as-a-Service (SaaS) apps to large compute-bound jobs to database applications. This means that only 14 percent of workloads will be running in traditional data centers. The old adage “A rising tide lifts all boats” applies to media system operations, too.

Compute virtualization allows dynamic deployment of workloads in a cloud environment. A related means to share compute resources by different workloads is called “containerization.” To get a handle on the differences and advantages see the Cloudspotters Journal article “Understanding Virtualization and Containers,” http://goo.gl/jrorHa.

Public cloud is growing faster than private cloud in terms of workload adoption. Many businesses want the resource agility that is offered by public clouds only. Plus, the pay-by-use model may not be available for private clouds. However, if uncompressed video data rates are used, then private has the edge for now. Security is always a concern, but in reality, security is a shared responsibility, part end user and part CSP. Mature developers know that cloud security can be excellent and the worry is often misplaced.

Fig. 1: Graphic shows a projection of the split between public and private cloud use. Fig. 1 shows a projection of the split between public and private cloud use. Note that public use is growing about three times faster than private, and in 2019, 155 million workloads will be executed in the public cloud.

Many media systems will use the hybrid cloud model where the most data intensive computing and storage will be done in-house likely using a private cloud and all other media processing executed in a public cloud setting. Very few systems will only be private-cloud only without a link to public SaaS, storage, API services or compute.

On the private-cloud front, companies such as EMC/Dell, HP, IBM, Cisco, NetApp, Nutanix, SimpliVity and others offer converged systems. The converged infrastructure (CI) vendors offer integrated compute, storage and networking in a racked, pretested configuration managed with “one pane of glass.” At the leading edge of convergence is the hyper-converged infrastructure (HCI). This paradigm relies on individual nodes with both storage and compute networked in a cluster (from a few to thousands of nodes) to create a “web-scale” architecture for executing workloads. CI and HCI are very friendly to on-premise media operations.

SERVICE WORKLOADS USING SAAS
SaaS apps are a huge disrupter to the current installed desktop (or workstation) model. Does any user really want to worry about updating applications and geo-distributed access to apps/data? SaaS provides automatic, invisible updates with universal access via a browser. It’s true that some high-end video editors, color correctors and related apps do need local processing due to elevated video data rates. But the majority of media apps can be SaaS-based and end users should be on the lookout for vendors that support this model.

The Cisco Global Cloud Index report predicts that public SaaS apps will grow at a 39-percent cumulative annual growth rate (CAGR) until 2019. Desktop apps have advantages for sure, but the SaaS model is so very compelling as a replacement.

CATCH THE WAVE
Both the private and public cloud will be a major part of many new or remodeled media facilities. Create a “cloud first” strategy—consider the cloud first; if not a fit, then look to alternatives. Stay educated in this area else you risk missing the wave that is sure to be our future.

Al Kovalick is the founder of Media Systems Consulting in Silicon Valley. He is the author of “Video Systems in an IT Environment (2nd ed.).” He is a frequent speaker at industry events and a SMPTE Fellow. For a complete bio and contact information, visitwww.theAVITbook.com.