Sunday, April 14, 2024

Generative AI’s impact on the Broadcast industry

By Jonathan Watson*

Media executives must find innovative efficiencies with technology, operations, and content. By integrating AI, media businesses can work better and faster. Content will be created and published with fewer operational assets. Generative AI will revolutionise multiple industries worldwide, contributing to a USD$7 trillion increase in global GDP over the next decade.

In this three-part series, we will explore integrations of GenAI into the broadcast industry and what organisations need to know to start a Generative AI project. Adopting Generative Artificial Intelligence (Gen AI) will revolutionise how content is created, produced, and distributed. I will give my impressions and examples of how Gen AI will impact operations and investment in technology. This first instalment aims to introduce you to Gen AI and provide relevant context for broadcasters, exploring its potential benefits and outcomes for the industry.

The second article will explore Gen AI services available for broadcasters and what your business needs to do to prepare. In the third article, we’ll present examples of Broadcasters who have deployed Gen AI projects and the outcomes, look into the future, and imagine some potential outcomes.

Decision-makers must plan strategically to embrace Generative AI and harness its opportunities, such as crucial decisions about data infrastructure, foundation model ownership, workforce structure, and AI governance.

What Generative AI is and How it Works

Generative AI captured the public’s imagination with Midjourney and Chat GPT. Its roots lie in machine learning, language models, deep learning, and neural net training. The leap from Machine Learning to Generative AI was an invention in how massive amounts of unstructured data could be analysed using what’s called ‘Transformers.’

An example of unstructured data in broadcast would be a video content repository with no metadata, so the Transformer must extract context via image recognition and training. Another example would be to collect all news scripts from the archives and train a model to write a script.

Generative AI foundation models take inputs such as text, image, audio, video, and code and generate new content in any format. The key to developing content is called ‘prompt engineering.’ Prompt Engineering is a new role that uses various conversational prompts to extract valuable outputs from the models.

Large Language Models (a foundation Model) are a considerable amount of textual data that understands the context of sentences and documents. Chat GPT is Generative AI, and the Large Language Model is called ChatGPT. Stable Diffusion is the FM behind Midjourney, making those saturated out-of-this-world images. Stable Diffusion is an Open Source model, and Open AI owns ChatGPT.

The Rabbit R1 is an AI-powered gadget that can navigate your apps. It uses a Large Action Model, which is a model that can interface and click through web interfaces via voice prompts.

Choosing between closed-source and open-source models depends on the specific requirements of a project, the level of customization needed, and the question of governance. Closed-source models may offer ease of use through APIs, while open-source models provide greater transparency and adaptability but require more technical expertise.

Foundation Models are published daily; there are thousands to choose from. When you speak with a vendor or project manager about your Gen AI project, it’s about the suitable foundation models and data transparency. If AI integration is on the roadmap, I will start with the data scientists in your org on what data is collected.

Hybrid models are when a smaller, proprietary data set is connected to a Foundation Model that is general purpose. And then ‘tuned’ to a specific task. Broadcasters would take the hybrid approach because the industry has very particular workflows and technology and thus must build their proprietary models.

An example of an applied data set is PDF documentation. An FM model is connected to a repository of PDFs that the AI would augment into the queries and analysis; this is called a RAG (Retrieval Augmented Generation).

Broadcasters have teams collecting data to create proprietary models to augment the larger FMs. However, the data collected is not in the operational and system data. Broadcasters must extract data from vendors’ systems to build operational user models. Automation systems will have excellent data for an AI engineer to make live production foundation models. Start identifying and collecting operational data from systems and workflows.

Broadcast executives planning for AI in the workflow will also use event-driven AI when data streams into the models, and the AI reacts in real-time. It is beneficial because Live production captures dynamic events, like a live cross.

AI Governance is an important consideration to be part of the decision-making process. Managing biases or hacker-inserted malicious results that tilt the model must be a factor in the AI-powered newsroom. ESG is all over Generative AI because the energy consumption and carbon footprint are immense.

Part of governance will be security. Chat GPT and Mid Journey are directly connected to the internet. Broadcasters have strict firewalls to prevent outside attacks; it raises questions about how a Gen AI project will be secure. Cloud vendors are the possible answer.

Broadcasters may seek a vendor with foundation models specifically tuned for broadcast and media. FMs that assist with writing scripts, software code, marketing, and audience analysis are common. Perhaps the future broadcaster is just one foundation model covering all broadcasters’ needs. Could the next-generation NRCS system be a Foundation Model?

Generative AI will impact these areas of Broadcast: content creation, workflow and production, commercial strategy, and operations.

Content Creation and Personalisation:

Generative AI’s noticeable impact on the broadcast industry is the evolution of how content gets created. AI systems can analyse vast amounts of data, understand viewer preferences, and generate personalised content tailored to individuals or groups. The FM will tailor news, sports, and opinions to the viewer’s profile. This personalization does raise ethical governance questions of singular echo chambers.

Script writing is the easiest and first impact a newsroom will have with Gen AI. Journalists will have personalised Gen AI editors that do fact-checking, copy editing, librarian, and source confirmation.

AI will build video packages; the prompt engineering would be the news story or script in the NRCS. As the journalist writes the script, the video package gets assembled in real-time, with video clips pulled from sources such as a MAM.

MAMs must have the data hooks that enable a newsroom FM model to connect via API, extract the clips, and drop them into a timeline. The MAMS of the future are foundation video models.

Where footage is unavailable, let’s say an interview for the story. Then, the newsroom AI assistant will coordinate the resources and production assets for the interview. The journalist does not need to initiate anything; this happens as ‘event-driven AI.’ I would even go so far as to suggest that if it’s an online interview, the Gen AI would book the guest, write and ask the questions, and record the interview. Ingesting into the MAM and inserting into the package or rundown.

Will personal AI assistants make for an ethical difficulty in what constitutes a source? Was that Quote from the actual person or their Virtual AI Assistant? It goes back to governance as something to consider.

News, opinion, and sports-specific foundation models will increase the variances of the automated content. Broadcasters will have families of foundation models, one for each Sport. Today, with AWS Gen AI services, Fox Sports in California is auto-generating sports highlights using the massive datasets Fox Sports has already been collecting. Following Sports highlight clips, news packages will be automated. GenAI will place the correct CG and branding, adjust colour correction and transitions, drop in the VO, create multiple versions, and publish.

An FM that captures the technical skills of an experienced live sports or news director will be an asset. An NHL producer will be different from an NFL producer. I predict the potential for the most skilled and expert live production producers and TDs to licence their ‘abilities’ into a foundation model. These would be available to buy for integration.

News and Sports are derived from live events; I don’t see any requirement to create a video package from the pixel level, but it could be possible. As a live production business, I would not get hung up on the fancy AI tools that create content from nothing, as broadcast content must remain authentic.

Virtual Sets and other 3D models will eventually be constructed with Generative AI, with some fine-tuning by a human technical artist. Virtual Sets will be designed, assembled, and deployed in near real-time. OpenAI has unveiled Point-E, a new text-to-3D model that can generate 3D point clouds from natural language inputs or Prompt Engineering. These Gen AI deployed models will be virtual studios or AR studio graphics rendered on VR/AR devices in your living or office space.

Virtual Sets will rebuild and assemble automatically live on air, depending on the script and prompts from the producers, scripts, or talent. Imagine watching a live news program, and the 3D elements in the virtual set are dynamic, reacting to the piece’s content.

Sports Broadcasters can leverage generative AI to enhance live sports production by automatically producing AR virtual overlays of player stats and CG without an operator. CG could be an example of event-driven AI. The type of CG and the information populating it are triggered by what happens on the Preview. Gen AI would build a broadcast map from the GIS data from a live feed.

A trimmed-down Sports OB van, as many tasks an operator may do, can be AI. A sports producer will create highlights not by spinning an edit wheel to find the right clip but by entering textual prompts.

News graphics are going to become spectacular! GenAI would create entire infographic storylines to explain complex pieces. News Infographics of this complexity are built by the script and prompts from the creative team. The future of News stories will become very infographic-heavy. Existing CG rendering platforms that are HTML-based have an advantage.

Digital Rights Management will be AI. Smart contracts on the blockchain will be the mechanism to trigger event-driven AI when it comes to monetization and content rights. I will write another series on how Blockchain tech can benefit broadcasters.

Enhanced Live Production Workflows

Generative AI streamlines workflows across all global industries and will do the same for Broadcast. Accounting, marketing, scheduling, research, and other traditional administrative tasks are more efficient and cost-effective—a reduced headcount or increased output, most likely a mix of both.

Real-time feedback into the production pipeline will speed up the news cycle. The circular news story, Go-live, clip, publish, feedback, repeat, etc.…is not new, but an organisation can manage many more stories in circulation with Gen AI.

Gen AI will have a more significant impact at the regional and hyper-local end of Broadcast networks, lowering and automating the cost of production for stories relevant to that local smaller market.

Content will be data-driven, meaning when content is published, consumer metrics will flow back in real-time, feeding the foundation model to impact content creators’ decisions. The AI will incorporate this data, automatically providing insights or adjusting the stories.

Broadcasters have virtualized control rooms since COVID-19, and platforms from switching, audio, and CG bundled into a single virtualized solution as an on-demand resource. The next evolution is an AI control room, which could become a foundation model. Or at least the TD and producers will become AI agents. But for this to happen, broadcasters need workflow data from live production. Log and user data must be recorded from the vendor’s platforms that comprise a control room.

The broadcast journalist could become a content strategist. Instead of writing a couple of stories, the journo builds a show by the prompts and queries that go into the NewsRoom AI. What comes out the other end is a series of scripts, video packages, infographics, social media, and playlists that get handed off to AI publishing.

Because the cost and volume of live production will drop, Gen AI Broadcast systems may start at the hyper-local regional markets. What would a national broadcaster look like if it had thousands of neighbourhood-level productions, all virtualized and automated?

Operations:

I believe that Gen AI will have the most significant impact on the operations of a broadcaster. How a vendor is selected, the way engineers manage technology stacks, and the management of systems used to put a production together. Operations will include HR and talent acquisition and training.

McKinsey Global Institutes projects that between 2030 and 2050, Gen AI will automate half of all knowledge tasks. What Microsoft is about to do to your Office 365 will blow you away. Gen AI is not replacing roles but simply making existing roles more time-effective and removing tasks within the roles. Did I mention a 4-day work week?

Writing code will be faster and more accurate, as LLMs can now offer real-time feedback and assistance to programmers. That will impact service and in-house teams’ levels of productivity. Vendors that write code for your business will also have efficiencies, shrinking project deadlines and delivery costs.

The daily operations of logistics to locations, booking interviews, and getting the right people in place. Scheduling OB vans and technical production crew will be automated. It may already be if your business is using a 3rd party vendor. Cloud-based project management tools already use Gen AI to help project managers.

Control Rooms are increasingly just software instances spun up on the cloud. These virtualized Operational nodes will be part of the same processes that assist the journalist in creating the content. The future technical director will be a Gen AI, ensuring the right graphic is on air and the right source is used with the correct audio levels. Once the show is complete, the control room spins down until the next Live show. But think about that: what it means is that broadcasters will produce 1 or 1000 live shows simultaneously with different branded graphics packages and target audiences.

Broadcast scheduling will be a monumental multi-dimensional task that no human can keep on top of. But this scheduling is across thousands of targeted personality profiles and pop-up channels. The entire playout infrastructure is best handed over to a playout foundation model.

Vendors supplying the technology that underpins a complex large broadcast organisation will open up data and metering capabilities. A Broadcast Engineer FM will manage all operations, including systems, SLA, licences, and commercials.

The above evolutions will create new data sets so that Gen AI can output a very accurate ROI on operations and output. Having end-to-end data, from edit tool to media throughput to live production, to deployment and feedback and click rate, will be the ultimate broadcast media foundation model; this will give the C-suite a macro ROI view of the entire business, be it a global brand or a regional broadcaster.

Conclusion

While the focus of content creation may be the flashy output, Foundation Models will significantly impact operations in terms of savings, efficiencies, and automation. Vendors and suppliers to broadcasters should be ready to expose system data as an additional product or feature. Service providers specialising in integrating Generative AI into their operations and cloud service providers will be knocking on the CTO and Engineering and Operations office doors if not already.

The interface with this level of AI is conversational, but it must also be thoughtful. Your existing workforce will need to learn ‘prompt engineering.’ Prompt engineering is the process of structuring text that can be interpreted and understood by a Generative AI model. Imagine that AI is a genie in a bottle. How you phrase your wish will be very important. The sooner you can get these training sessions on the go, the better.

Gen AI can leverage existing MAM systems to source and build a video package from the NRCS story script. Generating a video from nothing is not a requirement because of the live video nature of broadcasting. However, developing compelling infographics will be a relevant new feature in the news. In 2024/2025, HTML5 CG vendors will offer this feature of automated content creation with Data Storytelling and AI-generated infographics. The design team in your org will need to understand that it will not be about the tools, like After Effects or CG systems, but how news stories will be infused with infographics created from prompts.

There will not be one Foundation Model that does it all but a platform of different Foundation Models for parts of the broadcast production. This article mentioned ChatGPT
and Stable Diffusion, but over 300,000 open-source Foundation Models are already available on GitHub, with more added daily.

The following article in this series will look at the existing services and vendors offering solutions to broadcasters. And how a broadcaster should prepare for a project, defining the outcomes desired and which parts of the business to start connecting to a Gen AI Foundation Model.

* Jonathan Watson has spent over 20 years as a vendor to broadcasters globally working across production, graphics, and the commercial side of the business. With a focus on how Blockchain and Generative AI will impact broadcast and digital media, he brings an understanding of what’s on the horizon to how the broadcast industry can adapt and work more efficiently in the future.

Visit https://www.linkedin.com/in/watsonjon

Related Articles

Stay Connected

1,470FansLike
1,523FollowersFollow
120SubscribersSubscribe

Latest Articles