The State of AI in Media: From Hype to Reality
Note: The following was adapted from a keynote that I delivered at the Nordic AI in Media Summit on May 9, 2023 in Copenhagen.
AI has been a hot topic in the media industry for years, often generating hype and inflated expectations. IBM launched its first automatic news summarization demo in 1958. Given its long lineage, as the technology matures and becomes more integrated into news production, it’s helpful to take a step back and assess the real impact of AI on the media landscape. In this post I’ll talk about AI’s pervasiveness and hype in the news media, exploring what I think are some of the driving factors and implications.
Towards the Plateau of Productivity
Over recent years, AI has been steadily making its way into various aspects of news production, from data mining and automated content generation to distribution, many of which I documented in my book, Automating the News: How Algorithms are Rewriting the Media. Recent examples of successful AI implementations include the Washington Post using data mining to generate tip sheets during the 2020 US elections and Gannett’s Localizer project which uses automated content generation to create local content for its 230+ newsrooms. Techniques like A/B headline testing which leverage machine learning to determine the best-performing headline from a set of headlines, have proven to be effective in improving click-through rates for publishers and are in routine use in thousands of newsrooms. Likewise, smart paywalls, like the one the New York Times uses, are something that can be used to optimize the conversion of casual to paying subscribers.
For a while, at least for the past few years, it’s seemed to me like we were coming out of the hype curve and folks in industry were just doing the work to pin down exactly how AI technologies like natural language generation, recommender systems, and other machine learning approaches could produce actual value, both editorially and for news businesses.
AI applications like A/B testing, automated content production, audio transcription, language translation, and optimized paywalls have become fairly commonplace and increasingly inconspicuous. You don’t usually hear news organizations bragging about that stuff so much anymore. AI’s pervasiveness is evident in everyday tools like Gmail’s spelling, grammar, and personalized writing suggestions. As information processing increasingly involves some aspect of AI, avoiding its influence on news media will be unavoidable. Eventually, AI will become as ubiquitous as the phones in our pockets, and we’ll hardly notice its presence.
Generative AI Hype
Despite these advancements and tendency towards integrating “classic” AI in productive ways, generative AI has brought hype back to the forefront, with concerns about potential harms and misuse of the technology. Generative AI has already been used in various ways in news media and beyond, such as by creating synthetic news anchors or developing video content for advertising, including for political ads. Generative models do offer a range of capabilities which can support news work, from analytic tasks like data extraction and rating to generative tasks such as rewriting and summarizing documents, or for illustrating articles.
However, generative AI’s limitations, such as inaccuracies in output and potential for misinformation and pollution of the information environment are major factors. For instance, in some of my initial tests of Bing’s chat search results I found there were accuracy issues (including in source attribution) in about half of the queries tested. As generative AI becomes more prevalent, it’s important to focus on how to deploy the technology responsibly and distinguish between hyped uses and real risks and capabilities.
As we look out at all the possibilities that are getting hyped, like generative AI in search, in advertising, and in misinformation, really what we need is a more level-headed approach to thinking through the specific capabilities of generative AI, and how they enable specific newsroom tasks. What I think we need is to very quickly move into the stage of trying to “kick the tires” and see where it could actually be useful. That’s, in part, why I launched the Generative AI in the Newsroom Project.
Hype and the Level of Automation
So how do we make sense of the fact that there is both ongoing AI hype as well as AI technologies that are now routinely used throughout media production?
I think that the degree of automation achievable is a significant factor in distinguishing hyped AI from inconspicuous AI. Highly automatable tasks, which can substitute human labor, tend to fade into invisibility. Once we can delegate a task to AI and trust it to perform at our expectations, we forget about it. At the lower end of the scale, you have automation which complements human labor using interactive interfaces that keep the user in control.
The hype-cycle feeds on the uncertainty of whether or not a complete task (or a whole collection of tasks, such as comprises a job) is automatable. Capital likes hype because in uncertainty you can marshal resources based on unclear benefits and competitive fears. At the same time the hype is reinforced by fears from labor, which would prefer lower levels of automation (in general) to maintain autonomy and a sense of control, and also not to lose a sense of purpose or gainful employment.
To be sure, sometimes AI does advance enough to move a task up in automatability. But we constantly need to assess whether some task is REALLY automatable or if maybe parts of it are, or if perhaps it’s unreasonable to (try to) automate it at all. The hierarchical nature of tasks means we need to dig into tasks and sub-tasks to see if there’s some part that can be done with automation.
In short, pervasive and invisible applications of AI have graduated into the upper reaches of automation by proving largely reliable and trustworthy. Tasks that continue to generate hype, like summarization (which, again, IBM began working on in 1958), are not yet ready for high levels of automation while upholding rigorous editorial standards. But AI might be on the cusp, which is always the big question.
Automatability and Generative AI
So where is generative AI in terms of its ability to move up the automation hierarchy?
We can ask it, which is what I did here. I prompted it for tasks involved in news production, then asked it to do a hierarchical task analysis and rate automatability using generative AI while maintaining professional standards.
Where you see green that’s a task that GPT4 thinks could be automated while maintaining high professional standards. What you’ll see immediately is that there’s not a lot of green. It’s for the stuff that you’d probably consider to be pretty boring, certainly not the hyped stuff.
But the other trend is that you see a lot of yellow and orange. These are tasks where AI can definitely help but where you’ll need a human in the loop to supervise. Something like creating graphics, charts, or infographics. Yes, generative AI can definitely help with that, but you’ll need an editor.
And you’ll see some red which are tasks that AI is really just not going to be helpful for at all right now.
And of course, if we drill into these tasks, because tasks are always hierarchical, some sub-tasks will also be more or less automatable.
The main point here is that the vast majority of news tasks can be helped to some extent with AI, but they’re not yet at a point of high automatability. As a result, AI will often only disappear to the extent that it becomes invisible within the tools that journalists use.
Now you shouldn’t necessarily trust GPT-4’s ratings of these tasks. A good bit of the work before us is actually figuring out which of these tasks actually CAN be automated and which truly do need a human in the loop. But I think overall it’s probably a reasonable starting point.
Newsroom Tools: Design, Labor, and Management
Where this all leads is that what we really need to be creating are AI driven tools where people using the tools are left in control of overall tasks but are supported by AI that can automate sub-tasks at a high degree of reliability and trustworthiness. The key word to emphasize here is really DESIGN.
Newsrooms need to be thinking about designing the future of work and how to hybridize human abilities, control, and oversight with the state of the art in AI capabilities. As the frontier shifts in what can be fully automated, tasks may need to be re-architected. When the first electric motors were developed, it took years for factories to realize that they needed to entirely redesign their layout to make best use of those motors. The same is true here, we need to, in some cases, entirely re-imagine how news is made. This needs to be creative and iterative, and will require building specialized interfaces so that people can retain autonomy and control.
And newsrooms need to think about how to design AI technologies that incorporate news values, so that the outputs of these systems are trustworthy with respect to editorial needs, and so that ethical commitments of the profession are still met. In addition this could be important so that the news media can retain some independence from big tech companies.
This hybridization of work has significant implications for labor in the media industry. AI technologies often create new types of work related to maintaining and validating systems. As a result, individuals may need to reskill or upskill to adapt to these changes.
Additionally, managing the transition to AI-driven news production requires careful consideration of the impact on workers and the value derived from AI. Management is crucial here.
Let’s consider the management around something like transcription AI. If I’m a reporter using transcription AI, I can do, let’s say, twice as many interviews in a day because I don’t need to transcribe them myself. Management could say, well, now you’re going to write twice the number of stories. Or, it could say, now you’re going to have twice as many sources for each story, and improve the quality and diversity of the sourcing. Or it could also say, you’re going to write the same number of stories, but you’re also going to be asked to do some other job that wasn’t part of your original job description. All of these options have different implications for the news organization and the type of value it reaps from AI, and for the individual worker and how it changes their experience of the work.
We’re already seeing strikes in Hollywood from the Writers Guild of America, with AI being a main topic where they want to bring management to the table for negotiations. Unionized journalists will be better off in the long run (unionize now if you haven’t already!). Now could be the time to bring management to the table to ensure the transition to AI-augmented news production is managed in ways that are not unfavorable to labor.
In closing, moving past the hype cycle and focusing on the real potential of AI in media is crucial. This involves assessing the automatability of tasks, designing AI tools and interfaces that support human workers, support skills development, and managing the transition strategically and humanely. By doing so, the media industry can reap the benefits of AI technology while maintaining editorial standards and ensuring a smooth transition for its workforce.
