Artificial intelligence (AI) is developing rapidly and is being used increasingly often for audiovisual productions. But can AI also play a role in creating audio description (AD) for film and television? A recent study by Scribit.Pro, commissioned by the Bartiméus Fund and with support from CFAP in NPO Innovation, offers new insights. The research results show that AI offers promising applications, but still falls short for narrative content.
What is audio description and why is it important?
Audio description makes films and television accessible to people with visual impairments by adding descriptions of what is happening on screen. Good audio description not only conveys visual elements but also translates atmosphere, emotion, and nuances that are essential for experiencing a story.
AI and audio description: the possibilities and limitations
The study used ChatGPT-4o, currently the most advanced AI model for image description. Other models, such as Google Gemini and Anthropic Claude, proved unsuitable for generating usable audio description.
Through advanced prompt engineering, significant improvements were achieved in AI output. The use of a film script and subtitles provided more context, allowing descriptions to align better with the visuals. However, AI continued to struggle with narrative nuances and recognizing subtle visual details. This often resulted in descriptions that were either too superficial or too detailed, lacking the proper focus on what is truly important to the viewer.
How do users and professionals rate AI audio description?
A crucial part of the study was the assessment of AI-generated audio description by end users and professionals. The short film Der Kaiser (NTR Kort) was rated a 6.5 by people with visual impairments with AI audio description. Without audio description, the film received a 3.5, whereas the human audio description scored a 9.1. Thus, although AI offers a clear improvement over no audio description at all, users often found the AI descriptions too busy, repetitive, and less well-aligned with the film experience.
Professional audio describers at Scribit.Pro see potential for using AI as a tool, but point out that full automation is not yet feasible. Especially in animated films and drama productions, AI adds hardly any usable audio description, and extensive post-production is required.
Documentaries proved more suitable for AI audio description, where 60-80% of the generated text was usable with human review and editing.
Watch a preview of the short film Der Kaiser below. A version with audio description by AI and a version with audio description by a Scribit.Pro describer.
Source: Lemming film. Please note: the example fragment is only available in Dutch.
The future of AI in audio description
For the time being, AI is not suitable for audio description of drama films and other narrative content, because the lack of context and understanding of narrative structures is still too great. However, in documentary productions and informative content, AI can play a supporting role, provided a human editor remains involved. Scribit.Pro plans to integrate AI as a feature into its software editor as part of the AD generation to help describers with the production of audio description.
However, developments in AI are moving fast. Emerging models, such as the open-source DeepSeek, demonstrate that significant progress is being made. The researchers therefore advise continuing to closely monitor technological developments and keep experimenting with new AI tools.
Do you want to stay up to date on developments in AI and audio description? Subscribe to our newsletter.