Executive Summary

The threats posed by disinformation to security and democracy have been assessed as a significant and ongoing, if not habitual, concern. Spurred by advancements in artificial intelligence (AI), deepfakes are viewed as a modern evolution of disinformation which poses new challenges for governments, individuals, and societies. Safeguarding the integrity of the information ecosystem is a fundamental priority not only for democracy, but also for society as a whole.

Technological Advancements and Prosocial Applications

Deepfakes, originally a portmanteau of deep learning and fake media, is now used more broadly to refer to any impersonating media created or edited by deep learning algorithms.  Manipulated videos, images, audio/voice, and text created using generative AI techniques have quickly evolved to become increasingly accessible and realistic. In many ways, these advancements pose exciting opportunities.

• Developments in generative AI are significantly boosted by the availability of large-scale language and image generation models. Developments have focused on making the models more powerful, capable, and accessible while giving users more control over the style and content of the generated media through detailed text prompts.
• Deepfakes can be used for creating entertaining content, such as realistic face swaps or visual dubbing for film, TV, or video games, enabling new creative possibilities and bringing fictional characters to life. They can also be employed for visual effects and restoration purposes, recreating or enhancing scenes that are difficult or costly to produce practically. For example, deepfakes can be used to age or de-age actors or bring back deceased performers for a film or advertisement.
• As marketing and educational tools for training and simulations, deepfakes can enable realistic scenarios in fields like medicine, military training, or emergency response, allowing practitioners to practice skills in a safe and controlled environment.

Threats to Society and Security

As the capacity for generating media becomes more widely available and precise, the probability for misuse intensifies. Among the primary concerns with deepfakes is the potential for spreading disinformation and manipulating political discourse, leading to confusion, distrust, and social instability in democratic societies.

• Deepfakes raise serious privacy concerns as they can be used to create non-consensual explicit content by superimposing someone's face onto explicit material. This poses a threat to individuals' privacy and reputation while inflicting emotional distress. In addition, deepfakes present numerous legal and ethical challenges. They can infringe upon intellectual property rights and violate privacy laws.
• Deepfakes have the added potential to erode trust in visual media. As the technology becomes more sophisticated, it becomes increasingly challenging for people to distinguish between genuine and manipulated content, making it difficult to rely on video evidence and exacerbating the problem of disinformation.
• The widespread availability of deepfakes can have negative societal impacts, including cyberbullying, harassment, and potential for social unrest. Deepfakes can be weaponized to exploit or manipulate individuals, leading to reputational damage, psychological harm, or social divisions.

While deepfakes are more likely to advance already existing security threat-related activities rather than generating new concerns, it is important to recognize the potential risks associated with deepfakes and develop robust technological solutions, ethical guidelines, and legal frameworks to address these challenges and mitigate their negative consequences.

• Disinformation is a tool that has been used by state and non-state actors throughout history in their attempts to discredit and downplay democratic institutions, amplify conspiracies and radicalization, and encourage distrust of authority. Deepfakes facilitate the speed and effectiveness of these efforts, while expediting targeting of government/military personnel, phishing and social engineering, and mimicking of biometric data.
• Deepfakes can be used as a tool for creating noise to flood the intelligence collection space, causing distractions from true intelligence and/or distorting perceptions of human sources by creating artificially generated conversations, videos, or text. The increased reliance on open source intelligence (OSINT) makes deepfake information particularly impactful in the information ecosystem.
• Deepfakes can also be used to poison the data utilized for training of deep learning systems, intentionally compromising these systems with malicious information. For example, algorithms used to detect cyber-attacks could be compromised through data poisoning of the large-scale datasets on which they are trained.
• From a public safety and security perspective, deepfakes can be employed to commit fraud, engage in coercion and/or extortion, create fake evidence for criminal activities, or to impersonate and/or incriminate individuals in unlawful activities.

Outlook

Deepfakes are designed to deceive, and the human mind cannot consistently identify the outputs of sophisticated technologies. While tech giants have begun flagging deepfake content as disinformation, detection systems integrating both human and model predictions are of greater value. Governments have a role to play in facilitating the application of deepfake technologies that both benefit and protect citizens and democracy, and individual citizens have agency in protecting themselves and their communities.

• Deepfakes challenge existing legal frameworks in areas such as defamation, intellectual property, and privacy rights; and there is currently little distributor liability for social media platforms circulating deepfake content. Adapting and updating laws to account for the unique challenges posed by deepfakes while clarifying issues related to accountability, liability, and the rights of individuals affected by deepfake manipulation should be prioritized.
• Fostering research and development of technologies that can detect and mitigate deepfakes is a significant policy consideration. Increased collaboration with industry experts to establish standards and guidelines for responsible use of deepfake technology is essential, as is balancing innovation with necessary regulations to address the risks while fostering technological advancements.
• A promising approach is that of content authentication. Rather than trying to ‘detect’ AI generated content, the architecture of ‘authentication’ is instead embedded into the framework of the internet itself via a cryptographic marker embedded in the ‘DNA’ of the content.
• Societal norms and discourse on deepfakes should facilitate an environment where people are skeptical about what they see and are encouraged to challenge each others’ informational claims. Digital literacy training, especially if directed at societal thought leaders and influencers, assists in increasing awareness of risks as well as trust in media.

That deepfake technology will continue accelerating towards producing more realistic content more efficiently and more cost-effectively is a certainty. Considering deepfakes from a global perspective allows for comprehensive approaches to maximize the benefits of the evolving technology while addressing the associated individual and national security risks, upholding privacy rights, and maintaining public trust in media and information sources.