Microsoft and others prohibit the use of their generative AI systems to create various content. Off-limits content includes material that depicts or promotes sexual exploitation or abuse, is erotic or pornographic, or attacks, denigrates, or excludes people based on race, ethnicity, national origin, gender, gender identity, sexual orientation, religion, age, disability status or similar characteristics. It also does not allow the creation of content that contains threats, harassment, encouragement of physical harm, or other offensive behavior.
In addition to expressly prohibiting such uses of its platform, Microsoft has also developed guardrails that inspect both user-entered prompts and the resulting output for signs that the requested content violates any of these terms. These code-based restrictions have been circumvented repeatedly in recent years through hacks, some of which were benign and carried out by researchers and others by malicious threat actors.
Microsoft did not explain exactly how the defendants' software would be designed to bypass the guardrails the company created.
Masada wrote:
Microsoft AI services implement strong security measures, including built-in security restrictions at the AI model, platform, and application level. As alleged in our lawsuits released today, Microsoft observed a foreign-based group of threat actors developing sophisticated software that exploited publicly disclosed customer data harvested from public websites. In doing so, they attempted to identify and gain unlawful access to accounts on certain generative AI services and purposefully modify the capabilities of those services. Cybercriminals then used these services and sold access to other malicious actors with detailed instructions on how to use these customized tools to generate malicious and illegal content. Following discovery, Microsoft revoked access for cybercriminals, implemented countermeasures and enhanced security measures to further block such malicious activity in the future.
The lawsuit alleges that the defendants' agency violated the Computer Fraud and Abuse Act, the Digital Millennium Copyright Act, the Lanham Act, and the Racketeer Influenced and Corrupt Organizations Act, and that it committed wire fraud, access device fraud, trespass of customary law and unlawful interference. The complaint seeks an order prohibiting the defendants from engaging in “any activity herein.”
