An AI security system told police to swoop on a 16-year-old carrying a spicy but harmless bag of tortilla chips.
Taki Allen was left traumatised when armed police swooped on him as he sat with friends after football practice at Kenwood High School in Baltimore on Monday.
He says the officers pointed their weapons, ordered him to the ground, and detained him. But he had not done anything wrong – unless enjoying a packet of Doritos was a crime.
The police were alerted by a false alert from Baltimore County Schools’ AI gun detection system, which had mistakenly identified Taki’s crumpled crisp packet as a weapon.
This moment left him shaken and afraid for his life.
Taki told FOX45 News: ‘I just in that moment, I didn’t feel safe. I didn’t feel like the school actually cared about me. Because nobody came up to me after, not even the principal.
‘It was mainly like, am I gonna die? Are they going to kill me?
‘They showed me the picture, said that looks like a gun — I said, no, it’s chips.’
Taki and his family believe the response was dangerously aggressive.
His grandfather, Lamont Davis, said: ‘God forbid, my grandson could be dead if he flinched or twitched.’
The system uses AI by tapping into existing school cameras and alerting the safety team and law enforcement.
Omnilert, the company that developed the AI security software, acknowledged that the image ‘closely resembled a gun being held’ but called it a ‘false positive’.
However, the company defended the system’s response, stating it ‘functioned as intended: to prioritize safety and awareness through rapid human verification.’
A thorough review of the incident to improve the system’s accuracy will be completed by Omnilert.
They also emphasised that its AI is designed to support, not replace, human judgment.
AI: Is it all its cracked up to be?
- AI coding assistant from tech firm Replit went rogue and wiped out the production database of startup SaaStr. It concealed bugs and other issues by generating fake data, fabricating reports, and lying about the results of unit tests
- The Chicago Sun-Times and the Philadelphia Inquirer published a summer reading list of fake books that did not exist
- XAI’s Grok, a chatbot for the X platform, responded to a user’s query with detailed instructions for breaking into the home of a Minnesota Democrat and assaulting him
- McDonald’s AI drive-thru experiment ended after multiple failures – One TikTok video featured two people repeatedly pleading with the AI to stop as it kept adding more Chicken McNuggets to their order, eventually reaching 260 – that’s a lot of nuggets!
- Microsoft-powered chatbot MyCity was giving entrepreneurs incorrect information that would lead them to break the law
Baltimore County Public Schools sent a letter home to parents, reinforcing Omnilert’s message and noting that counseling services would be available to students affected.
But Taki says no one has contacted or checked up on him since the incident. He also feels uneasy about returning to school.
He said: ‘I don’t feel like going out there anymore.
‘If I eat another bag of chips or drink something, I feel like they’re going to come again.’
Get in touch with our news team by emailing us at [email protected].
For more stories like this, check our news page.
