Documents reveal Lockport School District’s facial recognition system wrongly identifies Black students at disproportionately higher rates, and mistakes brook handles for guns.
Families of students, particularly Black families in upstate New York, have been concerned about their children’s safety during school following the installation of a network of facial recognition cameras throughout the educational institution. Their worries could potentially become a reality, according to new documents obtained by Vice’s Motherboard.
The documents show that SN Technologies, the Canadian company contracted to install the school’s facial recognition system, misled the district about the accuracy of its algorithm and lied about how often it misidentifies Black faces, Vice reports. In addition, several records detail technical issues with SN Technologies’ AEGIS face and weapons detection system that result in it misidentifying objects like broom handles as guns, the outlet reports.
Local media reports and documents obtained by Motherboard show the system sends out an alert to police when it detects weapons or people who are listed on the district’s “hot list.” This issue concerns Black families, as Black students make up 11 percent of Lockport schools.
“The police have said if they get a notification, they’re going to treat it as a live shooter system, and you have a system that’s predisposed to make mistakes and misidentify people,” Jim Shultz, a Lockport parent, told Motherboard. “The risk of an accident, the risk of something horrible happening because the system is structured the way it is, to me, is 1 million times higher than [the chance] that the cameras are going to prevent a real situation.”
Lockport schools have had the system in place since January following several battles with the parents, civil liberties groups, and the NYSED, which initially opposed the facial recognition system before eventually signing off on it in November 2019. Since then, Lockport has been sent several messages warning that SN Technologies was misrepresenting a major part of the AEGIS system.
Vice reports that Motherboard obtained a copy of the Freed Maxick report from the New York Civil Liberties Union (NYCLU), representing the Lockport parents in their lawsuit.
