Google engineer uses AI to identify previously anonymous faces in Holocaust-era photos
From Numbers to Names founder Daniel Patt says his software can provide users with 10 possible matches to any random photo in a matter of seconds
Reporter at The Times of Israel
Madeline Paul’s grandfather died when she was 11, so finding a never-before-seen wartime photo of the Holocaust survivor was unexpected, she said.
“I wish I had asked him more questions,” said Paul. “Finding the photo of him in a Displaced Persons camp in Germany was meaningful to me,” the 18-year-old told The Times of Israel.
The discovery of the image with Paul’s grandfather — Stanley Leopold Paul — was facilitated by From Numbers to Names (N2N), which uses AI software to help people identify relatives and others in photographs from before, during and after the Holocaust.
“N2N allows you to search Holocaust-era photos by face. You upload a ‘query photo,’ which could be of a relative or someone you’re researching, and N2N will return the top 10 photos that are mostly likely to include the person you’re searching for,” said N2N founder Daniel Patt.
The fiscally-sponsored nonprofit project was featured on “The View,” NPR, and in The New York Times. Patt said the surge in global antisemitism following the October 7 massacres in Israel catalyzed him to accelerate volunteer work on N2N.
“Our latest estimate is that we’re able to search through a few million faces. We think there are at least another several million faces that have not yet been analyzed,” Patt told The Times of Israel. Many matches have been made this winter, added Patt.
Last year, N2N partnered with a New York high school where students helped identify people in photos using Patt’s custom-made, domain-specific machine learning system. Plans are underway to reach more schools.
“As students help confirm potential identifications provided by the software, they end up spending a lot of time looking at the people in these Holocaust photo collections; trying to determine who they were and what their lives were like, it’s hard for students not to feel a connection to the people in these photos,” said Patt, a software engineer at Google.
Family history reclaimed, one face at a time
Michael Paul — Madeline’s father — told The Times of Israel about his daughter’s efforts to help people in the community use N2N software following the discovery of the image with his father.
“Some elderly people asked Madeline to help and she found results — photos — for them to look through to identify a possible match,” said Paul.
“Finding these photos is a really great way to connect people to their parents and to each other,” said Paul. “It helped me feel closer to my dad and his memory.”
During the Holocaust, Paul’s father survived by his wits. He outran the Nazis and, even after being caught one time, escaped off a train headed for a concentration camp.
“So much of what drives me today is him, what he went through and how he persevered,” said Paul.
“It’s hard to know exactly how many matches we’ve helped facilitate because we encourage people using the site to confirm matches using any personal information they have,” said Patt.
“From our own internal searches, we’ve surfaced dozens of matches, many of which we have shared with living survivors and/or their descendants,” said Patt.
With funding, N2N believes it could scale discovery work and associated education programs to reach millions of high school and college students worldwide. The organization has already been contacted by dozens of schools throughout the United States. Additional small grants and donations would allow N2N to offset the cost of running searches, said Patt.
“While this work is fueled by passion, there is a cost associated with running our searches and making our discoveries. N2N is fully volunteer-staffed, supplemented by some donations and small grants,” said Wendy Weiss, the director of communications for N2N.
For Weiss, the growing project has the ability to “put names to faces. We are trying to recover the stories of their lives, saving pieces of history,” she said.
‘It was another woman’
Perhaps the most famous survivor to be identified in a previously lost photo was Rachel Auerbach, who compiled the secret “Oneg Shabbat” archive in the Warsaw Ghetto. Later, she founded the testimony department at Yad Vashem.
But to Sharon Ben-Shem, Auerbach was her grandfather’s second cousin who often visited their home outside Tel Aviv on weekends. Ben-Shem’s grandfather was the only relative of Auerbach’s to survive the Holocaust, Ben-Shem told The Times of Israel.
For years, a photograph purportedly of Auerbach — who died in 1975 — has circulated on the internet, said Ben-Shem.
In February, Patt’s technology identified the woman in the photo as a Polish-Jewish author of children’s books rescued by Irena Sendler, and not Auerbach.
“It was another woman all of these years,” Ben-Shem told The Times of Israel.
Fortunately for amateur genealogist Ben-Shem, Patt also found a previously unidentified photo of Auerbach. The image has Auerbach standing for a small group photo at a post-war conference on women in the military.
Coincidentally, Patt discovered a photo of Ruben Feldschu (Ben-Shem) — another of Auerbach’s relatives — a few weeks later. He confirmed the group photo included Feldschu through his granddaughter, Sharon Ben-Shem.
“I believe this could be the oldest individual who we’ve been able to identify — Ruben was born in 1900, and the photo was taken in 1924,” said Patt.
‘If you are here, so am I’
Auerbach was not the only prominent survivor whose image has been located by N2N’s platform in recent weeks.
Abraham Sutzkever, one of the greatest Yiddish poets of the last century, was identified in a group photo with other Vilna intellectuals.
During the war, Sutzkever was assigned by the Germans to the ghetto’s so-called “Paper Brigade,” a group of young intellectuals tasked with gathering the original writings of famous Jews. The writings were to be displayed at a museum commemorating the extinct Jewish race.
Through this assignment, Sutzkever managed to rescue large pieces of Europe’s Jewish heritage from Nazi clutches, including original writings of Herzl, Bialik, Shalom Aleichem and others.
After the war, Sutzkever and other “Paper Brigade” members returned to unearth manuscripts they had hidden in basements, within ghetto walls, and in the group.
While in the ghetto, Sutzkever made a “pact with the angel of poetry,” said Hadas Kalderon, his granddaughter.
“If you write brilliant poems that would persuade me. I will protect you with a flaming sword. But if not? Don’t blame me,” said Kalderon to relay what her grandfather said the angel told him.
In the ghetto, Sutzkever wrote poems each day. He watched his mother give birth to a baby who was immediately poisoned, in compliance with German orders about not reproducing. Sutzkever wrote a poem watching his baby brother murdered at birth, but he never became embittered or vengeful.
In a poem about his mother’s death, Sutzkever relayed that she came to him in a dream and urged him not to stay in grief because that is a sin, said Kalderon.
“If you are here, so am I,” wrote Sutzkever, who later testified at Nuremberg.
“The same way the core of the plum contains the tree, the nest, and the bird and everything around it,” wrote the poet.
Sutzkever’s diary will be published by Yad Vashem in April, said Kalderon.
“My grandfather believed the only role of poetry is to bring healing beauty in order to give others the strength to live,” said Kalderon.
Before and after her grandfather’s death, Kalderon, an Israeli actress, has told his story to audiences in Israel and Germany.
Before traveling to Germany in 2010, Kalderon asked his permission to share his story there. To craft the one-man show, Kalderon used her grandfather’s diaries, poems, Nuremberg testimony and interactions with him.
Sutzkever passed away the day the show premiered in Germany, said Kalderon.
“I was a candle of remembrance there in Germany,” said Kalderon. “And I made a form of closure.”
Seeing a new photo of her grandfather some 13 years after his death was unforgettable, said Kalderon. Surrounded by other young intellectuals in Vilna, Stutzkever’s photo could have been forgotten — if not for N2N making a match.
“It was quite a surprise to see this photo of him with other people who helped rescue the Jewish cultural treasures,” said Kalderon.
