New York’s Museum of Jewish Heritage is using AI for Holocaust education

Educational center unveils exhibit called ‘Survivor Stories’ that uses artificial intelligence to facilitate dialogue between survivors and visitors

Luke Tress is The Times of Israel's New York correspondent.

Holocaust survivor Toby Levy next to an interactive artificial intelligence display of herself, at the Museum of Jewish Heritage in New York City, January 27, 2025. (Luke Tress/Times of Israel)
Holocaust survivor Toby Levy next to an interactive artificial intelligence display of herself, at the Museum of Jewish Heritage in New York City, January 27, 2025. (Luke Tress/Times of Israel)

NEW YORK — A museum-goer asked Holocaust survivor Toby Levy about her time in hiding during the genocide at New York City’s Museum of Jewish Heritage on Monday, Holocaust Remembrance Day.

Levy promptly described her father’s efforts to guide the family through their ordeal hiding in a Polish family’s barn. Her father, she said, kept the group quiet, telling them not to cry, or talk aloud, or sneeze. When Levy’s uncle, who was sick, said he wanted to leave the barn, her father would not allow it.

“My father says, ‘No, nobody’s getting out. Either we go together or none,'” Levy said. “It’s his strength and his belief that pulled us through.”

Levy was not speaking in person, though. A new artificial intelligence system at the museum interpreted the question and delivered the response on a large, vertical screen. The real Levy stood to the side, beaming.

“When all the survivors go, the deniers will come out, big time, and complain, and say, ‘It never happened,’” Levy told an audience at the museum on Monday.

“We have witnesses,” she said, describing the AI program. “They’ll press a button, they’ll find a survivor. I am the witness.”

Holocaust educators have sought to preserve memories of the genocide for decades, as survivors grow old and their ranks thin. Researchers have collected testimony through audio, writing and video interviews. Artificial intelligence represents another effort to use technology to preserve the memory of the atrocities for future generations. The museum’s new project, called “Survivor Stories,” will give visitors the ability to ask survivors questions in a conversational format, providing the museum with another tool for education once survivors are gone.

“It’s a more remarkable opportunity to talk to survivors forever and ever,” said Bruce Ratner, the museum’s chairman. “Most survivors are in their 90s, and that’s why this is particularly important to do.”

The Museum of Jewish Heritage, in Lower Manhattan’s Battery Park, was established in 1997 and focuses on educating young people in the region about the Holocaust to foster tolerance and combat antisemitism. Tens of thousands of children and college students visit the museum each year. The museum’s mission has gained urgency in recent years, due to surging antisemitism and widespread ignorance about the Holocaust.

The museum started work on Survivor Stories with the nonprofit USC Shoah Foundation more than a year ago. Researchers filmed interviews with 10 survivors from the museum’s Speakers Bureau, volunteers who participate in educational programs, including Levy. The survivors answered 100 to 125 questions over several hours about their experiences before, during, and after the Holocaust, as well as personal questions about topics like their favorite foods and hobbies.

The project’s production team edited the interviews, removing the questions from the footage so only the survivors’ answers remained. The responses were transcribed and artificial intelligence was trained to match the answers to visitors’ questions.

To use the technology, museum visitors hold down a button on a small, mounted screen and verbally ask a question. The artificial intelligence interprets the question, matches the query to a response from a survivor, and plays the video clip on an adjacent, larger screen.

The project illustrated the technology’s potential for Holocaust education, but also its limitations.

The responses recorded by the survivors are not always a perfect match to the questions asked. Behind the scenes, the system assigns a score for how closely a question matches a response. If the response clears a threshold for a match, the system delivers a video of a survivor’s answer.

If none of the survivor’s responses falls within that threshold, the survivors deliver a pre-recorded “fallback response” that says something like, “That’s a great question. I was never asked that,” said Mike Jones, the director of web and automation technologies at USC libraries.

Ahead of the interviews, the survivors were coached on how to respond for the purposes of the AI system. They were told to avoid answering questions by saying “yes” or “no,” so the answers would be more flexible. They discussed their religious beliefs in general terms, for example, so the response could match a range of questions about religion. Asked about their age, the survivors stated their year of birth or their age at the time of the interview, since the videos will be in use for years to come.

“Someone might ask this question in 20 years. They’re no longer going to be 89 at the time. We have them respond in that way, rather than a very narrow answer that can only be answered by one specific way of asking that question,” Jones told The Times of Israel.

The researchers used non-generative artificial intelligence, meaning the technology does not write statements for the survivors or create any images. All of the responses are verbatim answers from the survivors and the video is actual footage from their interviews. The system does not mix and match statements from the survivors, only delivering full statements, as they were recorded.

An interactive display for the Museum of Jewish Heritage’s ‘Survivor Stories’ exhibit, in New York City, January 27, 2025. (Luke Tress/Times of Israel)

The project’s producers opted for non-generative artificial intelligence, due to concerns about ethics and historical accuracy, Jones said.

Sticking to survivors’ own words means the videos will only display “accurate historical representation of these exact folks,” he said.

Generative artificial intelligence could also be exploited for nefarious purposes. The developers had to get into the mind of those who would attempt to exploit the software to have the survivors say something sympathetic to the Nazis, for example.

“There’s a huge opportunity for abuse, and people know how to game this system. Unfortunately, there’s a lot of people out there that would love to be able to spin a Holocaust survivor into saying something antisemitic,” Jones said, adding that “the technology is not there yet” to prevent such abuse.

The model leans toward delivering the fallback responses if a question lands in a gray area.

“If someone tries to say, ‘Hitler wasn’t really that bad,’ and terrible questions like that that [the survivors] would never agree to, it goes to that fallback,” Jones said. “It’s the training on both sides to make sure it matches to the correct answers, but also doesn’t match to the wrong questions.”

There will be three displays open to the public in the museum, starting this week. The platform is also available online at the museum’s website.

The museum unveiled the project on Monday as part of its events for International Holocaust Memorial Day, marking the 80th anniversary of the liberation of the Auschwitz death camp. More than 200 survivors gathered at the museum to watch a live broadcast of a ceremony at Auschwitz. Local officials including Congressman Daniel Goldman, a Jewish New York Democrat, and Julie Menin, a descendant of survivors who serves on the New York City Council, were also in attendance.

Levy, who has volunteered with the museum for 15 years, said the event was the continuation of her father’s wish that the family would survive to tell the world about the Nazi atrocities.

“I remembered his words: ‘You will be the one to have to tell the world.’ Seventy-five years later, here I am,” she said.

Most Popular
read more: