IDF colonel discusses ‘data science magic powder’ for locating terrorists

A video has surfaced of a senior official at Israel’s cyber intelligence agency, Unit 8200, talking last year about the use of machine learning “magic powder” to help identify Hamas targets in Gaza.

The footage raises questions about the accuracy of a recent statement about use of artificial intelligence (AI) by the Israeli Defense Forces (IDF), which said it “does not use an artificial intelligence system that identifies terrorist operatives or tries to predict whether a person is a terrorist”.

However, in the video, the head of data science and AI at Unit 8200 – named only as “Colonel Yoav” – said he would reveal an “example of one of the tools we use” before describing how the intelligence division used machine learning techniques in Israel’s May 2021 offensive in Gaza for “finding new terrorists”.

“Let’s say we have some terrorists that form a group and we know only some of them,” he said. “By practising our data science magic powder we are able to find the rest of them.”

The descriptions in the video of technology used by Unit 8200 bear similarities with recent testimony from six IDF insiders about their use of an AI tool called “Lavender” during its offensive on Hamas. They said that the AI-generated database had been used to assist intelligence officers involved in the bombing campaign in Gaza, helping identify tens of thousands of potential human targets.

In its rebuttal, the IDF said some of the accounts were “baseless”. However, the accounts are consistent with the remarks by Yoav during an AI conference at Tel Aviv university in February last year. The video, in which Yoav can be heard talking but not seen, was hosted on the university’s YouTube channel, and until recently it had fewer than 100 views.

As he took to the stage wearing a military uniform, the audience was instructed not to take any photos of Yoav or record his presentation. “It’s good because it’s a bad hair day for me,” he joked.

In the 10-minute presentation – titled “digital transformation and artificial intelligence in the intelligence domain” – the colonel offered a rare insight into how opaque AI systems are being used by secretive military and intelligence bodies.

When using AI to predict whether someone is a terrorist, he explained, Unit 8200 takes information it has about people it believes are members of terrorist groups and aims “to find the rest of the group”.

A get in touch embed for Harry Davies

Referring to a specific example, the official said that in the IDF’s May 2021 military operation in Gaza, his department applied this principle to “find Hamas squad missile commanders and anti-tank missile terrorists in Gaza in order to operate against them”.

He explained that using a form of machine learning – known as “positive unlabelled learning” – “we take the original sub-group, we calculate their close circles, we then calculate relevant features, and at last we rank the results and determine the threshold.”

The colonel said intelligence officers’ feedback is used “to enrich and improve our algorithm” and stressed that “people of flesh and blood” make decisions. “Ethically speaking we put a lot of emphasis on this,” he said, adding that “these tools are meant to help break their barriers”.

According to Yoav, Unit 8200 was able to break “the human barrier” during the May 2021 offensive when it managed to produce more than 200 new targets. “There were times when this amount took us almost a year,” he said.

Contacted for comment about the video, the IDF said the colonel’s participation in the conference had been approved by the military. However, a spokesperson denied that his remarks conflict with the IDF’s recent denials about its use of AI. In a subtle change of wording not used in its original statement, the IDF told the Guardian its AI systems do not “choose targets” for attack.

“The IDF never denied the existence of a database of operatives in terrorist organisations, that cross-checks existing information on such operatives,” it said. “At the same time, the IDF fully stands by its statement that it does not use AI systems that choose targets for attack, and that the database in question is not a list of operatives eligible to attack. There is no contradiction.”

In their testimony, the six intelligence officers who spoke out last week said that the Lavender tool had been used to help identify potential targets at an unprecedented scale and pace. The IDF was not accused of using systems that automatically select targets for attack.

Four of the sources said that, at one stage early in the war, Lavender listed as many as 37,000 men in the enclave who had been linked by the AI system to Hamas or Palestinian Islamic Jihad. None of the sources denied that humans were involved in the process of authorising strikes, although some confessed to minimal human oversight.

“I would invest 20 seconds for each target at this stage, and do dozens of them every day,” one intelligence officer said. “I had zero added-value as a human, apart from being a stamp of approval. It saved a lot of time.”

Their accounts were published by the Israeli-Palestinian publication +972 magazine, its Hebrew-language outlet Local Call and the Guardian.

Col Yoav’s description of the partnership between AI and intelligence personnel echoes a model for targeting operations envisioned by his commander, the Unit 8200 chief Yossi Sariel, who the Guardian revealed left his identity exposed online when he secretly authored a book published in 2021, The Human Machine Team.

At one stage, the colonel explained that the AI and data science department he runs at Unit 8200 is also known internally as “the human machine integration centre”.

Speaking eight months before the IDF commenced operations in Gaza after the Hamas-led 7 October attacks, the colonel spoke optimistically about how the IDF is moving “from the postcard age to the digital era” where “suddenly you can react during battle with applied data-science driven solutions”.

Looking ahead, he added: “I’m curious with respect to how will the next operation look like, digitally speaking”.

The Guardian