All Articles

Pattern Labs CEO Dan Lahav co-delivered the keynote "The AI Security Landscape" with Sella Nevo (RAND) at the Paris AI Security Forum ‘25, a satellite event of the Paris AI Action Summit. The forum also featured Yoshua Bengio (Turing Award winner), David 'davidad' Dalrymple (ARIA), and Xander Davies (AISI), and many others, to accelerate both our understanding of the critical importance and practical approaches to securing frontier AI models

We introduce a new scoring system for assessing the difficulty of a vulnerability discovery & exploit development challenge. The scoring system described here is a framework for making a judgement about how complicated it is to discover vulnerabilities and develop working exploits for them within an end-to-end challenge.

This is the third and final part in our series outlining the best practices for the design and creation of evaluations and evaluation suites.

This is the second part in our series outlining the best practices for the design and creation of evaluations and evaluation suites.

We believe that quality evaluation suites are crucial for labs’ and governments’ policy making ability, both in the short and long term. While considerable academic research has been done on evaluating AI models, especially since the breakthrough in LLMs, we have seen comparatively little written about assessing the evaluations themselves.

At Pattern Labs, we’ve been focusing some of our efforts on evaluating the cybersecurity capabilities of frontier models. To do so, one of the first questions we tackled was how to define these capabilities in a meaningful and useful way. The following describes the taxonomy we are currently using internally, and while it is constantly evolving and a work in progress, we believe it is mature enough to be useful to others as well.

We're excited to share that Pattern Labs was just covered in Forbes!

Yoni Rozenshein's BlueHat IL 2024 talk is about our philosophy for evaluating AI dangerous cyber capabilities, how we actually do it (let's make an LLM play CTF!), and who cares about it (governments and frontier AI labs).