Discussion about this post

User's avatar
Chris Hughes's avatar

Appreciate the shout out and willingness to revisit your perspective.

Another of my frustrations with the recent discourse around this topic is all of the research on these topics, both from frontier corps and external teams such as Epoch and Irregular is a heavy emphasis and focus on offensive tasks and capabilities. (e.g. GLM-5.3 can do x, y and z for offensive tasks).

We're seeing very little by comparison in terms of research efforts to demonstrate the defensive capabilities of these models, which I think is tied to various factors, such as:

- Frontier corps incentivized to demonize open weight models due to competitive factors

- The fact that offense is far more verifiable than defense, especially as each environment is unique with different architectures, compensating controls etc.

- The reality that it would highlight the level of refusals legitimate defensive users/use cases encounter with closed models compared to open.

Almost all the hype and attention is on offensive use cases, and I suspect that is also influencing training and hill climbing, which inevitably and painfully ironically makes each model subsequently better at wreaking havoc vs. mitigating it.

Leo W.'s avatar

Attackers still need financial motivation and adequate compute to perform AI facilitated attacks. Why go after every connected device ever when there are still poorly secured piggy banks hither and thither you can pop for less work and more return? It might be more lucrative to monetize stolen credentials by buying and reselling compute credit than spending that compute on attacks with inchoate odds of payout, too. People need to stop thinking about bad actors like some reaction function and remember attackers are people with varied motivations.

2 more comments...

No posts

Ready for more?