I'll share the first-hand account I recently got from someone else.
> We've used it at work
> it is... not as hype as everyone is concerned about
> I'd argue the framework around it for security scanning is the arguably more useful side of the tool, definitely doesnt take a huge model to get all the issues it flagged on our systems
> For us, it absolutely flooded us with noise
> I mean hundreds if not thousands of false positives or minor issues or not applicable
> For every one reasonable issue
> The biggest issue it created was the execs treated every issue it produced like it was a drop everything and fix the issue type deal
> I'm talking company wide drop all things "we need to patch nginx because this module that no one uses and is disabled by default has this RCE vulnerability™
> Or "all ec2 AMIs need to be upgraded because it flagged a a version specific docker vulnerability", it flagged every single machine with docker regardless of if the actual vulnerability was relevant
> Vulnerability was with a very specific Auth plugin configuration you could enable with docker and specifically the Mosley docker compatible tool, but it is clear it only knew there was a vulnerability in docker, not if it was applicable or not
> Meanwhile dirtyfrag and friends not a single peep from btw despite it allowing for container escape
> Idk, I was underwhelmed with the quality of the reporting it gave really. If the company allowed me to get information about all the infrastructure in our entire organisation to run Claude over it repeatedly looking for recent CVEs I'm sure I could produce the same results...
There's a lot of speculation that it is indeed a marketing plot and the model is just a step improvement over current capabilities... and the real reason they aren't releasing the model is they are compute constrained and cannot serve the model. To my knowledge there's no proof of this however, but given the fact that literally 60 days ago they made Mythos out to be the end of the world and last Friday they announced that they will release the model in a few weeks, I feel like it was indeed something along those lines (marketing ploy).
The week before they released Mythos to governments they had all their source code stolen. It's all about improving their image and creating propoganda.
Not so sure I would want a company that does not see any issues with mass surveillance of my country [1] to have access to critical infrastructure or its source code where I live.
> We've used it at work
> it is... not as hype as everyone is concerned about
> I'd argue the framework around it for security scanning is the arguably more useful side of the tool, definitely doesnt take a huge model to get all the issues it flagged on our systems
> For us, it absolutely flooded us with noise
> I mean hundreds if not thousands of false positives or minor issues or not applicable
> For every one reasonable issue
> The biggest issue it created was the execs treated every issue it produced like it was a drop everything and fix the issue type deal
> I'm talking company wide drop all things "we need to patch nginx because this module that no one uses and is disabled by default has this RCE vulnerability™
> Or "all ec2 AMIs need to be upgraded because it flagged a a version specific docker vulnerability", it flagged every single machine with docker regardless of if the actual vulnerability was relevant
> Vulnerability was with a very specific Auth plugin configuration you could enable with docker and specifically the Mosley docker compatible tool, but it is clear it only knew there was a vulnerability in docker, not if it was applicable or not
> Meanwhile dirtyfrag and friends not a single peep from btw despite it allowing for container escape
> Idk, I was underwhelmed with the quality of the reporting it gave really. If the company allowed me to get information about all the infrastructure in our entire organisation to run Claude over it repeatedly looking for recent CVEs I'm sure I could produce the same results...
[1] https://www.anthropic.com/news/statement-department-of-war :
> But using these systems for mass domestic surveillance is incompatible with democratic values.
https://cyberplace.social/@GossiTheDog/116679693992983945