[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <CAM=PXV7qd66ceLZJmz2usDYJVunFPMPBL1m=s+bjUR6orL9GDA@mail.gmail.com>
Date: Tue, 28 Apr 2026 14:09:36 -0600
From: Greg Dahlman <dahlman@...il.com>
To: oss-security@...ts.openwall.com
Subject: Re: Coordinated Disclosure in the LLM Age
As I have been struggling about the ethics of releasing a POC I discussed
on this list (which just shared previously disclosed issues) let me provide
some feedback.
1) While most model providers are following the common dark pattern of
implicitly opting in non-enterprise users into data collection for
training, the maximum acceptable embargo period for issues disclosed to
these lists is 14 days, way shorter than the training period for the
foundational models. I don't think that cached data is used across
customers right now, it would seem too hard to avoid malicious prompts
etc... to do so. In fact 90 days for an upstream disclosure is probably
too short for that to be a *primary* concern.
2) Unless a bug happens to hit an exact match past the interpolation
threshold, it is already well represented in the training data, but
perhaps not in a pattern form that us humans will view as equal, so yes it
will change how you triage.
3) While I personally have a bright line, where I won't even put client
data on shared services, you should assume that any person who
receives content will potentially leak it.
4) The AI tooling is probably higher risk as many of these projects (and
traditional services) are passing the buck on security, note the below
quote from cursor from [0]
> Right now, the agent can read files without confirmation. That’s the
current behavior by design, and it’s described in the security docs. The
read restriction only applies to files listed in .cursorignore.
[0]
https://forum.cursor.com/t/cursor-agents-should-be-restricted-from-access-of-files-outside-the-project-without-permission/149418
Security/friction tradeoff is always a challenge, but let me address the
point below:
> any vulnerability reported as a result of research using an LLM is
trivially discoverable by others, and give up trying to pretend there's any
point to working it under embargo.
There is nuance here, was it discovered by someone pumping your code
into Qwen3.6-35B-A3B on a Mac Mini, or was it discovered by Anthropic
burning tens of thousands of dollars worth of tokens?
There is an almost uncountable amount of vulnerabilities, simply because
the threat landscape has changed, from the MIT AI labs needs, the rainbow
books, legacy code, indifference, etc.... While LLMs change the cost of
discovery and reporting, they unfortunately may actually exacerbate the
problem of mitigation. The corpus is just full of code that people are
just glad it works, and LLMs have dramatically improved the percentage of
code they produce that is functionally 'correct', they have not
dramatically improved the security of the code they produce at the same
rate.
Even the most simplistic synthetic benchmarks on LLM's ability to produce
correct *and* secure code at the same time are sitting at the coin flip
range today. Meaning the efficiency gains in reporting vulnerabilities
will be asymmetric with the ability to address them.
Any change in embargo timelines needs to respect that asymmetry, and
forgoing an embargo doesn't help lighten the load or get fixes in place; it
will just add context switches and reduce velocity more. Those disruptive
callouts need to be limited to the fire-drills where there is actually a
fire because they are super expensive.
Hopefully others have better feedback, but understand that with current
management practices at many peoples day jobs, you may need to
approach people in private for frank feedback.
Greg
On Tue, Apr 28, 2026 at 8:58 AM Jeremy Stanley <fungi@...goth.org> wrote:
> As I'm sure is the case for everyone, the projects I work in are
> under a seemingly unending deluge of vulnerability reports from
> researchers using LLMs to mine for security gold in our software. At
> the same time, we see maintainers on our projects relying on
> LLM-oriented tools to develop fixes for vulnerabilities and compose
> prose for advisories.
>
> While I take a moment to catch my breath, this new Bizarro World
> we're all living in has gotten me thinking about the risks of public
> LLM services to embargoed vulnerability handling workflows and
> traditional coordinated disclosure. The operators of these LLM
> services are known to feed prompts and results back into their
> training data, presumably making it faster and easier for the same
> information to be found later by other users of the same service.
> Would keeping embargoes short help to mitigate related risks of
> parallel rediscovery or outright disclosure to other LLM users? It
> seems to me that there must be some inherent lag in this process,
> but how much?
>
> I'm sorely tempted, both due to the increased volume and the risk of
> premature disclosure, to just assume that any vulnerability reported
> as a result of research using an LLM is trivially discoverable by
> others, and give up trying to pretend there's any point to working
> it under embargo. Similarly, it makes sense to me that patch
> development and descriptive prose shouldn't be produced with LLM
> assistance for any vulnerability that is being worked under an
> embargo.
>
> I can't be the only one whose been pondering this... what positions
> have the rest of you taken?
>
> [P.S. No LLMs were exploited in the making of this message.]
> --
> Jeremy Stanley
>
Powered by blists - more mailing lists
Please check out the
Open Source Software Security Wiki, which is counterpart to this
mailing list.
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
