Re: [PATCH] Documentation: document expectations around AI use

[Alyssa Ross <hi@alyssa.is>]

[-- Attachment #1: Type: text/plain, Size: 7035 bytes --]

On Wed, Aug 05, 2026 at 10:15:06AM +0200, Valentin Gagarin wrote:
> I like the nature of the argument, but from my own experience would
> suggest a different policy: don't submit LLM output.
>
> One piece of rationale behind that is in essence the same as yours,
> something along the lines of "contributors are assumed to understand
> what they're writing".  But even that still feels vague, because of
> course one can understand a given LLM output and package that as a
> contribution.  Therefore I would like to sharpen that line of reasoning
> before committing to that.

I don't think this is a feasible position for us to take into the
future.  Our goal is to create a secure desktop operating system, and in
the field of computer security we are currently facing a new reality
where an automated vulnerability discovery and exploitation machine
exists.  As the cost of vulnerability discovery and exploitation has
sharply declined, the task of delivering a "secure enough" system has
become much more difficult.  If the cost of exploitation reduces while
the cost of defense is held constant, eventually it's not going to be
realistic to even try to defend.  That doesn't mean we have to lower our
quality standards and fill Spectrum with slop, but it does mean that we
have to be open to new possibilities enabled by automation when they
would let us improve defensive robustness.  More on that below.

For other kinds of intellectual work, slower, but more thoughtful, 100%
human development is a tradeoff that can be worth making!  100%
artisanally human‐created security software that is eventually outpaced
by ever more efficient attackers, though, is not useful to anybody.  (Am
I happy to find myself in this position?  No, but it is what it is.)

> The other piece of rationale has more weight for me: you wouldn't submit
> search engine output verbatim either.  LLM output is qualitatively the
> same as generated code, plus randomness.  Why would you check that into
> version control?  That would also still need a constructive formulation
> for people further from this bubble to make sense immediately, but do
> you see what I'm getting at?

I've been thinking about this point in particular a lot over the last
couple of weeks.  I think it comes down to current form of collaboration
being source code, and so we expect LLMs to meet us where we're at, by
producing source code of equivalent quality to what we'd expect of
humans.

One can imagine a future world where our collaboration form has changed
from program source code to a formal specification of a program, and
LLMs are used like a compiler to generate source code implementing that
specification.  (This is very appealing for security software, because
the generated code could be written in a way that it proves certain
security properties hold — this is possible today, but it's mostly too
expensive for humans to do.  It might even be that this sort of method
is both the only way to produce software that is robust against ever
more capable adversaries, and only feasible with "AI" automation!)

In that world, we don't care about the specifics of the source code,
much like we don't much care about the machine code produced by a
compiler, and we just track the specification in version control.
That's quite a way off, though.  For now, we want source code to be our
canonical collaboration form, and we care about the exact content of
that source code, and we want it to be deterministic, so we keep source
code in version control.

> Certainly we should emphasize the part that sending LLM output
> needlessly shifts more burden onto reviewers, and the threshold to
> consider that spam is very easy to cross.

It doesn't necessarily do that.  One can use an LLM to produce the exact
same diff that could have been typed by hand.  Whether an LLM is used or
not, contributors have a responsibility not to overwhelm reviewers with
poor quality contributions.  LLMs do make it easier to do that,
especially without realising, so we might want to call attention to
that.

> This deliberately leaves open which tools you use and how you use them
> to arrive at your contributions.  Another thing also worth mentioning is
> that the contributor may be liable for all the legal implications of
> labeling LLM output as one's own contribution.  What if a particular
> piece turns out to violate copyright?  For that reason, maintainers
> should therefore not let into their codebase what is marked as LLM
> output, and otherwise take authorship claims for granted.

Yes, contributors have a responsibility to ensure (attested via the DCO)
that they have the necessary rights to submit their contributions.
Again this is the case regardless of how the contributions are produced.
I'd expect that most Spectrum contributions would be Spectrum‐specific
and (strongly encouraged by me to be) small enough that it would be
implausible for them to be substantially copying any prior creative
work.

> But again, I agree with the spirit of your proposal, which to me reads
> like that the project's communication channels are for people, period.
> Still figuring out how to state it so that spirit gets across despite
> cultural differences.  Maybe this would even allow us to bypass the LLM
> debate in documentation altogether, ideally without having to re-explain
> why e.g. checking in any sort of machine output is not a good idea in
> general.

If, some time in the future, a personable robot starts identifying and
fixing problems, without being extremely annoying, forgetful and
time‐sucking to interact with in the way that today's technology is, I'm
not going to reject those fixes!  We don't have that today, though, and
experience from other projects shows that contributors are not
necessarily good at realising that, hence the part about generated prose
being the most prescriptive part of my proposal.  But my hypothetical
robot demonstrates to me that "for people, period" is not quite the
principle I'm coming from.  We're probably close enough for the time
being, though.

> Thinking about it a bit more, the underlying theme is actually that of
> engineering, contribution, and communication culture.  It's easy to get
> along with people who happen to share it, but it's a lot of work for
> both sides when implicit assumptions don't match.  We may be even better
> served than by just policing LLM use if we state at least some of the
> core assumptions which drive the project, provide some guidance for how
> to soak up that culture, and ideally explicitly state positive examples.
>   That may substantially reduce the need to play catch-up with instances
> of the same phenomenon.

Yes, it would certainly be good if we could communicate the desired
culture effectively enough that people (and LLMs) can pick up what to do
from the principles without needing to be instructed on every specific
instance!

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]