Thread (7 messages) 7 messages, 5 authors, 3d ago

Re: [RFC] VEGA: a syzbot-like workflow for LLM-found kernel bugs

From: Yuan Tan <hidden>
Date: 2026-07-11 02:14:48
Also in: linux-crypto, lkml, netfilter-devel, workflows

Hi All,

Thanks for the thoughtful feedback, and apologies for the delayed reply.

1. Regarding Paolo's question, a bit more context about who we are.

I first got involved in the kernel community during undergrad with
guidance from Zhangjin Wu [off-list ref].
I started building VEGA earlier this year while I was a PhD student at
UC Riverside. I recently dropout and start a company called Nebula
Security with my labmates. We want to help fix bugs in open source
software.

The volunteer bug-fixing group that mainly includes students from
Lanzhou University, where I did my undergraduate studies, and UC
Riverside.

I will be at NetDev as well. See you there :)


2. To answer Laurent's question: All bug reports are human-reviewed.
For the past four months, we have been including a human-written and
reviewed patch when reporting bugs, with LLMs used only as an
assistive tool. However, because the volume is large and there are
some bugs we do not know how to fix well, we would like to make some
bug reports public in the interest of transparency.


3. To Hillf's point: if we start sending reports, the initial volume
will stay well below that level.


4. Regarding Andrew's point: Early on, we used syzbot config for
scanning and validation, and that did lead us to spend time on code
paths and features that may not matter much in practice. We should
definitely prioritize fixing bugs in actively maintained code.


5. And to Greg's point:

On Wed, Jul 8, 2026 at 7:55 AM Greg KH [off-list ref] wrote:
On Wed, Jul 08, 2026 at 02:22:47AM -0700, Yuan Tan wrote:
quoted
Hi all,

We would like to ask for feedback on a proposed workflow for reporting Linux
kernel bugs found by an LLM-assisted code auditing tool that we have
been developing since earlier this year.

Since February, we have been developing an LLM-driven kernel code auditing
tool called VEGA. It started as a side project, but the results became much
substantial than we expected: VEGA has found hundreds of valid bugs in Linux
kernel.

That immediately created a practical problem: we do not want to dump a large
pile of bug reports onto mail lists and annoy the maintainers.
True, which is why we all end up with long lists of issues/patches at
the moment.  The initial reaction is "we need a dashboard for everyone
to collab around!" like you did here, but I'd like to say this is not
the best thing to do at all.

syzbot can get away with a dashboard because someone is tending to it,
triaging the "serious" bugs before they become public, and only letting
the "would be nice to fix one day" type issues remain.  That's a huge
resource commitment that Google has made here, and that's great, but I
doubt that anyone else will have those resources to do this type of
thing.

Instead, let's just work to get these things fixed.  We all have
hundreds of patches/reports in our internal systems right now,
attempting to triage/rank/coordinate would just waste time.  In other
words, just grind through them, send patches out, and get these fixed.

I'm doing this now, and I know many others are as well.  We are all
running "different" tools, and so we find different issues, so we can
all just keep sending patches as we get them done.  It's going to take a
lot of effort (I've somehow convinced 8 interns to help me out with this
this summer), but once we get it done, we'll be much better off.
Yes, getting bugs fixed is the most important thing. The reason we
considered a syzbot-like workflow is that there are some validated
bugs which we currently do not know how to fix well ourselves. For
those cases we thought the community might have simpler ideas once the
report is made reproducible and concrete.

But we agree that any process around this should help move fixes
forward, not create another layer of overhead.
quoted
The first thing we tried was to fix as many as we could ourselves. We
started working with a group of student volunteers. Most of them are
college students, so we have been training them, reviewing their patches,
and trying to build an internal review process before anything is sent to
the mailing list. The goal is to turn these findings into useful fixes, and
also to help new contributors grow into people who can reduce maintainer
workload instead of adding to it.

The process was not perfect. Some patches were not good enough, and we also
made some mistakes early on when deciding what should be called a security
issue.  Our internal review process has been improving with the help of the
community.
That's great, keep it up!
quoted
But the remaining queue is still too large for us to handle.

Recently Jamal pointed out problems around our tags. That made me realize
that we should probably stop treating this as an ad-hoc patch effort and
build something closer to syzbot: public, reproducible, trackable,
deduplicated, and useful to maintainers.
Again, I think that effort is going to be larger than just getting the
patches fixed and pushed out.  It also turns into a central
point-of-failure, which is what we do not want to have at all for the
kernel.

But hey, I could be totally wrong.  Maybe some generous company that is
involved in unleashing this hell on us would be so kind as to pony up to
do the work to create this and help fix the issues that their tools are
finding.  Just like Google did in the past, there is precedent, but for
some reason people don't like learning from history...
We have also received some bug bounty rewards from Google, which gives
us some resources to put back into this effort.
We are prepared to invest more engineering time in fixing these bugs,
and we are also considering hiring engineers to help.

Will you also be attending NetDev in person? If so, perhaps we can chat there :)
It's going to be a long 18 months...

greg k-h
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help