No one can read 5,087 security articles a year

The request

The client is a B2B software company whose security team follows a public security-news feed. The feed delivered 5,087 articles in 12 months: about 420 a month, 14 a day, every day. At even one minute each, that is roughly 85 hours of reading before anyone has judged a single one.

No one does that. A person skimming the feed either misses the one article that matters or learns to ignore the feed. And skimming is not the real job: to say whether an article affects the client, you have to know what they run, how it is exposed, and whether the flaw works in their configuration.

So the brief was not how to read faster. It was how to make sure a person only ever reads the handful of articles that deserve it.

How we think

The design follows a few rules of thought, and they matter more than any tool we picked.

1. Spend the expensive reader last. Cheap, deterministic rules go first and shrink the pile. An LLM only reads what is left. We never send the whole feed to a model.
2. Judge against what the client actually runs. An article is not "important" in general. It is important if it concerns a product in the client's own inventory, in the way that product is exposed. The inventory is the anchor for every verdict.
3. Most articles should be GREEN. A review that finds something in everything is noise. The goal is that the large majority end as no impact, and only a small set gets a real impact estimate.
4. A rule may shrink the LLM queue, never grow it. Every new rule must move articles out of the queue. If a lookup can answer the question (does this package or plugin appear in the client's own lockfiles?), it is not an LLM job.
5. Scope is a decision, not an accident. Vendor-managed software and workstation tools are out of scope by policy, because this review is about the client's own product. The policy is written down and can be changed in one place.
6. The alarm must stay rare and credible. RED means "needed an emergency patch when published", and nothing softer. A security team that is warned too often stops listening.
7. One judge sees everything at once. Separate batches of ratings run hot and disagree. A single reviewer reading all the serious stories together applies one rule consistently.
8. Say what we do not know. Components the client is only assumed to run cannot reach RED, and the reviewer records why a story was held back. Article text is treated as untrusted data, never as instructions.
9. End in an action. Duplicate coverage is merged into one issue, and every issue carries a concrete next step, not just a score.

How it works

1. Arrived in the feed5,087Security-news feed, 12 months
2. Mention a product the client runs1,409Rule pre-filter: keyword and CVE match against the client inventory27.7% of the stage above
3. Security issue in product or infrastructure391Rule triage: security-related and in scope by policy27.7% of the stage above
4. Rules end here: the LLM reads only these 391Concrete flaw in software the client runs125LLM quick pass: a specific flaw, not a passing mention32.0% of the stage above
5. Rated RED or AMBER76LLM impact rating, then one consistency review60.8% of the stage above

4,696 articles (92.3%) never reach an LLM.

Plain rules handle the first 92%: 4,696 of 5,087 articles never reach a model. The 391 that remain are read by LLM subagents, which keep 125 for a full impact rating using the article text and public exploitation data. Anything a lookup can answer, such as a malicious package or a plugin on a list, becomes BLUE (a mechanical check) instead of an LLM job.

In order: the rule pre-filter matches each article against the client’s inventory, and rule triage sorts what matches into GREEN, BLUE, or grey for the LLM. The LLM quick pass reads titles and summaries and answers green, dependency, or assess. Assessed articles are grouped into stories, enriched with the article text and with exploitation data (CISA KEV and EPSS), and rated RED, AMBER, YELLOW or GREEN. One consistency review then reads all RED and AMBER stories together, merges duplicates into issues, and applies the strict RED rule. The output is one row per article with a final status, and one row per issue with first and last seen dates, article count, and a concrete action.

Results

93% of the year’s articles needed no action, and 76 articles (1.5%) were rated RED or AMBER. Those 76 articles collapse into 31 distinct issues, because several outlets cover the same flaw. The urgent reading shrank from 5,087 articles to 31 issues. Lower-priority work remains beside it: 43 YELLOW issues and 231 BLUE articles that need a mechanical check.

Status	Meaning	Articles	Share	Issues
RED	Needed an emergency patch when published	28	0.6%	7
AMBER	Real exposure, but needs a precondition (local access, credentials, a specific configuration)	48	0.9%	24
YELLOW	Unusual configuration, denial of service on an internal component, or low likelihood	44	0.9%	43
BLUE	Needs a mechanical lookup (dependency lockfiles, plugin lists)	231	4.5%	n/a
GREEN	No impact on what the client runs, or out of scope by policy	4,736	93.1%	n/a

GREEN is not one thing. 3,678 articles name nothing the client runs. 489 mention the client’s products but are informational, a passing mention, or judged not affected. 569 are out of scope by policy (vendor-managed SaaS and workstation software): they are not proven safe, they are simply not this review’s job.

Seven RED issues in 12 months is about one every seven weeks. That rarity is the point: when RED appears, it is believed.

What made the ratings trustworthy

One extra review step cut the RED count sharply. The unit here is the story, meaning the articles about one event; duplicate stories are later merged into one issue. Rated in separate batches, 21 stories came back RED, because each batch ran hot. A single reviewer then read all 44 RED and AMBER stories together and applied one strict RED rule. Two things followed: RED fell from 21 stories to 14, and merging duplicate coverage turned those 14 into 7 RED issues (all 44 stories became 32 issues). RED requires all of these:

• the component is in the client’s inventory, or it is a supply-chain compromise of a tool the client uses
• it is internet-facing
• no login is needed
• the impact is remote code execution or data and secret theft
• it works in a common configuration
• it is exploited, in CISA KEV, or scored CVSS 9 or higher

Anything else is AMBER at most. The reviewer records why a story was held back, for example “needs an authenticated account; customer accounts are issued, not self-registered”. The target was that RED means “needed an emergency patch when published”, so the security team is not warned for things that did not need one.

Both sides were checked by hand, within the capacity available. Every RED and AMBER issue was reviewed and confirmed as a true positive. Lower-rated articles were checked on a smaller scale, by a random sample plus a pick by severity. That is a spot check, not a measured miss rate: reviewing all 5,087 articles by hand was never possible with the people available, which is why the pipeline exists.

Terms used

• Inventory. The client’s own list of the products it runs, with how each is exposed (internal or external) and its type (SaaS, server, workstation, library).
• LLM queue. The articles that rules could not settle and that wait for an LLM to read them.
• Story and issue. A story is the set of articles about one event. Duplicate stories are merged into one issue, which is the unit a person acts on.
• CISA KEV. The public catalog, kept by the US cybersecurity agency CISA, of vulnerabilities known to be exploited in real attacks.
• CVSS. A 0 to 10 severity score for a vulnerability; 9 and above is critical.
• EPSS. A published estimate of how likely a vulnerability is to be exploited soon.
• Lockfile. A file that lists the exact package versions a software project uses, so a malicious package can be looked up mechanically.

Limits and open questions

• Assumed components cap the rating. 13 inventory entries are assumed, not confirmed, so their issues stop at AMBER. The reviewer marked two of them RED if the component is confirmed internet-facing.
• Scope is the product, by policy. 569 articles (176 vendor-managed SaaS, 393 workstation software) were marked GREEN because this review covers the product, not IT patching.
• One large source is read from its summary only. Its 2,322 articles are not fetched because the site blocks scripted access, so the rating rests on the feed summary.
• Ratings are model judgments. The numbers show how many issues were escalated, not whether each RED was patched in time.