Threat Feed methodology
A conservative selection process with exact-source approvals, honest date precision, immutable versions and failure retention.
The publication chain
- Original publisher
- Bounded extraction
- Normalized record
- Relevance review
- Exact-source approval
- Versioned static publication
Collection and normalization
Fetch offline over HTTPS using a named user agent and a response-size limit. Parse only the relevant article or machine-readable subject fields. Preserve original source identity, collection time, publisher dates where actually supplied, and a digest of reviewed content. Strip scripts and markup before subject classification. A changed heading, missing reviewed evidence phrase, unexpected redirect, or broken article selector fails closed for that source.
Dates retain their actual precision
Original publication, publisher update, first collection, successful source check and editorial review are separate fields. A date-only source keeps day precision, with no invented midnight or time zone. A catalog inclusion date is not vulnerability publication. Some migrated articles expose later platform metadata alongside an earlier original date; reviewed original publication is retained without treating migration as a new incident.
Classification and editorial decisions
Default-exclude generic items. Direct forensic integrity signals and context-sensitive candidates require explicit source support. CISA required-action text is not a classification input; its forensic-triage boilerplate cannot trigger inclusion. Review decisions bind the exact identifier, canonical source URL and content digest to a reason and original annotation. A changed subject digest or source identity returns the candidate to review.
Revision and failure handling
Source failure retains earlier good records and marks the source stale or unavailable. Reprocessing cached input does not update source-success timestamps. Public record versions distinguish source-content changes from annotation changes; earlier versions are retained. Selection snapshots are created for meaningful publication changes, including multiple genuine changes on one date. Rebuilding unchanged inputs does not create a new archive page.
Scope and rights
CISA KEV data is CC0; linked third-party material has its own terms. Other enabled reports are linked with concise evidence references and original annotations, not republished articles. CERT’s endpoint was verified but ingestion remains disabled pending appropriate reuse permission. No paid API, account signup, full-text syndication, or external aggregation dependency is required.
Refresh schedule and limitations
Normal checks are locally throttled to 23 hours per source. This is a conservative operating choice, not a representation of every publisher’s permitted rate. No scheduler or unattended deployment is active. A manual collection/build workflow is supplied for the maintainer. The fixed article sources are retrospective monitoring targets; they do not constitute continuous discovery of all new reports.
Corrections
Report the record ID, source URL, exact issue and supporting evidence through the existing contact route. A material correction needs a new publication version and actual review date. The library’s source policy distinguishes publisher observations from editorial interpretation and untested hypotheses.
Read the research and corrections standards · Contact and corrections