Advisory

GHSA-qx2g-xrx7-vfh8

Also known as: CVE-2026-72818, PYSEC-2026-3870

Summary

NLTK TweetTokenizer vulnerable to denial of service through catastrophic regex backtracking

Details

The URLS regular expression in nltk/tokenize/casual.py, compiled into TweetTokenizer.WORD_RE and applied by TweetTokenizer.tokenize, contains a naked-domain branch whose domain-label prefix [a-z0-9]+(?:[.\-][a-z0-9]+)* is unbounded. Input consisting of many alternating label separators can be partitioned in exponentially many ways, and because the branch also requires a trailing top-level domain that such input never supplies, the engine explores those partitions before failing at each offset. A few kilobytes of input therefore consumes seconds to minutes of single-threaded CPU, and the HANG_RE substitution performed before matching does not collapse the pattern. TweetTokenizer is intended for tokenizing untrusted social-media text, so any service that applies it, or the module-level casual_tokenize, to submitted text can be stalled per request without authentication. Version 3.10.1 bounds the label repetition.

Severity

High Basis: The source's own band: HIGH

Dates

Published
2026-08-21 00:31 UTC
Modified
2026-09-10 12:25 UTC

Affected packages and fixes

nltk

PyPI · All systems

Affected

  • introduced 0, fixed 3.10.1 (ecosystem)

Fixed versions

3.10.1

Fix

  • Fixed version 3.10.1
    pip install --upgrade nltk==3.10.1

Fix commands are the export's own generic commands. Ojuju never runs them. Check them against your setup first.

Check your own computer: guard packages check

View this record on OSV.dev

Source

Source: GitHub Advisory Database (reviewed advisories) · CC BY 4.0

Sources & licences