Advisory

PYSEC-2026-3906

Also known as: CVE-2026-55520, GHSA-wjmf-p669-5m5p

Summary

Protego has exponential backtracking ReDoS in robots.txt URL wildcard matching

Details

Problem description

Protego constructs regular expressions to match URLs against robots.txt Allow: and Disallow: directives, see protego._urlpattern._URLPattern._prepare_pattern_for_regex(). Every * in the directive value is translated into a lazy .*? regex piece, thus a specially crafted directive value with many asterisks may produce a regex that freezes the parser due to exponential backtracking.

Impact

Parsing a specially crafted robots.txt with protego.Protego.parse() and then trying to match an URL with protego.Protego.can_fetch() results in the latter call not returning for a period dependent on the length of the URL.

Proof of concept

from protego import Protego

robotstxt = f"""
User-agent: *
Disallow: /{"*1" * 12}*Z
"""
rp = Protego.parse(robotstxt)
url = "/" + "1" * 60
rp.can_fetch(url, "mybot")  # freezes

Severity

Not rated by the source.

Dates

Published
2026-09-10 09:44 UTC
Modified
2026-09-10 12:15 UTC

Affected packages and fixes

protego

PyPI · All systems

Affected

  • introduced 0, fixed 0.6.2 (ecosystem)

Fixed versions

0.6.2

Fix

  • Fixed version 0.6.2
    pip install --upgrade protego==0.6.2

Fix commands are the export's own generic commands. Ojuju never runs them. Check them against your setup first.

Check your own computer: guard packages check

View this record on OSV.dev

Source

Source: PyPA Advisory Database · CC BY 4.0

Sources & licences