Advisory

GHSA-cc65-xxvf-f7r9

Also known as: CVE-2024-1892, PYSEC-2024-162

Summary

Scrapy vulnerable to ReDoS via XMLFeedSpider

Details

Impact

The following parts of the Scrapy API were found to be vulnerable to a ReDoS attack:

  • The `XMLFeedSpider` class or any subclass that uses the default node iterator: iternodes, as well as direct uses of the scrapy.utils.iterators.xmliter function.

Handling a malicious response could cause extreme CPU and memory usage during the parsing of its content, due to the use of vulnerable regular expressions for that parsing.

Patches

Upgrade to Scrapy 2.11.1.

If you are using Scrapy 1.8 or a lower version, and upgrading to Scrapy 2.11.1 is not an option, you may upgrade to Scrapy 1.8.4 instead.

Workarounds

For XMLFeedSpider, switch the node iterator to xml or html.

For open_in_browser, before using the function, either manually review the response content to discard a ReDos attack or manually define the base tag to avoid its automatic definition by open_in_browser later.

Acknowledgements

This security issue was reported by @nicecatch2000 through huntr.com.

Severity

High Basis: The source's own band: HIGH

Dates

Published
2024-02-15 15:22 UTC
Modified
2025-01-14 17:20 UTC

Affected packages and fixes

scrapy

PyPI · All systems

Affected

  • introduced 2, fixed 2.11.1 (ecosystem)

Fixed versions

2.11.1

Fix

  • Fixed version 2.11.1
    pip install --upgrade scrapy==2.11.1

scrapy

PyPI · All systems

Affected

  • introduced 0, fixed 1.8.4 (ecosystem)

Fixed versions

1.8.4

Fix

  • Fixed version 1.8.4
    pip install --upgrade scrapy==1.8.4

Fix commands are the export's own generic commands. Ojuju never runs them. Check them against your setup first.

Check your own computer: guard packages check

View this record on OSV.dev

Source

Source: GitHub Advisory Database (reviewed advisories) · CC BY 4.0

Sources & licences