Raw f-string format spec undocumented behavior change between 3.11 and 3.12

[MeGaGiGaGon]

Bug report

Bug description:

I found this while working on astral-sh/ruff#19546 .

Between 3.11 and 3.12, the behavior of raw f-string format specifiers changed.

On 3.11 and earlier, the rawness of the f-string is respected in the format specifier:

PS ~>uvx python@3.11 -c @'
class UnchangedFormat:
    def __format__(self, format):
        return format

print("Non-raw output:", repr(f"{UnchangedFormat():\xFF}"))
print("Raw output:", repr(rf"{UnchangedFormat():\xFF}"))
'@
Non-raw output: 'ÿ'
Raw output: '\\xFF'

On 3.12 and later, the rawness of the f-string is not respected:

PS ~>uvx python@3.12 -c @'
class UnchangedFormat:
    def __format__(self, format):
        return format

print("Non-raw output:", repr(f"{UnchangedFormat():\xFF}"))
print("Raw output:", repr(rf"{UnchangedFormat():\xFF}"))
'@
Non-raw output: 'ÿ'
Raw output: 'ÿ'

This new behavior looks to have been carried to t-strings on 3.14 and later:

PS ~>uvx python@3.14 -c @'
class UnchangedFormat:
    def __format__(self, format):
        return format

print("Non-raw output:", repr(t"{UnchangedFormat():\xFF}"))
print("Raw output:", repr(rt"{UnchangedFormat():\xFF}"))
'@
Non-raw output: Template(strings=('', ''), interpolations=(Interpolation(<__main__.UnchangedFormat object at 0x00000183F9904EC0>, 'UnchangedFormat()', None, 'ÿ'),))
Raw output: Template(strings=('', ''), interpolations=(Interpolation(<__main__.UnchangedFormat object at 0x00000183F9908A50>, 'UnchangedFormat()', None, 'ÿ'),))

I could not find any information/documentation about how the rawness of f-strings affect their format specifiers, despite looking in:

• The input output tutorial f-string section
• The lexical analysis section on f-strings
• PEP 498 – Literal String Interpolation
• PEP 701 – Syntactic formalization of f-strings
• The 3.12 what's new section
• Multiple CPython github issue searches
• Asking in the Python discord

CPython versions tested on:

3.12, 3.11, 3.14

Operating systems tested on:

Windows

Linked PRs

• gh-137314: Fix incorrect treatment of format specs in raw fstrings #137328
• [3.14] gh-137314: Fix incorrect treatment of format specs in raw fstrings (GH-137328) #137344
• [3.13] gh-137314: Fix incorrect treatment of format specs in raw fstrings (GH-137328) #137345

[picnixz]

Unfortunately, 3.11 and 3.12 are both security-only and I don't think this is a security issue. AFAIK, there isn't an exceptional rule for such bugs, so even if this is a regression, we won't patch it :( However, we may fix it in 3.13+ (maybe?)

cc @pablogsal

[ericvsmith]

This looks like a bug to me. I think it should be fixed in 3.14 and backported to 3.13.

[pablogsal]

This looks like a bug to me. I think it should be fixed in 3.14 and backported to 3.13.

Agreed 👍