馃寪 US-Proxy
class="logged-out env-production page-responsive" style="word-wrap: break-word;" >
Skip to content

馃悰 fix(create): preserve config on encoding errors - #3400

Merged
gaborbernat merged 5 commits into
pypa:mainfrom
gaborbernat:fix/config-encode-before-write
Oct 17, 2026
Merged

gaborbernat merged 5 commits into
pypa:mainfrom
gaborbernat:fix/config-encode-before-write

Conversation

@gaborbernat

@gaborbernat gaborbernat commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

An encoding error while writing pyvenv.cfg left the file empty. The writer opened the file before it encoded the configuration text, and opening for writing truncates the file at once. A prompt supplied through the Python API that contained a character UTF-8 cannot encode, a lone surrogate such as \udcff, raised UnicodeEncodeError from the write and left a zero-byte pyvenv.cfg where an existing environment's configuration had been. 馃悰 After this PR the configuration is encoded before the file is opened, so a failed write leaves the previous contents in place.

The file was truncated before the text was encoded

PyEnvCfg.write built the configuration as a str and handed it to Path.write_text. That call opens the file in text mode with "w", which truncates it to zero bytes (open), and writes through a TextIOWrapper that encodes as it goes. A lone surrogate has no UTF-8 encoding, so the wrapper raised UnicodeEncodeError after the truncation and before any byte reached the file. The order dates from #1503.

The writer now encodes the whole text and passes the bytes to write_bytes. The encode fails before the open, with the file untouched. write_bytes opens with "wb", so a filesystem failure after the open can still leave a partial file, as before. write_text with its default newline=None also translated each \n to os.linesep (Path.write_text), so on Windows the file ended its lines with CRLF; write_bytes writes the bytes as given, and the file now ends its lines with LF on every platform.

sequenceDiagram
    participant W as PyEnvCfg.write
    participant C as UTF-8 codec
    participant F as pyvenv.cfg
    rect rgb(255, 214, 214)
    W->>F: open("w"): truncated to 0 bytes
    W->>C: TextIOWrapper.write encodes the text
    C-->>W: UnicodeEncodeError
    Note over F: previous contents gone
    end
    rect rgb(216, 243, 220)
    W->>C: text.encode("utf-8")
    C-->>W: UnicodeEncodeError, file untouched
    Note over W,F: open("wb") and the write run only after a successful encode
    end
Loading

Debug logs printed control characters raw

The same loop logged each line with %s. collapse_line_boundaries, from #3247, had already replaced every line boundary in the key and the value with a space, so the logged line could not split into two, but other control characters such as ESC went to the log as they were and a terminal rendered them. The line is now logged with %r, as a Python string literal, so a remaining control character appears as its escape: a prompt of a\x1bb logs as 'prompt = "a\x1bb"'.

flowchart LR
    V["value a ESC b"] --> B[collapse_line_boundaries]
    B --> S["%s: ESC reaches the terminal"]
    B --> R["%r: logged as 'prompt = #quot;a\x1bb#quot;'"]
    classDef bad fill:#ffd6d6,stroke:#c1121f,color:#5c0000;
    classDef ok fill:#d8f3dc,stroke:#2d6a4f,color:#1b4332;
    class S bad;
    class R ok;
Loading

Behavior changes

Unencodable values still raise UnicodeEncodeError, now before the file is opened, and an existing pyvenv.cfg keeps its contents. Written files end their lines with LF on every platform; on Windows they ended with CRLF before. Debug logs show each configuration line as a quoted Python string literal with control characters escaped. A filesystem failure after the open can still leave a partial file.

Reported by Anthropic's OSS Scanner: ANT-2026-RNWHCGDO.

@gaborbernat
gaborbernat force-pushed the fix/config-encode-before-write branch from 49f3de2 to 38bc704 Compare October 16, 2026 14:13
Reported by Anthropic's OSS Scanner: ANT-2026-RNWHCGDO.
Debug logs collapse line boundaries to spaces before the %r
formatting runs, so the changelog's claim that they stay visible
was wrong; what %r shows is the remaining control characters.
Reword the fragment, note that the file now uses LF line endings on
every platform, and add a test for the escaped log line.
@gaborbernat
gaborbernat merged commit a74295a into pypa:main Oct 17, 2026
76 checks passed
@gaborbernat
gaborbernat deleted the fix/config-encode-before-write branch October 17, 2026 10:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant