馃寪 US-Proxy
class="logged-out env-production page-responsive" style="word-wrap: break-word;" >
Skip to content

馃悰 fix(seed): preserve interpreter paths in console scripts - #3417

Merged
gaborbernat merged 7 commits into
pypa:mainfrom
gaborbernat:fix/console-script-paths
Oct 17, 2026
Merged

gaborbernat merged 7 commits into
pypa:mainfrom
gaborbernat:fix/console-script-paths

Conversation

@gaborbernat

@gaborbernat gaborbernat commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Every console script a seeder writes into bin/ starts with a launcher (the lines before the Python body that hand the file to the environment's interpreter). When the interpreter path contains a space, distlib falls back to a /bin/sh form and placed the path in double quotes. POSIX sh keeps $(...), backticks, $name and backslashes live inside double quotes, so a destination such as $(echo a)/env ran the substitution, the launcher exited with status 127, and a backslash in the path reached Python's tokenizer as an escape sequence. A tab in the path took a different route: it is not a space, so distlib wrote a direct #! line and the kernel split the interpreter name at the tab. 馃悰 After this PR both seeders write a launcher that sh and Python read as the same literal path, and any whitespace in the path selects the sh form. This addresses launcher generation during seeding for GHSA-c978-59m8-c77v.

Double quotes left shell expansion live in the exec line

Since #1880, ScriptMakerCustom.__init__ handed distlib the interpreter path through enquote_executable, which wraps a path containing a space in double quotes. distlib's ScriptMaker then writes a direct #!path line when the executable has no space and fits in the kernel's shebang length limit, and otherwise three lines: #!/bin/sh, then '''exec' <executable> "$0" "$@", then ' '''. Two readers parse the second line. The kernel starts /bin/sh, which runs exec with the quoted path. Python, when the file is passed to it as a script, reads lines two and three as one triple-quoted string literal, so the exec never runs there. Inside double quotes sh still performs command substitution and parameter expansion, and a backslash in the same line is an escape sequence to Python. A tab escaped both checks, since enquote_executable and _build_shebang test for a space only, so the direct line reached execve(2), which separates the interpreter from its optional argument at whitespace and failed with ENOENT.

_build_shebang now keeps distlib's result off POSIX and whenever that result is a single word, which is the direct line for a path without whitespace. Otherwise shell_shebang builds the three lines from the unquoted path. shlex.quote puts the path in single quotes, where sh expands nothing. Each backslash then becomes '"\\"': the single quote closes, a double-quoted \\ is one backslash to sh and a \\ escape to Python, and the single quote reopens. A direct line whose path contains a tab splits into two words, so it takes the sh form as well.

sequenceDiagram
    participant K as kernel
    participant S as /bin/sh
    participant P as python
    K->>S: execve: line 1 names /bin/sh
    rect rgb(255, 214, 214)
    Note over S: before: the path sits in double quotes
    S->>S: $( ), backticks and backslashes expand
    S->>P: exec of a different path
    end
    rect rgb(216, 243, 220)
    Note over S: now: the path sits in single quotes
    S->>P: exec of the literal path, "$0" "$@"
    end
    Note over P: lines 2 and 3 read as one string literal
Loading

Pip wrote the same launcher during pip seeding

The pip seeder runs python -m pip install inside the new environment. pip's vendored distlib builds the launcher the same way, from sys.executable passed through enquote_executable, so the launchers of the seeded distributions carried the same double-quoted exec line or the same unrunnable direct line. virtualenv did not touch them.

After the install, on POSIX, _rewrite_launchers walks the RECORD of each seeded distribution and looks at the entries that sit in the script directory. Two first lines qualify: the sh exec form, and a direct line naming the environment interpreter when that path contains whitespace. The launcher lines are replaced by shell_shebang for the environment's interpreter and the Python body is kept. The file is written back under its own mode masked to 0o755, and its row in RECORD gets the new sha256 and size, so the RECORD file matches the file on disk. A launcher whose direct line is one word stays as pip wrote it, so those environments do not gain a /bin/sh dependency.

flowchart LR
    R[launcher listed in RECORD] --> F{first line}
    F -- sh exec form --> W[requoted by shell_shebang]
    F -- direct shebang with whitespace --> W
    F -- direct shebang, one word --> K[kept as pip wrote it]
    W --> H[sha256 and size refreshed in RECORD]
    classDef bad fill:#ffd6d6,stroke:#c1121f,color:#5c0000;
    classDef ok fill:#d8f3dc,stroke:#2d6a4f,color:#1b4332;
    class W,H ok;
Loading

Behavior changes

On POSIX both seeders write, for an interpreter path with a space, a /bin/sh launcher that single-quotes the path; the double-quoted form is gone. A path with a tab now gets the sh form where it got a direct line the kernel could not run. The app-data seeder keeps distlib's direct line for a path without whitespace. After pip seeding, virtualenv rewrites the launchers of the seeded packages that use the sh form or a direct line with whitespace, and refreshes their RECORD rows. A later pip install -U pip, or any other install inside the environment, writes pip's own launcher form again. Environments created by an earlier virtualenv keep their old launchers until they are recreated. Windows keeps distlib's executable launcher and is not rewritten. Tests cover a space, an apostrophe, $(echo a) and backticks, a 120-character name, a backslash and a tab in the interpreter path for both seeders, started directly and through python, and check the RECORD hash and size of rewritten pip launchers.

Reported by Anthropic's OSS Scanner: ANT-2026-5DZVLGL1.

@gaborbernat gaborbernat added bug security Fixes a vulnerability or hardens the supply chain, CI or release labels Oct 16, 2026
@gaborbernat
gaborbernat force-pushed the fix/console-script-paths branch from 233397c to b8f76e9 Compare October 16, 2026 14:23
@gaborbernat
gaborbernat marked this pull request as draft October 16, 2026 14:28
@gaborbernat
gaborbernat force-pushed the fix/console-script-paths branch 3 times, most recently from 41e4ae2 to 44cbea4 Compare October 17, 2026 10:37
Reported by Anthropic's OSS Scanner: ANT-2026-5DZVLGL1.
Rewriting every pip-seeded launcher to the /bin/sh form made environments
whose interpreter path has no whitespace depend on a shell they never
needed, and diverged from the app-data seeder, which keeps distlib's
direct shebang there. Only launchers that already use the shell form, or
whose interpreter path the kernel would split, are rewritten now.

Add a command substitution case to the launcher test so it fails on the
unquoted launcher, and check which shebang form the pip seeder leaves.
@gaborbernat
gaborbernat force-pushed the fix/console-script-paths branch from 44cbea4 to d6fab38 Compare October 17, 2026 10:40
@gaborbernat
gaborbernat marked this pull request as ready for review October 17, 2026 10:41
@gaborbernat
gaborbernat merged commit 2519941 into pypa:main Oct 17, 2026
74 checks passed
@gaborbernat
gaborbernat deleted the fix/console-script-paths branch October 17, 2026 10:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug security Fixes a vulnerability or hardens the supply chain, CI or release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant