馃寪 US-Proxy
class="logged-out env-production page-responsive" style="word-wrap: break-word;" >
Skip to content

馃悰 fix(seed): isolate shared cache compilation - #3407

Merged
gaborbernat merged 3 commits into
pypa:mainfrom
gaborbernat:fix/seed-compile-isolation
Oct 17, 2026
Merged

gaborbernat merged 3 commits into
pypa:mainfrom
gaborbernat:fix/seed-compile-isolation

Conversation

@gaborbernat

@gaborbernat gaborbernat commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

The app-data seeder with --symlink-app-data builds one image of each seed wheel in the app-data cache (the per-user directory where virtualenv keeps wheels and extracted images) and symlinks that image into every environment it creates. Building the image ran the destination interpreter as python -m compileall <image> with its default startup. That startup puts the working directory first on sys.path and imports site, which runs the .pth files in the destination's site-packages. A compileall.py in the directory virtualenv was run from replaced the standard library module, and an import line in a .pth file of an existing destination ran before compilation started. 馃悰 The compile step now runs with -I -S, so neither the working directory nor the destination's site configuration takes part, and the bytecode in the shared image comes from the standard library's compileall alone.

The compile child started with the destination's full startup

SymlinkPipInstall._generate_new_files ran self._creator.exe -m compileall <image> with no interpreter flags. #1480 added the call when the shared image became read-only, so that the .pyc files exist before an import from a linked environment would try to write them and fail. The -m switch prepends the working directory to sys.path, so import compileall found a compileall.py next to where virtualenv was invoked before the standard library's. Default startup also imports site, which adds the destination's site-packages, executes every line beginning with import in the .pth files there, and then imports sitecustomize and usercustomize. On a re-run over an existing path, or with --system-site-packages, those directories hold whatever was there before. PYTHONPATH, PYTHONHOME and the other PYTHON* variables of the calling process applied as well. FromAppData.run keys the image by installer and wheel and builds it once, and every later environment created from the same cache links to it, so code that ran in the child could change the image before build_image made it read-only. A planted compileall.py that exits at once leaves the image with no bytecode, and the build goes on, since the child's exit status and output are discarded.

The command now carries -I and -S. -I keeps the working directory off sys.path, ignores the PYTHON* variables and skips the user site-packages. -S skips the site import, so no .pth file, sitecustomize or usercustomize runs. sys.path is then the standard library of the interpreter the destination was created from, which is where compileall comes from. The child is still the destination interpreter, so the bytecode still matches the target Python version. Its exit status and output are still not read, as before. The test plants a compileall.py in the working directory and a .pth file in the destination's site-packages, each of which would create a marker file, and checks that pip/__pycache__ is populated and no marker exists.

sequenceDiagram
    participant V as virtualenv
    participant C as destination python
    participant W as working directory
    participant S as destination site-packages
    V->>C: -m compileall image
    rect rgb(255, 214, 214)
    C->>W: sys.path[0]: a compileall.py here wins
    C->>S: import site: .pth import lines run, sitecustomize imported
    Note over C: PYTHONPATH and the other PYTHON* variables applied
    end
    V->>C: -I -S -m compileall image
    rect rgb(216, 243, 220)
    Note over C: sys.path is the stdlib only, site never imported
    C->>C: stdlib compileall writes image/**/__pycache__
    end
Loading

Behavior changes

Symlink seeding compiles its shared image without the destination's .pth files, sitecustomize and usercustomize, without the user site-packages, without the working directory on sys.path and without PYTHON* environment variables. A .pth file or sitecustomize that an existing destination relied on during that step no longer runs there. Startup of the completed environment is unchanged, and so is the copy installer, which has no compile step.

Reported by Anthropic's OSS Scanner: ANT-2026-9S579FUR.

@gaborbernat
gaborbernat force-pushed the fix/seed-compile-isolation branch from e19806d to 64ad260 Compare October 16, 2026 14:22
@gaborbernat
gaborbernat force-pushed the fix/seed-compile-isolation branch from 64ad260 to e97ae5a Compare October 16, 2026 14:25
The test asserted the compileall command line. It now plants a
compileall.py in the working directory and a .pth in the destination
site-packages, then checks that neither runs while the image still gets
its bytecode. Both cases fail with the previous command.
@gaborbernat
gaborbernat merged commit 4aaf75a into pypa:main Oct 17, 2026
144 of 146 checks passed
@gaborbernat
gaborbernat deleted the fix/seed-compile-isolation branch October 17, 2026 10:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant