馃寪 US-Proxy
class="logged-out env-production page-responsive" style="word-wrap: break-word;" >
Skip to content

馃悰 fix(seed): preserve shared cache when reseeding with pip - #3423

Merged
gaborbernat merged 6 commits into
pypa:mainfrom
gaborbernat:fix/pip-reseed-links
Oct 17, 2026
Merged

gaborbernat merged 6 commits into
pypa:mainfrom
gaborbernat:fix/pip-reseed-links

Conversation

@gaborbernat

@gaborbernat gaborbernat commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

An environment created with --symlink-app-data holds its seed packages as symlinks into the shared wheel image (the extracted copy of the wheel in the app-data cache, made read-only and shared by every environment seeded from it). Reseeding that environment with --seeder pip ran pip install --ignore-installed against the linked paths, so pip wrote through the links. 馃悰 After this PR the pip seeder removes the links in the seeded package roots, at their launcher names and in their metadata before pip starts, so pip installs local files and the image stays as it was. This builds on #3416.

pip installed through links into the shared image

The symlink seeder links each top-level image entry into the environment's site-packages and makes the image read-only, a layout from #1480. The pip seeder ran pip with --ignore-installed, added in #3064, against the existing destination. With site-packages/pip a link to the image's pip directory, pip's writes went to the image, and reinstalling through those links could change the cache's contents and permissions for every environment sharing it.

run now detaches the links between resolving the wheels and starting pip. _detach_links reads each selected wheel's entry names and takes the top-level names as the package roots, in purelib and platlib. _detach_tree unlinks a root that is itself a link; a root that is a real directory is walked with os.walk and followlinks=False, and every linked file or directory inside it is unlinked, so a package partly copied and partly linked is detached too. A root name that is not a plain file name ("", ., .., or anything with a separator) raises RuntimeError before pip runs. pip then installs the package into a directory that holds no links.

sequenceDiagram
    participant S as pip seeder
    participant E as env site-packages/pip
    participant I as shared image (read-only)
    Note over E,I: site-packages/pip is a symlink into the image
    rect rgb(255, 214, 214)
    S->>E: pip install --ignore-installed pip==X
    E->>I: writes follow the link into the image
    end
    rect rgb(216, 243, 220)
    S->>E: unlink the root, or the links inside it
    S->>E: pip install --ignore-installed pip==X
    Note over E: local files, image untouched
    end
Loading

Launchers and older metadata stayed linked

A launcher name taken from the wheel's entry_points.txt can be a link as well, and a script written through a link lands in the link's target. _detach_entry_points reads the console_scripts and gui_scripts sections of each selected wheel, and _detach_script unlinks the name in the script directory when it is a link, together with the pip, pipX and pipX.Y variants for pip and the .exe, -script.py and -script.pyw forms on Windows. A name that is not a plain file name raises RuntimeError before pip runs. Names the wheels do not declare are left alone, so an unrelated launcher link stays in place.

--ignore-installed does not uninstall, so the pip-<old>.dist-info link and the pip-<old>.virtualenv marker (the empty file the app-data seeder adds beside each dist-info, from _generate_new_files) survived a reseed to a newer version, and package listings reported both versions. _detach_metadata unlinks every linked <distribution>-*.dist-info and <distribution>-*.virtualenv entry of a seeded distribution before pip runs.

flowchart LR
    W[selected wheels] --> N[entry names] --> R[top-level roots in purelib and platlib]
    R --> V{plain name?}
    V -- no --> X[RuntimeError before pip]
    V -- yes --> T[unlink root link, or links inside the root]
    W --> EP[entry_points.txt] --> SC[console and gui script names] --> V2{plain name?}
    V2 -- no --> X
    V2 -- yes --> U[unlink linked launchers, pip variants included]
    T --> MD[unlink linked dist-info and .virtualenv of the distribution]
    U --> MD
    MD --> PIP[pip install --ignore-installed]
    classDef bad fill:#ffd6d6,stroke:#c1121f,color:#5c0000;
    classDef ok fill:#d8f3dc,stroke:#2d6a4f,color:#1b4332;
    class X bad;
    class T,U,MD,PIP ok;
Loading

Entry-point names lost their case and duplicates aborted the read

The app-data seeder parses entry_points.txt with a plain ConfigParser. Its default optionxform lowercases option names, so a script declared as MyTool was generated as mytool, and its default strict=True raises DuplicateOptionError on a repeated name. Both seeders now parse with EntryPointConfig, constructed with interpolation=None and strict=False (app-data seeder, pip seeder). Names keep their case, the last of duplicate entries wins, and a % in a value is read verbatim.

flowchart LR
    F["[console_scripts]\nMyTool = old:main\nMyTool = example:main"] --> A["ConfigParser()"]
    A --> A1["DuplicateOptionError, or mytool when unique"]
    F --> B["EntryPointConfig(interpolation=None, strict=False)"]
    B --> B1["MyTool = example:main"]
    classDef bad fill:#ffd6d6,stroke:#c1121f,color:#5c0000;
    classDef ok fill:#d8f3dc,stroke:#2d6a4f,color:#1b4332;
    class A1 bad;
    class B1 ok;
Loading

Behavior changes

Switching from app-data links to the pip seeder leaves the seeded packages installed locally and the shared image unchanged. Existing links inside those seed packages are removed and replaced as pip installs their contents. Linked launchers named by the seeded wheels are replaced by regular scripts; other launcher links are untouched. Invalid package-root or launcher names in wheel metadata fail before the pip subprocess starts. Linked metadata of older versions of a seeded distribution is removed, so package listings report the installed version. Console entry-point names retain their case in both seeders, and duplicate entries use the last value instead of failing. A symlinked site-packages parent requires the directory preparation in #3429; this branch alone does not protect that parent.

Reported by Anthropic's OSS Scanner: ANT-2026-14ZEQFWC.

@gaborbernat gaborbernat changed the title 馃悰 fix: preserve shared cache when reseeding with pip 馃悰 fix(seed): preserve shared cache when reseeding with pip Oct 17, 2026
@gaborbernat
gaborbernat force-pushed the fix/pip-reseed-links branch 3 times, most recently from dc0b1d8 to ee087d6 Compare October 17, 2026 10:31
@gaborbernat
gaborbernat merged commit 70a0102 into pypa:main Oct 17, 2026
73 of 74 checks passed
@gaborbernat
gaborbernat deleted the fix/pip-reseed-links branch October 17, 2026 10:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant