Repository navigation
馃悰 fix(seed): preserve shared cache when reseeding with pip - #3423
Merged
Merged
Conversation
gaborbernat
force-pushed
the
fix/pip-reseed-links
branch
from
October 16, 2026 14:23
dc45871 to
0936955
Compare
gaborbernat
force-pushed
the
fix/pip-reseed-links
branch
3 times, most recently
from
October 17, 2026 10:31
dc0b1d8 to
ee087d6
Compare
Reported by Anthropic's OSS Scanner: ANT-2026-14ZEQFWC.
gaborbernat
force-pushed
the
fix/pip-reseed-links
branch
from
October 17, 2026 10:35
ee087d6 to
306bcf1
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
An environment created with
--symlink-app-dataholds its seed packages as symlinks into the shared wheel image (the extracted copy of the wheel in the app-data cache, made read-only and shared by every environment seeded from it). Reseeding that environment with--seeder pipranpip install --ignore-installedagainst the linked paths, so pip wrote through the links. 馃悰 After this PR the pip seeder removes the links in the seeded package roots, at their launcher names and in their metadata before pip starts, so pip installs local files and the image stays as it was. This builds on #3416.pip installed through links into the shared image
The symlink seeder links each top-level image entry into the environment's site-packages and makes the image read-only, a layout from #1480. The pip seeder ran pip with
--ignore-installed, added in #3064, against the existing destination. Withsite-packages/pipa link to the image'spipdirectory, pip's writes went to the image, and reinstalling through those links could change the cache's contents and permissions for every environment sharing it.runnow detaches the links between resolving the wheels and starting pip._detach_linksreads each selected wheel's entry names and takes the top-level names as the package roots, inpurelibandplatlib._detach_treeunlinks a root that is itself a link; a root that is a real directory is walked withos.walkandfollowlinks=False, and every linked file or directory inside it is unlinked, so a package partly copied and partly linked is detached too. A root name that is not a plain file name ("",.,.., or anything with a separator) raisesRuntimeErrorbefore pip runs. pip then installs the package into a directory that holds no links.sequenceDiagram participant S as pip seeder participant E as env site-packages/pip participant I as shared image (read-only) Note over E,I: site-packages/pip is a symlink into the image rect rgb(255, 214, 214) S->>E: pip install --ignore-installed pip==X E->>I: writes follow the link into the image end rect rgb(216, 243, 220) S->>E: unlink the root, or the links inside it S->>E: pip install --ignore-installed pip==X Note over E: local files, image untouched endLaunchers and older metadata stayed linked
A launcher name taken from the wheel's
entry_points.txtcan be a link as well, and a script written through a link lands in the link's target._detach_entry_pointsreads theconsole_scriptsandgui_scriptssections of each selected wheel, and_detach_scriptunlinks the name in the script directory when it is a link, together with thepip,pipXandpipX.Yvariants for pip and the.exe,-script.pyand-script.pywforms on Windows. A name that is not a plain file name raisesRuntimeErrorbefore pip runs. Names the wheels do not declare are left alone, so an unrelated launcher link stays in place.--ignore-installeddoes not uninstall, so thepip-<old>.dist-infolink and thepip-<old>.virtualenvmarker (the empty file the app-data seeder adds beside each dist-info, from_generate_new_files) survived a reseed to a newer version, and package listings reported both versions._detach_metadataunlinks every linked<distribution>-*.dist-infoand<distribution>-*.virtualenventry of a seeded distribution before pip runs.flowchart LR W[selected wheels] --> N[entry names] --> R[top-level roots in purelib and platlib] R --> V{plain name?} V -- no --> X[RuntimeError before pip] V -- yes --> T[unlink root link, or links inside the root] W --> EP[entry_points.txt] --> SC[console and gui script names] --> V2{plain name?} V2 -- no --> X V2 -- yes --> U[unlink linked launchers, pip variants included] T --> MD[unlink linked dist-info and .virtualenv of the distribution] U --> MD MD --> PIP[pip install --ignore-installed] classDef bad fill:#ffd6d6,stroke:#c1121f,color:#5c0000; classDef ok fill:#d8f3dc,stroke:#2d6a4f,color:#1b4332; class X bad; class T,U,MD,PIP ok;Entry-point names lost their case and duplicates aborted the read
The app-data seeder parses
entry_points.txtwith a plainConfigParser. Its defaultoptionxformlowercases option names, so a script declared asMyToolwas generated asmytool, and its defaultstrict=TrueraisesDuplicateOptionErroron a repeated name. Both seeders now parse withEntryPointConfig, constructed withinterpolation=Noneandstrict=False(app-data seeder, pip seeder). Names keep their case, the last of duplicate entries wins, and a%in a value is read verbatim.flowchart LR F["[console_scripts]\nMyTool = old:main\nMyTool = example:main"] --> A["ConfigParser()"] A --> A1["DuplicateOptionError, or mytool when unique"] F --> B["EntryPointConfig(interpolation=None, strict=False)"] B --> B1["MyTool = example:main"] classDef bad fill:#ffd6d6,stroke:#c1121f,color:#5c0000; classDef ok fill:#d8f3dc,stroke:#2d6a4f,color:#1b4332; class A1 bad; class B1 ok;Behavior changes
Switching from app-data links to the pip seeder leaves the seeded packages installed locally and the shared image unchanged. Existing links inside those seed packages are removed and replaced as pip installs their contents. Linked launchers named by the seeded wheels are replaced by regular scripts; other launcher links are untouched. Invalid package-root or launcher names in wheel metadata fail before the pip subprocess starts. Linked metadata of older versions of a seeded distribution is removed, so package listings report the installed version. Console entry-point names retain their case in both seeders, and duplicate entries use the last value instead of failing. A symlinked site-packages parent requires the directory preparation in #3429; this branch alone does not protect that parent.
Reported by Anthropic's OSS Scanner: ANT-2026-14ZEQFWC.