packaging.python.org

PSA: qualifications to the source distribution specification I'm opening this DPO thread at @rgommers' request. PSA: this PR is going to make some clarifying changes to the source distribution specification. These changes clarify existing requirements, but packagers/build backend maintainers should be aware of them to ensure that they're conforming with the living standard for source distributions. The main changes: - The specification now emphasizes that source distributions MUST be a pax-style tar archive. This was already required, but has historically been loosely enforced by packaging tooling. - Source distributions MUST NOT use non-pax features, including GNU tar extensions or the V7 ("legacy") style tar format. - Source distributions MUST NOT use vendor extensions to pax, including VENDOR.attr style pax records that are not defined within the pax specification itself. These changes should have no impact on end users, since PyPI will continue to serve any source distributions that were created before these changes (as well as distributions that were created long before sdists were standardized at all). However, in the medium term, PyPI _may_ decide to begin rejecting new sdist uploads that do not strictly conform to the sdist specification. This will be its own protracted process, since users will need to be alerted and given ample time to fix their distributions before a breaking change can be made. (PyPI already rejects some sdist uploads as a matter of policy, but has not rejected non-pax sdists wholesale yet.)

[quote="Paul Ganssle, post:12, topic:109629, username:pganssle"] aligning tools on what they should and should not support. That way you don’t have weird discrepancies where PyPI accepts a certain length of version and pip rejects it or build backends produce it and it only fails at update, etc. [/quote] That's fine, but the PR in it's inception explicitly created a split between installers and other tools: packaging.python.org#2163 > Version identifiers for new releases MUST NOT exceed 128 characters after :ref:normalization <version-specifiers-normalization>, including any local version label. > > Installation tools MUST NOT reject otherwise valid versions solely because they exceed this limit. This kind of tool split is especially hard for libraries like packaging which have to serve both installers and other tools. Further, we've put a great deal of effort into the performance of constructing a Version objects in packaging. Implementing a rule like this would add significant cost, we would have to create a string normalized form and test it's length every construction. During a large resolve of requirements, a Python package dependency resolver might construct millions of Version objects. So I would be strongly -1 on this particular type of limit for all tools. And I don't think it should be surprising to users when publishing to a hosted service like PyPI that there exists some limit on name lengths and version lengths. The same is true for distribution size and project size, users can make arbitary large wheels locally, but they shouldn't be surprised when they can't use PyPI to backup their 8 TB hard drives encrypted as wheels. Likewise I could make a 10 MB reply to this post in a local markdown file, but I shouldn't be surprised for DPO to refuse it.

Amend PEP440 to limit version identifiers PEP440 does not contain a limit for version ids. Can we please add one? things like pypi.org happen without it, and other issues arise not least of which is user a11y related to trying to fit/show all of the wild versioning stats from pypi: we have 9.9mm releases, median version is 5 characters, 99th percentil is 20, and maximal is 217(!!) I propose a 128 char cap, 100 would affect ~9 current releases some releases in the last months with ~110-115 char versioning which I honestly don't understand but... whatever! 128 is fine. refs: - warehouse#12483 - warehouse#14965 - packaging.python.org#2163 - packaging-council#11