zlacker

Uv 0.3 – Unified Python packaging

submitted by mantap+(OP) on 2024-08-20 18:14:49 | 163 points 57 comments
[view article] [source] [go to bottom]

NOTE: showing posts with links only show all posts
◧◩
2. charli+X6[view] [source] [discussion] 2024-08-20 19:01:25
>>clayto+06
I wrote about it a bit here: https://github.com/astral-sh/rye/discussions/1342
◧◩
4. henbru+v8[view] [source] [discussion] 2024-08-20 19:12:14
>>clayto+06
There's no equivalent of https://rye.astral.sh/guide/pyproject/#toolryescripts yet, but it's apparently planned: https://github.com/astral-sh/uv/issues/5903#issuecomment-227...
◧◩
10. zanie+ge[view] [source] [discussion] 2024-08-20 19:48:24
>>frou_d+Da
Good thing we've been investing in our LSP server lately[1]! We're still a ways out from integrating uv into the LSP (and, more generally, providing auto-completions) but it's definitely on our minds.

The script dependency metadata _is_ standardized[2], so other LSP servers could support a good experience here (at least in theory).

[1] The Ruff Language Server: https://astral.sh/blog/ruff-v0.4.5

[2] Inline script metadata: https://peps.python.org/pep-0723/

◧◩
11. zanie+ig[view] [source] [discussion] 2024-08-20 20:00:10
>>crypto+Y8
We wanted to cover this in a dedicated discussion. See https://github.com/astral-sh/rye/discussions/1342
12. dang+ck[view] [source] 2024-08-20 20:27:42
>>mantap+(OP)
We changed the URL from https://github.com/astral-sh/uv/releases/tag/0.3.0 to the project page. Interested readers probably should look at both.
16. theano+tm[view] [source] 2024-08-20 20:44:58
>>mantap+(OP)
https://xkcd.com/927/
◧◩
21. simonw+Ap[view] [source] [discussion] 2024-08-20 21:05:12
>>pietz+Pn
Yeah, I think so. Their documentation includes a note on how to bootstrap from a fresh Docker image here: https://docs.astral.sh/uv/guides/integration/docker/
◧◩
22. millia+Qp[view] [source] [discussion] 2024-08-20 21:07:46
>>pietz+Pn
Not on the Astral team, but to the first step, I'd get uv from your distro package manager (e.g. https://build.opensuse.org/package/show/openSUSE:Factory/uv) and then the rest as you say ("manage python versions and projects from there and install ruff with uv").

If you have some other tool manager on your system (e.g. mise) then you can likely install uv through that.

25. ldubin+wr[view] [source] 2024-08-20 21:18:28
>>mantap+(OP)
I've been closely following the Astral team's work. I am excited by this release and look forward to trying out uv again.

I have been working with python for over 10 years and have standardized my workflow to only use pip & setuptools for all dependency management and packaging [1]. It works great as a vanilla setup and is 100% standards based. I think uv and similar tools mostly shine when you have massive codebases.

1: https://dubnest.com/blog/vanilla-python-packaging/

◧◩
36. westur+t51[view] [source] [discussion] 2024-08-21 04:01:07
>>0cf861+0l
I'm from Nebraska. Unfortunately if your Python is compiled in a datacenter in Iowa, it's more likely that it was powered with wind energy. Claim: Iowa has better Clean Energy PPAs for datacenters than Nebraska (mostly due to rational wind energy subsidies).

Anyways, software supply chain security and Python & package build signing and then containers and signing them too

Conda-forge's builds are probably faster than the official CPython builds. conda-forge/python-feedstock//recipe/meta.yml: https://github.com/conda-forge/python-feedstock/blob/main/re...

Conda-forge also has OpenBLAS, blos, accelerate, netlib, and Intel MKL; conda-forge docs > switching BLAS implementation: https://conda-forge.org/docs/maintainer/knowledge_base/#swit...

From "Building optimized packages for conda-forge and PyPI" at EuroSciPy 2024: https://pretalx.com/euroscipy-2024/talk/JXB79J/ :

> Since some time, conda-forge defines multiple "cpu-levels". These are defined for sse, avx2, avx512 or ARM Neon. On the client-side the maximum CPU level is detected and the best available package is then installed. This opens the doors for highly optimized packages on conda-forge that support the latest CPU features.

> We will show how to use this in practice with `rattler-build`

> For GPUs, conda-forge has supported different CUDA levels for a long time, and we'll look at how that is used as well.

> Lastly, we also take a look at PyPI. There are ongoing discussions on how to improve support for wheels with CUDA support. We are going to discuss how the (pre-)PEP works and synergy possibilities of rattler-build and cibuildwheel

Linux distros build and sign Python and python3-* packages with GPG keys or similar, and then the package manager optionally checks the per-repo keys for each downloaded package. Packages should include a manifest of files to be installed, with per-file checksums. Package manifests and/or the package containing the manifest should be signed (so that tools like debsums and rpm --verify can detect disk-resident executable, script, data asset, and configuration file changes)

virtualenvs can be mounted as a volume at build time with -v with some container image builders, or copied into a container image with the ADD or COPY instructions in a Containerfile. What is added to the virtualenv should have a signature and a version.

ostree native containers are bootable host images that can also be built and signed with a SLSA provenance attestation; https://coreos.github.io/rpm-ostree/container/ :

  rpm-ostree rebase ostree-image-signed:registry:<oci image>
  rpm-ostree rebase ostree-image-signed:docker://<oci image>
> Fetch a container image and verify that the container image is signed according to the policy set in /etc/containers/policy.json (see containers-policy.json(5)).

So, when you sign a container full of packages, you should check the package signatures; and verify that all package dependencies are identified by the SBOM tool you plan to use to keep dependencies upgraded when there are security upgrades.

e.g. Dependabot - if working - will regularly run and send a pull request when it detects that the version strings in e.g. a requirements.txt or environment.yml file are out of date and need to be changed because of reported security vulnerabilities in ossf/osv-schema format.

Is there already a way to, as a developer, sign Python packages built with cibuildwheel with Twine and TUF or sigstore to be https://SLSA.dev/ compliant?

37. carder+kd1[view] [source] 2024-08-21 05:48:08
>>mantap+(OP)
Love that you’ve kept the workspace support from Rye. Is there any support for workspace _builds_? That is, resolving the graph of interdependencies between packages to build a wheel/Dockerfile/whatever?

Fwiw I’m building a thing [1] that does this. Current docs suggest Rye but will s/rye/uv/ shortly. It’s basically just some CLI commands and a Hatch/PDM plugin that injects needed stuff at build-time.

[1] https://una.rdrn.me/

◧◩
39. carder+By1[view] [source] [discussion] 2024-08-21 09:37:07
>>carder+kd1
Just did some playing around and workspace support has definitely evolved from Rye [0] but there isn't yet (afaict) any mechanism that supports building workspace packages with their "internal" dependencies.

[1] https://docs.astral.sh/uv/concepts/workspaces/

◧◩
48. jdnier+mK3[view] [source] [discussion] 2024-08-22 04:40:48
>>clayto+06
See "Rye and Uv: August Is Harvest Season for Python Packaging"[1]

[1] https://lucumr.pocoo.org/2024/8/21/harvest-season/

◧◩
49. scotty+XM3[view] [source] [discussion] 2024-08-22 05:13:04
>>jimmcs+uE3
Seems they need to be on $PATH to get picked up. In Rye there's also a command to "register" a Python install at a specific path.

https://docs.astral.sh/uv/concepts/python-versions/#discover...

https://rye.astral.sh/guide/toolchains/#registering-toolchai...

◧◩
52. pantsf+235[view] [source] [discussion] 2024-08-22 16:21:49
>>tarask+UT
Ruff (the linter/formatter from Astral) has its own LSP right now: https://github.com/astral-sh/ruff-lsp. Although Ruff itself now ships with a language server already integrated, so I have no idea what the plan is long term.
◧◩◪◨⬒
56. jimmcs+5Y5[view] [source] [discussion] 2024-08-23 00:43:21
>>oblvio+4g5
Yep, have done https://github.com/astral-sh/uv/issues/6479
57. samuel+lK6[view] [source] 2024-08-23 12:28:20
>>mantap+(OP)
I'm happy to see uv is adopted in Pixi, which is a new personal favorite:

https://prefix.dev/blog/uv_in_pixi

Reasons for liking pixi, over e.g. poetry:

- Like poetry, it keeps everything in a project-local definition file and environment

But unlike poetry, it also:

- Can install python itself, even ancient versions

- Can install conda packages

- Is extremely fast

[go to top]