-
Notifications
You must be signed in to change notification settings - Fork 15.3k
Honor index_urls
when venv is created with uv
in PythonVirtualenvOperator
#52287
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Honor index_urls
when venv is created with uv
in PythonVirtualenvOperator
#52287
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the PR! Probably during time of UV integration we missed testing with a external index... thanks for reporting and providing the patch!
I just have two requests - which unfortunately would add a bit of complexity but...
(1) When we initially integrated the support for other index URLs we explicitly did not add the index/extra index URLs to the command line but decided to write the pip-conf to ensure that the command line will not spoil/expose secrets in a shared environment because credentials will be in command line. Even though secrets are usually masked in logs still there is a residual risk that other traces in logs would spoil credentials.
As it is a bit sad that UV does not pick-up the pip.cinf, the other alternative described is using ENV variables to define indexes and credentials (above the linked chapter in https://docs.astral.sh/uv/concepts/indexes/#defining-an-index e.g. via UV_INDEX
). Can you change the PR to use ENV's and prevent adding CLI parameters?
(2) I see with the change the passed index URLs are added as extra-index URLs - but for PIP we made it explicitly that they are not "just extra" but the first one listed is the primary and then further are going to be extra, otherwise maybe private package lookups would be exposed to pypi.org. So I would kindly request making this for UV in the same way, first index_url
should be the primary site and the next ones are extra.
Ah and I assume the CI failure is un-related to this PR...
Ah, and forgot - can you please also add a pytest? |
…-when-venv-is-created-with-uv
bbc57f0
to
fb6702c
Compare
Thanks for the review! 😊
|
Nice! |
Description
The
PythonVirtualenvOperator
offers the option to pass additionalindex_urls
to load Python packages from.It installs those Python packages either using
pip
oruv
. Which one to use is decided automatically depending on the system environment.The
index_urls
are put into apip.conf
file when the Python virtual environment is created.pip
is recognizing the configuration from that file automatically - howeveruv
is (currently) not. See: astral-sh/uv#1404This means that if a user provides
index_urls
when using thePythonVirtualenvOperator
, this may or may not work - depending on whetheruv
is available on the system.This PR fixes this issue by adding the provided
index_urls
as--extra-index-url
arguments touv
(as described here: https://docs.astral.sh/uv/concepts/indexes/#-index-url-and-extra-index-url)^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rst
or{issue_number}.significant.rst
, in airflow-core/newsfragments.