Bump min version for grpcio-status in spark provider (#36662)

potiuk · web-flow · commit ded01a5aba33 · 2024-01-08T18:51:24.000+01:00
Previously we limited grpcio minimum version to stop backtracking of `pip` from happening and we could not do it in the limits of spark provider, becaue some google dependencies used it and conflicted with it. This problem is now gone as we have newer versions of google dependencies and we can not only safely move it to spark provider but also bump it slightly higher to limit the amount of backtracking we need to do. Extracted from #36537
diff --git a/Dockerfile.ci b/Dockerfile.ci
@@ -1117,14 +1117,11 @@ RUN echo "Airflow version: ${AIRFLOW_VERSION}"
 # force them on the main Airflow package. Currently we need no extra limits as PIP 23.1+ has much better
 # dependency resolution and we do not need to limit the versions of the dependencies
 #
-# Without grpcio-status limit, pip gets into very long backtracking
-# We should attempt to remove it in the future
-#
 # Aiobotocore is limited for eager upgrade because it either causes a long backtracking or
 # conflict when we do not limit it. It seems that `pip` has a hard time figuring the right
 # combination of dependencies for aiobotocore, botocore, boto3 and s3fs together
 #
-ARG EAGER_UPGRADE_ADDITIONAL_REQUIREMENTS="grpcio-status>=1.55.0 aiobotocore>=2.5.4"
+ARG EAGER_UPGRADE_ADDITIONAL_REQUIREMENTS="aiobotocore>=2.5.4"
 ARG UPGRADE_TO_NEWER_DEPENDENCIES="false"
 ARG VERSION_SUFFIX_FOR_PYPI=""
 
diff --git a/airflow/providers/apache/spark/provider.yaml b/airflow/providers/apache/spark/provider.yaml
@@ -55,7 +55,7 @@ versions:
 dependencies:
   - apache-airflow>=2.6.0
   - pyspark
-  - grpcio-status
+  - grpcio-status>=1.59.0
 
 integrations:
   - integration-name: Apache Spark
diff --git a/generated/provider_dependencies.json b/generated/provider_dependencies.json
@@ -206,7 +206,7 @@
   "apache.spark": {
     "deps": [
       "apache-airflow>=2.6.0",
-      "grpcio-status",
+      "grpcio-status>=1.59.0",
       "pyspark"
     ],
     "cross-providers-deps": [
