Skip to content

Commit

Permalink
Add links between documentation related to Google Cloud Storage (#34994)
Browse files Browse the repository at this point in the history
* Add links between documentation related to Google Cloud Storage

* Fix links

* Fix documentation link in code

* Fix link from documentation to code
  • Loading branch information
PeWu committed Oct 18, 2023
1 parent 92204b3 commit e444bca
Show file tree
Hide file tree
Showing 3 changed files with 14 additions and 6 deletions.
2 changes: 1 addition & 1 deletion airflow/providers/google/cloud/operators/gcs.py
Original file line number Diff line number Diff line change
Expand Up @@ -919,7 +919,7 @@ class GCSSynchronizeBucketsOperator(GoogleCloudBaseOperator):
.. seealso::
For more information on how to use this operator, take a look at the guide:
:ref:`howto/operator:GCSSynchronizeBuckets`
:ref:`howto/operator:GCSSynchronizeBucketsOperator`
:param source_bucket: The name of the bucket containing the source objects.
:param destination_bucket: The name of the bucket containing the destination objects.
Expand Down
2 changes: 2 additions & 0 deletions docs/apache-airflow-providers-google/operators/cloud/gcs.rst
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@ Cloud Storage allows world-wide storage and retrieval of any amount of data at a
You can use Cloud Storage for a range of scenarios including serving website content,
storing data for archival and disaster recovery, or distributing large data objects to users via direct download.

See :doc:`/operators/transfer/index` for a list of specialized transfer operators to and from Google Cloud Storage.

Prerequisite Tasks
^^^^^^^^^^^^^^^^^^

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ Transfer data in Google Cloud Storage
The `Google Cloud Storage <https://cloud.google.com/storage/>`__ (GCS) is used to store large data from various applications.
Note that files are called objects in GCS terminology, so the use of the term "object" and "file" in this guide is
interchangeable. There are several operators for whose purpose is to copy data as part of the Google Cloud Service.
This page shows how to use these operators.
This page shows how to use these operators. See also :doc:`/operators/cloud/gcs` for operators used to manage Google Cloud Storage buckets.


Cloud Storage Transfer Service
Expand Down Expand Up @@ -49,6 +49,12 @@ modification date.

If you need information on how to use it, look at the guide: :doc:`/operators/cloud/cloud_storage_transfer_service`

Specialized transfer operators for Google Cloud Storage
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

See :doc:`/operators/transfer/index` for a list of specialized transfer operators to and from Google Cloud Storage.


Local transfer
~~~~~~~~~~~~~~

Expand Down Expand Up @@ -189,12 +195,12 @@ the ``delimiter`` argument apply to moves as well as copies.
:end-before: [END howto_operator_gcs_to_gcs_list_move]


.. _howto/operator:GCSSynchronizeBuckets:
.. _howto/operator:GCSSynchronizeBucketsOperator:

GCSSynchronizeBuckets
~~~~~~~~~~~~~~~~~~~~~
GCSSynchronizeBucketsOperator
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The :class:`~airflow.providers.google.cloud.transfers.gcs_to_gcs.GCSSynchronizeBuckets`
The :class:`~airflow.providers.google.cloud.operators.gcs.GCSSynchronizeBucketsOperator`
operator checks the initial state of the destination bucket, and then compares it with the source bucket.
Based on this, it creates an operation plan that describes which objects should be deleted from
the destination bucket, which should be overwritten, and which should be copied.
Expand Down

0 comments on commit e444bca

Please sign in to comment.