This website uses Cookies. Click Accept to agree to our website's cookie use as described in our Privacy Policy. Click Preferences to customize your cookie settings.
Good day, I have the following flex template that works perfectly when I have only one custom interface that extends from PipelineOptions: public class JobFlexTemplate {
public...
Hi all, I've been struggling with it for the past 2 weeks, going through GCP documentation and looking of the solution online. No luck. I simply do not know how to turn on (activate/enable) the Data...
Hello, I'm trying to create Composer2 environment but getting this error, see printscreen, I didn't find any answer, please help. CREATE operation on this environment failed 6 hours ago with th...
I need to perform incremental merge on a partitioned table. I am trying to using the below property updatePartitionFilter to limit the amount of data scanned on this large table during merge ope...
I currently have an Airflow task wherein I have several csvs in a bucket and I'm taking each csv, converting into a dataframe and then merging all dataframes to together and putting into BigQuery. It...
...ested resources. If the API supports cascading delete, set 'force' to true to delete it and its nested resources. Is there a way to destroy that google_dataform_repository using Terraform? I d...
Hi, How can we build a cloud function to read all entries from Data Loss Prevention data profile scans and accordingly tag PII columns with policy tags using data taxonomy? Do we need to use DLP AP...
Hi, Is there a way we can enforce lifecycle policies on the basis of Security classification "policy tags" that are implied on BigQuery tables using Dataplex or by any other method? Also how to imp...
I am struggling to achieve this logic in BigQuery. I shall be grateful if someone can help Consider these 2 CTEs WITH MaxCycle AS (
SELECT MAX(my_cycle) AS max_cycle
FROM `project_a.dataset_A.t...