Dataform: Release config PATCH from new compilatio...

brunogrcs

Hi there,

Our current CI/CD workflow for Dataform works by updating our release config with a new compilation result from a git commit SHA (using the Dataform API https://dataform.googleapis.com)

This broke yesterday (16.05.2024) and the API is consistently returning the following (on PATCH to

https://dataform.googleapis.com/v1beta1/projects/x/locations/y/repositories/z/releaseConfigs/name):

{
  "error": {
    "code": 400,
    "message": "A new current compilation result is not created from this release config",
    "status": "FAILED_PRECONDITION"
  }
}

Just wanted to check if this is a known issue? We're currently relying on manually updating the release configuration with the latest commit from our main branch through the UI, which is not ideal.

Thank you in advance for any help.

ms4446

The error you're encountering, "A new current compilation result is not created from this release config," with a FAILED_PRECONDITION status, is a bit generic

Dataform requires that a new compilation result be successfully created before you can update the release config with it. This ensures the config points to a valid and completed compilation.

Potential Causes

Compilation Failure: The compilation itself may have failed due to errors in your Dataform code.
API Race Condition: Your CI/CD script might be trying to update the release config too quickly, before the compilation result is fully available.
Dataform Issue: There might be an underlying issue with Dataform itself (though this is less likely).

Troubleshooting Steps

Verify Compilation Success: Double-check that your Dataform compilation is completing successfully. Look for any errors in the Dataform logs or in your CI/CD pipeline.
Check Compilation Result: After a successful compilation, ensure the compilation result is created and accessible in Dataform. You can usually find them in the "Compilation Results" section of the repository.
Introduce a Delay: If the compilation is successful, add a short delay (e.g., 5-10 seconds) in your CI/CD script between the compilation step and the API call to update the release config. This gives Dataform time to process the compilation result.
Inspect API Request: Make sure your API request is correctly formatted. Here's an example of a PATCH request to update a release config:

PATCH https://dataform.googleapis.com/v1beta1/projects/{project}/locations/{location}/repositories/{reposi...}
Content-Type: application/json

{
"compilationResult": "projects/{project}/locations/{location}/repositories/{repository}/compilationResults/{compilationResult}"
}

If the problem persists, reach out to Google Cloud Support.

martinroddam

Adding a +1 to this. Have already tried the suggestions but not managed a single successful patch since last week.

martinroddam

Ticket raised here FYI: https://issuetracker.google.com/issues/341900068

brunogrcs

Perfect thank you @martinroddam 👍

I wasn't expecting much from a beta API anyway, but given that there are 0 other solutions to have an effective deployment step, I'm hoping that it will get some attention

Dataform: Release config PATCH from new compilation result not working