Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Deploying Extensions with a Service Account on GH Actions and Locally fails #6060

Closed
abegehr opened this issue Jun 29, 2023 · 7 comments · Fixed by #6167
Closed

Deploying Extensions with a Service Account on GH Actions and Locally fails #6060

abegehr opened this issue Jun 29, 2023 · 7 comments · Fixed by #6167
Assignees
@joehan
Labels
api: extensions Needs: Attention type: bug

Comments

@abegehr
Copy link

abegehr commented Jun 29, 2023 •
edited

[REQUIRED] Environment info

firebase-tools:12.4.2

Platform:macOS

[REQUIRED] Test case

I'm trying to deploy Firebase extensions while authorized with a service account on a GH Action workflow and am getting the following error:

i  extensions: ensuring required API firebaseextensions.googleapis.com is enabled...
✔  extensions: required API firebaseextensions.googleapis.com is enabled
i  extensions: ensuring required API secretmanager.googleapis.com is enabled...
✔  extensions: required API secretmanager.googleapis.com is enabled
i  extensions: ensuring required API compute.googleapis.com is enabled...
✔  extensions: required API compute.googleapis.com is enabled
The following extension instances will be configured:
	firestore-bigquery-export (firebase/[email protected])
	firestore-send-email (firebase/[email protected])

i  extensions: By installing an extension instance onto a Firebase project, you accept the Firebase Extensions User Terms of Service: https://firebase.google.com/terms/extensions/user

Error: HTTP Error: 403, Firebase Extensions Terms of Service Private API has not been used in project XXX before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/firebaseextensionstos-pa.googleapis.com/overview?project=XXX then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
Error: Process completed with exit code 1.

Trying to access https://console.developers.google.com/apis/api/firebaseextensionstos-pa.googleapis.com/overview?project=XXX (with my project id) gives an error: "Failed to load".
I'm not sure why it asks for accepting terms of service, as I accepted the terms while deploying locally with my Google account logged in.

I've also tried deploying extensions locally, which worked when Firebase CLI is logged in with my Google account (Firebase project owner) but fails with the service account with a different error:

Error: Errors while reading 'extensions' in 'firebase.json'
- HTTP Error: 403, Your application is authenticating by using local Application Default Credentials. The firebaseextensions.googleapis.com API requires a quota project, which is not set by default. To learn how to set your quota project, see https://cloud.google.com/docs/authentication/adc-troubleshooting/user-creds.
- HTTP Error: 403, Your application is authenticating by using local Application Default Credentials. The firebaseextensions.googleapis.com API requires a quota project, which is not set by default. To learn how to set your quota project, see https://cloud.google.com/docs/authentication/adc-troubleshooting/user-creds.

How can I use a service account to deploy Firebase including extensions?

[REQUIRED] Steps to reproduce

On GH Actions:

Deploy a firebase project with extensions enabled on GH Actions with a service account:

name: Deploy to Firebase Staging
on:
  push:
    branches:
      - staging
jobs:
  deploy:
    name: Deploy Firebase
    runs-on: ubuntu-latest
    environment: staging
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-node@v3
        with:
          node-version: 18
      - uses: w9jds/[email protected]
        with:
          tools-version: 12.4.2
          gcp_sa_key: "${{ secrets.FIREBASE_SERVICE_ACCOUNT }}"
      - run: firebase deploy --except hosting --project staging
        working-directory: firebase

Locally:

  1. Open your firebase project with extensions enabled in terminal.
  2. Log out of Firebase CLI: firebase logout
  3. Authenticate with a service account key: export GOOGLE_APPLICATION_CREDENTIALS='path/to/key.json'
  4. Run firebase deploy --only extensions --project staging

[REQUIRED] Expected behavior

I would expect the service account to be able to deploy the whole Firebase project including extensions.

[REQUIRED] Actual behavior

Deploying with service account fails on GH action with:

Error: HTTP Error: 403, Firebase Extensions Terms of Service Private API has not been used in project XXX before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/firebaseextensionstos-pa.googleapis.com/overview?project=XXX then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
Error: Process completed with exit code 1.

and locally with:

Error: Errors while reading 'extensions' in 'firebase.json'
- HTTP Error: 403, Your application is authenticating by using local Application Default Credentials. The firebaseextensions.googleapis.com API requires a quota project, which is not set by default. To learn how to set your quota project, see https://cloud.google.com/docs/authentication/adc-troubleshooting/user-creds.
@aalej
Copy link
Contributor

aalej commented Jun 30, 2023

Hi @abegehr, thanks for the detailed report. The issue regarding deploying extensions via GitHub actions seems to be similar to #5855.

With regard to locally deploying, I consulted with our team, and they think that a call to firebaseextensions.googleapis.com is failing which is causing the error. To help us with investigating the issue, would it be okay if you could provide the contents of your firebase-debug.log after running firebase deploy --only extensions --project staging locally? This will give us a better overview of what causing this error:

Error: Errors while reading 'extensions' in 'firebase.json'
- HTTP Error: 403, Your application is authenticating by using local Application Default Credentials. The firebaseextensions.googleapis.com API requires a quota project, which is not set by default. To learn how to set your quota project, see https://cloud.google.com/docs/authentication/adc-troubleshooting/user-creds.

Please remove any PII (personal identifiable information) if needed.

@aalej aalej added the Needs: Author Feedback Issues awaiting author feedback label Jun 30, 2023
@KoheiKanagu
Copy link

I think I'm experiencing the same issue.

I'm attaching detailed logs.
According to the logs, the error occurred because I don't have the firebaseextensions.instances.list permission.

Details 

[2023-07-06T03:07:27.061Z] [iam] checking project ******** for permissions ["firebase.projects.get","firebaseextensions.instances.list"]
[2023-07-06T03:07:27.061Z] >>> [apiv2][query] POST https://cloudresourcemanager.googleapis.com/v1/projects/********:testIamPermissions [none]
[2023-07-06T03:07:27.066Z] >>> [apiv2][(partial)header] POST https://cloudresourcemanager.googleapis.com/v1/projects/********:testIamPermissions x-goog-quota-user=projects/********
[2023-07-06T03:07:27.066Z] >>> [apiv2][body] POST https://cloudresourcemanager.googleapis.com/v1/projects/********:testIamPermissions {"permissions":["firebase.projects.get","firebaseextensions.instances.list"]}
[2023-07-06T03:07:27.219Z] <<< [apiv2][status] POST https://cloudresourcemanager.googleapis.com/v1/projects/********:testIamPermissions 200
[2023-07-06T03:07:27.219Z] <<< [apiv2][body] POST https://cloudresourcemanager.googleapis.com/v1/projects/********:testIamPermissions {"permissions":["firebase.projects.get"]}
[2023-07-06T03:07:27.220Z] [iam] error while checking permissions, command may fail: FirebaseError: Authorization failed. This account is missing the following required permissions on project ********:

  firebaseextensions.instances.list
[2023-07-06T03:07:27.221Z] >>> [apiv2][query] GET https://firebaseextensions.googleapis.com/v1beta/projects/********/instances pageSize=100&pageToken=
✔  functions[asia-northeast1-onCreateAuthUser]: auth function initialized.
[2023-07-06T03:07:27.424Z] <<< [apiv2][status] GET https://firebaseextensions.googleapis.com/v1beta/projects/********/instances 403
[2023-07-06T03:07:27.424Z] <<< [apiv2][body] GET https://firebaseextensions.googleapis.com/v1beta/projects/********/instances {"error":{"code":403,"message":"The caller does not have permission","status":"PERMISSION_DENIED"}}

Error: HTTP Error: 403, The caller does not have permission
[2023-07-06T03:07:27.426Z] Error Context: {
  "body": {
    "error": {
      "code": 403,
      "message": "The caller does not have permission",
      "status": "PERMISSION_DENIED"
    }
  },
  "response": {
    "statusCode": 403
  }
}

My service account has the following roles:

  • roles/firebase.developAdmin
  • roles/cloudfunctions.serviceAgent
  • roles/cloudconfig.admin

It seems that firebaseextensions was updated in #2232, but I couldn't find the firebaseextensions.instances.list role even after checking IAM.

@gwp-rob
Copy link

gwp-rob commented Jul 6, 2023

I got the same error using Google Cloud Build using Stripe extension:

following extension instances will be configured:
firestore-stripe-payments (stripe/[email protected])

i extensions: By installing an extension instance onto a Firebase project, you accept the Firebase Extensions User Terms of Service: https://firebase.google.com/terms/extensions/user

Error: HTTP Error: 403, Firebase Extensions Terms of Service Private API has not been used in project 835040529702 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/firebaseextensionstos-pa.googleapis.com/overview?project=835040529702 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.

@KoheiKanagu
Copy link

#6060 (comment)

After adding roles/firebase.admin,
I also got the error Firebase Extensions Terms of Service Private API has not been used in project XXX before or it is disabled.

I'm attaching detailed logs.

Details 

2023-07-07T01:40:04.0660955Z [2023-07-07T01:40:03.174Z] Checking if latest AppDeveloper TOS has been accepted by ********...
2023-07-07T01:40:04.0661983Z i  extensions: By installing an extension instance onto a Firebase project, you accept the Firebase Extensions User Terms of Service: https://firebase.google.com/terms/extensions/user 
2023-07-07T01:40:04.0663250Z [2023-07-07T01:40:03.175Z] >>> [apiv2][query] GET https://firebaseextensionstos-pa.googleapis.com/v1/projects/********/appdevtos [none]
2023-07-07T01:40:04.0664478Z [2023-07-07T01:40:03.292Z] <<< [apiv2][status] GET https://firebaseextensionstos-pa.googleapis.com/v1/projects/********/appdevtos 403
2023-07-07T01:40:04.0669492Z [2023-07-07T01:40:03.293Z] <<< [apiv2][body] GET https://firebaseextensionstos-pa.googleapis.com/v1/projects/********/appdevtos {"error":{"code":403,"message":"Firebase Extensions Terms of Service Private API has not been used in project 275151145737 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/firebaseextensionstos-pa.googleapis.com/overview?project=275151145737 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.","status":"PERMISSION_DENIED","details":[{"@type":"type.googleapis.com/google.rpc.Help","links":[{"description":"Google developers console API activation","url":"https://console.developers.google.com/apis/api/firebaseextensionstos-pa.googleapis.com/overview?project=275151145737"}]},{"@type":"type.googleapis.com/google.rpc.ErrorInfo","reason":"SERVICE_DISABLED","domain":"googleapis.com","metadata":{"service":"firebaseextensionstos-pa.googleapis.com","consumer":"projects/275151145737"}}]}}
2023-07-07T01:40:04.0671833Z 
2023-07-07T01:40:04.0674628Z Error: HTTP Error: 403, Firebase Extensions Terms of Service Private API has not been used in project 275151145737 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/firebaseextensionstos-pa.googleapis.com/overview?project=275151145737 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
2023-07-07T01:40:04.0675912Z [2023-07-07T01:40:03.297Z] Error Context: {
2023-07-07T01:40:04.0676216Z   "body": {
2023-07-07T01:40:04.0676478Z     "error": {
2023-07-07T01:40:04.0676754Z       "code": 403,
2023-07-07T01:40:04.0678407Z       "message": "Firebase Extensions Terms of Service Private API has not been used in project 275151145737 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/firebaseextensionstos-pa.googleapis.com/overview?project=275151145737 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.",
2023-07-07T01:40:04.0679409Z       "status": "PERMISSION_DENIED",
2023-07-07T01:40:04.0679724Z       "details": [
2023-07-07T01:40:04.0679989Z         {
2023-07-07T01:40:04.0680378Z           "@type": "type.googleapis.com/google.rpc.Help",
2023-07-07T01:40:04.0680730Z           "links": [
2023-07-07T01:40:04.0680996Z             {
2023-07-07T01:40:04.0681412Z               "description": "Google developers console API activation",
2023-07-07T01:40:04.0682342Z               "url": "https://console.developers.google.com/apis/api/firebaseextensionstos-pa.googleapis.com/overview?project=275151145737"
2023-07-07T01:40:04.0683283Z             }
2023-07-07T01:40:04.0683534Z           ]
2023-07-07T01:40:04.0683781Z         },
2023-07-07T01:40:04.0684026Z         {
2023-07-07T01:40:04.0684428Z           "@type": "type.googleapis.com/google.rpc.ErrorInfo",
2023-07-07T01:40:04.0684856Z           "reason": "SERVICE_DISABLED",
2023-07-07T01:40:04.0685486Z           "domain": "googleapis.com",
2023-07-07T01:40:04.0685798Z           "metadata": {
2023-07-07T01:40:04.0686366Z             "service": "firebaseextensionstos-pa.googleapis.com",
2023-07-07T01:40:04.0686847Z             "consumer": "projects/275151145737"
2023-07-07T01:40:04.0687143Z           }
2023-07-07T01:40:04.0687396Z         }
2023-07-07T01:40:04.0687639Z       ]
2023-07-07T01:40:04.0687876Z     }
2023-07-07T01:40:04.0688111Z   },
2023-07-07T01:40:04.0688362Z   "response": {
2023-07-07T01:40:04.0688644Z     "statusCode": 403
2023-07-07T01:40:04.0688910Z   }
2023-07-07T01:40:04.0689153Z }

@abegehr
Copy link
Author

abegehr commented Jul 10, 2023

Hi @abegehr, thanks for the detailed report. The issue regarding deploying extensions via GitHub actions seems to be similar to #5855.

With regard to locally deploying, I consulted with our team, and they think that a call to firebaseextensions.googleapis.com is failing which is causing the error. To help us with investigating the issue, would it be okay if you could provide the contents of your firebase-debug.log after running firebase deploy --only extensions --project staging locally? This will give us a better overview of what causing this error:

Error: Errors while reading 'extensions' in 'firebase.json'
- HTTP Error: 403, Your application is authenticating by using local Application Default Credentials. The firebaseextensions.googleapis.com API requires a quota project, which is not set by default. To learn how to set your quota project, see https://cloud.google.com/docs/authentication/adc-troubleshooting/user-creds.

Please remove any PII (personal identifiable information) if needed.

@aalej, I have tried this again with the latest cli (12.4.3) and a service account that has the following roles:

API Keys Viewer
Cloud Datastore Index Admin
Cloud Functions Developer
Cloud Run Viewer
Cloud Scheduler Admin
Firebase Admin
Firebase Authentication Admin
Firebase Extensions Publisher - Extensions Admin
Firebase Hosting Admin
Firebase Rules Admin
Secret Manager Secret Accessor
Secret Manager Viewer
Service Account User

Running this command: export GOOGLE_APPLICATION_CREDENTIALS=./service-account-key.json && firebase deploy --only extensions --project staging

This time locally I got the same ToS issue as on GH actions:


=== Deploying to 'xxx-staging'...

i  deploying extensions
i  extensions: ensuring required API firebaseextensions.googleapis.com is enabled...
✔  extensions: required API firebaseextensions.googleapis.com is enabled
i  extensions: ensuring required API secretmanager.googleapis.com is enabled...
✔  extensions: required API secretmanager.googleapis.com is enabled
i  extensions: ensuring required API compute.googleapis.com is enabled...
✔  extensions: required API compute.googleapis.com is enabled
The following extension instances will be configured:
        firestore-bigquery-export-seo8 (firebase/[email protected])
        firestore-bigquery-export (firebase/[email protected])
        firestore-send-email (firebase/[email protected])

i  extensions: By installing an extension instance onto a Firebase project, you accept the Firebase Extensions User Terms of Service: https://firebase.google.com/terms/extensions/user

Error: HTTP Error: 403, Firebase Extensions Terms of Service Private API has not been used in project 1019787264662 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/firebaseextensionstos-pa.googleapis.com/overview?project=1019787264662 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.

No firebase-debug.log is generated.

@google-oss-bot google-oss-bot added Needs: Attention and removed Needs: Author Feedback Issues awaiting author feedback labels Jul 10, 2023
@joehan joehan mentioned this issue Jul 11, 2023
Closed
@davidstanke
Copy link

davidstanke commented Jul 13, 2023 •
edited

I've got the same issue, invoking via an Ubuntu worker on Cloud Build (using this config). Here's a log snippet:

Step #2: i  deploying firestore, functions, hosting, remoteconfig, extensions
Step #2: i  cloud.firestore: checking rules/firestore.rules for compilation errors...
Step #2: ✔  cloud.firestore: rules file rules/firestore.rules compiled successfully
Step #2: i  functions: preparing codebase inquiry-monitor for deployment
Step #2: i  functions: ensuring required API cloudfunctions.googleapis.com is enabled...
Step #2: i  functions: ensuring required API cloudbuild.googleapis.com is enabled...
Step #2: i  artifactregistry: ensuring required API artifactregistry.googleapis.com is enabled...
Step #2: ✔  artifactregistry: required API artifactregistry.googleapis.com is enabled
Step #2: ✔  functions: required API cloudbuild.googleapis.com is enabled
Step #2: ✔  functions: required API cloudfunctions.googleapis.com is enabled
Step #2: ⚠  functions: package.json indicates an outdated version of firebase-functions. Please upgrade using npm install --save firebase-functions@latest in your functions directory.
Step #2: i  functions: Loading and anaylzing source code for codebase inquiry-monitor to determine what to deploy
Step #2: Serving at port 8686
Step #2: 
Step #2: shutdown requested via /__/quitquitquit
Step #2: 
Step #2: i  functions: Loaded environment variables from .env.
Step #2: i  functions: preparing functions/inquiry-monitor directory for uploading...
Step #2: i  functions: packaged /workspace/functions/inquiry-monitor (95.9 KB) for uploading
Step #2: i  extensions: ensuring required API firebaseextensions.googleapis.com is enabled...
Step #2: ✔  extensions: required API firebaseextensions.googleapis.com is enabled
Step #2: i  extensions: ensuring required API secretmanager.googleapis.com is enabled...
Step #2: ✔  extensions: required API secretmanager.googleapis.com is enabled
Step #2: i  extensions: ensuring required API compute.googleapis.com is enabled...
Step #2: ✔  extensions: required API compute.googleapis.com is enabled
Step #2: The following extension instances will be configured:
Step #2: 	firestore-send-email (firebase/[email protected])
Step #2: 
Step #2: i  extensions: By installing an extension instance onto a Firebase project, you accept the Firebase Extensions User Terms of Service: https://firebase.google.com/terms/extensions/user
Step #2: 
Step #2: Error: HTTP Error: 403, Firebase Extensions Terms of Service Private API has not been used in project XXX before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/firebaseextensionstos-pa.googleapis.com/overview?project=XXX then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.

LMK if there's any other info I can provide that may be of use. Thanks!

@davidstanke davidstanke mentioned this issue Jul 13, 2023
Closed
@joehan joehan self-assigned this Jul 18, 2023
@joehan joehan mentioned this issue Jul 20, 2023
Merged
@samatcolumn
Copy link

@joehan is there a way I can accept the TOS via the console?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@joehan joehan

Labels
api: extensions Needs: Attention type: bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Make Extensions TOS calls best effort firebase/firebase-tools
8 participants
@davidstanke @joehan @KoheiKanagu @abegehr @google-oss-bot @gwp-rob @samatcolumn @aalej