I found a few problems with this issue:

• I couldn't figure out how to label this issue, so I've labeled it for a human to triage. Hang tight.
• This issue does not seem to follow the issue template. Make sure you provide all the required information.

@sudsingh438 :

Questions

1. Which version of the Firebase Instance ID are you using?
2. Are you able to share the complete IllegalStateException stack trace that might show the cause?
3. When initializing Firebase (i.e. class FirebaseApp) are you initializing with a full set of Firebase options (e.g. using a google-services.json from the Firebase Console)?

Background

If the initialization of Firebase (i.e. instance of FirebaseApp) is missing one of the following required parameters, this might explain the issue: Application ID, Project ID, API key.

The latest version of the Instance ID SDK depends on the new Firebase Installations SDK which is a change that might cause issues like this.

(@ankitaj224 : for visibility)

@andirayo

1. Firebase instance id version : 20.1.1
2. The one that I shared is the description is the complete stack trace from play console.

This stack trace is the one we got from Sentry

java.lang.IllegalStateException: null
    at com.google.firebase.iid.FirebaseInstanceId.o(com.google.firebase:firebase-iid@@20.1.1:12)
    at com.google.firebase.iid.FirebaseInstanceId.a(com.google.firebase:firebase-iid@@20.1.1:15)
    at com.google.firebase.perf.internal.f.c(com.google.firebase:firebase-perf@@19.0.5:5)
    at com.google.firebase.perf.internal.f.b(com.google.firebase:firebase-perf@@19.0.5:12)
    at com.google.firebase.perf.internal.f.a(com.google.firebase:firebase-perf@@19.0.5:52)
    at com.google.firebase.perf.internal.e.run(com.google.firebase:firebase-perf@@19.0.5:1)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
    at java.lang.Thread.run(Thread.java:764)

3. Yes we are using google-services.json from play console. We started seeing this issue in our app from last 1 week, before that we didnt encounter this issue before that.

Also, if it helps we are using below mentioned firebase component with version:
firebase-config:19.1.2(This we recently upgraded from 19.1.0)
firebase-dynamic-links:19.1.0
firebase-messaging:20.1.2(This we recently upgraded from 19.0.3)
firebase-perf:19.0.5

We have the exact same issue and it all started after we upgraded to:

com.google.firebase:firebase-bom:25.0.0

from 24.7.0

I have the same crash outburst. It started this week. I've updated:

com.google.firebase:firebase-messaging:20.1.0 -> 20.1.2
com.google.firebase:firebase-core:17.2.2 -> 17.2.3
com.google.firebase:firebase-config:19.1.1 -> 19.1.2

Downgraded today, will see if this going to fix situation.

I create custom Firebase instance with code that looks something like this:

val firebaseOptions = FirebaseOptions.fromResource(context)
FirebaseApp.initializeApp(context, firebaseOptions, "MyCustomInstance")

But Firebase Performance uses default Firebase instance.

FirebasePerformance.getInstance()

@asfdfdfd did downgrading solved this bug for you?

We started to experience the same crash after updating the following dependencies:

com.google.firebase:firebase-analytics 17.2.2 -> 17.2.3
com.google.firebase:firebase-messaging 20.1.0 -> 20.1.2
com.google.firebase:firebase-config 19.1.1 -> 19.1.2
com.google.firebase:firebase-firestore 21.4.0 -> 21.4.1

First of all:
We are very sorry for the inconvenience!

A downgrade to the old version of the SDKs, particularly a version of FirebaseInstanceId (a.k.a. firebase-iid) before 20.1.0 should solve the issue.

We are trying to identify the problem.
@asfdfdfd : We expect this or similar problems to occur if the FirebaseOptions are initialized without mandatory parameters Project ID, Application ID, or API key. However judging from @sudsingh438 's response before, I don't believe this is the issue here.

We believe that the underlying FirebaseInstallations SDK which is a new dependency of FirebaseInstanceId is causing this problem.

Is anybody able to reproduce this issue locally and can share with us steps to reproduce it?

We are also not able to reproduce this locally but we have some extra information, 97% of crashes happen in background. So I am not even sure that users get any crash dialog at all.

For us, also around 85%+ crashes are reported in background

Thanks for the information, @kozaxinan , @sudsingh438 !
We would appreciate any information that could help us pinpoint this problem:

Q1) Are your applications actually crashing (or is there a background thread crashing, but the application continues to run)?

Q2) How many crashes per app usage are there? If you write "97% of crashes" and "85%+ crashes" are you referring to 97%/85% of these IllegalStateException crashes or of all your app crashes? And (repeating Q1) by "crash" do you mean that the application is actually stopping?

Q3) Are the crashes happening randomly for end-users or repeatedly happening for the same end-user (e.g. crash-looping)?

Q4) If you are willing to share your Application ID or Project ID (which is non-secret information about your application), we would be able to look into our logs to see if the request patterns from your application to Firebase look healthy.

@andirayo
Q1) From the stack trace it looks like a background thread, but I can't tell if the application continues to run after the crash. In Crashlytics, I can see that 80% of those crashes occur when the application is in background. After examining the logs, it looks like some of those crashes occurred around the time when those users received push notifications, if it helps.
Q2) The crash is relatively rare. It is our most frequently occurring crash this week, but we still have 99.9% crash-free users.
Q3) There are 51 reported crashes reported by 48 users for the last 7 days, so it usually happens only once to each user.
Q4) Feel free to check our App ID: 1:499050530164:android:1508e54a25da7ef9

@andirayo thank you for looking into this. Hopefully this information is helpful:

Q1) I can't tell for sure but just like @MattSkala said the thread that is crashing does seem like a background thread. The trace is identical to the one in the issue description. Based on our logs it appears that some of the crashes might be occurring right after a push notification is received, but not all of them display that. Furthermore, we see the number of crashes increase significantly around the time that our daily push notifications are sent. However, this usually correlates with a rise in traffic anyway, so it's hard to say for sure that pushes are the cause.

Q2) For us the crash is affecting close to 1% of our total users. So while not super common, still high enough to cause serious concern (we'll be rolling back until there is a fix).

Q3) So far we have seen 128 crashes across 108 users. So they seem to be mostly unique, with a little bit of overlap.

Q4) I will follow up with my team and get back to you.

@MattSkala, @ygnessin :
Thank you so much for sharing this information. That really helps to better understand the situation.

@MattSkala : I looked into our server logs for the requests regarding Firebase identifiers from your application and can see exclusively successful requests.
I will involved the FCM/IID team into this, given that this might be related to push notifications arriving (while the application is in the background).

we had just released a new version of the app with these new versions when we started seeing this crash:
com.google.firebase:firebase-messaging from 20.1.0 -> 20.1.2
com.google.firebase:firebase-analytics from 17.2.2 -> 17.2.3

we were already using com.google.firebase:firebase-perf 19.0.5 before the crash happened so we have ruled out that it isn't that library.

we have decided to downgrade messaging and analytics for the time being while this is being investigated

@maxkohne , @ygnessin , and others:
By any chance, are you able to also share a StackTrace with us?

same exact stacktrace as what the author posted

Fatal Exception: java.lang.IllegalStateException at com.google.firebase.iid.FirebaseInstanceId.zzl(FirebaseInstanceId.java:75) at com.google.firebase.iid.FirebaseInstanceId.getId(FirebaseInstanceId.java:49) at com.google.firebase.perf.internal.zzf.zzbu(zzf.java:181) at com.google.firebase.perf.internal.zzf.zzbt(zzf.java:44) at com.google.firebase.perf.internal.zzf.zzc(zzf.java:195) at com.google.firebase.perf.internal.zze.run(zze.java:2) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641) at java.lang.Thread.run(Thread.java:764)

@maxkohne : Thank you so much!

internal: b/151480886

Update

We identified that the IllegalStateException is thrown in FirebaseInstanceId.getResultOrThrowException (FirebaseInstanceId.java:486) where task.getException() returns NULL.

We currently believe that some form of race condition and/or deadlock causes this issue.

We assume that the race condition occurs when multiple Firebase services require a client identifier (a.k.a. FID / Instance ID) at the same time, possibly in different threads or even different processes.
In the situation that multiple Firebase SDKs call FirebaseInstanceId.getId in parallel, a race condition might occur that leads to a deadlock.
Once the deadline for waiting for the identifier times out, an IllegalStateException is thrown.

Background

FirebaseInstanceId is an infrastructure service that is used by various Firebase services like Messaging (FCM), Google Analytics, and Performance Monitoring (FirePerf).
With the latest version (v20.1.0), FirebaseInstanceId introduced a new dependency: The Firebase Installations SDK (FIS).
We assume that the deadlock occurs in FIS and it is FirebaseInstanceId that times out waiting for the identifier, the response of FIS.

Next steps

To fix this issue, we need to be able to reproduce the issue locally.

At this point, we believe that an incoming push notification wakes up, i.e. triggers processes in an application that is running in background.
These processes could include Google Analytics recording the receival of the push notification and Firebase Performance Monitoring (FirePerf) recording some event or latency.
Thus, such an application would have to use at least Messaging (FCM) to receive push notifications and two Firebase services like Google Analytics and FirePerf that record events and thus need access to the client identifier.

Questions

Generally: Any information to achieve reproducing this crash would be highly appreciated!

Q5) Could you please share your Gradle Android dependency tree with us:
We would like to understand which Firebase SDKs (and which versions) you are using in your application.

Q6) Do you happen to have access to a full Android bugreport (including errors and logs from all threads)?

@andirayo Thanks for the update

Q5) Our dependency tree obtained from ./gradlew dependencies: https://pastebin.com/A8uGPzHh
Q6) I only have access to what I see in Crashlytics, these are stack traces for all threads: https://pastebin.com/XTve0qt3

@andirayo Please find the dependency list and stacktrace of one of the issue
Dependency list : https://pastebin.com/gcRZyxNQ
Stacktrace from crashlytics for all thread : https://pastebin.com/uJw7LgJm

@andirayo
We have more than 380 crashes. 98% of them are in background. According to threads, it is happening when we have notification. App uses network before notification displayed to fetch some information.

Q1) Not sure, we have crash report in firebase but not it looks like crash happens when notification received. FirebaseMessagingService is running at the moment of report.

Q2) 387 crashes for 370 reports. This 50 times more than 2nd most occurring crash. 98% percent of 387 reports happened in background. I am not sure but probably firebase messaging service fails to continue after this report.

Q3) Random users. 387 crashes for 370 reports

Q4) AppId : 1:208472424340:android:a243f98a00873753

Q5) Firebase dependencies :

object Firebase {

      const val analytics = "com.google.firebase:firebase-analytics:17.2.3"
      const val config = "com.google.firebase:firebase-config:19.1.2"
      const val core = "com.google.firebase:firebase-core:17.2.3"
      const val messaging = "com.google.firebase:firebase-messaging:20.1.2"
      const val perf = "com.google.firebase:firebase-perf:19.0.5"
    }

Q6) Stacktrace from firebase for all thread : https://0bin.net/paste/XYQs-BXx0WRDEe+g#F662Og7LOng22wN2W+Ob7UrBPg+5vOamqked4eRCex0

@MattSkala , @sudsingh438 :
Thanks so much for the stack traces! These are really valuable to us as they show what is happening in FirebaseInstallations during the crash.

@kozaxinan : Thanks a lot for the information. This is very helpful as it confirms our current assumptions. I also checked all client-server requests for the given AppId. Except for 1 request all ended successfully and thus I don't think server communication is the cause for this.

If possible, could you please resent the link to your stack trace as the link above does not seem to work for me.

@andirayo I updated the link.

@kozaxinan : Thank you for the updated link!

@MattSkala , @sudsingh438, @kozaxinan, and @ALL :
The stack traces containing all threads are the most helpful for us right now.

We would appreciate if you find the time to share more stack traces, given that the actual call stacks of the main thread seem to differ between cases, and also the call stacks of FirebaseInstallations differ.

@emanashraf :
Thank you for reporting this! I am sorry to hear that!

If possible, can you please share context with us:
Can you please share a new stack trace?
What percentage of users is affected or how many crashes do you see?
Do you have any indication when this is happening?
What Firebase SDKs and what versions are you using (in your Gradle file)?

@andirayo

--stack track ->https://0bin.net/paste/dCy-Q1r3exnQQk-z#LkY9TJO+11gfUbEI+96SWDmqvq8sQCAJWnFQcfUOLFm

--it happened in 20.1.3 to 14 users, and 1 user to 20.1.4

--I don't know when it's happening but occurrence is 100% background

-- implementation 'com.google.firebase:firebase-messaging:20.1.4'
implementation 'com.google.firebase:firebase-config:19.1.3'
implementation 'com.google.firebase:firebase-dynamic-links:19.1.0'
implementation 'com.google.firebase:firebase-inappmessaging-display:19.0.4'
implementation 'com.google.firebase:firebase-analytics:17.2.3'
implementation 'com.google.firebase:firebase-core:17.2.3'

is there anything i should do differently with this information?

@emanashraf :
Thanks so much for sharing this. This is really helpful.
I had a quick look at the stacktrace and noticed:

com.google.firebase.iid.FirebaseInstanceId.zzl (com.google.firebase:firebase-iid@@20.1.2:75)
com.google.firebase.iid.FirebaseInstanceId.getId (com.google.firebase:firebase-iid@@20.1.2:49)

The stacktrace seems to have been recorded by an instance running firebase-iid v20.1.2, but this issue was only fixed in firebase-iid v20.1.4.
Is it possible that some of your clients still run older versions of the application?

Where did you get the information from that the IllegalStateException crash happened to 1 user running firebase-iid v20.1.4?

@andirayo
from crashlytics , the 1 user had the new version from the apk which had the version 20.1.4

@emanashraf :
Thank you for the explanation!
Are you able to get and post the specific stacktrace for that use? That would be really helpful for us.

So, What was the problem and What's the solution?

@dcampogiani , @MattSkala , @ahinchman1 :
fyi: The fix for the LINT check has been published with firebase-iid v20.1.5:
https://firebase.google.com/support/release-notes/android#2020-04-03

@andirayo
i choose the version i want to see the stack track of from crashlytics , if that's what you're asking?

@emanashraf :
We would love to see a stacktrace of an application instance that uses firebase-iid v20.1.4 (or v20.1.5) to be able to look into this issue again.
You are saying that Crashlytics shows 1 user experiencing this crash with v20.1.4. However the stack trace you posted in your comment above shows firebase-iid v20.1.2.
Is it possible for you to share a stacktrace with us of an application instance that actually uses v20.1.4+?

After using v20.1.4 it seems that problem is resolved in our end until now. We have 50k++ average users and we didn't get any issue reported after having this version. Also will update this thread if I get any.

Not sure if my problem relates to this specific issue, but cross-posting my message to the Firebase-Talk group here: https://groups.google.com/forum/#!topic/firebase-talk/ShiZ2nLLVxU

To summarize, I found that FirebaseInitProvider does not always run. When the app is started in "restricted mode" (this happens when AutoBackup is enabled and running), ContentProviders are ignored, and the base Application is run instead of the class defined in the manifest. I validated that services can be started while in this mode. What I don't know is if Firebase sends messages/events while any app is in the mode or not.

@pablobaxter :
Thanks for posting.
I don't think the issue you describe is related.

The exception for this issue happen when FirebaseInstanceId.getId is executing.
In your case, it happens during the initialization of FirebaseInstanceId.

@bluetoothfx : Thank you for reporting that the change (#1355) has fixed the issue for you!

@ALL:
After @bluetoothfx's report I am about to close this issue as fixed, but I would love to have at least one more person chime in to be confident that this issue was indeed fixed.

@pablobaxter :
Thanks for posting.
I don't think the issue you describe is related.

The exception for this issue happen when FirebaseInstanceId.getId is executing.
In your case, it happens during the initialization of FirebaseInstanceId.

I've been working through any scenario as to why the FirebaseInitProvider was not running before my service (subclass of FirebaseInstanceIdService) was called. This issue was the closest I was able to find to my problem. I think i have enough evidence to prove this is an Android platform problem, not a Firebase problem. Thank you for the quick response!

@pablobaxter :
Anytime! Good luck! I hope you find someone that can give you better input than me! :)

@andirayo Hey. There isn't any crash on our app too; after upgrading to v20.1.4.

@adiamartya : Thank you so much for confirming! :)

We still see an issue with v20.1.5. Here is the stack trace:

Fatal Exception: java.lang.IllegalStateException
       at com.google.firebase.iid.FirebaseInstanceId.zzl(FirebaseInstanceId.java:75)
       at com.google.firebase.iid.FirebaseInstanceId.getId(FirebaseInstanceId.java:49)
       at com.google.firebase.perf.internal.zzf.zzbu(zzf.java:181)
       at com.google.firebase.perf.internal.zzf.zzbt(zzf.java:44)
       at com.google.firebase.perf.internal.zzf.zzc(zzf.java:195)
       at com.google.firebase.perf.internal.zze.run(zze.java:2)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1113)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:588)
       at java.lang.Thread.run(Thread.java:818)

@hetang is it the full stack trace? If not would you mind sharing full stack trace. Thanks :)

This is what I see in firebase crash reporting. We are not able to produce this locally and not able to see a full stack trace.

@hetang :
Thank you for sharing the stacktrace!
I ran the stacktrace through our deobfuscation mapper and the stacktrace has been recorded with either v20.1.1 or 20.1.2 of FirebaseInstanceId. Both these versions did not contain the fix that versions v20.1.4 and 20.1.5 contain. Thus, I have to assume that these exceptions have been recorded with old versions of FirebaseInstanceId.

I am not sure why Crashlytics shows the incorrect version of FirebaseInstanceId. I will investigate with them.

I am closing this issue as successfully fixed since we had reports of v20.1.5 no longer showing this issue and there have no longer been reports of this error.

Hi @andirayo, I'm still seeing this in 20.1.5:

Fatal Exception: java.lang.IllegalStateException
       at com.google.firebase.iid.FirebaseInstanceId.zzl(FirebaseInstanceId.java:70)
       at com.google.firebase.iid.FirebaseInstanceId.getId(FirebaseInstanceId.java:48)
       at com.google.firebase.perf.internal.zzf.zzbv(zzf.java:181)
       at com.google.firebase.perf.internal.zzf.zzbu(zzf.java:44)
       at com.google.firebase.perf.internal.zzf.zzc(zzf.java:195)
       at com.google.firebase.perf.internal.zze.run(zze.java:2)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
       at java.lang.Thread.run(Thread.java:919)

Are you certain it was fixed in that version? Are we hitting the same weird issue as @hetang where iid isn't resolving to a newer version (androidDependencies definitely shows 20.1.5)?

@andirayo do you have any update here? This is currently by far the largest open crash cluster in our app and it sounds like we're not alone. I'm happy to provide any additional details that might help track this down. Thank you!

Filed a new issue #1528 based on advice from @TKBurner