Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Firestore writing breaks in IOS background mode #3526

Closed
ebeling opened this issue Jul 31, 2020 · 16 comments
Closed

Firestore writing breaks in IOS background mode #3526

ebeling opened this issue Jul 31, 2020 · 16 comments
Assignees

Comments

@ebeling
Copy link

ebeling commented Jul 31, 2020

[REQUIRED] Describe your environment

  • Operating System version: iOS 13.5
  • Browser version: Mobile Safari 13.1
  • Firebase SDK version: 7.17.1
  • Firebase Product: firestore (auth, database, storage, etc)

[REQUIRED] Describe the problem

Steps to reproduce:

I implement an Ionic/Cordova App with Angular Fire (5.4.2) and Firestore. The app receives data from bluetooth interface and stores data periodically in firestore. Firestore is configured using offline persistence. All works fine as long the app is in the foreground. But if it goes to the background firestore breaks without error. The app must be restarted to make firestore working again. I can certainly exclude a bluetooth interface failure. Data are also received in the background mode! This only happens in IOS - in Android storing the data works fine. Logging I have enabled:

[Log] [2020-07-31T06:38:47.274Z]  @firebase/firestore: – "Firestore (7.17.1): IndexedDbPersistence" – "Starting transaction:" – "updateClientMetadataAndTryBecomePrimary" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:47.286Z]  @firebase/firestore: – "Firestore (7.17.1): SharedClientState" – "SET" – "firestore_sequence_number_firestore/[DEFAULT]/testdatabase-fs/" – "148" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:47.400Z]  @firebase/firestore: – "Firestore (7.17.1): SharedClientState" – "EVENT" – "firestore_sequence_number_firestore/[DEFAULT]/testdatabase-fs/" – "148" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:47.411Z]  @firebase/firestore: – "Firestore (7.17.1): SimpleDb" – "GET" – "owner" – "owner" – "{\"ownerId\":\"2p7ftHDHtDrt4sqxeLfn\",\"allowTabSynchronization\":true,\"leaseTimestampMs\":1596177523178}" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:47.414Z]  @firebase/firestore: – "Firestore (7.17.1): IndexedDbPersistence" – "Client '2p7ftHDHtDrt4sqxeLfn' is not zombied in LocalStorage" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:47.417Z]  @firebase/firestore: – "Firestore (7.17.1): SimpleDb" – "PUT" – "clientMetadata" – "<auto-key>" – "{\"clientId\":\"2p7ftHDHtDrt4sqxeLfn\",\"updateTimeMs\":1596177527417,\"networkEnabled\":true,\"inForeground\":false}" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:47.454Z]  @firebase/firestore: – "Firestore (7.17.1): SimpleDb" – "GET" – "owner" – "owner" – "{\"ownerId\":\"2p7ftHDHtDrt4sqxeLfn\",\"allowTabSynchronization\":true,\"leaseTimestampMs\":1596177523178}" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:47.458Z]  @firebase/firestore: – "Firestore (7.17.1): SimpleDb" – "GET" – "owner" – "owner" – "{\"ownerId\":\"2p7ftHDHtDrt4sqxeLfn\",\"allowTabSynchronization\":true,\"leaseTimestampMs\":1596177523178}" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:47.459Z]  @firebase/firestore: – "Firestore (7.17.1): IndexedDbPersistence" – "Client '2p7ftHDHtDrt4sqxeLfn' is not zombied in LocalStorage" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:47.463Z]  @firebase/firestore: – "Firestore (7.17.1): SimpleDb" – "PUT" – "owner" – "owner" – "{\"ownerId\":\"2p7ftHDHtDrt4sqxeLfn\",\"allowTabSynchronization\":true,\"leaseTimestampMs\":1596177527462}" (cordova.js, line 1540)
[Info] 08:38:48,800 TC.FirebaseProviderService _writeMeasurementSegment entry a0e6f8ce2f84 (cordova.js, line 1540)
[Debug] 08:38:48,828 TC.FirebaseProviderService segmentDocument assembled - try write to hdjjsdhjs83j37r474jdh4/measurements/measurements/1596177334703_a0e6f8ce3a86_a0e6f8ce2f84/a0e6f8ce2f84 ... (cordova.js, line 1540)
[Info] 08:38:48,838 TC.FirebaseProviderService _writeMeasurementSegment exit (cordova.js, line 1540)
[Log] [2020-07-31T06:38:48.848Z]  @firebase/firestore: – "Firestore (7.17.1): IndexedDbPersistence" – "Starting transaction:" – "Locally write mutations" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:48.859Z]  @firebase/firestore: – "Firestore (7.17.1): SharedClientState" – "SET" – "firestore_sequence_number_firestore/[DEFAULT]/testdatabase-fs/" – "149" (cordova.js, line 1540)
[Info] 08:38:48,870 TC.FirebaseProviderService _writeMeasurementSegment entry a0e6f8ce3a86 (cordova.js, line 1540)
[Debug] 08:38:48,882 TC.FirebaseProviderService segmentDocument assembled - try write to hdjjsdhjs83j37r474jdh4/measurements/measurements/1596177334703_a0e6f8ce3a86_a0e6f8ce2f84/a0e6f8ce3a86 ... (cordova.js, line 1540)
[Info] 08:38:48,884 TC.FirebaseProviderService _writeMeasurementSegment exit (cordova.js, line 1540)
[Log] [2020-07-31T06:38:48.893Z]  @firebase/firestore: – "Firestore (7.17.1): SharedClientState" – "EVENT" – "firestore_sequence_number_firestore/[DEFAULT]/testdatabase-fs/" – "149" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:49.017Z]  @firebase/firestore: – "Firestore (7.17.1): SimpleDb" – "GET" – "owner" – "owner" – "{\"ownerId\":\"2p7ftHDHtDrt4sqxeLfn\",\"allowTabSynchronization\":true,\"leaseTimestampMs\":1596177527462}" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:49.026Z]  @firebase/firestore: – "Firestore (7.17.1): IndexedDbPersistence" – "Client '2p7ftHDHtDrt4sqxeLfn' is not zombied in LocalStorage" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:49.046Z]  @firebase/firestore: – "Firestore (7.17.1): SimpleDb" – "ADD" – "mutations" – "{}" – "{}" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:49.053Z]  @firebase/firestore: – "Firestore (7.17.1): SimpleDb" – "PUT" – "mutations" – "<auto-key>" – "{\"userId\":\"yn4dKHMBQxPqGdG1RV6grcMuHkq2\",\"batchId\":5,\"localWriteTimeMs\":1596177528844,\"baseMutations\":[],\"mutations\":[{\"update\":{\"name\":\"pro…" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:49.065Z]  @firebase/firestore: – "Firestore (7.17.1): SimpleDb" – "PUT" – "documentMutations" – "[\"yn4dKHMBQxPqGdG1RV6grcMuHkq2\",\"hdjjsdhjs83j37r474jdh4\\u0001\\u0001measurements\\u0001\\u0001measurements\\u0001\\u00011596177334703_a0e6f8ce3a86_a0e6…" – "{}" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:49.259Z]  @firebase/firestore: – "Firestore (7.17.1): SharedClientState" – "SET" – "firestore_mutations_firestore/[DEFAULT]/testdatabase-fs/_5_yn4dKHMBQxPqGdG1RV6grcMuHkq2" – "{\"state\":\"pending\",\"updateTimeMs\":1596177529258}" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:49.267Z]  @firebase/firestore: – "Firestore (7.17.1): IndexedDbPersistence" – "Starting transaction:" – "notifyLocalViewChanges" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:49.273Z]  @firebase/firestore: – "Firestore (7.17.1): SharedClientState" – "SET" – "firestore_sequence_number_firestore/[DEFAULT]/testdatabase-fs/" – "150" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:49.415Z]  @firebase/firestore: – "Firestore (7.17.1): SharedClientState" – "EVENT" – "firestore_mutations_firestore/[DEFAULT]/testdatabase-fs/_5_yn4dKHMBQxPqGdG1RV6grcMuHkq2" – "{\"state\":\"pending\",\"updateTimeMs\":1596177529258}" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:49.425Z]  @firebase/firestore: – "Firestore (7.17.1): SharedClientState" – "EVENT" – "firestore_sequence_number_firestore/[DEFAULT]/testdatabase-fs/" – "150" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:49.539Z]  @firebase/firestore: – "Firestore (7.17.1): SimpleDb" – "GET" – "owner" – "owner" – "{\"ownerId\":\"2p7ftHDHtDrt4sqxeLfn\",\"allowTabSynchronization\":true,\"leaseTimestampMs\":1596177527462}" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:49.544Z]  @firebase/firestore: – "Firestore (7.17.1): IndexedDbPersistence" – "Client '2p7ftHDHtDrt4sqxeLfn' is not zombied in LocalStorage" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:49.592Z]  @firebase/firestore: – "Firestore (7.17.1): IndexedDbPersistence" – "Starting transaction:" – "Get next mutation batch" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:49.601Z]  @firebase/firestore: – "Firestore (7.17.1): SharedClientState" – "SET" – "firestore_sequence_number_firestore/[DEFAULT]/testdatabase-fs/" – "151" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:49.918Z]  @firebase/firestore: – "Firestore (7.17.1): SharedClientState" – "EVENT" – "firestore_sequence_number_firestore/[DEFAULT]/testdatabase-fs/" – "151" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:50.145Z]  @firebase/firestore: – "Firestore (7.17.1): SimpleDb" – "GET" – "owner" – "owner" – "{\"ownerId\":\"2p7ftHDHtDrt4sqxeLfn\",\"allowTabSynchronization\":true,\"leaseTimestampMs\":1596177527462}" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:50.152Z]  @firebase/firestore: – "Firestore (7.17.1): IndexedDbPersistence" – "Client '2p7ftHDHtDrt4sqxeLfn' is not zombied in LocalStorage" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:50.299Z]  @firebase/firestore: – "Firestore (7.17.1): IndexedDbPersistence" – "Starting transaction:" – "Get next mutation batch" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:50.313Z]  @firebase/firestore: – "Firestore (7.17.1): SharedClientState" – "SET" – "firestore_sequence_number_firestore/[DEFAULT]/testdatabase-fs/" – "152" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:50.319Z]  @firebase/firestore: – "Firestore (7.17.1): SharedClientState" – "EVENT" – "firestore_sequence_number_firestore/[DEFAULT]/testdatabase-fs/" – "152" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:50.322Z]  @firebase/firestore: – "Firestore (7.17.1): SimpleDb" – "GET" – "owner" – "owner" – "{\"ownerId\":\"2p7ftHDHtDrt4sqxeLfn\",\"allowTabSynchronization\":true,\"leaseTimestampMs\":1596177527462}" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:50.324Z]  @firebase/firestore: – "Firestore (7.17.1): IndexedDbPersistence" – "Client '2p7ftHDHtDrt4sqxeLfn' is not zombied in LocalStorage" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:50.492Z]  @firebase/firestore: – "Firestore (7.17.1): Connection" – "Creating WebChannel: https://firestore.googleapis.com/google.firestore.v1.Firestore/Write/channel [object Object]" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:50.503Z]  @firebase/firestore: – "Firestore (7.17.1): IndexedDbPersistence" – "Starting transaction:" – "Locally write mutations" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:50.505Z]  @firebase/firestore: – "Firestore (7.17.1): SharedClientState" – "SET" – "firestore_sequence_number_firestore/[DEFAULT]/testdatabase-fs/" – "153" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:50.775Z]  @firebase/firestore: – "Firestore (7.17.1): SharedClientState" – "EVENT" – "firestore_sequence_number_firestore/[DEFAULT]/testdatabase-fs/" – "153" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:51.004Z]  @firebase/firestore: – "Firestore (7.17.1): SimpleDb" – "GET" – "owner" – "owner" – "{\"ownerId\":\"2p7ftHDHtDrt4sqxeLfn\",\"allowTabSynchronization\":true,\"leaseTimestampMs\":1596177527462}" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:51.012Z]  @firebase/firestore: – "Firestore (7.17.1): IndexedDbPersistence" – "Client '2p7ftHDHtDrt4sqxeLfn' is not zombied in LocalStorage" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:51.154Z]  @firebase/firestore: – "Firestore (7.17.1): SimpleDb" – "ADD" – "mutations" – "{}" – "{}" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:51.168Z]  @firebase/firestore: – "Firestore (7.17.1): SimpleDb" – "PUT" – "mutations" – "<auto-key>" – "{\"userId\":\"yn4dKHMBQxPqGdG1RV6grcMuHkq2\",\"batchId\":6,\"localWriteTimeMs\":1596177530502,\"baseMutations\":[],\"mutations\":[{\"update\":{\"name\":\"pro…" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:51.180Z]  @firebase/firestore: – "Firestore (7.17.1): SimpleDb" – "PUT" – "documentMutations" – "[\"yn4dKHMBQxPqGdG1RV6grcMuHkq2\",\"hdjjsdhjs83j37r474jdh4\\u0001\\u0001measurements\\u0001\\u0001measurements\\u0001\\u00011596177334703_a0e6f8ce3a86_a0e6…" – "{}" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:51.250Z]  @firebase/firestore: – "Firestore (7.17.1): SharedClientState" – "SET" – "firestore_mutations_firestore/[DEFAULT]/testdatabase-fs/_6_yn4dKHMBQxPqGdG1RV6grcMuHkq2" – "{\"state\":\"pending\",\"updateTimeMs\":1596177531250}" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:51.256Z]  @firebase/firestore: – "Firestore (7.17.1): IndexedDbPersistence" – "Starting transaction:" – "notifyLocalViewChanges" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:51.259Z]  @firebase/firestore: – "Firestore (7.17.1): SharedClientState" – "SET" – "firestore_sequence_number_firestore/[DEFAULT]/testdatabase-fs/" – "154" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:51.274Z]  @firebase/firestore: – "Firestore (7.17.1): SharedClientState" – "EVENT" – "firestore_mutations_firestore/[DEFAULT]/testdatabase-fs/_6_yn4dKHMBQxPqGdG1RV6grcMuHkq2" – "{\"state\":\"pending\",\"updateTimeMs\":1596177531250}" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:51.277Z]  @firebase/firestore: – "Firestore (7.17.1): SharedClientState" – "EVENT" – "firestore_sequence_number_firestore/[DEFAULT]/testdatabase-fs/" – "154" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:51.285Z]  @firebase/firestore: – "Firestore (7.17.1): SimpleDb" – "GET" – "owner" – "owner" – "{\"ownerId\":\"2p7ftHDHtDrt4sqxeLfn\",\"allowTabSynchronization\":true,\"leaseTimestampMs\":1596177527462}" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:51.287Z]  @firebase/firestore: – "Firestore (7.17.1): IndexedDbPersistence" – "Client '2p7ftHDHtDrt4sqxeLfn' is not zombied in LocalStorage" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:51.421Z]  @firebase/firestore: – "Firestore (7.17.1): IndexedDbPersistence" – "Starting transaction:" – "Get next mutation batch" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:51.434Z]  @firebase/firestore: – "Firestore (7.17.1): SharedClientState" – "SET" – "firestore_sequence_number_firestore/[DEFAULT]/testdatabase-fs/" – "155" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:51.441Z]  @firebase/firestore: – "Firestore (7.17.1): SharedClientState" – "EVENT" – "firestore_sequence_number_firestore/[DEFAULT]/testdatabase-fs/" – "155" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:51.444Z]  @firebase/firestore: – "Firestore (7.17.1): SimpleDb" – "GET" – "owner" – "owner" – "{\"ownerId\":\"2p7ftHDHtDrt4sqxeLfn\",\"allowTabSynchronization\":true,\"leaseTimestampMs\":1596177527462}" (cordova.js, line 1540)
[Log] [2020-07-31T06:38:51.446Z]  @firebase/firestore: – "Firestore (7.17.1): IndexedDbPersistence" – "Client '2p7ftHDHtDrt4sqxeLfn' is not zombied in LocalStorage" (cordova.js, line 1540)
[Info] 08:40:25,606 TC.FirebaseProviderService _writeMeasurementSegment entry a0e6f8ce2f84 (cordova.js, line 1540)
[Debug] 08:40:25,634 TC.FirebaseProviderService segmentDocument assembled - try write to hdjjsdhjs83j37r474jdh4/measurements/measurements/1596177334703_a0e6f8ce3a86_a0e6f8ce2f84/a0e6f8ce2f84 ... (cordova.js, line 1540)
[Info] 08:40:25,644 TC.FirebaseProviderService _writeMeasurementSegment exit (cordova.js, line 1540)
[Info] 08:40:25,862 TC.FirebaseProviderService _writeMeasurementSegment entry a0e6f8ce3a86 (cordova.js, line 1540)
[Debug] 08:40:25,890 TC.FirebaseProviderService segmentDocument assembled - try write to hdjjsdhjs83j37r474jdh4/measurements/measurements/1596177334703_a0e6f8ce3a86_a0e6f8ce2f84/a0e6f8ce3a86 ... (cordova.js, line 1540)
[Info] 08:40:25,900 TC.FirebaseProviderService _writeMeasurementSegment exit (cordova.js, line 1540)
[Info] 08:41:40,375 TC.FirebaseProviderService _writeMeasurementSegment entry a0e6f8ce2f84 (cordova.js, line 1540)
[Debug] 08:41:40,382 TC.FirebaseProviderService segmentDocument assembled - try write to hdjjsdhjs83j37r474jdh4/measurements/measurements/1596177334703_a0e6f8ce3a86_a0e6f8ce2f84/a0e6f8ce2f84 ... (cordova.js, line 1540)
[Info] 08:41:40,385 TC.FirebaseProviderService _writeMeasurementSegment exit (cordova.js, line 1540)
[Info] 08:41:40,385 TC.FirebaseProviderService _writeMeasurementSegment entry a0e6f8ce3a86 (cordova.js, line 1540)
[Debug] 08:41:40,387 TC.FirebaseProviderService segmentDocument assembled - try write to hdjjsdhjs83j37r474jdh4/measurements/measurements/1596177334703_a0e6f8ce3a86_a0e6f8ce2f84/a0e6f8ce3a86 ... (cordova.js, line 1540)
[Info] 08:41:40,387 TC.FirebaseProviderService _writeMeasurementSegment exit (cordova.js, line 1540)

Could be related to #3495

Relevant Code:

public writeMeasurementSegment(...,  segment: MeasurementSegment): boolean {
   this.logger.entry('_writeMeasurementSegment', segment.id.serialNumber);
   ...
   const segmentDocument: AngularFirestoreDocument = this.afs.collection(pathToFsTDSensorMeasurements)
          .doc('' + segment.id.starttimestamp);
      this.logger.debug('segmentDocument assembled - try write to ' + pathToFsTDSensorMeasurements + ' ...');

      segmentDocument.set(segmentDocumentItem).then(() => {
        this.logger.entry('_handleSetSegmentDocumentSuccess', segment.id.starttimestamp);
        this.logger.exit('_handleSetSegmentDocumentSuccess', segment.id.starttimestamp);
      }).catch((reason) => {
        this.logger.entry('_handleSetSegmentDocumentError', segment.id.starttimestamp);
        this.logger.error(reason);
        this.logger.exit('_handleSetSegmentDocumentError', segment.id.starttimestamp);
      });
      this.logger.exit('_writeMeasurementSegment');

After going into the background 'segmentDocument.set(...)' never resolves.

@schmidt-sebastian
Copy link
Contributor

@ebeling Thanks for opening this issue. I don't see any error messages in the logs you provided. Do you happen to see any errors in the console?

@schmidt-sebastian schmidt-sebastian self-assigned this Jul 31, 2020
@wilhuff
Copy link
Contributor

wilhuff commented Jul 31, 2020

iOS locks the filesystem by default when your app is in the background. Firestore does not support this configuration. Either avoid writing to Firestore while in the background or change your entitlements to disable automatic filesystem locking.

See #2846 for more details including which entitlement to change.

Note that the file protection entitlement affects the default for new files but not existing ones. This means that changing the entitlement will not change any existing files and will not fix any users that already have created an instance. The same issue contains some example code for how to fix existing installations.

@wilhuff wilhuff closed this as completed Jul 31, 2020
@ebeling
Copy link
Author

ebeling commented Aug 3, 2020

Thank you for answer. Please help me: I don't use firestore messaging like descibed in #2846 and I don't use chrome extension, I use write to firestore. If firestore doesn't support this, why the document.set-method doesn't throw an error like "can not write because of no access to filesystem"? And firestore hangs, if the app comes back to the foreground, the app must be restarted ?! Do you mean firestore can not be used if writing in background is necessary?

@wilhuff
Copy link
Contributor

wilhuff commented Aug 3, 2020

I'm so sorry! I meant to link you over to an issue in the iOS repository--I can see how that would be confusing!

The issue to which I meant to refer you was firebase/firebase-ios-sdk#2846, wherein we discussed setting the com.apple.developer.default-data-protection entitlement to NSFileProtectionCompleteUntilFirstUserAuthentication. There's also code there that you can use to fix existing users.

If you need to write in the background, then either you need to set the entitlement that changes the filesystem protection level or you can't use Firestore. Fixing the SDK so that it provides an error in this case instead of throwing an Objective-C exception is on our list of things to do, but we haven't gotten to it yet.

@ebeling
Copy link
Author

ebeling commented Aug 4, 2020

Thank you for reply. I read your big discussion and try to implement the file entitlement solution. If I understand right, I have to do this:

  1. In Xcode (11.6) on app target using tab 'Signing & Capabilities' and add 'Data Protection'
  2. Under "https://developer.apple.com" expand App-ID with capability 'Data Protection -> Protected Until First User Authentication' and renew developer and distribution profile.
  3. Update profiles in Xcode project using tab 'Signing & Capabilities'.
    In my case there was an error:
    "Provisioning profile "XXX Development Profil" doesn't match the entitlements file's value for the com.apple.developer.default-data-protection entitlement." I found this in apple developer forum:
    https://developer.apple.com/forums/thread/95757

I determined that Xcode filled other key-value-pair in files "Entitlements-Debug.plist" and "Entitlements-Release.plist" (NSFileProtectionComplete is default value), I changed it to "NSFileProtectionCompleteUntilFirstUserAuthentication" outside of Xcode, because Entitlement-files were not visible in project navigator. After that no errors occurred with signature and profiles.
4. Delete App on mobile device (to enforce file protection entitlement can affect new app instance)
5. Clean, build and deploy app to mobile device

But nothing changed in term of storing data in firestore in background mode. Do you know whether I can see on mobile device if the permission for store files is correctly built into my app ?

@wilhuff
Copy link
Contributor

wilhuff commented Aug 4, 2020

The entitlement only affects the default for creating new files. Deleting the app should have cleaned up anything from before so this should have worked. You can check that it's taking effect with the code in

firebase/firebase-ios-sdk#2846 (comment)

Change the FileManager.default.setAttributes to attributesOfItem(atPath:) to read them instead.

However, I suspect that since you're still reproducing the failure this somehow hasn't taken effect. @morganchen12 do you have any insight into whether something might be missing from the provisioning steps taken?

@morganchen12
Copy link

The steps to update the app's entitlements look solid. You can force Xcode to refresh a provisioning profile by deleting that profile from the Xcode navigator and then re-downloading it, but based on the steps described I don't think this will have any effect. As @wilhuff mentioned you can use the FileManager class to check if the protection level on Firestore's leveldb files are correct.

@ebeling can you enable native logging via Firestore.enableLogging(true) in Swift and share the logs from reproing the issue with native logging enabled? This will give us a better picture of what's failing in Firestore.

@ebeling
Copy link
Author

ebeling commented Aug 5, 2020

Thank you for your engagement! I use cordova and because of this the native code is written in objective-c. @wilhuff I can't copy your unlock-function, I transfered it to objective-c

- (void) unlockFirestoreData
{
    NSDictionary<NSFileAttributeKey,id> *attributes = @{NSFileProtectionKey:NSFileProtectionNone};
    NSArray *dirs = NSSearchPathForDirectoriesInDomains(NSApplicationSupportDirectory, NSUserDomainMask, YES);
    NSString *applicationSupportDirectory = [dirs firstObject];
    NSLog(@"applicationSupportDirectory: '%@'", applicationSupportDirectory);
    
    
    NSString *firestoreDir = [applicationSupportDirectory stringByAppendingString:@"/firestore"];
    NSLog(@"firestoreDir: '%@'", firestoreDir);
    NSFileManager* sharedFM = [NSFileManager defaultManager];
    NSError *error = nil;
    if (![sharedFM fileExistsAtPath: firestoreDir isDirectory:nil]) {
        [sharedFM createDirectoryAtPath:firestoreDir withIntermediateDirectories:true attributes:attributes error:&error];
        if (error) {
            NSLog(@"ERROR while create firestore dir: '%@'", error);
            return;
        }
    }

    NSLog(@"state of firestore path: '%@'", [sharedFM attributesOfItemAtPath:firestoreDir error:&error]);
    NSDirectoryEnumerator<NSString *> *files = [sharedFM enumeratorAtPath:firestoreDir];
    NSString *file = nil;
    while (file = [files nextObject]) {
        [sharedFM setAttributes:attributes ofItemAtPath:file error:&error];
        if (error) {
            NSLog(@"ERROR while set file protection none '%@' with error '%@", file, error);
            error = nil;
        }
    }
}

The log for firestore path attributs is

state of firestore path: '{
    NSFileCreationDate = "2020-08-05 10:02:00 +0000";
    NSFileExtensionHidden = 0;
    NSFileGroupOwnerAccountID = 501;
    NSFileGroupOwnerAccountName = mobile;
    NSFileModificationDate = "2020-08-05 10:02:00 +0000";
    NSFileOwnerAccountID = 501;
    NSFileOwnerAccountName = mobile;
    NSFilePosixPermissions = 493;
    NSFileProtectionKey = NSFileProtectionNone;
    NSFileReferenceCount = 2;
    NSFileSize = 64;
    NSFileSystemFileNumber = 608518;
    NSFileSystemNumber = 16777218;
    NSFileType = NSFileTypeDirectory;
}'

I downloaded app-container with Xcode device manager. I asked me, wherefrom should the javascript-firebase-sdk which is executed in Webview know the firestore path which is set in native environment? As you can see in appended app-container the IndexedDB-files are located under "AppData/Library/WebKit/WebsiteData/IndexedDB/v1/ionic_localhost_0/..." But if they are visible they should not be protected?!
@morganchen12 I use javascript-sdk of firebase and there I have enabled logging ?!

Here is the app-container
202008051207appcontainer.xcappdata.zip

@morganchen12
Copy link

Based on the log you posted, the access level of Firestore files shouldn't be causing write issues in the background:

NSFileProtectionKey = NSFileProtectionNone;

You can enable Objective-C level logging with the same Swift call, translated to Objective-C:

[FIRFirestore enableLogging:YES];

@ebeling
Copy link
Author

ebeling commented Aug 6, 2020

I'm not sure that the problem has something to do with file entitlements. I performed some tests and observed logging in device console

Test 1 (without any write calls)

  • start app at 07:48:54 , don't call any firestore functions
  • lock device at 07:49:12 (logging in device console stops)
  • unlock device at 07:52:23 (logging in device console proceeds, among other -> SimpleDb Transaction failed with error: %s. Retrying: %s. Attempt to delete range from database without an in-progress transaction true
  • call AngulareFireCollection.valuechanges (07:52:27) - works as expected
  • change to view without firestore calls
  • lock device at 08:00:06 (logging in device console stops)
  • unlock device at 08:25:44 (logging shows SimpleDb Aborting transaction: An internal error was encountered in the Indexed Database server
    corresponding log:
    20200806074854_firestoreBackground.log

Test 2

  • start app at 08:45:11 (there are sometimes info '"inForeground":false' in log, for example 08:48:02 )
  • call AngulareFireCollection.valuechanges (08:50:33) - works as expected
  • change to view without firestore calls
  • lock device at 08:50:53 (logging in device console stops)
  • unlock device at 08:54:32 (logging in device console proceeds)
  • call AngulareFireCollection.valuechanges (08:55:42) - works as expected
  • change to view without firestore calls
  • lock device at 08:56:28 (logging in device console stops)
  • unlock device at 09:22:34 (logging in device console proceeds)
  • call document.set at 09:23:58
  • lock device at 09:24:59 (logging in device console proceeds until 09:27:39, then stops !!!)
  • unlock device at 09:30 -> no any firestore logging appears, firestore seems to be broken
    corresponding log:
    202008060845_firestoreBackground.log

note: under Android the firestore logging in console takes place without stopping independent of app state (foreground/ background)

@wilhuff wilhuff reopened this Aug 6, 2020
@wilhuff wilhuff closed this as completed Aug 6, 2020
@wilhuff
Copy link
Contributor

wilhuff commented Aug 6, 2020

Thanks for the logs. This clears up what's going on. The key part of the logs is

Failed to execute 'transaction' on 'IDBDatabase': The database connection is closing.

This means you're running into the same issue as #3495. The good news is that we have a fix submitted in #3535 which should be released soon. (I believe this was submitted after the cut-off for this week's release, but it should be in the next release if everything goes well.)

@ebeling
Copy link
Author

ebeling commented Aug 7, 2020

@wilhuff Thank you for new release (7.17.2) where the linked fix is included. I installed it. The behavior is different now. Unfortunately firestore still seems to crash.

I tested in Test 1

  • start app at 14:00:10 an do calls with firestore
  • lock device at 14:00:25 (logging in device console proceeds until 14:01:58)
  • unlock device at 14:03: firestore does not work even the app is back to foreground

corresponding log:
20200807140010_firebaseBackground.log

I tested my issue described in Test 2. Here is my test sequence:

  • start app at 12.01:27
  • call document.set 12:01:46 - works as expected
  • call AngulareFireCollection.valuechanges (12:01:58) - works as expected
  • call document.set at 12:03:16 - works as expected
  • lock device at 12:03.23 (different to release 7.17.1 logging in device console proceeds)
  • next call document.set works while app is in background (different to release 7.17.1 ) as logging shows (12:03.53 and 12:04:54 )
  • logging stops at 12:05.20 and firestore does not work even the app is back to foreground

corresponding lo file:
202008071201_firestoreBackground_xcode.log

@wilhuff
Copy link
Contributor

wilhuff commented Aug 7, 2020

The most recent release does not contain the fix in #3535. As I said, this change landed after the cut-off for yesterday's release. The fix will be in the next release.

In test 2, are you logging errors returned by the Firestore API while in the background? My understanding is that after IndexedDB locks, writes to Firestore should fail and we should be failing any writes or queries you try to make.

@ebeling
Copy link
Author

ebeling commented Aug 10, 2020

Sorry, but if I go to firebase tag 7.17.2, than there are 9 commits including the commit for #3535 ( https://github.com/firebase/firebase-js-sdk/compare/[email protected] ) ?

Both in test 1 and in test 2 no errors were thrown while writing in background, firestore freezes.

@schmidt-sebastian
Copy link
Contributor

@ebeling The link you posted shows the commits since 7.17.2. These are all the commits that are part of the release: https://github.com/firebase/firebase-js-sdk/commits/firebase%407.17.2

The changes in #3535 will be published this week.

@ebeling
Copy link
Author

ebeling commented Aug 10, 2020

@wilhuff Thank you! I will wait for this release. I tried to build master branch, but this failed for me, I don't know yarn and after the commands 'yarn' and 'yarn build' I don't see the distribution ...

@firebase firebase locked and limited conversation to collaborators Sep 6, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

5 participants