Skip to content

Errors importing lots of entities to DataStore using the emulator #95

Closed
@JustinBeckwith

Description

@JustinBeckwith

From @glenpike on May 23, 2018 10:26

[x] - Search the issues already opened: https://github.com/GoogleCloudPlatform/google-cloud-node/issues
[x] - Search StackOverflow: http://stackoverflow.com/questions/tagged/google-cloud-platform+node.js
[404] - Check our Troubleshooting guide: https://googlecloudplatform.github.io/google-cloud-node/#/docs/guides/troubleshooting
[404] - Check our FAQ: https://googlecloudplatform.github.io/google-cloud-node/#/docs/guides/faq

If you are still having issues, please be sure to include as much information as possible:

Environment details

  • gcloud SDK: 202.0.0
  • OS: OSX El Capitan (10.11.6) Using about 12GB / 16GB memory
  • Node.js version: v8.11.2
  • npm version: v5.6.0
  • google-cloud-node version:
├─┬ @google-cloud/[email protected]
│ ├─┬ @google-cloud/[email protected]
├─┬ @google-cloud/[email protected]
│ └─┬ @google-cloud/[email protected]
│   ├─┬ @google-cloud/[email protected]
│   ├─┬ @google-cloud/[email protected]
├─┬ @google-cloud/[email protected]
│ ├─┬ @google-cloud/[email protected]

Using DataStore via: [email protected]

Steps to reproduce

Looping through a list of data and creating a model for each one, then calling a function which
uses save

    const { body } = ctx;
    let promises = [];

    // Save new Object
    body.new.forEach((model) => {
        const createdEntity = FromModel.create(model, model.id);
        promises.push(createdEntity.upsert());
    });

    const response = await Promise.all(promises);

Trying to import about 2.5k models, we are getting lots of errors that look like they're coming from grpc maybe? Workaround is to split data into chunks, e.g. 1/4 works.

The log of errors looks like this ('...' is replacing several repeated events):

10:02:46.826Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8
10:02:46.826Z ERROR import: 13 INTERNAL: Half-closed without a request
10:02:46.827Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8
...
10:02:46.841Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8
10:02:46.841Z ERROR import: 13 INTERNAL: Half-closed without a request
10:02:46.841Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8
10:02:46.841Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8
10:02:46.841Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8
10:02:46.841Z ERROR import: 13 INTERNAL: Half-closed without a request
10:02:46.841Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8
...
10:02:46.858Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8
10:02:46.858Z ERROR import: 13 INTERNAL: Half-closed without a request
10:02:46.858Z ERROR import: 13 INTERNAL: Half-closed without a request
10:02:46.858Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8
10:02:46.858Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8
10:02:46.858Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8
10:02:46.858Z ERROR import: 13 INTERNAL: Half-closed without a request
10:02:46.859Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8
10:02:46.873Z ERROR import: 13 INTERNAL: Half-closed without a request
10:02:46.874Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8
...
10:02:46.898Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8
10:02:46.898Z ERROR import: 13 INTERNAL: Half-closed without a request
10:02:46.898Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8
10:02:46.898Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8
10:02:46.899Z ERROR import: 13 INTERNAL: Half-closed without a request
10:02:46.899Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8
10:03:43.145Z ERROR import: 4 DEADLINE_EXCEEDED: Deadline Exceeded
10:03:43.145Z ERROR import: 4 DEADLINE_EXCEEDED: Deadline Exceeded

The DEADLINE_EXCEEDED error seems to correspond with this in the emulator:

datastore] May 23, 2018 11:03:16 AM com.google.cloud.datastore.emulator.impl.LocalDatastoreFileStub$7 run
[datastore] INFO: Time to persist datastore: 198 ms
[datastore] Exception in thread "LocalDatastoreService-1" java.lang.OutOfMemoryError: unable to create new native thread
[datastore] 	at java.lang.Thread.start0(Native Method)
[datastore] 	at java.lang.Thread.start(Thread.java:714)
[datastore] 	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:950)
[datastore] 	at java.util.concurrent.ThreadPoolExecutor.processWorkerExit(ThreadPoolExecutor.java:1018)
[datastore] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1160)
[datastore] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[datastore] 	at java.lang.Thread.run(Thread.java:745)
[datastore] Exception in thread "LocalDatastoreService-4" java.lang.OutOfMemoryError: unable to create new native thread
[datastore] 	at java.lang.Thread.start0(Native Method)
[datastore] 	at java.lang.Thread.start(Thread.java:714)
[datastore] 	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:950)
[datastore] 	at java.util.concurrent.ThreadPoolExecutor.processWorkerExit(ThreadPoolExecutor.java:1018)
[datastore] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1160)
[datastore] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[datastore] 	at java.lang.Thread.run(Thread.java:745)

Copied from original issue: googleapis/google-cloud-node#2822

Metadata

Metadata

Labels

api: datastoreIssues related to the googleapis/nodejs-datastore API.type: questionRequest for information or clarification. Not an issue.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions