Earth Engine brings Vertex AI to the geospatial party
By Nicholas Clinton, Senior Developer Advocate from Google Earth Outreach
Vertex AI is the Google Cloud service for all things machine learning and artificial intelligence. Here at Earth Engine, we couldn’t wait to get connected to Vertex AI so we could fire a boatload of pixels through fancy new models. Well, the wait is over! We are stoked to announce that ee.Model.fromVertexAi() is now available and ready for you to try out. We’ve got documentation pages (overview, examples) to help you get started on your geospatial ML adventure. We’re excited to see folks leveraging the power of Vertex AI with geospatial data from Earth Engine.
One of the contexts in which we’ve been testing the new infrastructure is with sustainable sourcing through the Forest Data Partnership. For example, we trained a relatively simple neural network to map oil palm cultivation. When deployed on Vertex AI, you can interactively explore predictions in Earth Engine, analyze change and create visualizations. The inputs are composites made from publicly available data, specifically annual composites of Sentinel-1 and Sentinel-2 imagery. To get predictions anywhere the input data exists, you would host your model on Vertex AI, then connect to it using the new ee.Model.fromVertexAi() method. For example, in Javascript:
var model = ee.Model.fromVertexAi({
'endpoint': endpoint,
'inputTileSize': [1, 1],
'proj': ee.Projection('EPSG:4326').atScale(10),
'fixInputProj': true,
'outputBands': {'output': {
'type': ee.PixelType.float(),
'dimensions': 1
}
}
})Note that the inputs to this model are a 1x1xP vector, where P is the number of input bands from the Sentinel imagery (P=21 in this example). Using the Earth Engine Code Editor, it’s easy to display both inputs and outputs of the palm model:
For the palm model, we had some training data. But it’s also possible to host and use pre-trained models. For example, here’s a tree-counting model from the scientific literature (Li et al. 2023) running on public imagery in Estonia:
To deploy this model in Earth Engine, all we had to do was download the trained model, wrap the model in some base64 de/coding layers, host it on Vertex AI and then connect to it from Earth Engine. Unlike the palm model, this is a fully-convolutional network, so it accepts three dimensional inputs as a patch (256x256x3, for the three bands in RGB). Using the ee.Model.fromVertexAi() connector, you can specify tile overlap automatically, to help reduce tile boundary artifacts:
var model = ee.Model.fromVertexAi({
'endpoint': endpoint,
'inputTileSize': [128, 128],
'inputOverlapSize': [64, 64],
'proj': ee.Projection('EPSG:4326').atScale(0.2),
'fixInputProj': true,
'outputBands': {
're_serialize_output_2': {
'type': ee.PixelType.float(),
'dimensions': 1
},
're_serialize_output_2_1': {
'type': ee.PixelType.float(),
'dimensions': 1
}
}
})The entire workflow is demonstrated in this Colab notebook if you’d like to play around with the Li et al. (2023) model in your area of interest.
The shiny new Vertex AI connector comes with a few extra goodies. You get the same tiling and overlap control that was available in ee.Model.fromAiPlatformPredictor(). But with ee.Model.fromVertexAi() there’s an increased payload size, controllable with a new parameter maxPayloadBytes, that lets you really crank up the size of the patches you’re sending to Vertex AI for prediction! We’re working to make the integration with Vertex AI even more seamless and expect to have more features coming soon. For now, see the example workflow page for more information.
With Google Cloud, you can build your own end-to-end workflows to generate training data, train models, and visualize geospatially referenced model predictions. If you need pixels for training, we got ‘em! To train models, Vertex AI provides a suite of tools. And you can now get your results directly from Earth Engine.
Let us know what you think of these new features!
