GCP Batch noExternalIPAddres =True

[gradientopt]

Hi Team,

This post https://www.googlecloudcommunity.com/gc/Infrastructure-Compute-Storage/GCP-Batch-In-use-IP-addresses... mentioned that one could use GCP Batch service without attaching an IP address to each machine when running task by setting noExternalIPAddres =True. However, the additional network configuration seems complicated and I am wondering what would be the cost and speed/job scheduling efficiency comparison of  noExternalIPAddres =False (default, attach IP address) vs   noExternalIPAddres =True (do not attach ip address)?  We may run 40 thousand tasks in parallel and just want to make sure that would not cause things to be any extra expensive (cost of Batch API using ip address) or slow (hard to allocate so many ip addresses). Thanks!

[bolianyin · Accepted solution]

To clarify

>> the additional network configuration seems complicated

Do you mean setting noExternalIPAddres =True or something else?

Setting noExternalIPAddres =True should only provides benefits (easier to allocate VMs, etc) unless you do need external IPs. 

[gradientopt]

Thanks for the reply! 

1. Yes I meant noExternalIPAddres =True. We can not just set this parameter and expect it to work right? We also need to do steps here https://cloud.google.com/batch/docs/job-without-external-access

How much benefit in run time and cost could this provide? We are debating whether or not we should set this given the network configuration is quite complicated…

2. Is external ip required to make outgoing http download request?

[gradientopt]

Also, our Batch task uses docker that mounts gcs storage, if we disable external ip address, can we still access gcs? Thanks!

[wenyhu]

Hi @gradientopt,

Yes, in general, the external ip is required to make outgoing http download request.

However, if you only need to call Google APIs, you can consider setup internal network with Private Google Access. Since GCS is a Google service, you can still access to GCS if you setup network with private google access properly.

The other option to use internal only ip with outbound connections is to setup Cloud NAT with non-external ip.

You can find more detail in https://cloud.google.com/batch/docs/job-without-external-access.

Hope this helps!

- Wenyan

[gradientopt]

Thanks for the reply!  Sorry for keep asking these but I am a newbie to computer networking :-(. My confusion is the following. What else do I need to do if I set "noExternalIpAddress": true? The tutorial you pointed out says that I just need to specify the network and subnetwork. Can I just use the default vpc network for my project (I did not do any modification on default vpc after GCP initializes for me)? Or do I have to set up another new vpc network that has Private Google Access?

  "allocationPolicy": {
    "network": {
      "networkInterfaces": [
        {
          "network": "projects/HOST_PROJECT_ID/global/networks/NETWORK",
          "subnetwork": "projects/HOST_PROJECT_ID/regions/REGION/subnetworks/SUBNET",
          "noExternalIpAddress": true
        }
      ]
    }
  },

[gradientopt]

It seems that the only additional thing I need to do is to toggle Private Google Access to be on for my default vpc? The default is set to be off so I am wondering will turning it on affect my other more permanent VMs (such as my static dev server that I ssh to ) on this VPC that actually has an external ip address?

[wenyhu]

Hi @gradientopt, the default network and subnets are auto-generated when you enable a project with services. Although it should work, I would recommend you to create a new VPC for your `no external ip` cases instead of modifying the default VPC. Thanks!