This document provides an overview of how to use Cloud HPC Toolkit to deploy an HPC cluster on Google Cloud.
Before you begin
- If you are using a Linux or macOS workstation to deploy your cluster, install dependencies.
- From either your workstation or Cloud Shell, configure your environment.
- Ensure that you have created an HPC blueprint or selected one from the Cloud HPC Toolkit examples. See Prepare an HPC blueprint.
Overview
To deploy a cluster, you need to complete the following steps:
- Create the HPC deployment folder from the HPC blueprint file using the
ghpc create
command. - Deploy the cluster from the HPC deployment folder using the
ghpc deploy
command.
Create the HPC deployment folder
To create an HPC deployment folder, use the ghpc create
command command. Replace
PATH_TO_BLUEPRINT
with the location of your
HPC blueprint file.
./ghpc create PATH_TO_BLUEPRINT
Set deployment variables at the command line
The example HPC blueprints included with the Cloud HPC Toolkit don't set the project ID. You must supply a valid project ID by using one of the following options:
Using the
--vars
flag../ghpc create --vars project_id=PROJECT_ID PATH_TO_BLUEPRINT
Using a deployment file with the
--deployment-file
(or-d
) flag:./ghpc create -d DEPLOYMENT_FILE PATH_TO_BLUEPRINT
If the
--deployment-file
and--vars
flags are used at the same time, then the variables specified with--vars
take precedence. If the--vars
flag is specified multiple times, then later invocations take precedence.
The preceding example commands assume that you are in the
main working directory
and you want to use the example hpc-slurm.yaml
blueprint that is
located in the hpc-toolkit/examples/
directory.
Get help at the command line
For a full list of flags that you can use, run the --help
flag on
ghpc
or on any of the sub-commands.
./ghpc --help ./ghpc create --help
Deploy the cluster
To deploy the cluster, run the ghpc deploy
command as shown in the output of the
./ghpc create
command. For example to deplot the hpc-slurm
blueprint,
complete the following steps:
Run the
ghpc deploy
command to begin automatic deployment of your cluster:./ghpc deploy hpc-slurm
ghpc
reports proposed changes for your cluster.To deploy the cluster, accept the proposed changes by typing
a
and pressing enter. Optionally, you may review the proposed changes by typingd
and pressing enter.Summary of proposed changes: Plan: 37 to add, 0 to change, 0 to destroy. (D)isplay full proposed changes, (A)pply proposed changes, (S)top and exit, (C)ontinue without applying Please select an option [d,a,s,c]:
After accepting the changes,
ghpc
runsterraform apply
automatically. This takes approximately 5 minutes while it displays progress. If the run is successful, the output is similar to the following:Apply complete! Resources: 37 added, 0 changed, 0 destroyed.
You are now ready to submit jobs to your HPC cluster.