How to USE?

Important

You must follow this guide using this script.

Variables

Name	Description	Type	Default	Required
ssh_file	SSH public key path	string	"../.ssh/id_ed25519.pub"	yes
ssh_file_private	SSH private key path	string	"../.ssh/id_ed25519"	yes
env_file	Environment file path	string	"../.env"	yes
credentials_file	GCP credentials file path	string	"../credentials.json"	yes
project	GCP project name	string	"optimap-438115"	yes
region	GCP region name	string	"asia-east1"	yes
zone	GCP zone name	string	"asia-east1-c"	yes
machine_type	GCP machine type	string	"n1-standard-8"	yes
gpu_type	GPU type	string	"nvidia-tesla-t4"	yes
gpu_count	GPU count	number	1	yes
username	Google email ID	string	"sangyleegcp1"	yes
dockerhub_id	Docker Hub username	string	n/a	yes
dockerhub_pwd	Docker Hub password	string	n/a	yes

Warning

The dockerhub_id and dockerhub_pwd variables have no default values and must be provided when applying the Terraform configuration.

Prerequisite Setting

1. AWS (S3, DynamoDB)

create aws account
create aws IAM account access key
install aws cli For detailed installation instructions, please refer to the official AWS CLI documentation here.

2. Google Cloud

create google account
add billing account to google cloud platform (for getting 300$ credits)
Upgrade free trial account to paid Cloud Billing account. (for using GPU resources)
Create new project
Create Service account

Caution

You must set appropriate authority. IF you was confused which authority was better, Set the owner option.

Generate Service account Json key

Important

You must save this json file. This file will be used soon.

Set the Google Cloud Compute Engine API
Increase GPU Quota

Important

IF you do not increase gpu quota, you will not get gpu resource when creating compute engine with gpu.

3. Dockerhub

create dockerhub account

4. Add ssh key to github

generate ssh key in local -> guide
add ssh key to github -> guide

5. Install terraform

install terraform cli

Getting Started

With AWS Backend Case

Important

this section save terraform.tfstate file to aws s3 for collaborating. If you want to create own server locally, scroll down and follow Without Aws backend case.

1. set aws account to aws cli

aws configure --profile [your_profile_name]
> AWS Access Key ID [None]: [your id]
> AWS Secret Access Key [None]: [your secret key]

if Your setting is correct, you will show this screen

☁  ComputeEngineGPU-Terraform [main] cat ~/.aws/credentials
[your_profile_name]
aws_access_key_id = [your id]
aws_secret_access_key = [your secret key]

2. set google cloud cli

gcloud auth activate-service-account --key-file=./credentials.json

if Your setting is correct, you will show this screen

☁  ComputeEngineGPU-Terraform [main] gcloud auth list
                  Credentialed Accounts
ACTIVE  ACCOUNT
*       q2o3481298347123-compute@developer.gserviceaccount.com

Important

your credentials.json must be root directory and must follow that name.

set project to glcoud cli

glcoud config set project ${project-name}

3. set .ssh file

In previous setting section, move root directory to create pub key and private key wrapping .ssh directory in chapter 4

☁  ComputeEngineGPU-Terraform [main] ll .ssh  
total 8.0K
-rw------- 1 sangylee sangylee 419 Oct 11 00:18 id_ed25519
-rw-r--r-- 1 sangylee sangylee 104 Oct 11 00:18 id_ed25519.pub

Important

your .ssh must be root directory and must follow that name.

4. default gcp project setting

Open ./variables.tf, change default value (project, username) to your own enviroment

Caution

project name must specify numbers after project name dash (-)

5. create terraform.prod.tfvars

create terraform.prod.tfvars in root directory and contents is following

dockerhub_id = "your docker hub id"
dockerhub_pwd = "yout docker hub pwd"

Important

terraform.prod.tfvars is secret file. so you must not upload that files. I already add terraform.prod.tfvars files in gitignore

6. change aws profile

You must change your aws profile name following downscript ./main.tf

terraform {
    required_providers {
        google = {
            source = "hashicorp/google"
            version = "4.49.0"
        }
        aws = {
            source  = "hashicorp/aws"
            version = "~> 4.0"
        }
    }
    backend s3 {
        bucket         = "sangylee-s3-bucket-tfstate" # S3 버킷 이름
        key            = "terraform.tfstate" # tfstate 저장 경로
        region         = "ap-northeast-2"
        dynamodb_table = "terraform-tfstate-lock" # dynamodb table 이름
        profile = "(your aws profile name)" <--- change
    }
}

`./provider.tf

provider "google" {
    credentials = file(var.credentials_file)
    project = var.project
    region = var.region
    zone = var.zone
}

provider "aws" {
    region = "ap-northeast-2"
    profile = "(your aws profile name)" <--- change
}

`./s3_init/provider.tf

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 4.0"
    }
  }
}

provider "aws" {
    region = "ap-northeast-2"
    profile = "(your aws profile name)" <--- change
}

7. Initialize Terraform

make init

this command init s3 and dynamoDB with terraform backend

8. Deploy server

sudo chmod +x create_server_with_dynamic_zones.sh
bash ./create_server_with_dynamic_zones.sh

Without AWS Backend Case

1. set google cloud cli

gcloud auth activate-service-account --key-file=./credentials.json

if Your setting is correct, you will show this screen

☁  ComputeEngineGPU-Terraform [main] gcloud auth list
                  Credentialed Accounts
ACTIVE  ACCOUNT
*       q2o3481298347123-compute@developer.gserviceaccount.com

Important

your credentials.json must be root directory and must follow that name.

2. set .ssh file

In previous setting section, move root directory to create pub key and private key wrapping .ssh directory in chapter 4

☁  ComputeEngineGPU-Terraform [main] ll .ssh  
total 8.0K
-rw------- 1 sangylee sangylee 419 Oct 11 00:18 id_ed25519
-rw-r--r-- 1 sangylee sangylee 104 Oct 11 00:18 id_ed25519.pub

Important

your .ssh must be root directory and must follow that name.

3. default gcp project setting

Open ./variables.tf, change default value (project, username) to your own enviroment

Caution

project name must specify numbers after project name dash (-)

set project to glcoud cli

glcoud config set project ${project-name}

4. create terraform.prod.tfvars

create terraform.prod.tfvars in root directory and contents is following

dockerhub_id = "your docker hub id"
dockerhub_pwd = "yout docker hub pwd"

Important

terraform.prod.tfvars is secret file. so you must not upload that files. I already add terraform.prod.tfvars files in gitignore

5. comment aws configuration in `src/provider.tf`, `src/main.tf`

Open src/provider.tf, take a comment about aws provider

provider "google" {
    credentials = file(var.credentials_file)
    project = var.project
    region = var.region
    zone = var.zone
}

# comment this
# provider "aws" {
#     region = "ap-northeast-2"
#     profile = "falconlee236"
# }

Open src/main.tf, take a comment about aws backend

terraform {
    required_providers {
        google = {
            source = "hashicorp/google"
            version = "4.49.0"
        }
        # take comment
        # aws = { 
        #     source  = "hashicorp/aws"
        #     version = "~> 4.0"
        # }
    }
    # take comment
    # backend s3 {
    #     bucket         = "sangylee-s3-bucket-tfstate" # S3 버킷 이름
    #     key            = "terraform.tfstate" # tfstate 저장 경로
    #     region         = "ap-northeast-2"
    #     dynamodb_table = "terraform-tfstate-lock" # dynamodb table 이름
    #     profile        = "falconlee236"
    # }
}

Important

this section save terraform.tf.state file to local computer. So that you have to manage terraform state file.

6. terraform init without aws and Deploy server

sudo chmod +x create_server_with_dynamic_zones.sh
bash ./create_server_with_dynamic_zones.sh

how to change option?

If you want to change the following configurations, please refer to the information below:

GPU Type
CPU Type in create_server_with_dynamic_zones.sh change outside of for statement in bash shell script. So you can change following option

export TF_VAR_machine_type="n1-standard-8"
export TF_VAR_gpu_type="nvidia-tesla-t4"
export TF_VAR_gpu_count=2

For more information, please visit Dynamic GPU Provisioning Script for Terraform.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

USAGE.md

USAGE.md

How to USE?

Variables

Prerequisite Setting

1. AWS (S3, DynamoDB)

2. Google Cloud

3. Dockerhub

4. Add ssh key to github

5. Install terraform

Getting Started

With AWS Backend Case

1. set aws account to aws cli

2. set google cloud cli

3. set .ssh file

4. default gcp project setting

5. create terraform.prod.tfvars

6. change aws profile

7. Initialize Terraform

8. Deploy server

Without AWS Backend Case

1. set google cloud cli

2. set .ssh file

3. default gcp project setting

4. create terraform.prod.tfvars

5. comment aws configuration in `src/provider.tf`, `src/main.tf`

6. terraform init without aws and Deploy server

how to change option?

Files

USAGE.md

Latest commit

History

USAGE.md

File metadata and controls

How to USE?

Variables

Prerequisite Setting

1. AWS (S3, DynamoDB)

2. Google Cloud

3. Dockerhub

4. Add ssh key to github

5. Install terraform

Getting Started

With AWS Backend Case

1. set aws account to aws cli

2. set google cloud cli

3. set .ssh file

4. default gcp project setting

5. create terraform.prod.tfvars

6. change aws profile

7. Initialize Terraform

8. Deploy server

Without AWS Backend Case

1. set google cloud cli

2. set .ssh file

3. default gcp project setting

4. create terraform.prod.tfvars

5. comment aws configuration in src/provider.tf, src/main.tf

6. terraform init without aws and Deploy server

how to change option?

5. comment aws configuration in `src/provider.tf`, `src/main.tf`