Following the last article we are able to use gitlab runner to create AWS resources (VPC, subnets, rt tables, NGWs, IGWs, etc), in this article I will show:
- How to use AWS role instead of AWS_SECRET_ACCESS_KEY and AWS_ACCESS_KEY_ID to get authorisation to AWS resources creation, listing and delete, etc
- How to use AWS S3 bucket for gitlab-runner to sync tfstate file
How to use AWS role to get authorisation for AWS resource creation, listing and deletion.
In last article I have the following setup in .gitlab-ci.yml:
.base-terraform: image: name: "hashicorp/terraform:1.5.5" entrypoint: - '/usr/bin/env' - 'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin' - 'AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}' - 'AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}' - 'AWS_DEFAULT_REGION=${AWS_DEFAULT_REGION}'AWS_SECRET_ACCESS_KEY is attached to the AWS IAM user in AWS account (AWS_ACCESS_KEY_ID) .Where AWS_SECRET_ACCESS_KEY ,AWS_ACCESS_KEY_ID and AWS_DEFAULT_REGION is used to decide AWS account and region that this code will be implemented on. This setup requires us to specifically configure AWS_SECRET_ACCESS_KEY as parameter that can be reached by the the coder reader.
Now I want to modify the way of authentication and authorisation by using role instead. This requires the gitlab-runner that is supposed to implement change on AWS account owns correct role. so I modify the code as below:
in .gitlab-ci.yml, remove the following lines:
'AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}' - 'AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}' - 'AWS_DEFAULT_REGION=${AWS_DEFAULT_REGION}'In provider.tf, add the following script:
provider "aws" {
allowed_account_ids = ["awsaccountid"]
region = "region"
assume_role {
session_name = "terraform-aws-session"
role_arn = "arn:aws:iam::awsaccountid:role/assume-role"
}
}
Then I need define the “assume-role” in aws, I grant this role “administrator access” in permission policy. “administrator access”means that any entity with this role can do anything like administrator for this specifically AWS account. This is wide authorisation, and need to be narrow down to the resources that you want to grant your gitlab runner can assume.
I dont want to attach this role with such wider authorisation into the gitlab runner instance directly. In stead another role can be created and attached to gitlab-runner instance that was created in the last article. Let’s name this role as “gitlabrunnerole”, the role is defined as below:
Trusted Entity:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "ec2.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
Permissions:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:bucket",
"s3:bucket/*"
],
"Resource": "*"
}
]
}I granted this role for any actions in S3 bucket, this is for gitlab-runner able to visit, modify, push or delete tfstate file that I am going to save in S3 bucket. I will get that this part later.
Now I have 2 roles created “gitlabrunnerrole” and “assumerole”, I will allow “gitlabrunnerrole” to assume “assumerole” by adding the following trusted entity in to role “assumerole”:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Statement1",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::awsaccountid:role/gitlabrunnerrole"
},
"Action": "sts:AssumeRole"
}
]
}Now the last is to attch “gitlabrunnerrole” into the ec2 instance “gitlabrunner”
How to use AWS S3 bucket for gitlab-runner to sync tfstate file
Before go to the code we need understand how terraform save its tfstate file. here is the article about that.
A backend defines where Terraform stores its state data files.
Terraform uses persisted state data to keep track of the resources it manages. Most non-trivial Terraform configurations either integrate with Terraform Cloud or use a backend to store state remotely. This lets multiple people access the state data and work together on that collection of infrastructure resources.
To use s3 bucket for tfstate file saving, I have the following code:
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 4.18.0"
}
}
backend "s3" {
bucket = "terraformtf"
key = "path/"
region = "region"
}
}Before this I need S3 bucket with name “terraformtf”, and I will grant role “gitlabrunnerrole” to be able to do necessary actions to this bucket:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Statement1",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::awsaccountid:role/gitlabrunnerrole"
},
"Action": [
"s3:GetObject",
"s3:PutObject",
"s3:DeleteObject",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::pcnitf/*",
"arn:aws:s3:::pcnitf"
]
}It shows in aws management console as below:
With all above change I have removed AWS_SECRET_ACCESS_KEYAWS_ACCESS_KEY_ID from my code, and I have a tf.state file save in the S3 bucket.
Yings notebook 