AWS – Retrieve Instance Metadata

Because your instance metadata is available from your running instance, you do not need to use the Amazon EC2 console or the AWS CLI. This can be helpful when you’re writing scripts to run from your instance. For example, you can access the local IP address of your instance from instance metadata to manage a connection to an external application.

Instance metadata is divided into categories. For a description of each instance metadata category, see Instance metadata categories.

To view all categories of instance metadata from within a running instance, use the following URI.

http://169.254.169.254/latest/meta-data/

The IP address 169.254.169.254 is a link-local address and is valid only from the instance.

Note that you are not billed for HTTP requests used to retrieve instance metadata and user data.

Upgrading Aurora RDS Using Terraform

Source

Background

You created Aurora RDS databases using Terraform and need to do a major version upgrade. This blog post gives you one way to do it. These examples follow the Aurora PostgreSQL upgrade process from .9.x to .10.x. The steps would be similar when upgrading between other Aurora RDS versions.

Getting Started

Below is a code snippet from Terraform showing what our starting state is.

#---main.tf
provider "aws" {
  region  = var.aws_region
  version = "~> 2.0"
}## To create a lower version database cluster
resource "aws_rds_cluster_instance" "exampleclusterinstances" {
  count = 1
  identifier = "example-cluster-demo-${count.index}"
  cluster_identifier = aws_rds_cluster.default.id
  instance_class = "db.r4.large"
  engine = aws_rds_cluster.default.engine
  engine_version = aws_rds_cluster.default.engine_version
  publicly_accessible = true
  apply_immediately = true
}resource "aws_rds_cluster" "default" {
  cluster_identifier = "example-cluster-demo"
  availability_zones = ["us-east-1a", "us-east-1b", "us-east-1c"]
  database_name = "atestdb"
  engine = "aurora-postgresql"
  engine_version = "9.6.16"
  master_username = "zaphod"
  master_password = "breeblebox42"
  backup_retention_period = 1
  preferred_backup_window = "07:00-09:00"
  skip_final_snapshot = true
}

Upgrade Time

That 9.x database has treated us well, however, it is time to upgrade to a newer version. But, what is the best process? If I update my Terraform code from engine_version = "9.6.16" to engine_version = "10.11" my database will NOT be upgraded. Instead, the 9.6 version will be deleted and a new database with the same name will be created. This is not what we want!

Within Terraform, however, I cannot do a major version upgrade of the Aurora database engine.

Looking at the upgrade guide that AWS provides gives us a process that we can easily follow. Using either the awsCLI or the AWS Management Console, navigate to your database and start the upgrade process.

For this post, I am going to use the AWS Management Console.

• Navigate to your RDS instance in the AWS Console and select “Modify”.
• Select the DB version you want to upgrade to (in this instance 10.11).

Database Versions

• Select “continue”.
• Select when to modify (if testing, select “Apply immediately”).

Scheduling the Modification

• Modify Cluster & wait.

Depending on the size of your database this process may take a while. Be sure to test so you use a large enough outage window in production.

Getting Terraform in Sync

At this point, your Terraform configuration shows a database engine version of 9.6.16, but the AWS Console shows your database version as 10.11.

Let’s fix that!

From the location that you typically run terraform do a terraform show:

...
engine                       = "aurora-postgresql"
    engine_version               = "9.6.16"
    id                           = "example-cluster-demo-0"
    identifier                   = "example-cluster-demo-0"
...

If you ran a terraform plan, you would see something like this:

~ engine_version = “10.11” -> “9.6.16” # forces replacement`

However, if you run terraform refresh great things happen!

aws_rds_cluster.default: Refreshing state... [id=example-cluster-demo]
aws_rds_cluster_instance.exampleclusterinstances[0]: Refreshing state... [id=example-cluster-demo-0]

If you then runterraform show, you’ll see this:

...
   engine_version               = "10.11"
    id                           = "example-cluster-demo-0"
    identifier                   = "example-cluster-demo-0"
    instance_class               = "db.r4.large"
...

Almost Home!

The only step left is to update the Terraform configuration file (in my example, main.tf) to reflect that the updated database version. Specifically, change engine_versionin the awd_rds_cluster resource to 10.11.

Now, if you run terraform plan your output should be: “No changes. Infrastructure is up-to-date.”

Benefit

If you had simply attempted to update your main.tf file with a new database version, Terraform would have deleted the existing database and created a new database with the newer version. You would have needed to recover your data from a backup (you do take those, right?) in order to restore functionality.

By using the approach described here, you are able to minimize impact on your business and maintain correct versioning with Terraform.