An Introduction to the AWS Command Line Tool

Amazon Web Services has an extremely functional and easy to use web console called the AWS Management Console. It’s brilliant for performing complex tasks on your AWS infrastructure, although as a Linux sysadmin, you may want something more "console" friendly.

In early September 2013, Amazon released version 1.0 of awscli, a powerful command line interface which can be used to manage AWS services.

Amazon's awscli is a powerful command line interface that can be used to manage AWS services.

In this two-part series, I’ll provide some working examples of how to use awscli to provision a few AWS services. (See An Introduction to the AWS Command Line Tool Part 2.) We’ll be working with services that fall under the AWS Free Usage Tier. Please ensure you understand AWS pricing before proceeding.
For those unfamiliar with AWS and wanting to know a bit more, Amazon has excellent documentation on introductory topics.
Ensure you have a relatively current version of Python and an AWS account to be able to use awscli.

Installation & Configuration

Install awscli using pip. If you’d like to have awscli installed in an isolated Python environment, first check out virtualenv.

$ pip install awscli

Next, configure awscli to create the required ~/.aws/config file.

$ aws configure

It’s up to you which region you’d like to use, although keep in mind that generally the closer the region to your internet connection the less latency you will experience.
The regions are:

• ap-northeast-1
• ap-southeast-1
• ap-southeast-2
• eu-west-1
• sa-east-1
• us-east-1
• us-west-1
• us-west-2

For now, choose table as the Default output format. table provides pretty output which is very easy to read and understand, especially if you’re just getting started with AWS.
The json format is best suited to handling the output of awscli programmatically with tools like jq. The text format works well with traditional Unix tools such as grep, sed and awk.
If you’re behind a proxy, awscli understands the HTTP_PROXY and HTTPS_PROXY environment variables.

First Steps

So moving on, let’s perform our first connection to AWS.

$ aws ec2 describe-regions

A table should be produced showing the Endpoint and RegionName fields of the AWS regions that support Ec2.

$ aws ec2 describe-availability-zones

The output from describe-availability-zones should be that of the AWS Availability Zones for our configured region.
awscli understands that we may not just want to stick to a single region.

$ aws ec2 describe-availability-zones --region us-west-2

By passing the —region argument, we change the region that awscli queries from the default we have configured with the aws configure command.

Provisioning an Ec2 Instance

Let’s go ahead and start building our first Ec2 server using awscli.
Ec2 servers allow the administrator to import a SSH key. As there is no physical console that we can attach to for Ec2, SSH is the only default option we have for accessing a server.
The public SSH key is stored within AWS. You are free to allow AWS to generate the public and private keys or generate the keys yourself.
We’ll proceed by generating the keys ourselves.

$ ssh-keygen -t rsa -f ~/.ssh/ec2 -b 4096

After supplying a complex passphrase, we’re ready to upload our new SSH public key into AWS.

$ aws ec2 import-key-pair --key-name my-ec2-key \
    --public-key-material "$(cat ~/.ssh/ec2.pub)"

The —public-key-material option takes the actual public key, not the path to the public key.
Let’s create a new Security Group and open up port 22/tcp to our workstation's external IP address. Security Groups act as firewalls that we can configure to control inbound and outbound traffic to our Ec2 instance.
I generally rely on ifconfig.me to quickly provide me with my external IP address.

$ curl ifconfig.me
198.51.100.100

Now we know the external IP address of our workstation, we can go ahead and create the Security Group with the appropriate inbound rule.

$ aws ec2 create-security-group \
    --group-name MySecurityGroupSSHOnly \
    --description "Inbound SSH only from my IP address"
$ aws ec2 authorize-security-group-ingress \
    --group-name MySecurityGroupSSHOnly \
    --cidr 198.51.100.100/32 \
    --protocol tcp --port 22

We need to know the Amazon Machine Image (AMI) ID for the Linux Ec2 machine we are going to provision. If you already have an image-id then you can skip the next command.
AMI IDs for images differ between regions. We can use describe-images to determine the AMI ID for Amazon Linux AMI 2013.09.2 which was released on 2013-12-12.
The name for this AMI is amzn-ami-pv-2013.09.2.x86_64-ebs with the owner being amazon.

$ aws ec2 describe-images --owners amazon \
    --filters Name=name,Values=amzn-ami-pv-2013.09.2.x86_64-ebs

We’ve combined —owners and applied the name filter which produces some important details on the AMI.
What we’re interested in finding is the value for ImageId. If you are connected to the ap-southeast-2 region, that value is ami-5ba83761.

$ aws ec2 run-instances --image-id ami-5ba83761 \
    --key-name my-ec2-key --instance-type t1.micro \
    --security-groups MySecurityGroupSSHOnly

run-instances creates 1 or more Ec2 instances and should output a lot of data.

• InstanceId: This is the Ec2 instance id which we will use to reference this newly provisioned machine with all future awscli commands.
• InstanceType: The type of the instance represents the set combination of CPU, memory, storage and networking capacity that this Ec2 instance has. t1.micro is the smallest instance type available and for new AWS customers is within the AWS Free Usage Tier.
• PublicDnsName: The DNS record that is automatically created by AWS when we provisioned a new server. This DNS record resolves to the external IP address which is found under PublicIpAddress.
• GroupId under SecurityGroups: the AWS Security Group that the Ec2 instance is associated with.

If run-instances is successful, we should now have an Ec2 instance booting.

$ aws ec2 describe-instances

Within a few seconds, the Ec2 instance will be provisioned and you should be able to SSH as the user ec2-user. From the output of describe-instances, the value of PublicDnsName is the external hostname for the Ec2 instance which we can use for SSH. Once your SSH connection has been established, you can use sudo to become root.

$ ssh -i ~/.ssh/ec2 -l ec2-user \
    ec2-203-0-113-100.ap-southeast-2.compute.amazonaws.com

A useful awscli feature is get-console-output which allows us to view the Linux console of an instance shortly after the instance boots. You will have to pipe the output of get-console-output into sed to correct line feeds and carriage returns.

$ aws ec2 get-console-output --instance-id i-0d9c2b31 \
    | sed 's/\\n/\n/g' | sed 's/\\r/\r/g'