The Virtual Private Cloud (VPC) concept from Amazon Web Services creates an isolated boundary, defined by network rout-ability.  Said another way, computers (EC2 instances, for example) can be placed inside a VPC and, by default, network access is limited to other instances inside that VPC.  The idea is similar to most private networks (like a home network or corporate network).   The IP range chosen is usually one of the private IP ranges, see here: https://en.wikipedia.org/wiki/Private_network).

Of course, computers generally want to communicate with each other, and so the breaking of isolation begins with controlled access to other networks (such as Internet, for example).  As usual, XKCD captures this cycle : https://xkcd.com/2044/

In any event, I recently spent some time setting VPN-based connectivity between VPC’s.  Prior to any of the “VPC Peering” options, this would have been really the only mechanism possible.  I used OpenSwan for the VPC Initiator and AWS VPN for the receiver.  I’m not really sure why AWS chose to only implement the receiver side of things; it would have been nice to have had a managed initiator as well.    I got things working smoothly with no huge challenges, other than some user-inflicted typos.

I’d like to now consider the differences between peering and using VPN to connect AWS VPC’s

Internet Access.  With peering, no Internet access is necessary.  With a VPN setup, at least the initiator-side must have Internet access.  Note that the when using AWS VPN, the receiving side, doesn’t need Internet access to the VPC.

Availability. With peering, there are multiple levels of redundancy, and specifically not reliance on a single EC2 instance.  In the VPN case, for the initiator the routing rules must go to a single ENI/instance.  While there are ways to setup two instances and have a third monitoring instance update route tables when it detects a failure, this can be brittle and require minutes of convergence time.  On the managed VPN side, there are two instances (one per tunnel) which helps, though the version of OpenSwan that I used didn’t seem to support two tunnels.

Cost.  The cost of t3.small for a month is ~$14 and the cost of managed VPN is ~$36, so that is ~$50/month, plus bandwidth charges on both network egress from the imitator side and ingress on the reciever side.

ipv6.  ipv6 just works across peering (along side ipv4).  For a VPN-based solution, it may work, but doesn’t out of the box.  AWS VPN doesn’t work with ipv6 at the time of this writing.

DNS resolution.  EC2 provides a default DNS name for instances (a la ip-10-48-19-13.us-east-2.compute.internal).  This works across peering, but not VPN-based solutions.

Jumbo Frames. Jumbo frames enables the Ethernet frame to carry 9001 bytes, instead of the default 1500 for the internet.  VPN-based solutions will be limited to 1500 and also, potentially require a lot of processing to reassemble jumbo frames, which is default in the EC2 networking.  With peering, Jumbo Frames just work.

Route Propagation.  Here is a case where VPN-based solution has one advantage.  There is a feature of the Virtual Gateway (VGW) that propagates routes automatically to associated route tables.  So, if on the initiator side of things new routes are added, and advertised, the AWS route tables are automated updated.  Not so with peering.  If you add a new CIDR range to a VPC, you need to update all the route tables in the peered VPC’s to reflect the new CIDR range.  I will point out though in the VPN-case, you need to manually update any new CIDR’s ranges manually on the initiator side.

Scale and Limits.  Per the docs (https://docs.aws.amazon.com/vpc/latest/userguide/amazon-vpc-limits.html) , there can be a maximum of 10 VPN connections per VGW (and there is only one VGW per VPC).  With Peering, there is currently a maximum of 125 active peering connections per VPC, a 10x increase.  That said, to get around these limits, you can setup a “Transit VPC” configuration, which doesn’t require a mesh-configuration that Peering encourages.

Security Groups. Both currently work the same — you must manually update your security groups with the CIDR in question. Neither scenario supports security group references to the other VPC.

Security. Peering comes with a couple of distinct advantages.  One the traffic is over the Amazon backbone network, note the Internet.  This provides increased availability, bandwidth and security.  Also, because the traffic is over the Amazon backbone, the customer’s traffic (i.e. yours) is mixed in with other traffic and thus less susceptible to attacks specific to your traffic.

To learn more about CloudFront, I recently moved an S3-based web-site (one containing an experimental picture application) to be distributed via CloudFront.  The goal was to enable faster downloads, based on the end-user’s geography.

Setting up the CF distribution was pretty straightforward.  However, the app wasn’t working correctly: when entering the URL with just the host name, the default object (i.e. usually index.html) wasn’t loading.  It turns that this needs to be configured explicitly in the CF UI.  Here is the field in the “General” properties:

All good: now the web-site would load when going to the base URL.  However, subsequent directories in the URL path wouldn’t load this document.

When served directly from S3, this setting:

was sufficient to have the index file be used for each directory. Specifically

curl http://host.com/

curl http:/host.com/05-02-2012/

would both return the contents of s3-list.html.  However, when the DNS name for the CloudFront endpoint was used, s3-list.html was only returned for the first URL.

I researched a bit; I couldn’t determine if the S3 model is too permissive, or if the CF model was limited.  I ended up changing my app a bit to accommodate the CF model.  Namely, I changed internally created URL to reference the specific page, rather than assuming the server would return the default.