Private connectivity from DDVE to S3 leveraging VPC S3 Endpoints.

Where we are at ?

In Part 1 we talked about securing the relationship between the DDVE instance and the target S3 instance. This was a permissions based approach leveraging the very powerful native IAM features and key management capabilities of AWS. A very Zero-Trust approach, truth being told… always authenticate every API call, no implicit trust etc.

We have a little problem though, our IAM stuff won’t work yet, but the reason is by design. Referring back to our original diagram ( Forgive the rather crude mark up – but it serves a purpose). Before we do that, just a brief word on the format of this series. The first few will introduce concepts such as IAM, VPC endpoints, Security Groups etc. The last in the series, will tie everything together and we will run through a full deployment, leveraging CloudFormation. First things first however!

Public versus Private Subnets

The DDVE appliance is considered a back-end server. It should never be exposed directly to the internet, hence that is why it sits in a ‘Private Subnet’, as per the diagram. A private subnet is one that is internal to the VPC and has no logical route in or out of the environment. At most it can see all the other internal devices within its local VPC. The purpose of course is that this minimises the attack surface by not exposing these devices to the internet.

Of course we have the concept of a ‘Public Subnet’ also. Referring to the diagram you can see our ‘Bastion host’ (fancy name for a jump box), sitting in the public subnet. As its name implies, it is facing the public or internet. There are various ways we can achieve this leveraging IGW, NAT etc., that we won’t delve into here. Suffice to say our ‘Bastion Host’ can reach the internet and devices private to the VPC.

The above is somewhat simplistic, in that we can get much more granular in terms of reachability leveraging Security Groups and Access Control Lists (ACLs). You will see how we further lock down the attack surface of the DDVE appliance in the next post leveraging Security Groups. For now, we have enough to progress with the video example below.

So what’s the problem?

S3 is a publicly accessible, region based AWS offering. It is accessed via what is called a ‘Public Service Endpoint’. To reach this endpoint an EC2 device must have access to this ‘Public Service Endpoint’, external to its VPC. By definition Private Subnets have no way out of their VPC so S3 access will fail.

Possible Solutions

1. Move the DDVE EC2 instance to the Public VPC.
   • I’ve included this as a possible option, clearly we won’t do this. Bad idea!
2. Leverage a NAT gateway deployed in the Public Subnet.
   • This is a valid option in that the private address is ‘obscured’ by the NAT translation process. It’s IP address still remains private and not visible externally.
   • Traffic from the private subnet would be routed towards the NAT device residing in the Public subnet
   • Once in the public subnet, then it can reach the S3 Public Service Endpoint via a route through the VPC Internet Gateway (IGW).
   • It is important to note here that even though traffic destined for S3 Public Service Endpoint, traverses the Internet Gateway, it does not leave the AWS network. So there is no security implication in this regard.

Considerations around using NAT

So yes we have a solution… well kind of. You have two NAT options

1. NAT Instance: Where you manage your own EC2 instance to host the NAT software. Technically you aren’t paying anything for the NAT service from AWS, but this is going to be complicated in terms of configuration, performance and lifecycle management. Even so, dependent on the throughput you require, you may need a beefy EC2 instance. This of course will be billed.
2. AWS NAT Gateway. An AWS managed service, so complications around performance, configuration, and lifecycle management are offloaded to AWS. of course the issue now becomes cost. You will be charged for the privilege. The cost structure is based on throughput, processing and egress, so if you are shifting a lot of data, as you may well be, then the monthly cost may come as a bit of a surprise. Scalability shouldn’t be too much of a concern, a gateway can scale to 100Gbps, but who knows!

A Better Solution: Leveraging VPC Gateway Endpoints (A.K.A S3 Endpoint)

Thankfully, the requirement for private subnet connectivity to regional pan AWS services is well known use case. AWS have a solution called Gateway Endpoints, to allow internally routed access to services such as S3 and DynamoDB. Once deployed, traffic from your VPC to Amazon S3 or DynamoDB is routed to the gateway endpoint.

The process is really very straightforward and is essentially just a logical routing construct managed by AWS directly. When a Gateway endpoint is stood up, a route to the S3 service endpoint ( defined by a prefix list), via the assigned gateway, is inserted in the Private subnets routing table. We will see this in more detail via the video example. Suffice to say the construct has many other powerful security features baked in leveraging IAM etc., that we will discuss during a later post. In summary:

• Endpoints allow you to connect to AWS services such as S3 using a Private network instead of the Public network. No need for IGW’s, NAT Gateways, NAT instances etc., Plus they are free!
• Endpoint devices are logical entities that scale horizontally, are highly redundant/available and add no additional bandwidth overhead to your environment. No need to worry about Firewall throughput or packet per second processing rates.
• There are two types of endpoints. The first we have discussed here, the Endpoint Gateway. The second, the Interface Endpoint which leverages PrivateLink and ENI’s, and for which there is charge. These are architected differently but add more functionality in terms of inter region/inter VPC connectivity etc. In most, if not all cases for DDVE to S3, the Endpoint Gateway will suffice.

Video Demo

In the video demo we will follow on from the last post.

• VPC setup is already in place, including private and public subnets, internet gateways, S3 bucket and the S3 IAM policy we created in the previous post.
• We will deploy a bastion host as per the diagram, apply the IAM policy and test connectivity to our S3 bucket. All going well this will work.
• We will then deploy an EC2 instance to mirror the DDVE appliance in the private subnet, apply the same IAM policy and test connectivity to the same S3 bucket. This will of course fail as we have no connectivity to the service endpoint.
• Finally we will deploy a VPC Endpoint Gateway for S3 and retest connectivity. Fingers crossed all should work!

Next Steps

The next post in the series will examine how we lock down the surface attack area of the DDVE appliance even further using Security Groups.

DISCLAIMER
The views expressed on this site are strictly my own and do not necessarily reflect the opinions or views of Dell Technologies. Please always check official documentation to verify technical information.

#IWORK4DELL

.