Appendix A. Recommended S3 Configuration

Follow this quick guide for our recommended configuration of your N2WS copy to S3 setup to make the most of AWS cost-optimization capabilities.

Contents:

Appendix A – Recommended Configuration for Copy to S3

Appendix A – Recommended Configuration for Copy to S3

When ‘worker’ instances are using public IP, NAT, or IGW within a VPC to access S3 buckets within the same region/account, it results in network transfer fees:

https://www.linkedin.com/pulse/keep-s3-traffic-private-your-vpc-aws-travis-haag/

https://medium.com/nubego/how-to-save-money-with-aws-vpc-endpoints-9bac8ae1319c

If the bucket is in another region or in another account, the transport charges will be incurred anyway.

Using VPC endpoint enables instances to use their private IP to communicate with resources in other services, such as S3, within the AWS network without incurring network transfer fees.

To create a subnet associated with a route table that will direct connections to S3 in the same region as the VPC endpoint:

In AWS, create a subnet within VPC of the region.

C:\Users\Janet\AppData\Local\Temp\image002.jpg

After successful creation, the successful creation message appears.

C:\Users\Janet\AppData\Local\Temp\image003.jpg

The subnet is automatically associated with the default route table.

Create a new route table.

C:\Users\Janet\AppData\Local\Temp\image004.jpg

Change the subnet association by associating the previously created subnet with this route table.

Create a VPC endpoint for S3 in the region and associate it with the previously created route table.

C:\Users\Janet\AppData\Local\Temp\image006.jpg

Choose a region.

C:\Users\Janet\AppData\Local\Temp\image007.jpg

Then choose the previously defined route table.

C:\Users\Janet\AppData\Local\Temp\image008.jpg

The permissions to access the bucket will be defined by the IAM polcies attached to the roles of the CPM. Grant Full Access.

C:\Users\Janet\AppData\Local\Temp\image009.jpg

The route table of the subnet now looks like the following:

C:\Users\Janet\AppData\Local\Temp\image010.jpg

If CPM is in a different account/region/VPC, add to the route table an Internet Gateway so the ‘worker’ can communicate with CPM. Add the following rule:

C:\Users\Janet\AppData\Local\Temp\image011.jpg

The route table will look like:

C:\Users\Janet\AppData\Local\Temp\image012.jpg

In this configuration, the connection to S3 will be routed to the VPC endpoint. See Note at the end of this section.

In CPM, open the Configure workers for S3 operations screen and set this subnet to be used in the specific region and the VPC where it is defined.

C:\Users\Janet\AppData\Local\Temp\image015.jpg

Note: For additional information about setting up VPC Gateway Endpoints, see https://docs.aws.amazon.com/vpc/latest/userguide/vpce-gateway.html

Share this post →