Microsoft Fabric Updates Blog

Creating a shortcut to a VPC-protected Amazon S3 bucket

This guide will show you how to create a OneLake shortcut to a VPC-protected Amazon S3 bucket.

Why use the On-premises-data gateway?

Today, organizations are protecting data by leveraging network security capabilities like virtual networks, firewalls and virtual protected clouds (VPC). To access data securely and to provide a bridge between protected environments and Microsoft Fabric, an on-premises data gateway can be used.

Although the name might suggest that the on-premises data gateway can only be used to access your data that is on-premises, it can actually be used to access any data that is protected by any type of firewall or virtual network, including the virtual protected clouds on AWS. More information about the on premises data gateway is available here.

Setting up a gateway is an easy process. You need to provision an EC2 instance within your virtual private cloud; and configure (or open) appropriate ports to securely communicate with Microsoft Fabric. In this tutorial, we will walk you through the steps to complete end-to-end setup.

If you are already using the on-premises data gateway within Fabric for other items, like Pipelines, dataflows or Power BI, you can use the same instance of on-premises data gateway, as long as it also has access to your S3 bucket inside the VPC as shown in the diagram below.

At a high-level, the setup process consists of the following steps:

  1. Create a public subnet in your VPC environment and assign security groups to the S3 bucket subnet
  2. Create an EC2 instance within the public subnet
  3. Install the on-premises data gateway on the EC2 instance
  4. Open the right ports to Fabric service
  5. Create a shortcut using the on-premises data gateway

Prerequisites

Step-by-step set up

1.     Create a public subnet in your VPC environment and assign NSG’s.

  • If you don’t have a public subnet follow this guide to create an internet gateway for a subnet in your VPC.

2.     Create an EC2 instance within the public subnet

  • Launch EC2 instance in the public subnet of your VPC. Be sure to save the private key file in a secure place. You will need this in the next step.

3. Install the on-premises data gateway on the EC2 instance

4. Open the right ports to the Fabric service

  • If a firewall blocks outbound connections, configure the firewall to allow outbound connections from the gateway to its associated Azure region. The firewall rules on the gateway need to be updated to allow outbound traffic from the gateway server to the following endpoints.

5. Create a shortcut to S3

Entradas de blog relacionadas

Creating a shortcut to a VPC-protected Amazon S3 bucket

octubre 22, 2024 por Elizabeth Oldag

Shortcuts in Microsoft OneLake allow you to unify your data across domains and clouds by creating a single virtual data lake for your entire enterprise. With shortcuts, data can be reused multiple times, making it simple to consolidate data, without data movement, data duplication or changing ownership of the data. The consumption of data via … Continue reading “Use OneLake shortcuts to access data across capacities: Even when the producing capacity is paused!”

octubre 4, 2024 por Jason Himmelstein

We had an incredible time in our host city of Stockholm for FabCon Europe! 3,300 attendees joined us from our international community, and it was wonderful to meet so many of you in person. Throughout the week of FabCon Europe, our teams published a wealth of valuable content, and we want to ensure you have … Continue reading “Fabric Community Conference Europe Recap”