Secure data streaming: Custom CA and mTLS in Fabric Eventstream connectors (Preview)
Security is non-negotiable when it comes to real-time data streaming. In regulated industries such as banking, healthcare, and telecommunications, organizations must ensure that every data connection is encrypted and mutually authenticated. But many enterprises rely on internal Certificate Authorities (CAs) or require mutual TLS (mTLS) to meet their security policies, which means that the standard, publicly trusted certificates that come pre-installed with most platforms simply aren’t enough.
Microsoft Fabric Eventstream, part of Real-Time Intelligence (RTI), provides nearly 20 streaming connectors that ingest real-time data from popular sources such as Apache Kafka, Amazon Managed Streaming for Apache Kafka, and Confluent Cloud for Apache Kafka. Until now, these connectors only supported TLS encryption using system-predefined CA certificates from a trusted CA list. If your source systems used certificates signed by a custom CA—or if your infrastructure required mTLS for two-way authentication—you couldn’t use Eventstream to connect.
We heard this loud and clear from customers across industries, and today we’re announcing Custom CA and mTLS support for Kafka-based sources in Eventstream, now available in preview for Kafka-based sources, including Apache Kafka, Amazon Managed Streaming for Apache Kafka, and Confluent Cloud for Apache Kafka, as well as Confluent Schema Registry.
Bring your own certificates with Azure Key Vault
With this new feature, Eventstream integrates with Azure Key Vault to let you bring your own certificates. Import your custom CA certificate and—if mTLS is required—your client certificate and private key into your Azure Key Vault. When you configure a Kafka-based source in Eventstream, simply reference these certificates from your Key Vault. Eventstream’s connector service then fetches the certificates at runtime and uses them to establish an encrypted and mutually authenticated connection with your source.
This approach offers significant advantages over traditional file-based certificate management. Azure Key Vault serves as a centralized, secure store for all your certificates. Multiple data engineers across your organization can reference the same certificates without needing individual file handoffs. When it’s time to rotate or update a certificate, you update it once in Key Vault—Eventstream connectors that reference it will automatically pick up the new version.
Figure 1 Architecture custom CA and mTLS in Eventstream.
Essentials and getting started
To ensure secure streaming connections using custom certificates or mTLS, the following information provides the necessary steps to begin. Prior to configuring a custom Certificate Authority (CA) or mutual TLS (mTLS) in Eventstream, please confirm that the following prerequisites are in place:
- Source system readiness: Confirm your source server is configured to accept connections with your custom CA certificates and, if using mTLS, is set up to validate client certificates.
- Azure Key Vault with certificates: Import your required certificates (CA certificate for TLS, or both CA and client certificates for mTLS) into an Azure Key Vault in .pem format. Ensure you have the appropriate role assigned, such as ‘Key Vault Administrator’ or ‘Key Vault Certificate User.’
- Private network setup (if applicable): If your source resides in a private network, you’ll need to use Eventstream’s vNet injection solution. This requires an Azure virtual network connected to your source’s private network. Additionally, create a private endpoint from your Azure Key Vault to the virtual network so that the vNet-injected connector can securely access your certificates.
After setting up prerequisites, you can easily configure your streaming connector to use certificates from your Azure Key Vault in the wizard. The following animation demonstrates the complete end-to-end flow in Eventstream.
Figure: Custom CA and mTLS configuration end to end flow.
For detailed guide, please refer to the specific eventstream source configuration documentations in Add and Manage Eventstream Sources, such as Apache Kafka source, Confluent Cloud for Apache Kafka, and Amazon Managed Streaming for Apache Kafka.
Next steps and resources
Try this feature with your Fabric account, if you don’t have one, sign up for Power BI with a new Microsoft 365 trial and start a free Fabric trial capacity. Learn more about Eventstream.
We welcome your feedback through the community forum, idea submission, or via email.
