Build data-driven agents with curated data from OneLake
Innovation doesn’t always happen in a straight line. From the invention of the World Wide Web, to the introduction of smartphones, technology often makes massive leaps that transform how we interact with the world almost overnight. Now we’re seeing the next great shift: the era of AI. This shift has been decades in the making, but the opportunity of AI is right now. Already, organizations are using AI agents to augment their workforce and execute business processes.
With services like Azure AI Foundry, you can not only access generative AI, but build your own agents, tailor-made for your use cases. Creating these custom AI experiences requires data—lots of it. Data is the foundation on which AI is built, and the simple fact is AI is only as good as the data it’s based on. As you enter a future built on AI, you need a data estate capable of fueling AI innovation across your organization. This can be a challenging prospect for most organizations whose data environments have grown organically over time with specialized and fragmented solutions.
That’s why we introduced Microsoft Fabric and Microsoft OneLake, Fabric’s unified data lake. With OneLake, you can access your entire multi-cloud data estate from a single data lake that spans the entire organization. OneLake can act as the central, accessible location for comprehensive data access and management. And once connected to OneLake, your teams can use the array of data and analytics tools in Fabric to integrate, transform, model, and prepare your data for any AI project—all in a pre-integrated and optimized SaaS environment.
Today we are going to focus on why Fabric and OneLake are the ideal data tools to fuel your AI projects in AI Foundry. First, we will talk through how you can unify your data estate on OneLake, then cover how Fabric’s workloads can help you prepare your data for AI projects. Finally, we’ll show you how easy it is to connect OneLake to Azure AI Foundry so you can start building data-driven agents in seconds.
Unifying your data estate on OneLake
For teams tasked with building new AI solutions, finding and accessing the necessary data across a sea of disconnected data services can be challenging at the best of times. To lay the foundation for long-term success, organizations need a more unified, flexible data estate based on a lake-centric approach. The right data lake foundation can help you unify all of your multi-cloud sources and allow your data professionals to work from the same data—reducing data duplication, improving collaboration, and streamlining analysis.
OneLake is designed as the single point to discover and explore data for everyone in your entire organization. You can unify all of your multi-cloud and on-premise sources using zero ETL shortcuts and mirroring in OneLake without data duplication or movement. Alternatively, you can leverage the 180+ connectors in Fabric Data Factory to move your data in from any other source. OneLake is automatically wired into every Fabric workload and since data is stored in an open format, you can use data in OneLake for all your data projects, no matter the vendor or service. You can also save time and reduce data duplication by loading data into OneLake only once and using a single copy across every Fabric engine and even other engines like from Snowflake.
Once enabled in OneLake, you can use domains and the OneLake catalog to organize your data into a logical data mesh and empower everyone to easily explore, manage, and govern their data. Take a look at the OneLake catalog:
Preparing and curating your data for AI projects
For generative AI solutions to be as accurate as possible, they need to be built with clean data and in a semi-structured way. You’ll need an analytics platform that can help you prepare your data before building custom AI experiences. With your data in OneLake, you can use Fabric’s various workloads to make the data AI-ready. Fabric has tools for data integration and engineering, data warehousing, data science, real-time analytics, data modeling and visualization, and even has native, industry-specific and partner-created workloads to help you accelerate your data projects.
All Fabric workloads work together seamlessly out-of-the-box without the myriad of infrastructure and configuration settings you typically find in data platforms, so you can focus on getting results. Advanced security, governance, and continuous integration and continuous delivery (CI/CD) capabilities are woven into the platform with personalized experiences for admins and users alike. Copilot in Fabric and other AI capabilities are built into every layer of Fabric to help data professionals and business users automate routine tasks and get more done. Fabric also comes with category-leading performance, instant scalability, shared resilience, and built-in security, governance, and compliance so you can feel confident using Fabric for your mission-critical workloads.
Connecting OneLake data to AI products
Now that your data is AI-ready, you need to connect it to your AI platforms like Azure AI Foundry to build and scale data-driven GenAI apps. We’ve built native integration between OneLake and Azure AI Foundry to make this as seamless as possible. Azure AI Foundry can operate directly on OneLake, opening endless possibilities for AI and app developers, data engineers, data scientists, and business users to interact using natural language to uncover insights from their data.
Azure AI Foundry
Azure AI Foundry is a platform designed to empower your developers, AI engineers, and IT professionals to customize, host, run, and manage AI solutions with greater ease and confidence. Similar to Fabric, Azure AI Foundry’s unified approach simplifies the development and management process, helping all stakeholders focus on driving innovation and achieving strategic goals. It’s designed to help your developers build more technical, customized AI solutions.
The integration between Azure AI Foundry and OneLake is built on the same shortcut technology that allows you to virtualize data in OneLake from your cloud sources like Amazon S3 and Google Cloud without having to move and duplicate the data. You can immediately work with your structured and unstructured data from OneLake in Azure AI Foundry without creating copies and adding more data sprawl. OneLake also directly integrates with Azure AI Search, which can store, index, and retrieve data, including vector embeddings, from your data sources including OneLake.
Finally, you can ground your Azure AI Agent’s responses with data from Fabric using Fabric data agents to unlock powerful data analysis capabilities. Data agents (formally known as AI skills) in Fabric are AI-powered assistants that can learn, adapt, and deliver insights, allowing users to interact with the data through chat. With out-of-the-box authorization, this integration simplifies access to enterprise data in Fabric while maintaining robust security, ensuring proper access control and enterprise-grade protection. Check out this full demo:
Popular applications for data-driven agents
This seamless integration offers many opportunities for generative AI use cases across various industries, including:
- Enhancing data insights: Build agents that can help your business users explore and better understand critical data using natural language from structured, unstructured, and real-time data.
- Analyzing customer interactions: Build agents trained on your customer interaction data to enhance customer service, tailor support responses, and make data-driven decisions. These agents can detect language, summarize content, analyze sentiment, and convert insights into vector embeddings for future access in search queries.
- Customizing machine learning models: Tailor models to specific business needs, whether it’s predictive maintenance, fraud detection, or customer sentiment analysis. Azure AI Foundry, Azure Machine Learning, and Microsoft Fabric empower developers and data scientists to create custom models that fit their business requirements, grounded on their enterprise data in OneLake.
- Department-specific agents: Build agents that automate budget and expenses, increase up-sell and conversion opportunities, and improve operational efficiency
- Industry-specific agents: Build data-driven agents to streamline operations and manage OEE in manufacturing, optimize logistics and interact with customers in retail, and reduce patient-practitioner contact time in healthcare.
Ready to learn more?
Unlock a realm of new possibilities for your organization in the era of AI with the integration of Microsoft Fabric and Azure AI. Explore the potential, innovate, and thrive in the new digital landscape.
If you want to learn more about these tools, consider:
- Signing up for the Microsoft Fabric free trial.
- Visiting the Microsoft Fabric website.
- Visiting the Azure AI Foundry website
