Microsoft Fabric: A new paradigm in how we work with data
Continuous innovation (frenetic, but controlled, I dare say) has been a hallmark of Microsoft in recent years. After the explosion of AI at the end of the first quarter of this year and its integration into Bing, Azure Open AI, and all the co-pilots that are still to come, the Redmond company is taking another turn in Data management, covering subjects it had “pending” in its range of services with the presentation of Microsoft Fabric.
“All the data. Every computer. All in one place”, this is the motto with which Microsoft sums up Fabric, a suite that allows us to:
- Unify our data structure with data lakes as its single center.
- Manage AI models on this single structure.
- Train and help everyone in our company in the decision-making process based on the conclusions provided by Fabric after analyzing the data.
- And, of course, all with integrated security, compliance, and data governance.
Fabric allows us to take advantage of these 4 points by offering us this architecture:
A single source, a better partnership
We can see that the fundamental foundation, the center of the entire Fabric ecosystem is OneLake: a single, unified data lake for the entire organization. Similar to OneDrive and Microsoft tenants, each Microsoft Fabric tenant has its OneLake which should be the single place to store all analytics and data.
This unification of data greatly improves collaboration between all data professionals in our organization. It links together all the tools tailored to each of our data experts’ roles, experiences, and technologies. As a result, it enables people to work with them instantly:
In this way Microsoft Fabric puts at our disposal:
- Synapse
- Power BI
- Data Factory
- Data Advisor (currently in preliminary version).
Throughout the article we will go deeper into each of the above points.
AI-Based Capabilities
In Microsoft Fabric we have all the AI features available, both in Azure OpenAI and Copilot. Using natural language for operations from data integration to reporting. This way we can focus on getting value out of our data. Copilot plays a crucial role: Helping with DataFlows, and Data Pipelines and guiding in the creation of ML models.
In this video we can see a few glimpses of these features:
And above all, most importantly, the most important premise and mantra is that it is a trusted environment:
- Our data is ours and ours alone.
- Our data is not used to train, retrain, or improve Microsoft or third-party models.
- Data is protected by the most comprehensive security and compliance controls.
- Commitment to store and process data within the EU (EU Data Boundary)
Microsoft Synapse
Another of the elements that make up Microsoft Fabric is Synapse which, in turn, is divided into 4 important aspects:
Data engineering: Large-scale data transformation:
- Create and manage data using a lake shore.
- Design pipelines to copy data to the lake store.
- Use Spark job definitions to submit a batch or streaming job to a Spark cluster-
- Use notebooks to write code for data ingestion, preparation, and transformation.
- Data warehousing: Delivering maximum SQL performance with the ability to scale compute and storage independently.
- Data Science: Creating predictive models without losing the power of collaboration in training, deploying, and managing machine learning models. Accompanying us throughout the process from problem formulation to result.
- Real-time analytics: That is, having the conclusions of our data in real time from an analysis of large volumes of data from applications, websites and IoT. This point has its peak of usefulness in that it answers questions such as:
- Do I need a high update of the data ingest to perform queries?
- Do I want to transform streaming data?
- Do I have a service that needs to access data with low query latency (within seconds)?
- Do I need to search or access data in different formats, such as structured data, semi-structured data (including complicated data such as JSON or other arrays), or unstructured data (e.g. free text)?
- Do I want the ability to query large amounts of data?
- Does my data have a time component that can benefit from the optimized database structure for time series?
- Do I want the ability to create ad hoc queries on any field or row without prior optimization?
Microsoft Power BI
Of all the Fabric universe, perhaps the best-known actor. In this new iteration of Power BI we will be able to have:
- Data Hub: the creation of a secure and reliable data repository that provides the opportunity for the entire organization to find and analyze the right data.
- Creation of reports that allow us to draw key conclusions while visually appealing through a very easy-to-use report designer:
- Copilot for report creation and analysis
- Integration with Microsoft 365
Microsoft Data Factory
We could summarise Microsoft Data Factory as Fabric’s ETL tool.
Its functionalities range from data integration by collecting data from different sources (both local and On the Cloud), with more than 100 connectors available; through the autonomous ETL and without code of Data Flows and Data Pipelines to the management of the pipeline in the consolidation of hybrid data to be monitored and administered.
Data Activator
Data Activator is a no-code product in Microsoft Fabric that monitors data and triggers actions when business conditions are met.
For example, you can send email alerts when sales revenue data in your Power BI report falls below a threshold. Similarly, you can send a notification in Microsoft Teams when the temperature of a perishable package exceeds a threshold.
So we can see what Data Activator offers us:
- Real-time data monitoring
- Creation of configurable alerts
- Triggering human and automated actions in response to these alerts
All without the need for code.
Microsoft Fabric is a big step in the evolution of data management and analysis as we take advantage of the ever-improving capabilities of AI.
Microsoft Fabric is currently still in preview. That’s why we encourage you to try it out and see first-hand all the features we have reviewed in this article: Try Fabric.
Microsoft Fabric responds to Unified Data Management and AI Power
In short, Microsoft Fabric integrates data lakes, AI, and security into a single platform. It enables organizations to consolidate their information, drive deep analytics, and make informed decisions. With tools like Synapse, Power BI, Data Factory, and Data Activator, Microsoft is not only demonstrating its commitment to innovation but also facilitating the path to a future where data management and analytics cohesively merge.
If you would like to learn more about Microsoft Fabric or if you have any additional questions, please do not hesitate to contact us. We will be happy to help you and provide you with the information you need.