We are excited to announce new improvements to Power BI dataflows releasing this month including an enhanced compute engine to drastically improve data refresh times and a new set of dataflow templates to help you easily connect to common data sources mapped against the common data model.
Enhanced compute engine
As organizations accumulate more and more data, it becomes increasingly important to quickly perform ETL at scale over these datasets. Today we are announcing a major step forward in overcoming these pains points with a new compute engine for dataflows. Now you will be able to efficiently combine and transform different data sources and manage TB’s of data using dataflows. With the new engine, calculation of computed entities can now be up to x20 faster when working with large scale data volumes.
To enable the enhanced compute engine, you will first need to go to the capacity admin setting and locate a new option called Enhanced Dataflows Compute Engine (Preview). By toggling this option on you will enable the new engine. Bear in mind you will need to restart the capacity before the new engine will be activated.
In order to fully utilize and benefit from the new compute engine, split ingestion of data into separate dataflows and put transformation logic into computed entities in different dataflows. This is recommended because the compute engine works on dataflows which reference an existing dataflow (but not on ingestion dataflows). Following this guidance will ensure that transformation steps such as joins or merges are handled by the new compute engine for optimal performance. We hope you enjoy this new capability and see an improvement in refresh times. If you have any feedback on this feature, we would love to hear it please email the team.
Dataflow templates
A common challenge that our customers face while integrating data is the differences between the source and destination schema. Mapping entities and fields becomes a tedious and expensive process. A dataflow template, saves you time and effort by providing you an out-of-the-box predefined set of entities and mappings to enable flow of data from the source to the destination, in the Common Data Model. It provides you a head start to ingest data wherein you don’t need to worry mapping the source and destination entities and fields – We do it for you, through dataflow templates.
This month, we are releasing a set of dataflow templates that help you unify data from popular CRM apps such as Dynamics 365 for Sales and Salesforce. A few clicks will help you build your sales leaderboard dashboard and get visibility over your sales pipeline. Connecting to quotes, order and invoice data in Dynamics 365 for Sales or gaining insights by analyzing lead conversion and opportunity closures from Salesforce has never been easier.
That’s all for this month! We hope that you enjoy these updates for the month. if you have any feedback feel free to contact the dataflows team