Power BI Premium was announced generally available less than a year ago as the platform of choice for enterprises deploying high-performance, scalable BI systems in the cloud. In the May release of Power BI Desktop, we announced the next step in our vision: Power BI Premium supports incremental refresh (preview)! Incremental refresh enables large datasets in the Power BI Premium service with the following benefits.
- Refreshes are faster. Only data that has changed needs to be refreshed. For example, refresh only the last 5 days of a 10-year dataset.
- Refreshes are more reliable. For example, it is not necessary to maintain long-running connections to volatile source systems.
- Resource consumption is reduced. Less data to refresh reduces consumption of memory and other resources.
Please see this article to learn how to use incremental refresh.
In the following video, Adam Saxton from the Power BI CAT Team and Christian Wade from the Power BI Premium product team discuss the significance of incremental refresh.
Incremental refresh has traditionally been in the realm of Azure Analysis Services for high-end, scalable BI models. Whilst one of Analysis Services’ greatest strengths, it’s a feature that can require considerable time and effort to set up. With Power BI Premium, incremental refresh is simplified to essentially a dialog box with a few checkboxes and dropdowns!
Related items coming soon
Update metadata
Once the dataset is published and refreshed, if a change needs to be made to the model or reports, it needs to be republished from Power BI Desktop to the service. The current publish workflow detects when there is already a dataset with the same name and prompts if it should be replaced. Replacing a dataset in this way replaces the data within it. This could mean having to reload historical data. We plan to provide the ability to update and retain the data on publish. Incremental refresh will stay in public preview until this feature is released.
Increased dataset size
We plan to remove the 10 GB dataset-size limit in the Power BI Premium service. This will allow datasets utilizing incremental refresh to be limited only by the capacity size.
Override effective date
We plan to allow setting the current date for a refresh operation. This will be useful to use with datasets like Adventure Works that don’t have data up to the current date, and for testing purposes.
XMLA Endpoints
XMLA Endpoints will enable connectivity, manageability and programmability in Power BI Premium comparative to that of Azure Analysis Services. For example, you will be able to use SQL Server Management Studio (SSMS) to connect to datasets with incremental-refresh policies and generate scripts to refresh specific historical periods (partitions).
Shared capacity support
Based on customer feedback, we plan to make incremental refresh available on shared-capacity (non Premium) workspaces.