Introduction
Datamarts are a preview feature in Power BI Premium (and Premium Per User) that enable self-service users to securely collect, store, analyze, and share their data in a unified, low code, solution – delivering SQL and unified analytics –ย all in a web-based experience, for the first time.
At the same time, datamarts provide an auto-generated dataset, enabling downstream data visualization and analysis via the “clicky-clicky-draggy-droppy” Power BI reports you love.
As part of Power BI Premium, datamarts ship updates monthly โ but also as often as weekly! Being a 100% web-based, SaaS solution allows the team to iterate very quickly. This blog serves as a round-up of all the features and new releases weโve shipped since we launched in May, including:
- Datamart availability in new regions
- Data connectivity and preparation updates
- Collaboration enhancements
- Performance acceleration for Datasets
- Datamart consumption updates
- New Blogs and Docs updates
- Datamart Community updates
- Updates to your Ideas
- Ideas weโve shipped
- Ideas moved to planned
- Ideas under review
You can continue reading the rest of this blog post for more details about each of these items.
Datamarts Available in More Regions
Announcing Availability in Southeast Asia
Weโve added Southeast Asia as an additional region where Datamarts can be created. Youโll notice Datamarts are now available in Premium capacities here. Try it out and let us know what you think!
Data connectivity and preparation
Named Connection support in Power BI datamarts
Datamarts now allow users to create multiple connections against the same data source with different credentials. This enhancement to Power Query Online enables organizations to centrally manage connectivity to enterprise data sources, such as a critical data warehouse or line of business application.
Certified connectors available in Power BI datamarts
As mentioned in the Power Query and Power BI Blogs, we are excited to announce the availability ofย 80+ additional data sourcesย (all certified connectors) to the Power Query Online experience in Power BI datamarts. This means that Power BI datamart creators now have the option to connect to many more data sources in Power Query Online experiences, like Power BI desktop today. In the future, all newly released certified connectors will also now be available by default in Power BI datamarts.
To access these data sources, see Get started with datamartsย to learn more.
Notable additions
- Amazon Athena
- Anaplan
- Asana
- Azure Databricks and Databricks
- Denodo
- Dremio
- Google Sheets
- HexagonSmartAPI
- MariaDB
- Palantir Foundry
- Planview Projectplace
- Starburst Enterprise
- SurveyMonkey
See Certified connectors available in Power BI dataflows and datamarts to learn more.
Enhancements to the Data Prep and Save Experience
Weโve heard your feedback and delivered an improved experience around creating data preparation steps and loading data into your datamart. With this update, datamarts will save your progress automatically, recover your data preparation steps if you unexpectedly leave your work, and provide a better experience when troubleshooting issues.
Enhancements to Incremental Refresh
Weโve improved the incremental refresh behavior to provide better performance and a better experience in working with all data sources. This can provide improved performance when you use bigger data volumes and need to control refresh so that it appends, instead of replaces, data. This is especially powerful when used in conjunction with relational data sources (databases), where we can perform query folding.
Collaboration Enhancements
Multiple contributors for datamarts
Weโre providing the ability for workspace admins, contributors, or members to open the datamart and perform collaborative steps with the datamart owner. This enables measure editing, reviewing PQ steps, enhancing existing PQ steps, performing semantic modeling, and working with queries. For data and analytics teams, this will enable greater productivity working with datamarts. To take advantage of this feature, leverage workspace roles of at least contributor.
Performance acceleration for datasets
Automatic Import Mode
Weโve delivered a new feature, called proactive caching, which will provide performance acceleration to the auto-generated dataset by leveraging the super-fast Vertipaq engine. Behind the scenes of this feature โ we change the storage mode of your model to import โ the default, super-fast in memory engine in Power BI and Analysis Services today. By design, this will also automatically manage the switching between Direct Query and Import modes, so that your data is always up-to date, and you donโt need to worry about storage modes. This feature is already on for all datasets.
Datamart Consumption and Analysis
Create a report from the datamart ribbon
We’ve heard your feedback that it can be hard to validate measures and work with the auto-generated dataset within the datamart experience as you want to be able to see your results and also build reports in 1-click from a datamart. With this update, weโre working to make that experience easier. Users will now be able to create a report while editing or analyzing data within a datamart, which will open a report connected to the dataset in a new tab. This enables a few things:
- End to end report creation, starting from a datamart to an end report in a seamless experience
- Easier development of measures with the ability to refresh the report as new datasets are added
Creating a report is available in the home tab in the ribbon:
This launches a Power BI report, connected to the auto-generated dataset from the datamart, in a new tab:
Create and Save Multiple Queries
Datamarts allow you to build no-code or SQL queries, enabling analysis across wide tables and query patterns that are non-trivial with datasets in Power BI via:
- Visual Queries, identified with:ย
- SQL Queries, identified with:ย
Behind the scenes both of these generate SQL queries that we save within the datamart experience. To make this experience better, we are also enabling query tabs and saving these queries within a specific collection within the object explorer:
To learn more about this experience and walk-through specific scenarios, check out our blog post: Announcing improvements to Datamart Query editor | Microsoft Power BI Blog | Microsoft Power BI
Connect to datamarts and the SQL endpoint in Power BI desktop
Datamart connectivity in Power BI Desktop is an important step towards re-using and enhancing the valuable analytical assets users in your organization create. As announced in the July 2022 Power BI Update, ย you will notice that you can now easily discover the entry point to connect to datamarts from the โGet Dataโ experience (Power Platform section) within Power BI desktop:
You can also search for datamarts within the โGet dataโ experience:
With this new update, we are excited to share that clicking on a datamart provides you with two options:
- Connect (Auto-generated dataset):ย Connect to the datamartโs underlying auto-generated dataset using live connect so that you can easily create reports
- Connect to SQL endpoint:ย Connect to the datamartโs SQL endpoint using direct query or import and build datasets or reports. This provides an easy discovery experience and reduces the friction associated with finding and copy-pasting the datamartโs SQL connection string from datamart settings or info page.
Note that just like other artifacts, datamarts can be certified and promoted, and that endorsement label is front and center. Take advantage of this feature to make it even easier to people in your organization to do more, with the right set of data, in your organization!
Check out the announcement blog for more details.
New Blogs and Documentation Updates
How to Guides on the Power BI Blog:
Weโre releasing new blogs aimed at helping you to get started with datamarts and walk through key scenarios:
Available Now:
Coming Soon:
- Consumption
- Administration
Donโt see a topic weโve covered yet? Let us know what youโre looking for in the comments.
Documentation:
We recognize that documentation can be helpful for understanding datamart capabilities and is equally important in helping you to deliver key scenarios. Weโve made some updates to our docs we want to call attention to below.
New
Weโve added these new areas to our documentation:
Updates
Weโve updated this documentation
Coming Soon
- Best Practices
- Incremental Refresh
Need more technical documentation? Give us feedback in the comments section.
Datamarts Community Updates
Our community is super important to evolving the shape of the product and the features that make datamarts useful for YOU! Weโre sharing some upcoming events here that caught our attention and we hope youโll be able to make it!
Recordings
The Power BI Community Show is a live show hosted by Kelly Kaye, the Community Engagement Lead for Power BI. On this episode, Kelly Kaye interviews members of the Power BI product team, who discuss and demo datamarts, as well as provide an outlook for the future.
Here’s the recording link on YouTube: Power BI Community Show – Datamarts
ย
Podcasts
These podcasts, delivered in livestream format will provide an interactive Q&A session with members of the Power BI datamarts product team and the Power BI community.
Explicit Measures – Podcast and Livestream
The Explicit Measure Podcast is a fun and informative conversation around the Power BI ecosystem. Microsoft MVPs, Tommy Puglia, Seth Bauer, and Mike Carlo provide a ย “water cooler” take on delivering business intelligence within organizations.
Join this special podcast, tomorrow, 8/31/2022 at 11:00am Pacific (-8 UTC) for a discussion on All things datamarts, including answering key questions like: โWhat is the Audienceโ, โWho is this for?โ โWhere does it fit in the Ecosystem? and โWhere does Microsoft see datamarts going?โ
Explicit Measures – All Things Datamarts YouTube Link
Separately, check out the Explicit Measures Podcast live every Tuesday and Thursday at 7:30 am CST (12:30 pm UTC)
KasperOnBI – Podcast and Livestream
On September 16th, join the Power BI datamarts team, including Kasper de Jonge from the Power BI CAT team on his KasperOnBI show on Everything datamarts. In this one hour show we will discuss โwhat are datamarts in detail?โ, โwho are datamarts for?โ, โwhen to use datamarts?โ and much more! Follow along here to get notified when the episode releases.
Virtual Conferences
Data Platform Virtual Summit 2022ย is 5-day,ย virtual, round-the-clock learning event onโ Microsoft Azure Data Platform, from September 19-23, 2022. Data Platformย Virtual Summit 2022, the 8th edition of Dataโ Platform Summit, will focus on sevenย technology tracks: Data Science, Businessย Intelligence & Analytics, Databaseย Administration, Database Development, Cloud,ย IoT & NoSQL and Big Data. The sessions scaleย out on knowledge level from 200 through 500ย from experts around the globe. Theย conference is for the Data & Analyticsย community to learn new skillsย andย technologies and enhance existing skills onย Microsoft Azure Data Stack.
Datamarts will have a dedicated, pre-recorded session โ Whatโs new in Datamarts, together with a Live Q&A session.
Separately, the virtual Microsoft booth will host experts and resources for deeper dives on datamarts as well as other products.
Live Conferences
Kicking off in-person on September 18ย & 19, with pre-conference workshops, the Microsoft Power Platform Conference 2022 runs September 20-22, 2022. This event will feature pre-conference workshops, keynotes, sessions, breakouts, and moreโall dedicated to Microsoft Power Platform products: Power Automate, Power Apps, Power BI, Power Pages, and Power Virtual Agents. Come and get inspired by Microsoft’s senior thought leaders, software engineers, and community experts who are focused on uniting humanity and technology. You’ll also have the chance to hear from users that solve real business problems daily
Datamarts will have a dedicated session, Datamarts Deep Dive and Roadmap, which will dive deep on critical capabilities, architectural patterns and use cases, and more. Come to this session to learn more about Power BI datamarts and what’s coming soon!
Call to action โ let us know what community events to highlight next!
Want to discuss datamarts and engage with the community? Weโre always looking for activities to highlight! Please reach out to us in the comments below if you have a blog, podcast, webinar series, live or virtual event you’d like us to highlight in future blogs.
Updates to Your Ideas
Thank you for all of your enthusiasm, feedback, and feature requests you’ve provided onย Power BI Ideas, theย Microsoft Power BI Community,ย and other forums, like Twitter. We really appreciate it! Below are some updates to your top ideas, where weโll share dates and statuses along the way.
Items that have just shipped:
- Power BI Desktop Connector โ improved connectivity to datamart SQL
- Multiple Contributors โ improved team collaboration when editing datamarts
- Save Queries – allowing datamart contributors to collaborate, save, and re-use ad-hoc analysis in the web
Items that have changed date or status:
- Open connectivity to Datamart SQL endpoint โ enabling secure AAD authentication and connections via notebooks, client applications, BI tools, and more (Planned for end of September)
- Takeover ownership โ Enabling full control over data prep and datamart connectivity (Planned for end of September)
- Datamarts DML / DDL โ SQL write support (Under review by the team, date to be confirmed)
- This will enable other ideas weโve merged like views, table-valued-functions,ย and stored procedures
- This will also enable more complex ETL scenarios and pro-devs to be more productive in datamarts
- Dataflows (Gen 2) output to datamarts (Under review by the team, date to be confirmed)
- Datamarts DML/DDL feature is a pre-requisite
- Enable datamarts for select users โ improved access control (Under review, date to be confirmed)
- Other Ideas โ We updated multiple items on the ideas site and marked many of them under review. Once a top voted idea reaches an under review or planned state, it will be mentioned here for future updates.
Thatโs all for this month! Please continue sending us your feedback and do not forget to vote for other features that you would like to see in Power BI! We hope that you enjoy the update! As always, keep voting onย Ideasย to help us determine what to build next.