Mastering Power BI Dataflows: Essential Guide to Create, Configure & Consume

Mastering Power BI Dataflows Essential Guide to Create Configure Consume

Mastering Power BI Dataflows: Essential Guide to Create, Configure & Consume

Power BI Dataflows offer a low-code, cloud-based solution for data preparation and transformation. Built on Azure Data Lake and powered by Power Query, dataflows enable self-service data engineering by allowing users to ingest, clean, transform, and enrich data—outside of traditional Power BI datasets.

Dataflows are especially powerful when you want to centralize data transformation logic, reuse data preparation across multiple reports, or create a consistent, governed data source for reporting purposes.

What is dataflow?

dataflow is a collection of entities (entities are similar to tables) that are created and managed in workspaces in the Power BI service. An entity/Table is a set of fields that are used to store data, much like a table within a database. You can add and edit entities/tables in your dataflow, as well as manage data refresh schedules, directly from the workspace in which your dataflow was created.

image 66

When to use dataflows

  • Create reusable transformation logic that can be shared by many datasets and reports inside Power BI. It means using One Power query table in Multiple Power BI reports.
  • Expose the data in your own Azure Data Lake Gen 2 storage, enabling you to connect other Azure services to the raw underlying data
  • Create a single source of the truth by forcing analysts to connect to the dataflows, rather than connecting to the underlying systems, providing you with control over which data is accessed, and how data is exposed to report creators.
  • If you want to work with large data volumes and perform ETL at scale, dataflows with Power BI Premium scales more efficiently and gives you more flexibility. Dataflows supports a wide range of cloud and on-premise sources.

Where the output stored

  • Dataflows stores the data in the Azure Data lake storage.
  • Dataflow manages the Data Lake configurations internally so we need only Power BI accounts and subscriptions.

Dataflow vs Dataset

DataflowDataset
Replacement of your Power QueryReplacement of DAX Calculations and Relationships
ETL LayerModelling Layer
Feeds Data into the DatasetFeeds Data into Visualizations
Access the Data Source DirectlyAccess the Data from the Dataflow
Developer Needs Power Query SkillsDeveloper Needs DAX and Modelling Skills
Dataflow are Data ModelersDataset are Report Visualizers
Row level security – NoRow level security – Yes
Data access method – ImportData access method – Import, DirectQuery
Primary purpose – Data reusePrimary purpose – Data analysis
Development – Power query onlineDevelopment – Power query in Power BI desktop

Create and configure dataflows

Creating and configuring dataflows in Power BI is a straightforward process, designed to empower business users and analysts with minimal technical complexity. Here’s how it typically works:

Step 1 : Create a new workspace

image 67

Step 2 : Create a new dataflow

To create a dataflow, launch the Power BI service in a browser then select a Demo workspace.

image 68
image 69

There are a multiple of ways to create or build on top of a new dataflow:

  1. Create a dataflow using define new entities
  2. Create a dataflow using linked entities
  3. Create a dataflow using a computed entity
  4. Create a dataflow using import/export

We will only cover Point 1 : Create a dataflow using define new entities.

Using the Define new entities option lets you define a new entity/table and connect to a new data source.

image 70

Select common data service.

image 71

Provide the CDS url.

image 72

After login successful, Select the data entities as below.

image 73

We will remove the columns which are not needed now. Now, below entities are available in Dataflows.

image 74

Step 3: Configure a dataflow

To configure the refresh of a dataflow, select the More menu (the ellipsis) and select Settings.

image 75

The Settings options provide many options for your dataflow, as the following sections describe.

image 76

We will see some of important settings below.

  • Gateway Connection: In this section, you can choose whether the dataflow uses a gateway, and select which gateway is used.
  • Data Source Credentials: In this section you choose which credentials are being used, and can change how you authenticate to the data source.
  • Sensitivity Label: Here you can define the sensitivity of the data in the dataflow. To learn more about sensitivity labels, see how to apply sensitivity labels in Power BI.
  • Scheduled Refresh: Here you can define the times of day the selected dataflow refreshes. A dataflow can be refreshed at the same frequency as a dataset.

Step 4 : Refreshing a dataflow

Dataflows act as building blocks on top of one another. When the schedule refresh for the dataflow triggers, it will trigger any dataflow that references it upon completion. This functionality creates a chain effect of refreshes, allowing you to avoid having to schedule dataflows manually.

There are a few limitations when dealing with linked entities refreshes:

  • A linked entity will be triggered by a refresh only if it exists in the same workspace
  • A linked entity will be locked for editing if a source entity is being refreshed. If any of the dataflows in a reference chain fail to refresh, all the dataflows will roll back to the old data (dataflow refreshes are transactional within a workspace).
  • Only referenced entities are refreshed when triggered by a source refresh completion. To schedule all the entities, you should set a schedule refresh on the linked entity as well. Avoid setting a refresh schedule on linked dataflows to avoid double refresh.

Incremental Refresh (Premium only) Dataflows can be also set to refresh incrementally. To do so, select the dataflow you wish to set up for incremental refresh, and then select the incremental refresh icon.

image 77

Setting incremental refresh adds parameters to the dataflow to specify the date range. For detailed information on how to set up incremental refresh, see the incremental refresh in Power Query article.

There are some circumstances under which you should not set incremental refresh:

  • Linked entities should not use incremental refresh if they reference a dataflow. Dataflows do not support query folding (even if the entity is Direct Query enabled).
  • Datasets referencing dataflows should not use incremental refresh. Refreshes to dataflows are generally performant, so incremental refreshes shouldn’t be necessary. If refreshes take too long, consider using the compute engine, or DirectQuery mode.

Consume dataflows

Once a dataflow is created and refreshed, it can be easily consumed across the Power BI ecosystem:

A dataflow can be consumed in the following three ways:

  1. Create a linked entity from the dataflow to allow another dataflow author to use the data
  2. Create a dataset from the dataflow to allow a user to utilize the data to create reports
  3. Create a connection from external tools that can read from the CDM format

To consume a dataflow, run Power BI Desktop and select the Power BI dataflows connector in the Get Data dialog.

image 78

Login with Organization account.

image 79

Select Demo dataflow and Select all entities.

image 80

Here, We will show the information  about employee, position and department. Also we will create some headcount of employees, department, Positions and FTE.

Once it is ready then report should like below.

image 81

Licensing

image 82

Dataflows best practices

It can be found here for detailed explanation

https://docs.microsoft.com/en-us/power-bi/transform-model/dataflows/dataflows-best-practices

Dataflows limitations

https://docs.microsoft.com/en-us/power-bi/transform-model/dataflows/dataflows-features-limitations

Reference link :

https://docs.microsoft.com/en-us/power-bi/transform-model/dataflows/dataflows-introduction-self-service

What are Power BI Dataflows and their Use Cases?

If you like this article, feel free to share it with others who might find it helpful! If you have any questions, feel free to reach out to me.

Parag Chapre

Parag Chapre is a Microsoft MVP in the fields of Dynamics 365 Finance & Operations, Human Resources, and Power Platform, recognized for his outstanding contributions to the Microsoft Dynamics community.

With over 15 years of hands-on expertise in various Microsoft Dynamics 365 areas, Parag has designed and delivered complex, innovative solutions for customers across industries and geographies. He has also provided leadership and technical guidance to project teams, managed offshore and onshore resources, and worked closely with Microsoft Product teams. Parag is passionate about sharing his knowledge and insights through his personal website, blog posts, articles, and community events. He is a member of the Microsoft Biz Apps Community Advisory Board, a Dynamics 365 Human Resource Community star, a Dynamics 365 Community contributor, and a Dynamics 365 Community Spotlight honoree.

1 Comment

Leave a Reply