Exploring Data Lineage

Foundation requires Data Product developers to specify the inputs that will be required to populate it. Without those inputs, a Data Product would not be able to run transformations, for example.

Data Product owners and developers can also specify the consumers of their data products, if known.

With all this information, users can easily explore the dependencies between different assets across what we call the "Landscape".

Exploring Lineage through the Landscape UI

1. Navigate to the Data Landscape UI or your Pinned Mesh

2. Select the right Mesh (you will skip this step if you already clicked on a Pinned Mesh) by clicking on the top bar

3. Browse the Landscape

4. Hover over an Asset to see a handful of details.

5. Click on "More" of you want to read more information about that asset on its Catalog page.

The landscape represents dependencies between assets by showing how data flows left to right.

Exploring Lineage through the Data Catalog UI

The Data Catalog UI is more suitable when trying to understand the lineage for a particular asset.

1. Navigate to the Data Catalog page

2. Search for the Asset you need to understand the dependencies for

3. For Data Products, click on the Lineage tab. Other Assets will display it already.

For all Asset types you will find the direct input and consumer for the asset.

For Data Products, you will find, additionally, information about the transformations configured.

Exploring Lineage through the APIs

Foundation offers API endpoints that return lineage information about each entity:

/api/data/data_product

/api/data/data_system

/api/data/data_source

/api/data/application

/api/data/data_object

Read the Using the Foundation APIs page to leverage this option in the best way possible.