Finding Data Assets for a Use Case
When looking for data, there are four different options.
Data Catalog UI
Data Landscape UI
SQL Workspace
APIs
Data Discovery through the Data Catalog
In most cases, Data Products will be the best option, as they will already be curated, maintained and described for immediate consumption. However, if the data you are looking for has not been packaged yet, you might need to look for data sources. Both constitue examples of Assets.
Looking for Assets Using the Search Bar
Navigate to the Data Catalog
Click on the top search bar
You can choose to either filter by one of the three categories (Asset Type, Owner, Tag) or type in the name of the asset you are looking for.
Click on the asset if you found it, or press Enter to update your results on the second column of the Catalog.
Looking for Assets Using the Left Bar
Navigate to the Data Catalog
You can narrow down your search using the following buttons:
Data System, Data Source, Data Product, Application: they will return all the assets that you have access to in that particular category.
Pinned Meshes: after pinning a Mesh in the Landscape, you will be able to click on it and browse through the Assets that belong to that Mesh.
For You: look for assets that you own or that have been created recently (in the past 30 days).
Mesh Systems and Sources: You can select one of the Meshes in your Organization and either click on a particular System or Data Source, either of them will return you their dependent Assets.
Choosing the right Asset
Data Products aim to make data more easily consumable, and therefore deliver a lot more information about themselves to the user, so that they can decide whether it is the right Asset or not for them.
Data Products will display the following details to a user with full access:
Overview: description, health, quality score, tags, owner, last updated, total rows and columns, consumers
Metadata: identifiers, state, mesh information, timestamps
Dictionary: for every column: PK, Name, Description, Optional, Data Type, Classification, Sensitivity
Lineage: Diagram representing parents and children, list of transformations
Quality: Score for each dimension, last updated, failed quality checks per dimension
Profile: details about each column such as example values, data type, unique values, percentage of nulls, max and min.
Other Assets (Data Systems, Data Sources, Applications) display information specific to their nature that can help select them as well. For example, for Data Sources, users will find the Data Objects that are returned from the Connectors, including their file Type, Path, Header and Delimiter.
Data Discovery through the Landscape
The Landscape might be a more suitable option for those who think of data from a lineage perspective, and who know which System or Source it will be coming from.
Navigate to the Landscape or your Pinned Mesh
Select the right Mesh (you will skip this step if you already clicked on a Pinned Mesh)
Browse the Landscape
Hover over an Asset to see a handful of details.
Click on "More" of you want to read more information about that asset on its Catalog page.
Data Discovery through the SQL Workspace
The SQL Workspace is the least recommended option to find the data you need, but for those users with SQL knowledge, it might be a useful one.
Navigate to the SQL Workspace
Search for a data product
Click on a data product to find its schema
Click on the table name for the data product to automatically populate it on the SQL statement box.
Optional: customize the SQL query
Run the query to explore the data set.
Data Discovery through the APIs
Foundation offers a few API endpoints to help find the right Asset:
/api/data/search/api/data/application/list/api/data/data_system/list/api/data/data_product/list/api/data/data_object/list/api/data/data_source/listRead the Using the Foundation APIs page to leverage this option in the best way possible.
Last updated