Updates and Clarifications on Snowflake CLI Ingestion Permissions

ingestion
1

Original Slack Thread

<@U06TM7M40RK> In snowflake cli ingestion. Did some require permission changed?
in version https://securitiai.testrail.io/index.php?/cases/view/408415|“0.12.1.5”

2

Yes, there have been some updates and clarifications to the required permissions for Snowflake ingestion in DataHub, but the core set of required privileges has remained largely consistent across recent versions, including 0.12.1.5. The main permissions needed are for reading metadata, extracting lineage, and (optionally) profiling and usage statistics. However, there have been some improvements and clarifications in the documentation and error handling around permissions.

Key Points on Required Permissions

  1. Core Permissions:
    The DataHub Snowflake ingestion source requires a Snowflake user with a role that has the following privileges:

    • USAGE and OPERATE on the warehouse used for ingestion.
    • USAGE on the databases and schemas you want to ingest.
    • REFERENCES or SELECT on tables, views, and external tables (depending on whether profiling is enabled).
    • IMPORTED PRIVILEGES on the SNOWFLAKE database for lineage, usage, and tags extraction.

    These requirements are documented in the <https://datahubproject.io/docs/0.14.0/quick-ingestion-guides/snowflake/setup|official DataHub Snowflake setup guide> and <https://datahubproject.io/docs/0.14.0/generated/ingestion/sources/snowflake#prerequisites|prerequisites section>.

  2. Recent Documentation and Error Handling Improvements:

  3. Dynamic Tables and New Features:

  4. No Major Breaking Changes in 0.12.1.5:

Example: Granting Required Permissions

3 
-- Create role and user
create or replace role datahub_role;
grant operate, usage on warehouse "&lt;your-warehouse&gt;" to role datahub_role;
grant usage on database "&lt;your-database&gt;" to role datahub_role;
grant usage on all schemas in database "&lt;your-database&gt;" to role datahub_role;
grant references on all tables in database "&lt;your-database&gt;" to role datahub_role;
grant references on all views in database "&lt;your-database&gt;" to role datahub_role;
grant imported privileges on database snowflake to role datahub_role;
create user datahub_user password='&lt;your-password&gt;' default_role=datahub_role default_warehouse='&lt;your-warehouse&gt;';
grant role datahub_role to user datahub_user;

(https://datahubproject.io/docs/0.14.0/quick-ingestion-guides/snowflake/setup|Reference)

Summary:
The required permissions for Snowflake ingestion in DataHub have not fundamentally changed in version 0.12.1.5, but the documentation and error handling have improved. If you are seeing permission errors, double-check that all the required grants (especially IMPORTED PRIVILEGES on the SNOWFLAKE database) are in place, and consult the latest <https://datahubproject.io/docs/0.14.0/quick-ingestion-guides/snowflake/setup|setup guide> for details.

Would you like a step-by-step breakdown of the permission requirements or troubleshooting for a specific error message?

Sources:

4

Hi <@U07J218Q1M0>, how did my answer work out for you? Need any more assistance? You can also tag <@U07FV3UHE4D> to let your fellow community members know you’re still looking for guidance.