Help with Icons in DataHub

ui
1

Original Slack Thread

<@U06TM7M40RK> help with icons

2

It looks like you need help with icons in DataHub. Could you please provide more context or specify what exactly you need help with regarding icons? For example:

  1. Are you looking to change or customize existing icons?
  2. Do you need to add new icons for specific entities or features?
  3. Are you encountering issues with icons not displaying correctly?

Providing more details will help me give you a more precise answer. In the meantime, here are some relevant resources that might be helpful:

Would you like more details on any of these topics or something else related to icons in DataHub?

Sources:

3

<@U06TM7M40RK> 3. Are you encountering issues with icons not displaying correctly? Yes

4

If you are encountering issues with icons not displaying correctly in DataHub, there are a few common causes and solutions you can try:

  1. Ingesting data_platforms.json:
    • In some versions of DataHub, icons for data platforms are not displayed correctly because the data_platforms.json file has not been ingested. This file contains the necessary metadata for platform icons.
    • You can ingest the data_platforms.json file using the following configuration:
source:
  type: "file"
  config:
    filename: ./examples/mce_files/data_platforms.json

sink:
  type: "datahub-kafka"
  config:
    connection:
      bootstrap: localhost:9092

  1. Check for Correct URLs:

    • Ensure that the URLs for the icons are correct. Sometimes, the issue might be due to incorrect URLs being used for the icons. For example, the URL might point to a non-existent resource or an incorrect path.
    • Verify that the URLs in your data_platforms.json or other configuration files are correct and accessible.

  2. Local vs. External Resources:

    • There might be a mix-up between local and external resources. For instance, some logos might be fetched from GitHub while others are expected to be served locally.
    • Ensure that your configuration is consistent and that all icons are either served locally or fetched from a valid external source.

  3. Browser Console Errors:

    • Check the browser console for any errors related to loading resources. This can provide clues about what might be going wrong.
    • Look for 404 errors or other network-related issues that indicate the icons are not being found.

  4. Version-Specific Issues:

    • Some issues might be specific to certain versions of DataHub. Make sure you are using a version that does not have known issues with icon display.
    • If you are using an older version, consider upgrading to a newer version where the issue might have been fixed.

Would you like more detailed steps on any of these troubleshooting points or further assistance on a specific part of the process?

Sources:

5

<@U06TM7M40RK> i need a pratical exemple about this configuration data_platforms

6

Certainly! Here is a practical example of how to configure and ingest a custom data platform using the data_platforms.json file in DataHub.

Step-by-Step Guide

  1. Create the data_platforms.json File:
    • Create a JSON file named data_platforms.json with the necessary metadata for your custom data platform. Here is an example configuration:
[
  {
    "auditHeader": null,
    "proposedSnapshot": {
      "com.linkedin.pegasus2avro.metadata.snapshot.DataPlatformSnapshot": {
        "urn": "urn:li:dataPlatform:MyCustomDataPlatform",
        "aspects": [
          {
            "com.linkedin.pegasus2avro.dataplatform.DataPlatformInfo": {
              "datasetNameDelimiter": "/",
              "name": "My Custom Data Platform",
              "type": "OTHERS",
              "logoUrl": "https://example.com/my-custom-logo.png"
            }
          }
        ]
      }
    },
    "proposedDelta": null
  }
]
  1. Create the Ingestion Recipe:
    • Create a YAML file named data_platform_ingestion_recipe.yaml to configure the ingestion process. This file will specify the source as the data_platforms.json file and the sink as DataHub.
source:
  type: "file"
  config:
    filename: "./data_platforms.json"

sink:
  type: "datahub-rest"
  config:
    server: "http://localhost:8080"  # Update with your DataHub server URL
  1. Run the Ingestion:
    • Use the DataHub CLI to run the ingestion process. Ensure you have the DataHub CLI installed. If not, you can install it using pip install acryl-datahub.
datahub ingest -c data_platform_ingestion_recipe.yaml

Example Code Snippet

Here is the complete example in a structured format:

data_platforms.json:

[
  {
    "auditHeader": null,
    "proposedSnapshot": {
      "com.linkedin.pegasus2avro.metadata.snapshot.DataPlatformSnapshot": {
        "urn": "urn:li:dataPlatform:MyCustomDataPlatform",
        "aspects": [
          {
            "com.linkedin.pegasus2avro.dataplatform.DataPlatformInfo": {
              "datasetNameDelimiter": "/",
              "name": "My Custom Data Platform",
              "type": "OTHERS",
              "logoUrl": "https://example.com/my-custom-logo.png"
            }
          }
        ]
      }
    },
    "proposedDelta": null
  }
]

data_platform_ingestion_recipe.yaml:

source:
  type: "file"
  config:
    filename: "./data_platforms.json"

sink:
  type: "datahub-rest"
  config:
    server: "http://localhost:8080"  # Update with your DataHub server URL

Ingestion Command:

datahub ingest -c data_platform_ingestion_recipe.yaml
7

Explanation

  • data_platforms.json: This file defines the custom data platform with its metadata, including the name, type, and logo URL.
  • data_platform_ingestion_recipe.yaml: This YAML file configures the ingestion process, specifying the source as the data_platforms.json file and the sink as the DataHub REST API.
  • Ingestion Command: The datahub ingest command runs the ingestion process using the specified YAML configuration.

Would you like more detailed steps or troubleshooting tips?

Sources:

8

<@U06TM7M40RK> this is the issue![attachment]({‘ID’: ‘F07QZ2HG738’, ‘EDITABLE’: False, ‘IS_EXTERNAL’: False, ‘USER_ID’: ‘U07N5E066G2’, ‘CREATED’: ‘2024-10-09 18:16:05+00:00’, ‘PERMALINK’: ‘Slack’, ‘EXTERNAL_TYPE’: ‘’, ‘TIMESTAMPS’: ‘2024-10-09 18:16:05+00:00’, ‘MODE’: ‘hosted’, ‘DISPLAY_AS_BOT’: False, ‘PRETTY_TYPE’: ‘PNG’, ‘NAME’: ‘image.png’, ‘IS_PUBLIC’: True, ‘PREVIEW_HIGHLIGHT’: None, ‘MIMETYPE’: ‘image/png’, ‘PERMALINK_PUBLIC’: ‘https://slack-files.com/TUMKD5EGJ-F07QZ2HG738-495c71e10f’, ‘FILETYPE’: ‘png’, ‘EDIT_LINK’: None, ‘URL_PRIVATE’: ‘Slack’, ‘HAS_RICH_PREVIEW’: False, ‘TITLE’: ‘image.png’, ‘IS_STARRED’: False, ‘PREVIEW_IS_TRUNCATED’: None, ‘URL_PRIVATE_DOWNLOAD’: ‘Slack’, ‘PREVIEW’: None, ‘PUBLIC_URL_SHARED’: False, ‘MESSAGE_TS’: ‘1728497797.735699’, ‘PARENT_MESSAGE_TS’: ‘1728497412.821019’, ‘MESSAGE_CHANNEL_ID’: ‘CV2UXSE9L’, ‘_FIVETRAN_DELETED’: False, ‘LINES_MORE’: None, ‘LINES’: None, ‘SIZE’: 7572, ‘_FIVETRAN_SYNCED’: ‘2024-10-13 12:43:20.604000+00:00’})

9

Sure thing! I’m looking through the available data to answer your question. Hold on a second… :hourglass_flowing_sand:

10

<@U06TM7M40RK> shoud be like![attachment]({‘ID’: ‘F07R86AASSG’, ‘EDITABLE’: False, ‘IS_EXTERNAL’: False, ‘USER_ID’: ‘U07N5E066G2’, ‘CREATED’: ‘2024-10-09 18:17:31+00:00’, ‘PERMALINK’: ‘Slack’, ‘EXTERNAL_TYPE’: ‘’, ‘TIMESTAMPS’: ‘2024-10-09 18:17:31+00:00’, ‘MODE’: ‘hosted’, ‘DISPLAY_AS_BOT’: False, ‘PRETTY_TYPE’: ‘PNG’, ‘NAME’: ‘image.png’, ‘IS_PUBLIC’: True, ‘PREVIEW_HIGHLIGHT’: None, ‘MIMETYPE’: ‘image/png’, ‘PERMALINK_PUBLIC’: ‘https://slack-files.com/TUMKD5EGJ-F07R86AASSG-4118e5904d’, ‘FILETYPE’: ‘png’, ‘EDIT_LINK’: None, ‘URL_PRIVATE’: ‘Slack’, ‘HAS_RICH_PREVIEW’: False, ‘TITLE’: ‘image.png’, ‘IS_STARRED’: False, ‘PREVIEW_IS_TRUNCATED’: None, ‘URL_PRIVATE_DOWNLOAD’: ‘Slack’, ‘PREVIEW’: None, ‘PUBLIC_URL_SHARED’: False, ‘MESSAGE_TS’: ‘1728497854.096059’, ‘PARENT_MESSAGE_TS’: ‘1728497412.821019’, ‘MESSAGE_CHANNEL_ID’: ‘CV2UXSE9L’, ‘_FIVETRAN_DELETED’: False, ‘LINES_MORE’: None, ‘LINES’: None, ‘SIZE’: 10784, ‘_FIVETRAN_SYNCED’: ‘2024-10-13 12:43:20.683000+00:00’})