Troubleshooting Issue with Displaying Entities on the Search Page

troubleshoot
1

Original Slack Thread

Hey everyone. I could use some direction in troubleshooting an issue with displaying entities on the search page. In the navigation bar, I can see entities listed but an open search (*) results in no results found. Any idea what I should look at to find the root cause?

attachment
attachment2076×878 70.6 KB

2

I’ve experienced this going from 0.10.4 to 0.12.1 (also experienced in 0.12.0)

3

Hey there! :wave: Make sure your message includes the following information if relevant, so we can help more effectively!

  1. Which DataHub version are you using? (e.g. 0.12.0)
  2. Please post any relevant error logs on the thread!
4

The system update and reindexing jobs have been completed. I don’t see any error logs

5

I tried turning off BrowseV2 with SHOW_BROWSE_V2=false and was able to see entities through the classic browser but the search page continued to show nothing

6

Hey Andrew - hmm sorry to hear about this

7

To clarify, did searching work in the past & then an upgrade caused it to stop working? Or did you never get it to work?

8

Great question. Between this being a dev environment and the old browse UI being the default, I can’t say for sure. I know our prod deployment’s search is working on the older version

9

I’ve rolled back to 0.10.4 and can confirm that search works on that version

10

Ok got it. Let me think about this

11

A couple other notes that may be useful:
• We’re using Docker Compose to deploy Datahub. I’m wondering if there may be a new env var that we’ve missed. I plan to look through changes to the Helm Chart as part of my troubleshooting
• ACL groups aren’t coming in from our OIDC provider (Okta). We don’t have any RBAC in place that would require the groups to sync over and the groups weren’t coming through in our previous (working) version

12

A new twist! I tried the path /browse/dataset which isn’t a link available on the home page anymore. That path still works and there’s data available there

13

<@U01JUBAPZPS> I stepped away from the problem for a while and have come back to play with the GraphQL. This has brought me to an interesting finding that might help us find the solution:
The query getSearchResultsForMultiple is used to populate the search results on the /search endpoint. The unfiltered query uses the below variables as inputs.

  "input":
  {
    "types":[],
    "query":"*",
    "start":0,
    "count":10,
    "searchFlags":{"getSuggestions":true}
  }
}```
With these variables, I receive no results (`"total": 0`).
If I remove the line `"searchFlags":{"getSuggestions":true}` and try the query again, I receive the expected results (`"total": 10000`). From the search UI, I don't see an option to enable/disable the `getSuggestions` flag
14

<datahub/datahub-web-react/src/app/search/SearchPage.tsx at 16d3df620f07c4d41118be9c8f38dc0cf46df76f · datahub-project/datahub · GitHub at the code>, I see that getSuggestions: true is hardcoded on the page

15

I’ve figured out how to turn on and view debug logs. Doing that has given me the exact query that’s submitted to ES which has allowed me to try querying ES directly. When submitting the logged query to ES, it returns results as expected

16

I’m not sure what virtual fields are in ES but see references to a virtual field called _entityName. Should _entityName be found in the documents returned by ES? I don’t see that field anywhere

17

Final update (with resolution!): Mappings needed an update. That update is only applied if the env var ELASTICSEARCH_INDEX_BUILDER_MAPPINGS_REINDEX=true is set. Once I set that and re-ran SystemUpdate, search results began to load :slightly_smiling_face: