Making an array of Objects searchable within `MdTest` in PDLs and advice needed

troubleshoot
1

Original Slack Thread

Hi All!

I have an array of MdChannels within MdTest . MdChannels has an object called mdChannelMetadata. I am unable to make mdChannelMetadata searchable. Can someone have a look at the PDLs and please advise?

Note : - mdTestMetadata as well as mdChannelName are searchable. Basically how to make an array of Objects searchable is the question.

MdTest.pdl


@Aspect = {
  "name": "mdTest",
  "autoRender": true,
  "renderSpec": {
    "displayType": "tabular", // or properties
    "key": "mdTest",
    "displayName": "MdTest"
  }
}

record MdTest {
  
  @Searchable = {
        "fieldType": "KEYWORD",
        "fieldName": "mdTestName"
    }
  mdTestName: string


  mdChannels: array[MdChannel]


  @Searchable = {
    "/*": {
      "queryByDefault": true,
      "fieldType": "OBJECT"
    }
  }
  mdTestMetadata: map[string, string]
}```


MdChannel.pdl
```namespace com.mycompany.dq

record MdChannel {
  @Searchable = {
        "fieldType": "KEYWORD",
        "fieldName": "mdChannelName"
    }
  mdChannelName: string

    @Searchable = {
      "/*": {
        "queryByDefault": true,
        "fieldType": "OBJECT"
      }
    }
    mdChannelMetadata: map[string, string]
}```
cc <@U06CMSR2S1E> <@U02RXEEJQBH>
2

Hey there! :wave: Make sure your message includes the following information if relevant, so we can help more effectively!

  1. Which DataHub version are you using? (e.g. 0.12.0)
  2. Please post any relevant error logs on the thread!
3

DataHub version = 0.13.1

4

CFBR

5

ESMapping is showing mdChannelMetadata as of type text. Is it possible to have Object inside an array, will they get indexed properly in ES.

6

Have you tried it using the override pattern?: https://datahubproject.io/docs/metadata-modeling/extending-the-metadata-model/#annotating-collections--annotation-overrides

7

<@UV5UEC3LN> Firstly, thanks for sending the docs!
Below is sample data for our pdls. Basically with the above PDL structure, we can query mdTestName, mdTestMetadata and mdChannelName but not properties like “weather” or “foobar” within mdChannelMetadata . We are unable to understand few things from above documentation.

Could you kindly help with below qns:

  1. Is @Searchable required in front of mdChannels array in MdTest.pdl?
  2. Is “fieldName” required within @Searchable annotation?
  3. What is the “/*” path that we give in the annotation? In our case, what should be the path in the @Searchable?
  4. Should we define @Relationship also for mdChannels array?
  "mdTestName": "test-test",
  "mdChannels": [
    {
      "mdChannelName": "test-channel-1",
      "mdChannelMetadata": {
        "foobar": "sunny"
      }
    },
    {
      "mdChannelName": "test-channel-2",
      "mdChannelMetadata": {
        "foobar": "sunny"
      }
    }
  ],
  "mdTestMetadata": {
    "weather": "sunny",
    "foo": "bar"
  }
}```