Differentiating between Batch Ingestion and Introspection in System Metadata Filter Stability

Original Slack Thread

Also need suggestions about the stability of the filter created on the basis of run Id in system metadata. We want to differentiate between batch ingestion and introspection.

<@UV5UEC3LN> might be able to speak to this!

Not really sure what this means, <@U04N9PYJBEW> this sounds like something on the actions framework side?

Correct we are filtering event envelope that comes from actions framework

Can you elaborate on “introspection”? Are you trying to differentiate between aspects created by batch ingestion vs stuff manually added via the UI or SDK?

<@U04N9PYJBEW> exactly! That is what we are looking for. Currently we found out that MCL that got created by ingestion has the run id but not for the stuff added via UI or SDK. Is there any better way to differentiate between them?

Yeah, runId looks like the best to me. Note that if you create an aspect with one runId, and then emit the same aspect again (with contents exactly the same), it will update lastRunId in system metadata to the latter run’s run id, but retain the original run’s run id in runId.