Discussing Data Retention Aspect and Pruning in Datahub Project

all-things-deployment
1

Original Slack Thread

re: data retention.
i noticed that deploying 0.10.3 inserted a datahubretention aspect into the mysql DB with a value of 20 for all aspects and entities.
https://datahubproject.io/docs/advanced/db-retention/#how-to-configure
I noticed that my GMS didnt have ENTITY_SERVICE_ENABLE_RETENTION=true as an environmental variable, yet it was still pruning my entries

whats the best way to disable pruning? remove the aspect?

2

i hard deleted the entry using the delete cli in the end

3

<@U03MF8MU5P0> might be able to speak to this!

4

As of <Blaming datahub/metadata-service/configuration/src/main/resources/application.yml at 7b443e559e82760b31172866fcdbe1fb3fe9a5c8 · datahub-project/datahub · GitHub a year ago>, the default value for entity retention enable is true. You’ll need to set ENTITY_SERVICE_ENABLE_RETENTION=false in order to disable retention. I’ll create a https://github.com/datahub-project/datahub/pull/8797|PR to update the documentation with this information.

5

thanks for confirming and updating the docs!