Getting below exception when running stateful ingestion from CLI
raise PipelineInitError(f"Failed to {step}: {e}") from e
datahub.ingestion.run.pipeline.PipelineInitError: Failed to configure the source (mysql): pipeline_name must be provided if stateful ingestion is enabled.```
The error you’re encountering indicates that the pipeline_name is missing in your configuration file, which is required when stateful ingestion is enabled. The pipeline_name is a mandatory field that uniquely identifies the ingestion pipeline and is used to save and retrieve checkpoint states.
Hey there! Make sure your message includes the following information if relevant, so we can help more effectively!
Are you using UI or CLI for ingestion?
Which DataHub version are you using? (e.g. 0.12.0)
What data source(s) are you integrating with DataHub? (e.g. BigQuery)
<@U06TM7M40RK> Can you help with the questions in this thread?
We’re experimenting with a new AI assistant for questions in the <#CUMUWQU66> channel to help you track down helpful resources more quickly. Keep in mind that responses may not always be spot-on; feel free to ask it follow-up questions and don’t forget to provide its responses a or so other folks in the Community know if you’re still looking for help on the topic!