How to Create a Single Dataset When Running Ingestions by Recipe

getting-started
1

Original Slack Thread

Hi guys, Can I skip to make containers (or folders) when I run ingestions by recipe?
For examples,

When I set like

table_name: ‘bucket_name/service/name={name}/{table}```
Ingestion creates containers of each date_id so that multiple datasets are created.

I’d like to create only one dataset. Is this possible?
2

<@U01GZEETMEZ> could you confirm if this is possible?

3

Yup it’s possible - you’ll need to use the partition_key pattern in the path spec. We have some examples here https://datahubproject.io/docs/generated/ingestion/sources/s3/#example-3---folder-of-files-as-dataset-with-partitions