Hi all,
Has anyone used Ingestion with profiling on the S3?
As my folder (Table) has several files, this has generated several entities, and I would like to unify them into a single object.
source:
type: s3
config:
path_specs:
-
include: ‘s3://bucket/table_folder/*.*’
table_name: kinesis_folder
aws_config:
aws_region: eu-central-1
aws_role: ‘arn:aws:iam:acc:role/s3_data_catalog_read_only’
profiling:
enabled: true
<@UV14447EU> might be able to help here!
<@U068C8YJLBY> please, can you change the include to:
include: '<s3://bucket/{table}/*.*>'
This way it won’t ingest individual files.