Unable to delta_scan for delta lake on GCS

[yohei1126]

What happens?

delta_scan tried to access S3 bucket and failed even if I give GCS bucket file path.

To Reproduce

Insert GCS HMAC key https://duckdb.org/docs/guides/network_cloud_storage/gcs_import.html

$ INSERT INTO duckdb.secrets (type, key_id, secret)
  VALUES ('GCS', 'xx'x, 'yyy');
INSERT 0 1

# SELECT * FROM  delta_scan ('gs://databricks-xxx-unitycatalog/yyy') AS (color text,  price numeric);
ERROR:  (PGDuckDB/CreatePlan) Prepared query returned an error: 'IO Error: Hit DeltaKernel FFI error 
(from: While trying to read from delta table: 'gs://databricks-xxx-unitycatalog/yyy/'): Hit error: 8 (ObjectStoreError)
with message (Error interacting with object store: Generic S3 error: Error after 10 retries in 2.43329041s, 
max_retries:10, retry_timeout:180s, source:error sending request for url (https://s3..amazonaws.com/databricks-xxxx-unitycatalog/yyy/_delta_log/_last_checkpoint))

OS:

MacOS

pg_duckdb Version (if built from source use commit hash):

0.2.0

Postgres Version (if built from source use commit hash):

16

Hardware:

No response

Full Name:

Yohei

Affiliation:

Onishi

What is the latest build you tested with? If possible, we recommend testing with the latest nightly build.

I have tested with a stable release

Did you include all relevant data sets for reproducing the issue?

No - I cannot share the data sets because they are confidential

Did you include all code required to reproduce the issue?

• Yes, I have

Did you include all relevant configuration (e.g., CPU architecture, Linux distribution) to reproduce the issue?

• Yes, I have

[Y--]

Hi @yohei1126! Thanks for the report. Just to double check, does the same work on vanilla DuckDB? (using the CLI for example?) Thanks!

[yohei1126]

@Y-- Hi, delta_scan work on vanilla DuckDB
https://duckdb.org/docs/extensions/delta.html