Skip to content

Conversation

@subkanthi
Copy link
Collaborator

… files.

@subkanthi subkanthi linked an issue Oct 15, 2025 that may be closed by this pull request
@subkanthi
Copy link
Collaborator Author

Testing:
local file

 describe-parquet iris.parquet -a
2025-10-28 19:00:00 [2119751-100] INFO c.a.i.r.c.i.r.RESTCatalogServlet > @token:anonymous GET v1/config
---
summary:
  rows: 150
  rowGroups: 1
  compressedSize: 1885
  uncompressedSize: 0
  createdBy: "DuckDB"
  columnCount: 5
columns:
- name: "sepal.length"
  type: "DOUBLE"
  repetition: "OPTIONAL"
- name: "sepal.width"
  type: "DOUBLE"
  repetition: "OPTIONAL"
- name: "petal.length"
  type: "DOUBLE"
  repetition: "OPTIONAL"
- name: "petal.width"
  type: "DOUBLE"
  repetition: "OPTIONAL"
- name: "variety"
  type: "BINARY"
  repetition: "OPTIONAL"
  logicalType: "STRING"
rowGroups:

@subkanthi
Copy link
Collaborator Author

remote file

describe-parquet   https://d37ci6vzurychx.cloudfront.net/trip-data/yellow_tripdata_2025-01.parquet
2025-11-06 20:34:28 [12843412-49] INFO c.a.i.r.c.i.r.RESTCatalogServlet > @token:anonymous GET v1/config
---
summary:
  rows: 3475226
  rowGroups: 4
  compressedSize: 59138125
  uncompressedSize: 92602414
  createdBy: "parquet-cpp-arrow version 16.1.0"
  columnCount: 20

@subkanthi
Copy link
Collaborator Author

 describe-parquet -a   s3://aws-public-blockchain/v1.0/btc/transactions/date=2025-01-01/part-00000-33e8d075-2099-409b-a806-68dd17217d39-c000.snappy.parquet
2025-11-06 20:41:15 [12843412-78] INFO c.a.i.r.c.i.r.RESTCatalogServlet > @token:anonymous GET v1/config
---
summary:
  rows: 292213
  rowGroups: 5
  compressedSize: 539840873
  uncompressedSize: 717049859
  createdBy: "parquet-mr version 1.10.1 (build 65f31597b18a0f2718a129fd2d69af0168952c55)"
  columnCount: 19
columns:
- name: "hash"
  type: "BINARY"
  repetition: "OPTIONAL"
  logicalType: "STRING"
- name: "version"
  type: "INT64"
  repetition: "OPTIONAL"
- name: "size"
- ```

@subkanthi subkanthi marked this pull request as ready for review November 7, 2025 02:42
@subkanthi subkanthi requested a review from shyiko November 7, 2025 02:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Provide convenient introspection of Parquet files

2 participants