Skip to content

cover mapping for INT8/INT16 and isSigned(INT32) to Integer and Unsig… #71

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft

cover mapping for INT8/INT16 and isSigned(INT32) to Integer and Unsig… #71

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 13 additions & 3 deletions ice/src/main/java/org/apache/iceberg/parquet/MessageTypeToType.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -226,11 +226,21 @@ public Optional<Type> visit(LogicalTypeAnnotation.IntLogicalTypeAnnotation intTy
intType.isSigned() || intType.getBitWidth() < 64,
"Cannot use uint64: not a supported Java type");
*/
if (intType.getBitWidth() < 32) {
return Optional.of(Types.IntegerType.get());
} else if (intType.getBitWidth() == 32 && intType.isSigned()) {

// Preserve logical type information by choosing appropriate Iceberg types
// that can represent the full range of the original type
int bitWidth = intType.getBitWidth();
boolean isSigned = intType.isSigned();

// Determine the appropriate Iceberg type based on bit width and signedness
if (bitWidth <= 16) {
// INT8/INT16, UINT8/UINT16 all fit in INT32
return Optional.of(Types.IntegerType.get());
} else if (bitWidth <= 32) {
// INT32 fits in INT32, UINT32 needs INT64 to avoid overflow
return Optional.of(isSigned ? Types.IntegerType.get() : Types.LongType.get());
} else {
// INT64 needs INT64, UINT64 - best effort with INT64, may have range issues
return Optional.of(Types.LongType.get());
}
}
Expand Down