Skip to content

Commit 74e8981

Browse files
authored
GH-709: Correct length calculation of value buffers of variable-sized arrays (#707)
## What's Changed For variable-size binary layout arrays, BufferImportTypeVisitor currently derives the length of the value buffer by calculating the difference between the last and first offset. When the first offset is not zero, this is actually incorrect and leads to out of bounds errors when attempting to read values from the imported array. Instead, BufferImportTypeVisitor should simply use the last offset value as the length of the value buffer. This PR makes that change. Just FYI, I bumped into this issue when attempting to import an array originating from DataFusion. A test query of the form `SELECT column1 FROM VALUES ('a'), ('b'), ('c'), ('d') LIMIT 2 OFFSET 1;` returns a slice of the full set of values. The values buffer contains all the original values, and the offsets buffer contains 1 and 2 as values to handle the offset from the query. Closes #709.
1 parent 3e135f4 commit 74e8981

File tree

1 file changed

+4
-8
lines changed

1 file changed

+4
-8
lines changed

c/src/main/java/org/apache/arrow/c/BufferImportTypeVisitor.java

+4-8
Original file line numberDiff line numberDiff line change
@@ -228,9 +228,8 @@ public List<ArrowBuf> visit(ArrowType.Utf8 type) {
228228
type,
229229
start,
230230
end);
231-
final int len = end - start;
232231
offsets.getReferenceManager().retain();
233-
return Arrays.asList(maybeImportBitmap(type), offsets, importData(type, len));
232+
return Arrays.asList(maybeImportBitmap(type), offsets, importData(type, end));
234233
}
235234
}
236235

@@ -279,9 +278,8 @@ public List<ArrowBuf> visit(ArrowType.LargeUtf8 type) {
279278
type,
280279
start,
281280
end);
282-
final long len = end - start;
283281
offsets.getReferenceManager().retain();
284-
return Arrays.asList(maybeImportBitmap(type), offsets, importData(type, len));
282+
return Arrays.asList(maybeImportBitmap(type), offsets, importData(type, end));
285283
}
286284
}
287285

@@ -296,9 +294,8 @@ public List<ArrowBuf> visit(ArrowType.Binary type) {
296294
type,
297295
start,
298296
end);
299-
final int len = end - start;
300297
offsets.getReferenceManager().retain();
301-
return Arrays.asList(maybeImportBitmap(type), offsets, importData(type, len));
298+
return Arrays.asList(maybeImportBitmap(type), offsets, importData(type, end));
302299
}
303300
}
304301

@@ -320,9 +317,8 @@ public List<ArrowBuf> visit(ArrowType.LargeBinary type) {
320317
type,
321318
start,
322319
end);
323-
final long len = end - start;
324320
offsets.getReferenceManager().retain();
325-
return Arrays.asList(maybeImportBitmap(type), offsets, importData(type, len));
321+
return Arrays.asList(maybeImportBitmap(type), offsets, importData(type, end));
326322
}
327323
}
328324

0 commit comments

Comments
 (0)