Skip to content

Prefer com.palantir.common:streams MoreIterables.partition over Iterables.partition#68

Open
schlosna wants to merge 3 commits into
developfrom
davids/partition
Open

Prefer com.palantir.common:streams MoreIterables.partition over Iterables.partition#68
schlosna wants to merge 3 commits into
developfrom
davids/partition

Conversation

@schlosna
Copy link
Copy Markdown
Contributor

@schlosna schlosna commented Jan 22, 2026

Before this PR

Uses of Guava's Iterables.partition preallocates an array of partition size, which can introduce excessive allocations in cases where the input iterable is smaller than the partition size.

After this PR

==COMMIT_MSG==
Prefer use of atlasdb-commons IterableUtils.partition over Iterables.partition

The com.palantir.common:streams MoreIterables.partition implementation avoids this and provides several optimizations when the input iterable is an ImmutableCollection to return sublists.
==COMMIT_MSG==

Possible downsides?

@changelog-app
Copy link
Copy Markdown

changelog-app Bot commented Jan 22, 2026

Generate changelog in changelog/@unreleased

Type (Select exactly one)

  • Feature (Adding new functionality)
  • Improvement (Improving existing functionality)
  • Fix (Fixing an issue with existing functionality)
  • Break (Creating a new major version by breaking public APIs)
  • Deprecation (Removing functionality in a non-breaking way)
  • Migration (Automatically moving data/functionality to a new system)

Description

Prefer use of atlasdb-commons IterableUtils.partition over Iterables.partition

The atlasdb-commons IterableUtils.partition implementation avoids this and provides several optimizations when the input iterable is an ImmutableCollection to return sublists.

Check the box to generate changelog(s)

  • Generate changelog entry

@schlosna schlosna requested a review from Copilot January 22, 2026 20:00
@changelog-app
Copy link
Copy Markdown

changelog-app Bot commented Jan 22, 2026

Successfully generated changelog entry!

Need to regenerate?

Simply interact with the changelog bot comment again to regenerate these entries.


📋Changelog Preview

💡 Improvements

  • Prefer use of atlasdb-commons IterableUtils.partition over Iterables.partition

    The atlasdb-commons IterableUtils.partition implementation avoids this and provides several optimizations when the input iterable is an ImmutableCollection to return sublists. (#68)

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@schlosna schlosna requested a review from Copilot January 22, 2026 22:51
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@schlosna schlosna requested a review from Copilot January 23, 2026 01:16
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

.build();
return fix(tree, state, "com.google.common.collect.Lists");
}
return fix(tree, state, "com.palantir.common.collect.IterableUtils");
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that atlasdb is closed-source, I'm not sure it's a good thing to put fixes using internal classes in our public error-prone packages, as this makes it less reusable for others.
Do you think we could put IterableUtils in some of our OSS packages instead? (if needed, maybe having a collection-utils repo/library could be useful?)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at landing this in streams' MoreIterables palantir/streams#653

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated to use MoreIterables.partition in streams 2.6.0+

@schlosna schlosna changed the title Prefer atlasdb-commons IterableUtils.partition over Iterables.partition Prefer com.palantir.common:streams MoreIterables.partition over Iterables.partition Jan 26, 2026
Copy link
Copy Markdown
Contributor

@aldexis aldexis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lgtm - just one nit and one question. I'll schedule a reminder to come back to this in a week to check progress of the streams 2.6.0 rollout

.build();
return fix(tree, state, "com.google.common.collect.Lists");
}
return fix(tree, state, "com.palantir.common.streams.MoreIterables");
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of curiosity, is there a way to check whether that class is on the classpath? Otherwise, let's wait a bit before merging this such that streams 2.6.0 is upgraded in more places (I'll track to come back to this in a week) - see https://pl.ntr/2zk

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@schlosna I haven't forgotten about this by the way, and have been checking every few days on the rollout progress for streams 2.6.0. It seems like we still have ~30 repos that are somehow still on 2.5.0, so I'm still a bit hesitant to merge something that will break baseline-error-prone upgrades due to failing compilation, but the number continues to go down over time. I'll be ooto for the next 10 days, so I'll check back in after I'm back

Comment on lines 71 to 76
"import static com.google.common.collect.Iterables.partition;",
"class Test {",
" void test(Iterable<String> items) {",
" // BUG: Diagnostic contains: Prefer com.palantir.common.streams.MoreIterables.partition",
" partition(items, 10);",
" }",
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: At least for new ones (but ideally for this entire file), let's use code blocks, to make it easier to read

Suggested change
"import static com.google.common.collect.Iterables.partition;",
"class Test {",
" void test(Iterable<String> items) {",
" // BUG: Diagnostic contains: Prefer com.palantir.common.streams.MoreIterables.partition",
" partition(items, 10);",
" }",
// language=Java
"""
import static com.google.common.collect.Iterables.partition;
class Test {
void test(Iterable<String> items) {
// BUG: Diagnostic contains: Prefer com.palantir.common.streams.MoreIterables.partition
partition(items, 10);
}
"""

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will update this in a bit. may create a separate 🤖 PR to migrate all the old tests

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#93

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants