Add scope based feature flags #4812

adinauer · 2025-10-16T10:59:28Z

📜 Description

addFeatureFlag can be used to track feature flag evaluations and have them show up on errors in Sentry

💡 Motivation and Context

Scope based part of #4763

💚 How did you test it?

Manually + E2E tests + unit tests

📝 Checklist

I added GH Issue ID & Linear ID
I added tests to verify the changes.
No new PII added or SDK only sends newly added PII if sendDefaultPII is enabled.
I updated the docs if needed.
I updated the wizard if needed.
Review from the native team if needed.
No breaking change or entry added to the changelog.
No breaking change for hybrid SDKs or communicated to hybrid SDKs.

🔮 Next steps

Test (de)serialization
Add maxFeatureFlags to external options (sentry.properties)
Add span based API
Make params on addFeatureFlag nullable

github-actions · 2025-10-16T10:59:58Z

	Messages
📖	Do not forget to update Sentry-docs with your feature once the pull request gets approved.

Generated by 🚫 dangerJS against 7c4449a

sentry/src/main/java/io/sentry/IScope.java

github-actions · 2025-10-16T11:21:53Z

Performance metrics 🚀

	Plain	With Sentry	Diff
Startup time	306.06 ms	368.79 ms	62.73 ms
Size	1.58 MiB	2.12 MiB	555.24 KiB

Baseline results on branch: main

Startup times

Revision	Plain	With Sentry	Diff
`806307f`	357.85 ms	424.64 ms	66.79 ms
`ee747ae`	554.98 ms	611.50 ms	56.52 ms
`17a0955`	372.53 ms	446.70 ms	74.17 ms
`2124a46`	319.19 ms	415.04 ms	95.85 ms
`9fbb112`	404.51 ms	475.65 ms	71.14 ms
`b6702b0`	395.86 ms	409.98 ms	14.12 ms
`ee747ae`	357.79 ms	421.84 ms	64.05 ms
`27d7cf8`	314.17 ms	347.00 ms	32.83 ms
`9fbb112`	361.43 ms	427.57 ms	66.14 ms
`96449e8`	361.30 ms	423.39 ms	62.09 ms

App size

Revision	Plain	With Sentry	Diff
`806307f`	1.58 MiB	2.10 MiB	533.42 KiB
`ee747ae`	1.58 MiB	2.10 MiB	530.95 KiB
`17a0955`	1.58 MiB	2.10 MiB	533.20 KiB
`2124a46`	1.58 MiB	2.12 MiB	551.51 KiB
`9fbb112`	1.58 MiB	2.11 MiB	539.18 KiB
`b6702b0`	1.58 MiB	2.12 MiB	551.79 KiB
`ee747ae`	1.58 MiB	2.10 MiB	530.95 KiB
`27d7cf8`	1.58 MiB	2.12 MiB	549.42 KiB
`9fbb112`	1.58 MiB	2.11 MiB	539.18 KiB
`96449e8`	1.58 MiB	2.11 MiB	539.35 KiB

Previous results on branch: feat/feature-flags-on-scope

Startup times

Revision	Plain	With Sentry	Diff
`79c6335`	342.31 ms	399.16 ms	56.85 ms
`0983aa0`	369.83 ms	412.26 ms	42.43 ms
`3128a6f`	319.66 ms	386.50 ms	66.84 ms
`8dfdb03`	349.64 ms	423.82 ms	74.18 ms

App size

Revision	Plain	With Sentry	Diff
`79c6335`	1.58 MiB	2.12 MiB	553.10 KiB
`0983aa0`	1.58 MiB	2.11 MiB	540.70 KiB
`3128a6f`	1.58 MiB	2.12 MiB	553.11 KiB
`8dfdb03`	1.58 MiB	2.12 MiB	549.53 KiB

romtsn · 2025-10-16T12:14:16Z

sentry/src/main/java/io/sentry/featureflags/FeatureFlagBuffer.java

+        tmpList.remove(0);
+      }
+
+      flags = new CopyOnWriteArrayList<>(tmpList);


Does it need to be CopyOnWrite if we never actually call flags.add()?

I opted for CopyOnWriteArrayList due to it being a good choice for scope cloning without affecting parent scopes.

Gotta think about whether using something else would work too, probably yes but what's the benefit?

Should be possible to replace with ArrayList but then we must never modify the collection. I was still hoping for some enlightenment on how to speed up the add method. I've written this down and depending on how the add method turns out, we can replace it.

We could in theory wrap it with Collections.unmodifyableList but I'd rather send corrupted data as opposed to crashing the customers application.

but then we must never modify the collection.

why is that? sorry for the questions I guess i'm missing some context 😅

Discussed in more detail internally.

We currently optimized feature flags storage for scope cloning performance (where CopyOnWriteArrayList simply re-uses the internal array, when cloning the list, which is O(1)).
Merging was our second priority.
Add is currently rather inefficient.

We wanna release as is and if customers request better performance, we can revisit and create an Android specific implementation, that prioritizes differently and/or optimizes for fewer allocations.

sentry/src/main/java/io/sentry/SentryOptions.java

lcian · 2025-10-17T12:50:15Z

sentry/src/main/java/io/sentry/protocol/FeatureFlag.java

+  private @NotNull String flag;
+
+  /** Evaluation result of the feature flag. */
+  private boolean result;


Flag evaluation is allowed to be an arbitrary JSON according to Relay https://github.com/getsentry/relay/blob/21c2e18ebe6f834a4ce4e149c6a43e4bec1e90f8/relay-event-schema/src/protocol/contexts/flags.rs#L30
However in docs and frontend this is typed as bool.
I think it will be easy to extend this with an overload that takes Object or we just change this to take it into account from the get go.
Let's also double check with @cmanallen on how to proceed here.

Relay is set up to accommodate remote-configs/feature-flags of any JSON type. The UI currently expects boolean though. I'm not sure how resilient it is to non-boolean types. The goal is to expand our UI to accommodate more types in the future but AFAIK that's not been realized yet. Something I can find out later today.

@cmanallen can we release this as is with Boolean param exposed to customers?

Checked internally, Boolean is OK for now.

lcian · 2025-10-17T13:12:34Z

sentry/src/main/java/io/sentry/featureflags/FeatureFlagBuffer.java

+      final int size = flags.size();
+      final @NotNull ArrayList<FeatureFlagEntry> tmpList = new ArrayList<>(size + 1);
+      for (FeatureFlagEntry entry : flags) {
+        if (!entry.flag.equals(flag)) {
+          tmpList.add(entry);
+        }
+      }
+      tmpList.add(new FeatureFlagEntry(flag, result, System.nanoTime()));
+
+      if (tmpList.size() > maxSize) {
+        tmpList.remove(0);
+      }
+
+      flags = new CopyOnWriteArrayList<>(tmpList);


Suggested change

final int size = flags.size();

final @NotNull ArrayList<FeatureFlagEntry> tmpList = new ArrayList<>(size + 1);

for (FeatureFlagEntry entry : flags) {

if (!entry.flag.equals(flag)) {

tmpList.add(entry);

}

}

tmpList.add(new FeatureFlagEntry(flag, result, System.nanoTime()));

if (tmpList.size() > maxSize) {

tmpList.remove(0);

}

flags = new CopyOnWriteArrayList<>(tmpList);

final int size = flags.size();

for (int i = 0; i < size; i++) {

if (flags.get(i).equals(flag)) {

flags.remove(i);

break;

}

}

flags.add(new FeatureFlagEntry(flag, result, System.nanoTime()));

if (flags.size() > maxSize) {

flags.remove(0);

}

Maybe this is better?

Nice!
Yes, it's better, see #4812 (comment) and #4812 (comment)
This is called "Mutable CopyOnWriteArrayList" in the tables.

adinauer · 2025-10-20T13:38:55Z

Table compares ops/s for add method of different implementations

Implementation	Unique	Duplicates	Duplicates in Full Buffer
Immutable CopyOnWriteArrayList	74,071.034	363,250.814	29,828.637
Immutable ArrayList	67,329.692	425,473.814	26,404.218
Mutable CopyOnWriteArrayList	146,023.283	155,988.324	51,812.315

Immutable CopyOnWriteArrayList = current state in this PR, where we create a temporary ArrayList for creating the new state

            final int size = flags.size();
            final @NotNull ArrayList<FeatureFlagEntry> tmpList = new ArrayList<>(size + 1);
            for (FeatureFlagEntry entry : flags) {
                if (!entry.flag.equals(flag)) {
                    tmpList.add(entry);
                }
            }
            tmpList.add(new FeatureFlagEntry(flag, result, System.nanoTime()));

            if (tmpList.size() > maxSize) {
                tmpList.remove(0);
            }

            flags = new CopyOnWriteArrayList<>(tmpList);

Immutable ArrayList = replaced CopyOnWriteArrayList with ArrayList which we replace in add

            final int size = flags.size();
            final @NotNull ArrayList<FeatureFlagEntry> tmpList = new ArrayList<>(size + 1);
            for (FeatureFlagEntry entry : flags) {
                if (!entry.flag.equals(flag)) {
                    tmpList.add(entry);
                }
            }
            tmpList.add(new FeatureFlagEntry(flag, result, System.nanoTime()));

            if (tmpList.size() > maxSize) {
                tmpList.remove(0);
            }

            flags = tmpList;

Mutable CopyOnWriteArrayList = iterate flags, remove existing entry if found

            final int size = flags.size();
            for (int i = 0; i < size; i++) {
                if (flags.get(i).equals(flag)) {
                    flags.remove(i);
                    break;
                }
            }
            flags.add(new FeatureFlagEntry(flag, result, System.nanoTime()));

            if (flags.size() > maxSize) {
                flags.remove(0);
            }

Here's the code I used to benchmark:

  @Benchmark
  public void addUniqueFeatureFlag(Blackhole blackhole) throws InterruptedException {
    IFeatureFlagBuffer buffer = FeatureFlagBufferCOWAL.create(100);

    for (int i = 0; i < 100; i++) {
      buffer.add("unique_" + i, true);
    }

    blackhole.consume(buffer);
  }

  @Benchmark
  public void addNonUniqueFeatureFlag(Blackhole blackhole) throws InterruptedException {
    IFeatureFlagBuffer buffer = FeatureFlagBufferCOWAL.create(100);

    for (int i = 0; i < 100; i++) {
      buffer.add("unique_" + 1, true);
    }

    blackhole.consume(buffer);
  }

  @Benchmark
  public void addNonUniqueToFullFeatureFlagBuffer(Blackhole blackhole) throws InterruptedException {
    IFeatureFlagBuffer buffer = FeatureFlagBufferCOWAL.create(100);

    for (int i = 0; i < 100; i++) {
      buffer.add("unique_" + i, true);
    }

    for (int i = 0; i < 100; i++) {
      buffer.add("unique_" + 1, true);
    }

    blackhole.consume(buffer);
  }

adinauer · 2025-10-20T14:39:42Z

Reran benchmarks for a (hopefully) more realistic scenario

NOTE: I just updated these after fixing a bug with the duplicate check

Implementation	Realistic Usage
Immutable CopyOnWriteArrayList	100,522.600
Immutable ArrayList	89,760.930
Mutable CopyOnWriteArrayList	135,685.613
Mutable CopyOnWriteArrayList (reverse)	133,285.356

  @Benchmark
  public void realisticFeatureFlagUsage(Blackhole blackhole) throws InterruptedException {
    IFeatureFlagBuffer buffer = FeatureFlagBufferCOWAL.create(100);

    for (int i = 0; i < 50; i++) {
      buffer.add("flag_" + i, i % 2 == 0);
    }

    int[] duplicateIndices = {25, 0, 49, 12, 37, 5, 44, 18, 31, 2, 47, 8, 41, 15, 28, 22, 35, 10, 39, 20};
    for (int i = 0; i < 20; i++) {
      buffer.add("flag_" + duplicateIndices[i], true);
    }

    for (int i = 50; i < 70; i++) {
      buffer.add("flag_" + i, i % 2 == 0);
    }

    blackhole.consume(buffer);
  }

Added a new implementation which looks for the duplicate flag in reverse order:

            final int size = flags.size();
            for (int i = size - 1; i >= 0; i--) {
        		final @NotNull FeatureFlagEntry entry = flags.get(i);
        		if (entry.flag.equals(flag)) {
                    flags.remove(i);
                    break;
                }
            }
            flags.add(new FeatureFlagEntry(flag, result, System.nanoTime()));

            if (flags.size() > maxSize) {
                flags.remove(0);
            }

Co-authored-by: Lorenzo Cian <[email protected]>

cursor · 2025-11-05T14:18:26Z

sentry/src/main/java/io/sentry/featureflags/FeatureFlagBuffer.java

+      featureFlags.add(entry.toFeatureFlag());
+    }
+    return new FeatureFlags(featureFlags);
+  }


Bug: Return null when feature flags are empty

The getFeatureFlags() method always returns a non-null FeatureFlags object even when the buffer is empty (containing zero flags). This causes unnecessary bloat in event payloads because an empty FeatureFlags object with an empty list will be added to events even when no feature flags have been recorded. The method should return null when the flags list is empty to avoid adding empty FeatureFlags objects to events, similar to how NoOpFeatureFlagBuffer.getFeatureFlags() returns null.

cursor

Bug: Contexts Cloning: FeatureFlags State Shared

The Contexts copy constructor doesn't handle FeatureFlags type when cloning contexts. All other context types like App, Browser, Device, Spring, etc. have explicit copy constructor handling, but FeatureFlags is missing. This causes FeatureFlags to fall through to the else branch where it gets shallow copied instead of deep copied, potentially leading to shared mutable state between the original and cloned contexts.

sentry/src/main/java/io/sentry/protocol/Contexts.java#L39-L72

sentry-java/sentry/src/main/java/io/sentry/protocol/Contexts.java

Lines 39 to 72 in 7c4449a

    
           public Contexts(final @NotNull Contexts contexts) { 
        
             for (final Map.Entry<String, Object> entry : contexts.entrySet()) { 
        
               if (entry != null) { 
        
                 final Object value = entry.getValue(); 
        
                 if (App.TYPE.equals(entry.getKey()) && value instanceof App) { 
        
                   this.setApp(new App((App) value)); 
        
                 } else if (Browser.TYPE.equals(entry.getKey()) && value instanceof Browser) { 
        
                   this.setBrowser(new Browser((Browser) value)); 
        
                 } else if (Device.TYPE.equals(entry.getKey()) && value instanceof Device) { 
        
                   this.setDevice(new Device((Device) value)); 
        
                 } else if (OperatingSystem.TYPE.equals(entry.getKey()) 
        
                     && value instanceof OperatingSystem) { 
        
                   this.setOperatingSystem(new OperatingSystem((OperatingSystem) value)); 
        
                 } else if (SentryRuntime.TYPE.equals(entry.getKey()) && value instanceof SentryRuntime) { 
        
                   this.setRuntime(new SentryRuntime((SentryRuntime) value)); 
        
                 } else if (Feedback.TYPE.equals(entry.getKey()) && value instanceof Feedback) { 
        
                   this.setFeedback(new Feedback((Feedback) value)); 
        
                 } else if (Gpu.TYPE.equals(entry.getKey()) && value instanceof Gpu) { 
        
                   this.setGpu(new Gpu((Gpu) value)); 
        
                 } else if (SpanContext.TYPE.equals(entry.getKey()) && value instanceof SpanContext) { 
        
                   this.setTrace(new SpanContext((SpanContext) value)); 
        
                 } else if (ProfileContext.TYPE.equals(entry.getKey()) && value instanceof ProfileContext) { 
        
                   this.setProfile(new ProfileContext((ProfileContext) value)); 
        
                 } else if (Response.TYPE.equals(entry.getKey()) && value instanceof Response) { 
        
                   this.setResponse(new Response((Response) value)); 
        
                 } else if (Spring.TYPE.equals(entry.getKey()) && value instanceof Spring) { 
        
                   this.setSpring(new Spring((Spring) value)); 
        
                 } else { 
        
                   this.put(entry.getKey(), value); 
        
                 } 
        
               } 
        
             } 
        
           }

cursor · 2025-11-11T13:14:42Z

sentry/src/main/java/io/sentry/protocol/FeatureFlags.java

+  FeatureFlags(final @NotNull FeatureFlags featureFlags) {
+    this.values = featureFlags.values;
+    this.unknown = CollectionUtils.newConcurrentHashMap(featureFlags.unknown);
+  }


Bug: Shallow Copy Breaks FeatureFlags Isolation

The FeatureFlags copy constructor performs a shallow copy by directly assigning this.values = featureFlags.values, causing both the original and copied FeatureFlags instances to share the same List object. Modifications to the list in one instance will be visible in the other, breaking the expected isolation between copied objects. Should create a new list instance containing the same elements.

Add scope based feature flags

5b475d1

adinauer requested review from lcian, markushi, romtsn and stefanosiano as code owners October 16, 2025 10:59

This comment was marked as outdated.

Sign in to view

adinauer commented Oct 16, 2025

View reviewed changes

sentry/src/main/java/io/sentry/IScope.java Outdated Show resolved Hide resolved

fix equals

74cc427

This comment was marked as outdated.

Sign in to view

romtsn reviewed Oct 16, 2025

View reviewed changes

adinauer added 6 commits October 17, 2025 13:51

add serialization tests

fb42140

Create new reference on buffer clone

045337a

add comment explaining the merge method

c1061ca

optimize merge method

74acb10

make flag and result nullable params

eeae5c6

handle empty/noop buffer on merge and add tests for it

21806f2

lcian reviewed Oct 17, 2025

View reviewed changes

adinauer added 2 commits October 21, 2025 08:10

optimize add method

95550ec

format; fix test

865ad5d

This comment was marked as outdated.

Sign in to view

adinauer and others added 4 commits October 21, 2025 09:42

Fix duplicate check

a5c7aef

Update sentry/src/main/java/io/sentry/SentryOptions.java

0ab241e

Co-authored-by: Lorenzo Cian <[email protected]>

Merge branch 'main' into feat/feature-flags-on-scope

4b757b8

Format code

ab1a146

lcian approved these changes Oct 21, 2025

View reviewed changes

Merge branch 'main' into feat/feature-flags-on-scope

1cbda53

cursor bot reviewed Nov 5, 2025

View reviewed changes

adinauer and others added 4 commits November 6, 2025 09:56

changelog

15544cd

nullable getFeatureFlags

9ac80b5

Merge branch 'main' into feat/feature-flags-on-scope

839d878

Merge branch 'main' into feat/feature-flags-on-scope

7c4449a

cursor bot reviewed Nov 11, 2025

View reviewed changes

adinauer merged commit 5d8a2df into main Nov 11, 2025
60 of 62 checks passed

adinauer deleted the feat/feature-flags-on-scope branch November 11, 2025 13:31


	public Contexts(final @NotNull Contexts contexts) {
	for (final Map.Entry<String, Object> entry : contexts.entrySet()) {
	if (entry != null) {
	final Object value = entry.getValue();
	if (App.TYPE.equals(entry.getKey()) && value instanceof App) {
	this.setApp(new App((App) value));
	} else if (Browser.TYPE.equals(entry.getKey()) && value instanceof Browser) {
	this.setBrowser(new Browser((Browser) value));
	} else if (Device.TYPE.equals(entry.getKey()) && value instanceof Device) {
	this.setDevice(new Device((Device) value));
	} else if (OperatingSystem.TYPE.equals(entry.getKey())
	&& value instanceof OperatingSystem) {
	this.setOperatingSystem(new OperatingSystem((OperatingSystem) value));
	} else if (SentryRuntime.TYPE.equals(entry.getKey()) && value instanceof SentryRuntime) {
	this.setRuntime(new SentryRuntime((SentryRuntime) value));
	} else if (Feedback.TYPE.equals(entry.getKey()) && value instanceof Feedback) {
	this.setFeedback(new Feedback((Feedback) value));
	} else if (Gpu.TYPE.equals(entry.getKey()) && value instanceof Gpu) {
	this.setGpu(new Gpu((Gpu) value));
	} else if (SpanContext.TYPE.equals(entry.getKey()) && value instanceof SpanContext) {
	this.setTrace(new SpanContext((SpanContext) value));
	} else if (ProfileContext.TYPE.equals(entry.getKey()) && value instanceof ProfileContext) {
	this.setProfile(new ProfileContext((ProfileContext) value));
	} else if (Response.TYPE.equals(entry.getKey()) && value instanceof Response) {
	this.setResponse(new Response((Response) value));
	} else if (Spring.TYPE.equals(entry.getKey()) && value instanceof Spring) {
	this.setSpring(new Spring((Spring) value));
	} else {
	this.put(entry.getKey(), value);
	}
	}
	}
	}

Uh oh!

Add scope based feature flags #4812

Add scope based feature flags #4812

Conversation

adinauer commented Oct 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📜 Description

💡 Motivation and Context

💚 How did you test it?

📝 Checklist

🔮 Next steps

Uh oh!

github-actions bot commented Oct 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as outdated.

Uh oh!

Uh oh!

This comment was marked as outdated.

Uh oh!

github-actions bot commented Oct 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Performance metrics 🚀

Baseline results on branch: main

Startup times

App size

Previous results on branch: feat/feature-flags-on-scope

Startup times

App size

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

adinauer Oct 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

adinauer commented Oct 20, 2025

Uh oh!

adinauer commented Oct 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as outdated.

Uh oh!

cursor bot Nov 5, 2025

Choose a reason for hiding this comment

Bug: Return null when feature flags are empty

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

Bug: Contexts Cloning: FeatureFlags State Shared

Uh oh!

cursor bot Nov 11, 2025

Choose a reason for hiding this comment

Bug: Shallow Copy Breaks FeatureFlags Isolation

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

adinauer commented Oct 16, 2025 •

edited

Loading

github-actions bot commented Oct 16, 2025 •

edited

Loading

github-actions bot commented Oct 16, 2025 •

edited

Loading

adinauer Oct 20, 2025 •

edited

Loading

adinauer commented Oct 20, 2025 •

edited

Loading