Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: org release limits #8682

Open
wants to merge 41 commits into
base: next
Choose a base branch
from
Open

feat: org release limits #8682

wants to merge 41 commits into from

Conversation

jordanl17
Copy link
Member

@jordanl17 jordanl17 commented Feb 17, 2025

Description

With the API returning

Scenario Demo
Free project with no trial freeNOTrialWithApi
Free project with growth trial withTrialWithAPI
Enterprise project under limit WITHAPI-UNDER_LIMIT
Enterprise project at limit withAPI-atLimit

With the API not existing

Scenario Demo
Free project with no trial freeNOTrialWITHOUTApi
Free project with growth trial actualGRWNOAPI
Enterprise project under limit WITHOUTAPI-underlimit
Enterprise project at limit WITHOUTAPI-atlimit

Releases+
Free project after trial:
Screenshot 2025-02-21 at 14 13 40

Free project with trial:
Screenshot 2025-02-21 at 14 13 40

Enterprise project:
Screenshot 2025-02-21 at 14 30 39

What to review

  • The 2 new persistent stores useOrgActiveReleaseCount and useReleaseLimits, and the way they emit their data
    • How graceful handing of errors works for these hook - they support both the error cases from the endpoint, where data will still be returned an we assume limits not reached and releases+ not enabled, and internal errors on the server, eg when it goes away or is down.
  • The caching on useOrgActiveReleaseCount - it caches for a TTL (60s) or until the activeReleases.length` changes
  • How the guard now works - it fetches the limits then uses comparisons of the limits against the activeReleases.length and data.orgActiveReleaseCount
  • Determining releases+ support from the org limits and the default

Testing

Manually tested by above scenarios for now - unlikely we will fast-follow tests given this solution will go away in the close future

Notes for release

N/A

Copy link

vercel bot commented Feb 17, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
page-building-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Feb 21, 2025 5:37pm
performance-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Feb 21, 2025 5:37pm
test-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Feb 21, 2025 5:37pm
2 Skipped Deployments
Name Status Preview Comments Updated (UTC)
studio-workshop ⬜️ Ignored (Inspect) Visit Preview Feb 21, 2025 5:37pm
test-next-studio ⬜️ Ignored (Inspect) Feb 21, 2025 5:37pm

Copy link
Contributor

github-actions bot commented Feb 17, 2025

Coverage Report

Status Category Percentage Covered / Total
🔵 Lines 43.02% 55108 / 128090
🔵 Statements 43.02% 55108 / 128090
🔵 Functions 48.13% 2801 / 5819
🔵 Branches 79.5% 10673 / 13424
File Coverage
File Stmts Branches Functions Lines Uncovered Lines
Changed Files
packages/sanity/src/core/perspective/navbar/ReleasesList.tsx 96.47% 88.46% 100% 96.47% 140-145
packages/sanity/src/core/releases/components/documentHeader/contextMenu/VersionContextMenu.tsx 100% 91.07% 100% 100%
packages/sanity/src/core/releases/contexts/upsell/ReleasesUpsellProvider.tsx 10.25% 0% 0% 10.25% 22-32, 44-260
packages/sanity/src/core/releases/contexts/upsell/fetchReleaseLimits.ts 62.06% 100% 50% 62.06% 16-22, 57-67
packages/sanity/src/core/releases/contexts/upsell/types.ts 0% 0% 0% 0%
packages/sanity/src/core/releases/contexts/upsell/useReleasesUpsell.ts 95% 66.66% 11.11% 95% 15
packages/sanity/src/core/releases/hooks/useIsReleasesPlus.ts 92.85% 42.85% 100% 92.85% 28
packages/sanity/src/core/releases/store/createReleaseOperationStore.ts 93.51% 100% 100% 93.51% 179-180, 189-199, 362-363
packages/sanity/src/core/releases/store/createReleasePermissionsStore.ts 82.85% 66.66% 66.66% 82.85% 13, 59-62, 64-65
packages/sanity/src/core/releases/store/useOrgActiveReleaseCount.ts 13.23% 100% 0% 13.23% 17-65, 78-102
packages/sanity/src/core/releases/store/useReleaseLimits.ts 86.48% 100% 100% 86.48% 18-20, 24-26
packages/sanity/src/core/releases/store/useReleasePermissions.ts 100% 100% 100% 100%
packages/sanity/src/core/releases/tool/components/releaseCTAButtons/ReleaseRevertButton/ReleaseRevertButton.tsx 19.63% 55.55% 50% 19.63% 34-216, 233-235, 255-264, 272-281
packages/sanity/src/core/releases/tool/overview/ReleasesOverview.tsx 94.73% 86.9% 75% 94.73% 100, 142-143, 161, 177, 246-250, 284-286, 293-297
Generated in workflow #30828 for commit e3a3478 by the Vitest Coverage Report Action

Copy link
Contributor

No changes to documentation

Copy link
Contributor

github-actions bot commented Feb 18, 2025

⚡️ Editor Performance Report

Updated Fri, 21 Feb 2025 17:42:55 GMT

Benchmark reference
latency of sanity@latest
experiment
latency of this branch
Δ (%)
latency difference
article (title) 24.7 efps (41ms) 25.3 efps (40ms) -1ms (-2.5%)
article (body) 67.6 efps (15ms) 67.6 efps (15ms) +0ms (-/-%)
article (string inside object) 25.6 efps (39ms) 26.3 efps (38ms) -1ms (-2.6%)
article (string inside array) 22.5 efps (45ms) 23.3 efps (43ms) -2ms (-3.4%)
recipe (name) 45.5 efps (22ms) 45.5 efps (22ms) +0ms (-/-%)
recipe (description) 52.6 efps (19ms) 51.3 efps (20ms) +1ms (+2.6%)
recipe (instructions) 99.9+ efps (5ms) 99.9+ efps (5ms) +0ms (-/-%)
synthetic (title) 19.2 efps (52ms) 20.4 efps (49ms) -3ms (-5.8%)
synthetic (string inside object) 19.6 efps (51ms) 20.4 efps (49ms) -2ms (-3.9%)

efps — editor "frames per second". The number of updates assumed to be possible within a second.

Derived from input latency. efps = 1000 / input_latency

Detailed information

🏠 Reference result

The performance result of sanity@latest

Benchmark latency p75 p90 p99 blocking time test duration
article (title) 41ms 70ms 83ms 494ms 889ms 14.5s
article (body) 15ms 18ms 26ms 225ms 216ms 5.7s
article (string inside object) 39ms 41ms 52ms 157ms 344ms 7.6s
article (string inside array) 45ms 48ms 60ms 171ms 592ms 8.3s
recipe (name) 22ms 23ms 26ms 42ms 0ms 7.5s
recipe (description) 19ms 21ms 35ms 41ms 0ms 5.0s
recipe (instructions) 5ms 6ms 7ms 30ms 0ms 3.1s
synthetic (title) 52ms 55ms 64ms 108ms 973ms 12.5s
synthetic (string inside object) 51ms 55ms 68ms 400ms 1300ms 9.1s

🧪 Experiment result

The performance result of this branch

Benchmark latency p75 p90 p99 blocking time test duration
article (title) 40ms 62ms 77ms 446ms 843ms 10.6s
article (body) 15ms 17ms 43ms 211ms 219ms 5.9s
article (string inside object) 38ms 39ms 42ms 149ms 225ms 7.4s
article (string inside array) 43ms 46ms 56ms 179ms 436ms 7.9s
recipe (name) 22ms 24ms 27ms 64ms 8ms 7.5s
recipe (description) 20ms 21ms 26ms 37ms 0ms 5.0s
recipe (instructions) 5ms 7ms 9ms 31ms 0ms 3.1s
synthetic (title) 49ms 52ms 66ms 414ms 838ms 13.1s
synthetic (string inside object) 49ms 53ms 63ms 581ms 1156ms 8.5s

📚 Glossary

column definitions

  • benchmark — the name of the test, e.g. "article", followed by the label of the field being measured, e.g. "(title)".
  • latency — the time between when a key was pressed and when it was rendered. derived from a set of samples. the median (p50) is shown to show the most common latency.
  • p75 — the 75th percentile of the input latency in the test run. 75% of the sampled inputs in this benchmark were processed faster than this value. this provides insight into the upper range of typical performance.
  • p90 — the 90th percentile of the input latency in the test run. 90% of the sampled inputs were faster than this. this metric helps identify slower interactions that occurred less frequently during the benchmark.
  • p99 — the 99th percentile of the input latency in the test run. only 1% of sampled inputs were slower than this. this represents the worst-case scenarios encountered during the benchmark, useful for identifying potential performance outliers.
  • blocking time — the total time during which the main thread was blocked, preventing user input and UI updates. this metric helps identify performance bottlenecks that may cause the interface to feel unresponsive.
  • test duration — how long the test run took to complete.

Copy link
Contributor

github-actions bot commented Feb 18, 2025

Component Testing Report Updated Feb 21, 2025 5:47 PM (UTC)

❌ Failed Tests (4) -- expand for details
File Status Duration Passed Skipped Failed
comments/CommentInput.spec.tsx ✅ Passed (Inspect) 1m 21s 15 0 0
formBuilder/ArrayInput.spec.tsx ✅ Passed (Inspect) 14s 3 0 0
formBuilder/inputs/PortableText/Annotations.spec.tsx ❌ Failed (Inspect) 2m 3s 4 0 2
formBuilder/inputs/PortableText/copyPaste/CopyPaste.spec.tsx ✅ Passed (Inspect) 1m 1s 11 7 0
formBuilder/inputs/PortableText/copyPaste/CopyPasteFields.spec.tsx ✅ Passed (Inspect) 0s 0 12 0
formBuilder/inputs/PortableText/Decorators.spec.tsx ✅ Passed (Inspect) 30s 6 0 0
formBuilder/inputs/PortableText/DisableFocusAndUnset.spec.tsx ✅ Passed (Inspect) 17s 3 0 0
formBuilder/inputs/PortableText/DragAndDrop.spec.tsx ✅ Passed (Inspect) 31s 6 0 0
formBuilder/inputs/PortableText/FocusTracking.spec.tsx ❌ Failed (Inspect) 1m 57s 14 0 1
formBuilder/inputs/PortableText/Input.spec.tsx ✅ Passed (Inspect) 1m 57s 21 0 0
formBuilder/inputs/PortableText/ObjectBlock.spec.tsx ❌ Failed (Inspect) 3m 0s 20 0 1
formBuilder/inputs/PortableText/PresenceCursors.spec.tsx ✅ Passed (Inspect) 15s 3 9 0
formBuilder/inputs/PortableText/Styles.spec.tsx ✅ Passed (Inspect) 31s 6 0 0
formBuilder/inputs/PortableText/Toolbar.spec.tsx ✅ Passed (Inspect) 2m 2s 21 0 0
formBuilder/tree-editing/TreeEditing.spec.tsx ✅ Passed (Inspect) 0s 0 3 0
formBuilder/tree-editing/TreeEditingNestedObjects.spec.tsx ✅ Passed (Inspect) 0s 0 3 0

@@ -184,7 +184,8 @@ export function ReleasesList({
text={t('release.action.create-new')}
data-testid="create-new-release-button"
tooltipProps={{
content: !hasCreatePermission && t('release.action.permission.error'),
disabled: hasCreatePermission === true,
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@RitaDias checking that this (and other uses elsewhere) is true explicitly so that the tooltip doesn't show whilst we are pending fetching the permissions

@jordanl17 jordanl17 marked this pull request as ready for review February 21, 2025 15:04
@jordanl17 jordanl17 requested review from a team as code owners February 21, 2025 15:04
@jordanl17 jordanl17 requested review from RitaDias and juice49 and removed request for a team February 21, 2025 15:04
catchError((error: ClientError) => {
console.error(error)

if (typeof error.response.body !== 'string' && 'data' in error.response.body) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this handle the scenario of the quota being reached? If so, would it be safer to additionally check the response status code?

I'm just wondering whether there are any other errors that could produce a response body containing a data property that would incorrectly be caught here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I can do - on this particular endpoint, as it currently stands, data will always be of our contractually expected format. I'll need to check on all the error codes to allow as there are a handful depending on what the particular error is

Copy link
Contributor

@juice49 juice49 Feb 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool. I don't think we necessarily need it, just wanted to check! 🙂

@@ -355,9 +355,11 @@ export function createRequestAction(
},
})
} catch (e) {
if (isReleaseLimitError(e)) {
// if dryRunning then essentially this is a silent request
// so don't want to create disruptive upsell because of limit
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice 🙌

const activeReleasesCount = activeReleases?.length || 0

// dependencies must be objects not primitives, so nesting activeReleasesCount in an object
const count = useMemo(() => ({activeReleasesCount}), [activeReleasesCount])
Copy link
Contributor

@juice49 juice49 Feb 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By creating an object in the useOrgActiveReleaseCount hook to use as a key, won't this code create and cache a new instance of the createOrgActiveReleaseCountStore store for every component that invokes it?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeh that's true. This is a bug for sure! It doesn't get surfaced as a bug currently because useorgActiveReleaseCount is only used in ReleaseUpsellProvider. Perhaps useActiveReleases should be returning a memoized object instead?

Copy link
Contributor

@juice49 juice49 Feb 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I would consider having the observable produced by createOrgActiveReleaseCountStore consume the observable produced by useActiveReleases and deriving its state based on that, rather than injecting a snapshot of it at creation-time.

It's not necessarily a problem for now. Just thinking out loud.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants