You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: documentation/under-the-hood/ranking-notes.md
+8-4
Original file line number
Diff line number
Diff line change
@@ -99,10 +99,11 @@ We currently assign notes a "Not Helpful" status if the max (upper confidence bo
99
99
**Supervised confidence modeling**
100
100
101
101
We also employ a supervised model to detect low confidence matrix factorization results.
102
-
If the model predicts that a note will lose Helpful status, then the note will remain in Needs More Ratings status for an additional 30 minutes to allow it to gather a larger set of ratings.
103
-
If after 30 minutes the note still meets Helpful standards based on the matrix factorization scoring, the note will be rated Helpful and shown on X.
102
+
If the model predicts that a note will lose Helpful status, then the note will remain in Needs More Ratings status for up to an additional 180 minutes to allow it to gather a larger set of ratings.
103
+
If after that time the note still meets Helpful standards based on the matrix factorization scoring, the note will be rated Helpful and shown on X.
104
104
In all cases, the final status of the note is determined by matrix factorization.
105
-
The maximum effect of the supervised model is no more than a 30 minute delay.
105
+
The maximum effect of the supervised model is no more than a 180 minute delay.
106
+
All notes will receive a 30 minute delay to gather additional ratings.
106
107
This helps reduce notes briefly showing and then returning to Needs More Rating status.
107
108
108
109
The training data for the supervised confidence model includes all notes that meet the criteria for Helpful status _at some point in time_.
@@ -113,7 +114,7 @@ The features employed by the model include:
113
114
- Statistics summarizing the Helpful ratings for a note (e.g. standard deviation of user factors from Helpful ratings)
114
115
- Bucket counts of Helpful, Somewhat Helpful and Not Helpful ratings, partitioned by user factor $f_u$ as positive ($f_u >.3$), neutral ($-.3 \leq f_u \leq .3$) and negative ($f_u <-.3$)
115
116
116
-
The model uses logistic regression to predict note status outcomes, and is calibrated to delay Helpful status for no more than 25% of notes that ultimately stabilize to Helpful status.
117
+
The model uses logistic regression to predict note status outcomes, and is calibrated to delay Helpful status for no more than 60% of notes that ultimately stabilize to Helpful status.
117
118
118
119
## Tag Outlier Filtering
119
120
@@ -354,6 +355,9 @@ For not-helpful notes:
354
355
355
356
## What’s New?
356
357
358
+
**Mar 3, 2025**
359
+
- Update supervised confidence modeling to (1) allow notes with higher flip probability to gather ratings for a longer time before being set to Currently Rated Helpful (CRH), (2) identify more such notes with higher flip probability by adjusting supervised modeling thresholds, and (3) add a minimum delay for all notes that reach CRH criteria to gather more ratings before being set to CRH. Notes are shown as a note preview during that time, to help gather ratings.
360
+
357
361
**Feb 28, 2025**
358
362
- Update note assignment to topic threshold, increasing probability that notes with topic seed words are assigned to associated topic.
359
363
- Additional Not Helpful scoring logic (RatioCRNH scoring rule) to identify more notes that are widely rated as Not Helpful.
0 commit comments