-
Notifications
You must be signed in to change notification settings - Fork 2.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
docs(memorystore): added valkey leaderboard guide #10006
Open
dackers86
wants to merge
1
commit into
GoogleCloudPlatform:main
Choose a base branch
from
invertase:@invertase/memorystore-valkey-leaderboard-guide
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,166 @@ | ||
# Building a Leaderboard Service on Google Cloud using Valkey, Spring Boot, and PostgreSQL | ||
|
||
Leaderboards are a useful way to display ranking data in applications. This guide explains how to create a scalable leaderboard system using Spring Boot, PostgreSQL, and Valkey (or Memorystore on GCP). By using a caching layer, developers can deliver real-time leaderboard rankings while reducing database load. | ||
|
||
## Benefits of a Cached Leaderboard | ||
|
||
- **Performance:** Leaderboards store ranking data in memory, enabling near-instantaneous retrieval of scores and rankings, reducing the time required to query and sort data from the database. | ||
- **Scalability:** Cached leaderboards can handle a high volume of reads and writes, providing consistent update and response times. | ||
- **Database Efficiency:** Caching frequently accessed leaderboard data minimizes the need for repetitive, and high volume database queries. | ||
|
||
## What You’ll Build | ||
|
||
You’ll set up a leaderboard service that: | ||
|
||
1. **Stores leaderboard data in PostgreSQL** for persistence and historical analysis. | ||
2. **Uses Valkey (Memorystore)** as an in-memory cache for updating scores and quick lookups. | ||
3. **Spring Boot Applications** to expose RESTful APIs for adding scores, retrieving rankings, and filtering leaderboards. | ||
4. **Deploys on Google Cloud Platform (GCP)** using services like Cloud Run, Cloud SQL, and Memorystore. | ||
|
||
By following this guide, you’ll implement a high-performing, scalable leaderboard system. | ||
|
||
## Architecture Overview | ||
|
||
- **Spring Boot Application:** Manages leaderboard logic and provides APIs for interaction. | ||
- **Valkey (In-Memory Cache):** Stores active leaderboard data for quick lookups. | ||
- **PostgreSQL Database:** Acts as the persistent storage for leaderboard data. | ||
- **Google Cloud Platform Services:** Hosts the application and its dependencies. | ||
|
||
## Leaderboard Workflow | ||
|
||
1. **Score Submission:** A user submits a score, which is added to the cache and database. | ||
2. **Rank Retrieval:** By default rankings are returned, displaying data ordered by the highest scores. | ||
3. **Rank Retrieval (Filtered):** Rankings are displayed based on any applied filters. | ||
|
||
## Step-by-Step Guide | ||
|
||
To begin, generate an API with the following routes: | ||
|
||
_addScore_: Creates a new score. | ||
_getScores_: Returns the leaderboard rankings. | ||
|
||
### Creating a new application | ||
|
||
The first step is to initialize a Spring Boot application. The [official guide](https://spring.io/guides/gs/spring-boot) demonstrates how to generate a new project using [Spring Initializer](https://start.spring.io/). | ||
|
||
1. Choose `Maven` as the project type for this demonstration.. | ||
2. Select Sprint Boot version 3.4.1 | ||
3. Complete the appropriate metadata. | ||
4. Choose your preferred `Packaging` for downloading. | ||
5. Select `Java 17` for your Java version. | ||
6. Finally, generate and extract the files. | ||
|
||
### Installing additional dependencies | ||
|
||
Next, ensure the following dependencies have been added to your POM.xml file. | ||
|
||
#### Jedis | ||
|
||
Add the folowing snippet toconnect directly to the Memorystore for Valkey instance. | ||
|
||
```xml | ||
<!-- Jedis: Redis Java Client --> | ||
<dependency> | ||
<groupId>redis.clients</groupId> | ||
<artifactId>jedis</artifactId> | ||
<version>4.3.0</version> <!-- Use the latest version --> | ||
</dependency> | ||
``` | ||
|
||
#### Jakarta | ||
|
||
To ensure that our api routes are correctly validated. Add the following dependency. | ||
This dependency enables the use of annotations like `@NotNull` and `@Size` on classes to automatically enforce input constraints, reducing the need for manual validation logic. | ||
|
||
```xml | ||
<!-- Add Validation support--> | ||
<dependency> | ||
<groupId>jakarta.validation</groupId> | ||
<artifactId>jakarta.validation-api</artifactId> | ||
<version>3.0.2</version> | ||
</dependency> | ||
``` | ||
|
||
### Connecting the Service Layer | ||
|
||
Next, add the following to the routing logic in the API. | ||
|
||
#### Initalizing the cache | ||
|
||
To ensure data is always availabe in the caching layer, a request is made to the database to update the cache if it is empty. | ||
|
||
```java | ||
private boolean initializeCache() { | ||
if (this.jedis.zcard(Global.LEADERBOARD_ENTRIES_KEY) > 0) { | ||
return false; | ||
} | ||
|
||
List<LeaderboardEntry> entries = this.leaderboardRepository.getEntries(); | ||
|
||
if (!entries.isEmpty()) { | ||
for (LeaderboardEntry entry : entries) { | ||
this.jedis.zadd( | ||
Global.LEADERBOARD_ENTRIES_KEY, entry.getScore(), entry.getUsername()); | ||
} | ||
} | ||
|
||
return true; | ||
} | ||
|
||
``` | ||
|
||
#### Fetching the leaderboard with zrangeWithScores (Ascending) | ||
|
||
```java | ||
if (!isDescending) { | ||
entries = new ArrayList<>(jedis.zrangeWithScores(cacheKey, position, maxPosition)); | ||
} | ||
``` | ||
|
||
#### Fetching the leaderboard with zrevrangeWithScores (Descending) | ||
|
||
```java | ||
if (isDescending) { | ||
entries = new ArrayList<>(jedis.zrevrangeWithScores(cacheKey, position, maxPosition)); | ||
} | ||
``` | ||
|
||
##### Fetching the leaderboard based on a user search using zrevrank | ||
|
||
```java | ||
if (username != null) { | ||
Long userRank = jedis.zrevrank(cacheKey, username); | ||
if (userRank != null) { | ||
position = userRank; | ||
maxPosition = userRank + pageSize - 1; | ||
|
||
return new LeaderboardResponse( | ||
getEntries(cacheKey, position, maxPosition, true), cacheStatus); | ||
} | ||
} | ||
``` | ||
|
||
##### Creating or updating a user score | ||
|
||
```java | ||
public void createOrUpdate(String username, Double score) { | ||
this.leaderboardRepository.update(username, score); | ||
this.jedis.zadd(Global.LEADERBOARD_ENTRIES_KEY, score, username); | ||
} | ||
``` | ||
|
||
## Scaling and Optimization | ||
|
||
As traffic increases, the architecture can scale horizontally: | ||
|
||
- **Cloud Run** can automatically scale instances based on load. | ||
- **Memorystore (Valkey)** can be sized or upgraded to handle more cached data or higher throughput. | ||
- **Cloud SQL** can scale vertically or horizontally (with read replicas) as needed. | ||
|
||
You can fine-tune cache expiration strategies (TTL values) and eviction policies, depending on your data access patterns. | ||
|
||
## Conclusion | ||
|
||
By implementing this leaderboard system, you can easily display large amounts of sorted datasets, while also having the support to effectively filter the data. Leveraging caching with Valkey (Memorystore) significantly reduces database load while maintaining fast and reliable user experiences. Running it in Google Cloud extends these benefits further, providing managed services and easy scaling. | ||
|
||
For more information check out the [repository](https://github.com/GoogleCloudPlatform/java-docs-samples/tree/main/memorystore/valkey/leaderboard) for the full project details and follow the instructions to get started. |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why MemoryStore code sample requires Jakarta?