Skip to content
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions build/render_hook_docs/DECISION_TREE_FORMAT.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,29 @@ Each answer (`yes` or `no`) contains:

- **`label`** (required): The recommendation text
- **`id`** (required): Unique identifier (e.g., `jsonOutcome`, `hashOutcome`)
- **`sentiment`** (optional): Indicates the nature of the outcome for visual styling
- `"positive"`: Renders with green styling (e.g., "Use this option")
- `"negative"`: Renders with red styling (e.g., "Don't use this option")
- Omitted: Defaults to red (neutral/warning styling)

**Note**: The `sentiment` field is particularly useful for **suitability trees** (where outcomes are binary: suitable vs. unsuitable) as opposed to **selection trees** (where all outcomes are valid options). See [Tree Types](#tree-types) below.

## Tree Types

Decision trees can serve different purposes, which affects how you structure outcomes:

### Selection Trees
All paths lead to valid recommendations. Users choose between options.
- **Example**: "Which data type should I use?" → JSON, Hash, or String (all valid)
- **Outcome styling**: Typically all neutral (no sentiment field needed)
- **Use case**: Helping users choose among alternatives

### Suitability Trees
Paths lead to binary outcomes: suitable or unsuitable for the use case.
- **Example**: "Should I use RDI?" → Yes (good fit) or No (various reasons why not)
- **Outcome styling**: Use `sentiment: "positive"` for suitable outcomes and `sentiment: "negative"` for unsuitable ones
- **Use case**: Determining if a technology/approach is appropriate
- **Benefit**: Visual distinction (green vs. red) helps users quickly understand if something is recommended

## Multi-line Text

Expand Down Expand Up @@ -112,6 +135,8 @@ scope: documents
5. **Consistent naming**: Use camelCase for IDs, end question IDs with "Question"
6. **Match fence and YAML IDs**: The `id` in the code block fence should match the `id` field in the YAML for consistency
7. **Use meaningful scopes**: Choose scope values that clearly indicate the tree's domain (e.g., `documents`, `collections`, `sequences`)
8. **Add sentiment for suitability trees**: If your tree determines whether something is suitable (not just choosing between options), use `sentiment: "positive"` and `sentiment: "negative"` to provide visual feedback
9. **Be consistent with sentiment**: In a suitability tree, ensure all positive outcomes have `sentiment: "positive"` and all negative outcomes have `sentiment: "negative"` for clarity

## Example: Redis Data Type Selection

Expand Down
27 changes: 27 additions & 0 deletions build/render_hook_docs/DECISION_TREE_IMPLEMENTATION_NOTES.md
Original file line number Diff line number Diff line change
Expand Up @@ -140,3 +140,30 @@ const maxCharsPerLine = Math.floor(maxBoxWidth / charWidth);

**Benefit**: Helps future implementers and enables AI agents to understand the format.

## 11. Sentiment-Based Styling for Suitability Trees

**Discovery**: Not all decision trees are "selection trees" (choose between options). Some are "suitability trees" (determine if something is appropriate).

**Problem**: Selection trees and suitability trees have fundamentally different semantics:
- **Selection trees**: All outcomes are valid recommendations (e.g., "Use JSON" vs. "Use Hash" vs. "Use String")
- **Suitability trees**: Outcomes are binary (suitable vs. unsuitable) (e.g., "RDI is a good fit" vs. "RDI won't work")

**Solution**: Add optional `sentiment` field to outcomes:
```yaml
outcome:
label: "✅ RDI is a good fit for your use case"
id: goodFit
sentiment: "positive" # Green styling
```

**Implementation Details**:
- Extract `sentiment` field during YAML parsing in JavaScript
- Apply conditional styling in SVG rendering:
- `sentiment: "positive"` → Green background (`#0fa869`) and border
- `sentiment: "negative"` → Red background (`#d9534f`) and border
- No sentiment → Red (default, maintains backward compatibility)

**Key Insight**: Explicit metadata is better than heuristics. Don't try to infer sentiment from emoji (✅/❌) or label text. Use explicit fields for reliability and AI agent compatibility.

**Backward Compatibility**: Existing trees without sentiment fields continue to work with default red styling. This allows gradual adoption.

25 changes: 4 additions & 21 deletions content/integrate/redis-data-integration/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,27 +80,10 @@ RDI provides enterprise-grade streaming data pipelines with the following featur

## When to use RDI
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm happy to do this myself, but I'd recommend also adding this section somewhere on this page: https://redis.io/docs/latest/operate/rc/databases/rdi/ - I think it would go great before the Data Pipeline Architecture section.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cmilesb Good idea! I don't mind doing it while I've got this PR open (it may need other changes anyway). Do you think a link to this page in the main RDI section is OK, or would you prefer to have an embed and use it in both the main section and in Cloud?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@andy-stark-redis An embed is probably the better option. Thanks!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cmilesb The content has changed quite a bit now, but I've added all the new stuff in the Cloud page too, using an embed. Let me know what you think :-)


RDI is designed to support apps that must use a disk based database as the system of record
but must also be fast and scalable. This is a common requirement for mobile and web
apps with a rapidly-growing number of users; the performance of the main database is fine at first
but it will soon struggle to handle the increasing demand without a cache.

You should use RDI when:

- You must use a slow database as the system of record for the app .
- The app must always *write* its data to the slow database.
- You already intend to use Redis for the app cache.
- The data changes frequently in small increments.
- Your app can tolerate *eventual* consistency of data in the Redis cache.

You should *not* use RDI when:

- You are migrating an existing data set into Redis only once.
- The data is updated infrequently and in big batches.
- Your app needs *immediate* cache consistency rather than *eventual* consistency.
- The data is ingested from two replicas of Active-Active at the same time.
- The app must *write* data to the Redis cache, which then updates the source database.
- Your data set will only ever be small.
RDI is highly configurable but it is not intended to be a general
solution for all data integration tasks. See
[When to use RDI]({{< relref "/integrate/redis-data-integration/when-to-use" >}})
to find out if your use case is a good fit for RDI's features.

## Supported source databases

Expand Down
2 changes: 1 addition & 1 deletion content/integrate/redis-data-integration/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ linkTitle: Architecture
summary: Redis Data Integration keeps Redis in sync with the primary database in near
real time.
type: integration
weight: 3
weight: 30
---

## Overview
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ linkTitle: Data pipelines
summary: Redis Data Integration keeps Redis in sync with the primary database in near
real time.
type: integration
weight: 30
weight: 40
---

RDI uses *pipelines* to implement
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ linkTitle: Install/upgrade
summary: Redis Data Integration keeps Redis in sync with the primary database in near
real time.
type: integration
weight: 2
weight: 20
---

The guides in this section explain the options you have for installing and upgrading RDI on your own servers. See the [Redis Cloud RDI guide]({{< relref "/operate/rc/databases/rdi" >}}) to
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
Title: Quickstart
linkTitle: Quickstart
description: Get started with a simple pipeline example
weight: 1
weight: 10
alwaysopen: false
categories: ["redis-di"]
aliases: /integrate/redis-data-integration/ingest/quick-start-guide/
Expand Down
169 changes: 169 additions & 0 deletions content/integrate/redis-data-integration/when-to-use.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,169 @@
---
Title: When to use RDI
alwaysopen: false
categories:
- docs
- integrate
- rs
- rdi
description: Understand when (and when not) to use RDI.
group: di
hideListLinks: false
linkTitle: When to use RDI
summary: Redis Data Integration keeps Redis in sync with the primary database in near
real time.
type: integration
weight: 5
---

RDI is designed to support apps that must use a disk based database as the system of record
but must also be fast and scalable. This is a common requirement for mobile and web
apps with a rapidly-growing number of users; the performance of the main database is fine at first
but it will soon struggle to handle the increasing demand without a cache.

## When to use RDI

You should use RDI when:

- You must use a slow database as the system of record for the app.
- The app must always *write* its data to the slow database.
- You already intend to use Redis for the app cache.
- The data changes frequently in small increments.
- Your app can tolerate *eventual* consistency of data in the Redis cache.
- RDI throughput during
[full sync]({{< relref "/integrate/redis-data-integration/data-pipelines#pipeline-lifecycle" >}}) would not exceed 30K records per second and during
[CDC]({{< relref "/integrate/redis-data-integration/data-pipelines#pipeline-lifecycle" >}})
would not exceed 10K records per second.

## When not to use RDI

You should *not* use RDI when:

- You are migrating an existing data set into Redis only once.
- Your app needs *immediate* cache consistency rather than *eventual* consistency.
- The data is ingested from two replicas of Active-Active at the same time.
- The app must *write* data to the Redis cache, which then updates the source database.
- Your data set will only ever be small.
- Your data is updated by some batch or ETL process with long and large transactions - RDI will fail
processing these changes.

## Decision tree for using RDI

Use the decision tree below to determine whether RDI is a good fit for your architecture:

```decision-tree {id="when-to-use-rdi"}
id: when-to-use-rdi
scope: rdi
rootQuestion: systemOfRecord
questions:
systemOfRecord:
text: |
Does your app require a disk-based database as the system of record?
whyAsk: |
RDI is designed to keep Redis in sync with a primary database. If you don't need a primary database, RDI is not necessary.
answers:
yes:
value: "Yes"
nextQuestion: writeLocation
no:
value: "No"
outcome:
label: "RDI is not necessary for your use case"
id: notNecessary
sentiment: "negative"
writeLocation:
text: |
Does your app write data directly to the disk-based database?
whyAsk: |
RDI requires the primary database to be the system of record. If your app writes to Redis first, RDI won't work.
answers:
yes:
value: "Yes"
nextQuestion: consistency
no:
value: "No"
outcome:
label: "RDI won't work - your app must write to the primary database"
id: wrongWritePattern
sentiment: "negative"
consistency:
text: |
Can your app tolerate eventual consistency in the Redis cache?
whyAsk: |
RDI provides eventual consistency, not immediate consistency. If your app needs real-time cache consistency, RDI is not suitable.
answers:
yes:
value: "Yes"
nextQuestion: throughput
no:
value: "No"
outcome:
label: "RDI is not suitable - you need immediate cache consistency"
id: needsImmediate
sentiment: "negative"
throughput:
text: |
Will your throughput stay within RDI limits (≤30K records/sec during full sync, ≤10K records/sec during CDC)?
whyAsk: |
RDI has throughput limits. Exceeding these limits will cause processing failures and data loss.
answers:
yes:
value: "Yes"
nextQuestion: dataPattern
no:
value: "No"
outcome:
label: "RDI throughput limits will be exceeded"
id: exceedsLimits
sentiment: "negative"
dataPattern:
text: |
Is your data updated frequently in small increments (not by batch/ETL with large transactions)?
whyAsk: |
RDI captures changes from the database transaction log. Large batch transactions or ETL processes can cause RDI to fail.
answers:
yes:
value: "Yes"
nextQuestion: dataSize
no:
value: "No"
outcome:
label: "RDI will fail with batch/ETL processes and large transactions"
id: batchProcessing
sentiment: "negative"
dataSize:
text: |
Is your data set large enough to benefit from caching?
whyAsk: |
RDI adds operational complexity. If your data set is small, you may not need caching at all.
answers:
yes:
value: "Yes"
nextQuestion: dataSource
no:
value: "No"
outcome:
label: "RDI is not necessary - your data set is too small"
id: tooSmall
sentiment: "negative"
dataSource:
text: |
Is your data ingested from a single source (not from two Active-Active replicas simultaneously)?
whyAsk: |
RDI cannot handle data ingested from two Active-Active replicas at the same time, as this creates conflicting change events.
answers:
yes:
value: "Yes"
outcome:
label: "✅ RDI is a good fit for your use case"
id: goodFit
sentiment: "positive"
no:
value: "No"
outcome:
label: "RDI won't work with Active-Active replicas"
id: activeActive
sentiment: "negative"
```


31 changes: 24 additions & 7 deletions static/js/decision-tree.js
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,8 @@
depth: depth + 1,
type: 'outcome',
text: question.answers.yes.outcome.label || '',
answer: 'Yes'
answer: 'Yes',
sentiment: question.answers.yes.outcome.sentiment || null
});
}
}
Expand All @@ -110,7 +111,8 @@
depth: depth + 1,
type: 'outcome',
text: question.answers.no.outcome.label || '',
answer: 'No'
answer: 'No',
sentiment: question.answers.no.outcome.sentiment || null
});
}
}
Expand Down Expand Up @@ -204,11 +206,26 @@
rect.setAttribute('height', item.boxHeight);

if (item.type === 'outcome') {
// Outcomes: pale red background, dashed border
rect.setAttribute('fill', '#ffe6e6');
rect.setAttribute('stroke', '#d9534f');
rect.setAttribute('stroke-width', '1');
rect.setAttribute('stroke-dasharray', '3,3');
// Outcomes: color based on sentiment
if (item.sentiment === 'positive') {
// Green for positive outcomes
rect.setAttribute('fill', '#e6f7f0');
rect.setAttribute('stroke', '#0fa869');
rect.setAttribute('stroke-width', '1');
rect.setAttribute('stroke-dasharray', '3,3');
} else if (item.sentiment === 'negative') {
// Red for negative outcomes
rect.setAttribute('fill', '#ffe6e6');
rect.setAttribute('stroke', '#d9534f');
rect.setAttribute('stroke-width', '1');
rect.setAttribute('stroke-dasharray', '3,3');
} else {
// Default red for outcomes without explicit sentiment
rect.setAttribute('fill', '#ffe6e6');
rect.setAttribute('stroke', '#d9534f');
rect.setAttribute('stroke-width', '1');
rect.setAttribute('stroke-dasharray', '3,3');
}
} else {
// Questions: standard styling
rect.setAttribute('fill', '#f5f5f5');
Expand Down
Loading