feat(story): multi-story library with completion tracking, longer content, and translation tooling

Author: CuplexUser

README.md

@@ -13,7 +13,7 @@ A focused language learning app built with React + Express. Adaptive CEFR progre
 - Post-session mistake review — optional mini-session drilled from session errors
 - Session autosave and resume — in-progress sessions survive page refresh or tab close
 - Per-language course switching — independent progress tracked per language
-- **Story Reader** (Read tab) — comprehensible-input reading mode with leveled short stories (A1–B2). Tap any word for a bottom lookup drawer (gloss, part of speech, optional grammar note, listen button) backed by the same three-tier word system as exercise tooltips. Glossary words get a solid accent underline, grammar-hint words a dotted underline. Includes sentence-level audio, a Show/Hide English toggle, an inline cultural note, and looked-up/saved counters. "Save to review" persists unknown words so they resurface in practice sessions via spaced repetition. Reading grants no XP (anti-score-inflation).
+- **Story Reader** (Read tab) — comprehensible-input reading mode with a leveled story library (A1–B2). Each level offers several stories; level tabs drive a per-level list that marks finished stories and opens to the first unread one. Tap any word for a bottom lookup drawer (gloss, part of speech, optional grammar note, listen button) backed by the same three-tier word system as exercise tooltips. Glossary words get a solid accent underline, grammar-hint words a dotted underline. Stories run from short A1 pieces to multi-paragraph B1 reads, with sentence-level audio, a Show/Hide English toggle, an inline cultural note, and looked-up/saved counters. "Finish story" records completion and suggests the next unread story; "Save to review" persists unknown words so they resurface in practice sessions via spaced repetition. Reading grants no XP (anti-score-inflation).
 
 ### Exercise Types
 
@@ -56,7 +56,9 @@ npm run translate:language
 
 The tool reads `server/.env`, lets you select a supported target language and category files, rate-limits API calls if needed, and writes only new files under `server/content/languages/<language>/`. It never overwrites existing category files.
 
-The wizard first asks **what to generate** — course categories or the **practice word** pool. Choosing practice words translates the English word list in `server/content/practice_words/_template.json` into the target language, writing `server/content/practice_words/<language>.json`. Translations are batched (`TRANSLATE_BATCH_SIZE` words per request), so a ~1000-word pool costs roughly 20 API calls rather than one per word.
+The wizard first asks **what to generate** — course categories, the **practice word** pool, or **Story Reader stories**. Choosing practice words translates the English word list in `server/content/practice_words/_template.json` into the target language, writing `server/content/practice_words/<language>.json`. Translations are batched (`TRANSLATE_BATCH_SIZE` words per request), so a ~1000-word pool costs roughly 20 API calls rather than one per word.
+
+Choosing **Stories** reads the hand-authored `server/content/stories/english.json`, translates each sentence and title English → target, then runs a reverse target → English pass to build a per-word glossary (part-of-speech left blank for machine glosses), writing `server/content/stories/<language>.json`. As with all jobs it never overwrites an existing file, so delete a language's short-seed story file first if you want to regenerate it from the longer English source.
 
 The generator code is split for clarity under `scripts/libretranslate/`: `terminal-menu.ts` (generic interactive prompts), `content-generator.ts` (translation + JSON file IO), and `index.ts` (the wizard that wires them together).
 
@@ -327,6 +329,7 @@ npm run verify          # Lint + client tests
 | `DELETE` | `/api/bookmarks/:questionId` | Remove a bookmark |
 | `GET` | `/api/stories?language=<id>&level=<lvl>&category=<cat>` | List story summaries (filterable) |
 | `GET` | `/api/stories/:id` | Fetch a full story (sentences, glossary, cultural note) |
+| `POST` | `/api/stories/:id/complete` | Mark a story finished (idempotent, per user) |
 | `GET` | `/api/saved-words?language=<id>` | List words saved from the Story Reader |
 | `POST` | `/api/saved-words` | Save a word to review (idempotent; enters the SRS queue) |
 | `DELETE` | `/api/saved-words/:word?language=<id>` | Remove a saved word |

TODO.md

@@ -376,11 +376,47 @@ pipeline feeding the spaced-repetition system.
   as the learner re-encounters the word.
 - [ ] **Native/Forvo audio** — Story Reader uses browser `SpeechSynthesis` only; real recorded audio
   remains out of scope (shared with the existing audio TODO).
-- [ ] **More stories per language** — each language has the three seed stories (A1/A2/B1). Add more
-  per level and a B2 tier, keeping the concrete high-frequency vocabulary + factual cultural-note bar.
+- [ ] **More stories per language** — English and Russian now ship six longer stories each (2 per
+  A1/A2/B1, see Phase 18.1). Generate the remaining five languages from the English source via the
+  LibreTranslate Stories job, then add a B2 tier.
 - [ ] Optionally surface saved words directly in the mistake-review drill, and add a saved-words
   management view (reuse the Bookmarks page pattern).
 
+#### Phase 18.1: Longer content, progressive library & translation pipeline — shipped
+
+Expands the MVP from one short story per level to a multi-story library that tracks progress, with
+a tooling path to scale authored content across languages.
+
+**Done**
+- [x] **Progressive library + completion tracking.** New `story_completions` table
+  (`UNIQUE(user_id, story_id)`) with `markStoryComplete` / `getCompletedStoryIds` in `db.ts`.
+  `POST /api/stories/:id/complete` records a finish; `GET /api/stories` is now user-aware and returns
+  a `completed` flag per summary. "Finish story" calls the endpoint and the modal offers **Read next**
+  (the next unread story, lowest level first).
+- [x] **Client `StoryPage.tsx`** rebuilt around a two-tier selector: level tabs (A1/A2/B1) drive a
+  per-level story list (`sr-library`) showing each story's title + a completed check. On load it
+  defaults to the first unread story; completed stories persist across reloads.
+- [x] **Longer stories with paragraph breaks.** `storyLoader.ts` accepts an optional `break` flag per
+  sentence (rendered as a paragraph gap) and treats glossary `pos` as optional. English + Russian
+  authored as the reference set: six stories each (Daily life, Family, Market, Weekend, Travel letter,
+  First day at work) at ~10–20 sentences with full grammar-aware glossaries.
+- [x] **English as dual-purpose source.** `server/content/stories/english.json` ships as the
+  English-course story set and is the canonical translation source.
+- [x] **LibreTranslate Stories job.** `scripts/libretranslate` gained a "Stories" content type:
+  forward (English → target) for sentences/titles plus a reverse (target → English) glossary pass
+  (`translateStories` in `content-generator.ts`). Never overwrites existing files; `pos` left blank
+  for machine glosses.
+- [x] Tests: completion endpoint (idempotent, per-user, 404), loader `break`/optional-`pos`, updated
+  content-stats counts; client Finish→complete + Read next flow.
+
+**Remaining**
+- [ ] **Generate es/it/sv/fr/de stories** from the English source. Run
+  `node --experimental-strip-types scripts/libretranslate/index.ts` → *Stories*, after deleting each
+  language's existing short-seed `server/content/stories/<lang>.json`. Requires `LIBRETRANSLATE_URL` /
+  `LIBRETRANSLATE_API_KEY` in `server/.env`. Spot-check and hand-fix any unresolved glosses.
+- [ ] Optional: vocab-mastery–gated story ordering (the `story_completions` schema is compatible with
+  layering this on later).
+
 ## Completed archive
 
 ### Phase 1: Reliability and anti-trivial-cheat

client/src/__tests__/story_page.test.tsx

@@ -4,6 +4,7 @@ import { describe, it, expect, vi, beforeEach } from "vitest";
 
 const getStories = vi.fn();
 const getStory = vi.fn();
+const completeStory = vi.fn();
 const getSavedWords = vi.fn();
 const fetchWordTranslations = vi.fn();
 const saveWord = vi.fn();
@@ -13,6 +14,7 @@ vi.mock("../api", () => ({
   api: {
     getStories: (...args: unknown[]) => getStories(...args),
     getStory: (...args: unknown[]) => getStory(...args),
+    completeStory: (...args: unknown[]) => completeStory(...args),
     getSavedWords: (...args: unknown[]) => getSavedWords(...args),
     fetchWordTranslations: (...args: unknown[]) => fetchWordTranslations(...args),
     saveWord: (...args: unknown[]) => saveWord(...args),
@@ -59,6 +61,7 @@ const SUMMARIES = [
 beforeEach(() => {
   vi.clearAllMocks();
   getStories.mockResolvedValue(SUMMARIES);
+  completeStory.mockResolvedValue({ ok: true });
   getSavedWords.mockResolvedValue([]);
   fetchWordTranslations.mockResolvedValue({});
   saveWord.mockResolvedValue({ ok: true, saved: true });
@@ -69,8 +72,9 @@ beforeEach(() => {
 describe("StoryPage", () => {
   it("renders the first story for the active language", async () => {
     render(<StoryPage language="russian" languageLabel="Russian" />);
-    expect(await screen.findByText("Утро Анны")).toBeInTheDocument();
-    expect(screen.getByText("Anna's morning")).toBeInTheDocument();
+    // The title shows in the story heading (the library list repeats it as a span).
+    expect(await screen.findByRole("heading", { name: "Утро Анны" })).toBeInTheDocument();
+    expect(screen.getAllByText("Anna's morning").length).toBeGreaterThan(0);
     // English translations are hidden until toggled on.
     expect(screen.queryByText("Anna lives in Moscow.")).not.toBeInTheDocument();
   });
@@ -113,9 +117,22 @@ describe("StoryPage", () => {
 
   it("switches stories when a different level is selected", async () => {
     render(<StoryPage language="russian" languageLabel="Russian" />);
-    await screen.findByText("Утро Анны");
+    await screen.findByRole("heading", { name: "Утро Анны" });
     await userEvent.click(screen.getByRole("button", { name: "A2" }));
-    expect(await screen.findByText("На рынке")).toBeInTheDocument();
+    expect(await screen.findByRole("heading", { name: "На рынке" })).toBeInTheDocument();
+    expect(getStory).toHaveBeenCalledWith("ru-a2");
+  });
+
+  it("marks the story complete on finish and jumps to the next unread story", async () => {
+    render(<StoryPage language="russian" languageLabel="Russian" />);
+    await screen.findByRole("heading", { name: "Утро Анны" });
+
+    await userEvent.click(screen.getByRole("button", { name: /Finish story/ }));
+    await waitFor(() => expect(completeStory).toHaveBeenCalledWith("ru-a1"));
+
+    // The completion modal offers the next unread story (the A2 one).
+    await userEvent.click(await screen.findByRole("button", { name: /Read next/ }));
+    expect(await screen.findByRole("heading", { name: "На рынке" })).toBeInTheDocument();
     expect(getStory).toHaveBeenCalledWith("ru-a2");
   });
 });

client/src/api.ts

@@ -506,6 +506,8 @@ export const api = {
     return request<StorySummary[]>(`/stories${query ? `?${query}` : ""}`);
   },
   getStory: (id: string): Promise<Story> => request<Story>(`/stories/${encodeURIComponent(id)}`),
+  completeStory: (id: string): Promise<{ ok: boolean }> =>
+    request<{ ok: boolean }>(`/stories/${encodeURIComponent(id)}/complete`, { method: "POST" }),
   getSavedWords: (language?: string): Promise<SavedWord[]> =>
     request<SavedWord[]>(`/saved-words${language ? `?language=${encodeURIComponent(language)}` : ""}`),
   saveWord: (payload: { language: string; word: string; translation: string; storyId: string; category: string }) =>

client/src/components/StoryPage.tsx

@@ -8,6 +8,24 @@ type StoryPageProps = {
   languageLabel: string;
 };
 
+const LEVEL_SEQUENCE = ["a1", "a2", "b1", "b2"];
+
+// First unread story in a list, falling back to the first story so a level tab
+// always resolves to something selectable.
+function firstUnreadId(list: StorySummary[]): string {
+  const unread = list.find((summary) => !summary.completed);
+  return (unread ?? list[0])?.id ?? "";
+}
+
+// Next unread story across the whole library (lowest level first), used to power
+// the "Read next" affordance after finishing a story.
+function nextUnreadStory(list: StorySummary[], currentId: string): StorySummary | null {
+  const ordered = [...list].sort(
+    (a, b) => LEVEL_SEQUENCE.indexOf(a.level) - LEVEL_SEQUENCE.indexOf(b.level)
+  );
+  return ordered.find((summary) => !summary.completed && summary.id !== currentId) ?? null;
+}
+
 type DrawerEntry = {
   word: string;
   gloss: string;
@@ -66,6 +84,7 @@ function Icon({ name }: { name: string }) {
 export function StoryPage({ language, languageLabel }: StoryPageProps) {
   const [stories, setStories] = useState<StorySummary[]>([]);
   const [storiesError, setStoriesError] = useState("");
+  const [selectedLevel, setSelectedLevel] = useState("");
   const [selectedId, setSelectedId] = useState("");
   const [story, setStory] = useState<Story | null>(null);
   const [storyLoading, setStoryLoading] = useState(false);
@@ -90,7 +109,14 @@ export function StoryPage({ language, languageLabel }: StoryPageProps) {
       .then((list) => {
         if (cancelled) return;
         setStories(list);
-        setSelectedId(list[0]?.id ?? "");
+        // Default to the lowest level that still has an unread story (progressive).
+        const present = LEVEL_SEQUENCE.filter((lvl) => list.some((s) => s.level === lvl));
+        const level =
+          present.find((lvl) => list.some((s) => s.level === lvl && !s.completed)) ??
+          present[0] ??
+          "";
+        setSelectedLevel(level);
+        setSelectedId(firstUnreadId(list.filter((s) => s.level === level)));
       })
       .catch(() => {
         if (cancelled) return;
@@ -187,6 +213,7 @@ export function StoryPage({ language, languageLabel }: StoryPageProps) {
     return story.sentences.map((sentence) => ({
       en: sentence.en,
       target: sentence.target,
+      brk: Boolean(sentence.break),
       tokens: tokenizeSentence(sentence.target).map((token) => {
         const interactive = tokenHasLetter(token.core);
         const key = interactive ? token.core.toLowerCase() : "";
@@ -250,14 +277,42 @@ export function StoryPage({ language, languageLabel }: StoryPageProps) {
     }
   }
 
-  const levels = useMemo(() => {
-    const seen = new Map<string, string>();
-    for (const summary of stories) {
-      if (!seen.has(summary.level)) seen.set(summary.level, summary.id);
-    }
-    return [...seen.entries()].map(([level, id]) => ({ level, id }));
+  const availableLevels = useMemo(() => {
+    const present = new Set(stories.map((summary) => summary.level));
+    return LEVEL_SEQUENCE.filter((level) => present.has(level as StorySummary["level"]));
   }, [stories]);
 
+  const storiesInLevel = useMemo(
+    () => stories.filter((summary) => summary.level === selectedLevel),
+    [stories, selectedLevel]
+  );
+
+  const readNext = useMemo(() => nextUnreadStory(stories, selectedId), [stories, selectedId]);
+
+  function selectLevel(level: string) {
+    setSelectedLevel(level);
+    setSelectedId(firstUnreadId(stories.filter((summary) => summary.level === level)));
+  }
+
+  function selectStory(summary: StorySummary) {
+    setSelectedLevel(summary.level);
+    setSelectedId(summary.id);
+    setFinished(false);
+  }
+
+  async function handleFinish() {
+    setFinished(true);
+    if (!story) return;
+    const id = story.id;
+    // Optimistically mark the story complete; completion is best-effort.
+    setStories((prev) => prev.map((s) => (s.id === id ? { ...s, completed: true } : s)));
+    try {
+      await api.completeStory(id);
+    } catch {
+      /* the modal still shows; completion will retry on the next finish */
+    }
+  }
+
   const savedList = [...saved];
 
   return (
@@ -268,15 +323,15 @@ export function StoryPage({ language, languageLabel }: StoryPageProps) {
             <p className="eyebrow">Read · {languageLabel}</p>
             <h2>Story Reader</h2>
           </div>
-          {levels.length > 0 && (
+          {availableLevels.length > 0 && (
             <div className="sr-levels" role="group" aria-label="Reading level">
-              {levels.map(({ level, id }) => (
+              {availableLevels.map((level) => (
                 <button
                   key={level}
                   type="button"
                   className="sr-level"
-                  aria-pressed={stories.find((s) => s.id === selectedId)?.level === level}
-                  onClick={() => setSelectedId(id)}
+                  aria-pressed={selectedLevel === level}
+                  onClick={() => selectLevel(level)}
                 >
                   {level.toUpperCase()}
                 </button>
@@ -285,6 +340,31 @@ export function StoryPage({ language, languageLabel }: StoryPageProps) {
           )}
         </div>
 
+        {storiesInLevel.length > 0 && (
+          <ul className="sr-library" aria-label="Stories in this level">
+            {storiesInLevel.map((summary) => (
+              <li key={summary.id}>
+                <button
+                  type="button"
+                  className={`sr-lib-item ${summary.id === selectedId ? "active" : ""} ${
+                    summary.completed ? "done" : ""
+                  }`}
+                  aria-pressed={summary.id === selectedId}
+                  onClick={() => selectStory(summary)}
+                >
+                  <span className="sr-lib-title">{summary.title}</span>
+                  <span className="sr-lib-sub">{summary.titleEn}</span>
+                  {summary.completed ? (
+                    <span className="sr-lib-check" aria-label="Completed">
+                      <Icon name="check" />
+                    </span>
+                  ) : null}
+                </button>
+              </li>
+            ))}
+          </ul>
+        )}
+
         {storiesError ? <div className="status">{storiesError}</div> : null}
         {!storiesError && !stories.length ? (
           <p className="sr-empty">No stories yet for {languageLabel}. Check back soon.</p>
@@ -303,7 +383,7 @@ export function StoryPage({ language, languageLabel }: StoryPageProps) {
 
             <div className="sr-story">
               {tokenized.map((sentence, sentenceIndex) => (
-                <p className="sr-sent" key={sentenceIndex}>
+                <p className={`sr-sent${sentence.brk ? " sr-break" : ""}`} key={sentenceIndex}>
                   {sentence.tokens.map((token, tokenIndex) => {
                     const trailingSpace = tokenIndex < sentence.tokens.length - 1 ? " " : "";
                     if (!token.entry || !token.key) {
@@ -433,7 +513,7 @@ export function StoryPage({ language, languageLabel }: StoryPageProps) {
                 <b>{saved.size}</b> saved
               </span>
             </div>
-            <button type="button" className="sr-finish" onClick={() => setFinished(true)}>
+            <button type="button" className="sr-finish" onClick={handleFinish}>
               Finish story <Icon name="arrow" />
             </button>
           </div>
@@ -464,9 +544,14 @@ export function StoryPage({ language, languageLabel }: StoryPageProps) {
               )}
             </div>
             <div className="sr-modal-actions">
-              <button type="button" className="sr-action primary" onClick={() => setFinished(false)}>
+              <button type="button" className="sr-action" onClick={() => setFinished(false)}>
                 <Icon name="check" /> Done
               </button>
+              {readNext ? (
+                <button type="button" className="sr-action primary" onClick={() => selectStory(readNext)}>
+                  Read next <Icon name="arrow" />
+                </button>
+              ) : null}
             </div>
           </div>
         </div>