Skip to content

feat: web search, fetch, and citation UI - #280

Open
cpsievert wants to merge 33 commits into
mainfrom
feat/web-citations-rendering
Open

feat: web search, fetch, and citation UI#280
cpsievert wants to merge 33 commits into
mainfrom
feat/web-citations-rendering

Conversation

@cpsievert

@cpsievert cpsievert commented Jul 27, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Why this matters

Web-enabled models can return search activity, fetched pages, and structured citations. Without dedicated rendering, that context competes with the answer itself. This PR keeps responses readable while users can inspect evidence beside a claim or from one message-wide Sources summary.

User experience

  • Web search and web fetch steps render as compact activity UI.
  • Each ContentCitation renders through the public <shiny-aside> component from feat: <shiny-aside> markup for source pills + popovers #278.
  • When a provider supplies grounded_span, shinychat maps it to the public grounded-span attribute. Opening or paging the aside highlights the matching answer text.
  • Grounded spans match their rendered text when provider metadata contains Markdown formatting, including spans that begin or end at an emphasis boundary.
  • Repeated citation occurrences remain separate in their local aside group. The Sources summary lists each source URL once.

Component boundary

  • feat: <shiny-aside> markup for source pills + popovers #278 owns generic aside rendering, including block-local grounded-span matching and active-popover highlighting.
  • This PR adapts provider web activity and ContentCitation data into that public component.
  • data-citation opts an aside into the message-wide Sources summary. It is not required for grounded highlighting.
  • Citation metadata such as cited_quote remains separate from generic aside state.

R support

The R path uses ellmer::ContentCitation and depends on the corresponding citation-content support in ellmer.

Verification

  • cd js && npm test -- --run — 994 passed, 23 skipped
  • cd js && npm run lint
  • cd js && npm run build
  • uv run pytest pkg-py/tests/pytest pkg-py/tests/test_*.py -q — 410 passed
  • uv run pytest pkg-py/tests/test_normalize_web.py -q — 13 passed
  • uv run pytest pkg-py/tests/playwright/chat/web_citations/test_web_citations.py -q — 2 passed
  • cd pkg-r && Rscript -e "devtools::test()" — 841 passed, 1 browser-dependent skip
  • A live Gemini 2.5 Flash request in Chromium confirmed search, successful fetch, citation placement, grounded highlighting, and source paging.

@cpsievert cpsievert changed the title feat: render web citations as shiny-aside pills with a whole-message Sources summary feat: web search, fetch and citation UI Jul 27, 2026
@cpsievert cpsievert changed the title feat: web search, fetch and citation UI feat: web search, fetch, and citation UI Jul 27, 2026
@cpsievert
cpsievert force-pushed the feat/web-citations-rendering branch from 1f8e035 to 0fd0c6e Compare July 27, 2026 21:08
@cpsievert
cpsievert marked this pull request as ready for review July 28, 2026 21:37
@cpsievert
cpsievert marked this pull request as draft July 28, 2026 21:37
cpsievert
cpsievert commented Jul 29, 2026
View reviewed changes
Comment thread pyproject.toml Outdated
cpsievert added 5 commits July 30, 2026 09:20
@cpsievert
feat: add a generalized <shiny-sidenote> markup convention
ffdeb2a 
Introduces a reusable sidenote pill + popover component: any assistant
message can carry inline <shiny-sidenote label="..." url="..." icon="...">
tags anywhere in its markdown, with the popover body as the tag's own
markdown children (supporting both a simple inline body and, via a
blank-line-separated block body, richer content like lists and
paragraphs). Entries are grouped per end-of-block into a labeled chip
(or count-fallback pill) that expands into a hover/click/focus popover
with prev/next navigation across grouped entries. This is a
general-purpose building block, not tied to any particular content
source (e.g. chatlas web search/fetch citations, a follow-up PR, will
build on top of it).

Anonymous sidenotes are numbered sequentially across the whole message,
and that message-scoped index is shown on the pill. The popover opens
after a short grace period and is rendered through a @floating-ui/react
portal so it escapes the message list's scrolling container, flipping
and shifting into view rather than clipping.

Accessibility and safety details:
- Multi-source labeled pills include the overflow count ("+N more") in
  their accessible name.
- The `icon` attribute is URL-sanitized the same way `url` is.
- rehypeGroupSidenotes runs in both the markdown and HTML content
  pipelines, so sidenotes also work under content_type="html".
- A literal `"` inside label/url/body must be HTML-entity-escaped, since
  these are ordinary HTML attributes parsed by rehype-raw/parse5.

The component was originally built and named <shiny-footnote>, then
renamed to <shiny-sidenote> across the JS component, markdown pipeline,
Python/R packages, and docs/tests before landing.

Includes JS/CSS unit tests, a Python Playwright app + test, R docs, and
rebuilt web assets for the Python and R packages.
@cpsievert
feat(sidenote): delay pill display until streaming settles, fix self-…
0585384 
…closing tags

While a response streams, a sidenote pill no longer flashes mid-sentence
and jitters as text streams past it. It now appears only once its
surrounding block has settled — either a later block has started or the
stream has ended.

Mirrors the existing streaming suggestion-card pattern: a new
rehypeMarkTrailingSidenotes marks groups in the still-open trailing block
as data-pending (SidenoteGroup renders nothing while pending), and
finalizePendingSidenotes clears the markers on the non-streaming render
path in hastToReact via an immutable path-copy that never mutates the
cached HAST.

Also fixes a bug where a self-closing <shiny-sidenote/> was rewritten to
a lone <template …/>, which — like any non-void element — ignores the
slash and swallows the text after it into the popover body. Normalizes
<shiny-sidenote …/> to an open/close pair before the template rewrite,
skipping over quoted attribute values so a slash inside url="https://…"
isn't mistaken for the self-close. Also corrects the broken-icon test to
assert the icon <img> unmounts (matching the intentional switch away from
display:none hiding), and documents in the Python/R sidenote help that the
favicon is fetched at render time from DuckDuckGo and that an explicit
`icon` bypasses that third-party request.
@cpsievert
feat(sidenote): keep every same-label sidenote instead of deduping
8aaee11 
Previously, grouping same-label sidenotes within a block kept only the
first entry and discarded the rest. Now every entry is kept and the group
pages through them (prev/next), matching the existing multi-source
popover behavior. The overflow "+N more" badge is now hidden when all
grouped entries share a single label, since paging already conveys that
there's more than one entry — the badge is only shown for mixed-label
groups where the count isn't otherwise visible.

Updates the Python/R chat_append docs to describe same-label paging and
the now-conditional overflow badge, corrects CHANGELOG/NEWS entries that
still described the old dedup behavior, adds a test covering entry-count
overflow in a mixed-label group, and rebuilds the JS assets for both
packages.
@cpsievert
feat(sidenote): rename <shiny-sidenote> to <shiny-aside>
1dd15ee 
"Aside" better describes the convention: a small pill/popover attached
to a paragraph or list item, not limited to citation-style notes.
@cpsievert
refactor(sidenote): use Floating UI interaction hooks for AsideGroup
aa2f1a1 
Replace hand-rolled hover/pin/blur/keydown handling with useHover,
useFocus, useClick, useDismiss, useRole, and FloatingFocusManager so
open/close/pin state, outside-click dismissal, and focus trapping are
handled by Floating UI instead of bespoke DOM containment checks.
@cpsievert
cpsievert force-pushed the feat/sidenote-markup branch from 7b961a2 to aa2f1a1 Compare July 30, 2026 14:24
@cpsievert
cpsievert force-pushed the feat/web-citations-rendering branch from 0fd0c6e to e3c8fde Compare July 30, 2026 14:32
@cpsievert cpsievert mentioned this pull request Jul 31, 2026
Open
cpsievert added 13 commits August 7, 2026 17:53
@cpsievert
fix(sidenote): address blocking rendering issues
aec8b83
@cpsievert
fix(sidenote): add deployment-wide favicon policy
3a631e9
@cpsievert
refactor(asides): normalize renderer input
1fb8d8d
@cpsievert
docs(asides): clarify grouping behavior
84b1dd2
@cpsievert
Merge remote-tracking branch 'origin/main' into feat/sidenote-markup
f2b9d57 
# Conflicts:
#	js/dist/shinychat.css
#	js/dist/shinychat.css.map
#	js/dist/shinychat.js
#	js/dist/shinychat.js.map
#	js/src/chat/ChatApp.tsx
#	js/src/chat/chat-entry.ts
#	js/src/chat/chat-tools.scss
#	js/tests/markdown/MarkdownContent.test.tsx
#	pkg-py/CHANGELOG.md
#	pkg-py/src/shinychat/www/GIT_VERSION
#	pkg-py/src/shinychat/www/shinychat.css
#	pkg-py/src/shinychat/www/shinychat.css.map
#	pkg-py/src/shinychat/www/shinychat.js
#	pkg-py/src/shinychat/www/shinychat.js.map
#	pkg-r/NEWS.md
#	pkg-r/inst/lib/shiny/GIT_VERSION
#	pkg-r/inst/lib/shiny/shinychat.css
#	pkg-r/inst/lib/shiny/shinychat.css.map
#	pkg-r/inst/lib/shiny/shinychat.js
#	pkg-r/inst/lib/shiny/shinychat.js.map
#	pkg-r/tests/testthat/test-chat.R
@cpsievert
feat: carry web search/fetch UI onto the shiny-aside base
115f9be
@cpsievert
feat: render web citations as shiny-aside markup with a whole-message…
43ccf96 
… Sources summary

Web search/fetch citations now render as deduped <shiny-aside> pills
inline, plus a single Sources summary pill per message that aggregates
every citation so users get an at-a-glance list without re-reading the
whole response.

- Add rehypeDedupeCitations and wire it into the markdown/html processors
- Normalize web citations (incl. chatlas) to shiny-aside markup on the Python side
- Add a pure citation model, message-scoped collector context, and whole-message merge
- Add the SourcesSummary pill + popover, sharing useDismissiblePopover with AsideGroup
- Cover the new behavior with unit, integration, and Playwright/e2e tests
- Rebuild and distribute JS/CSS assets to pkg-py and pkg-r
@cpsievert
Update pyproject.toml
386cbe7
@cpsievert
fix: update chatlas citation handling for 0.20.0 API
d66b698 
chatlas 0.20.0 (posit-dev/chatlas#318) dropped WebSource.domain and
replaced ContentCitation's flat url/title with a nested, optional
source. Adjust to the new shape so streaming web-search results and
citations no longer raise AttributeError.
@cpsievert
fix(web-activity): align chevron, rail, and dots to one column
8e789c2 
The disclosure chevron, the timeline's rail, and its dots were each
positioned with independently eyeballed offsets, so the rail missed
the dot centers and sat off from the chevron. Extract the chevron
(already solved correctly in ThinkingDisplay) into a shared component
and derive every rail/dot offset in SCSS from its fixed box, so the
column is correct by construction. Also draw the rail per node instead
of spanning the whole timeline, so it always terminates on a dot
instead of overshooting into a stub below the last one.
@cpsievert
Render ellmer web content in shinychat
e2d1fd0
@cpsievert
Require dev ellmer
f5eb493
@cpsievert
feat(citations): highlight grounded response spans
d8f4e13
@cpsievert
cpsievert force-pushed the feat/web-citations-rendering branch from 27d575b to d8f4e13 Compare August 7, 2026 23:53
@cpsievert
fix(asides): preserve theme and announce source changes
441da9f
@cpsievert
Merge branch 'feat/sidenote-markup' into feat/web-citations-rendering
e649447 
# Conflicts:
#	js/dist/shinychat.css
#	js/dist/shinychat.css.map
#	js/dist/shinychat.js
#	js/dist/shinychat.js.map
#	js/src/chat/AsideGroup.tsx
#	pkg-py/src/shinychat/www/GIT_VERSION
#	pkg-py/src/shinychat/www/shinychat.css
#	pkg-py/src/shinychat/www/shinychat.css.map
#	pkg-py/src/shinychat/www/shinychat.js
#	pkg-py/src/shinychat/www/shinychat.js.map
#	pkg-r/inst/lib/shiny/GIT_VERSION
#	pkg-r/inst/lib/shiny/shinychat.css
#	pkg-r/inst/lib/shiny/shinychat.css.map
#	pkg-r/inst/lib/shiny/shinychat.js
#	pkg-r/inst/lib/shiny/shinychat.js.map
@cpsievert cpsievert mentioned this pull request Aug 8, 2026
Merged
@cpsievert
feat(asides): support grounded spans
9bf5da3
@cpsievert
docs: clarify grounded aside use cases
dfb860a
@cpsievert
Merge public grounded asides into web citations
12fae80
@cpsievert
Merge branch 'feat/sidenote-markup' into feat/web-citations-rendering
189b987 
# Conflicts:
#	pkg-py/CHANGELOG.md
#	pkg-r/NEWS.md
@cpsievert
chore: refresh built JavaScript assets
df6567c
@cpsievert
Merge branch 'feat/sidenote-markup' into feat/web-citations-rendering
ab463b1 
# Conflicts:
#	js/dist/shinychat.js
#	js/dist/shinychat.js.map
#	pkg-py/src/shinychat/www/GIT_VERSION
#	pkg-py/src/shinychat/www/shinychat.js
#	pkg-py/src/shinychat/www/shinychat.js.map
#	pkg-r/inst/lib/shiny/GIT_VERSION
#	pkg-r/inst/lib/shiny/shinychat.js
#	pkg-r/inst/lib/shiny/shinychat.js.map
@cpsievert
chore: normalize built source map paths
8fcb8df
@cpsievert
Merge branch 'feat/sidenote-markup' into feat/web-citations-rendering
af8baa9 
# Conflicts:
#	js/dist/shinychat.js.map
#	pkg-py/src/shinychat/www/shinychat.js.map
#	pkg-r/inst/lib/shiny/shinychat.js.map
Base automatically changed from feat/sidenote-markup to main August 10, 2026 14:28
@cpsievert
Merge remote-tracking branch 'origin/main' into feat/web-citations-re…
3143a8d 
…ndering

# Conflicts:
#	js/dist/shinychat.css
#	js/dist/shinychat.css.map
#	js/dist/shinychat.js
#	js/dist/shinychat.js.map
#	js/src/chat/AsideGroup.tsx
#	js/src/chat/chat-tools.scss
#	js/src/chat/chatTagToComponentMap.ts
#	js/src/markdown/processors.ts
#	js/src/markdown/urlSanitize.ts
#	js/tests/chat/AsideGroup.test.tsx
#	pkg-py/CHANGELOG.md
#	pkg-py/src/shinychat/www/GIT_VERSION
#	pkg-py/src/shinychat/www/shinychat.css
#	pkg-py/src/shinychat/www/shinychat.css.map
#	pkg-py/src/shinychat/www/shinychat.js
#	pkg-py/src/shinychat/www/shinychat.js.map
#	pkg-r/NEWS.md
#	pkg-r/inst/lib/shiny/GIT_VERSION
#	pkg-r/inst/lib/shiny/shinychat.css
#	pkg-r/inst/lib/shiny/shinychat.css.map
#	pkg-r/inst/lib/shiny/shinychat.js
#	pkg-r/inst/lib/shiny/shinychat.js.map
@cpsievert
Merge remote-tracking branch 'origin/main' into feat/web-citations-re…
70ecab3 
…ndering

# Conflicts:
#	js/dist/shinychat.css
#	js/dist/shinychat.css.map
#	js/dist/shinychat.js
#	js/dist/shinychat.js.map
#	pkg-py/src/shinychat/www/GIT_VERSION
#	pkg-py/src/shinychat/www/shinychat.css
#	pkg-py/src/shinychat/www/shinychat.css.map
#	pkg-py/src/shinychat/www/shinychat.js
#	pkg-py/src/shinychat/www/shinychat.js.map
#	pkg-r/inst/lib/shiny/GIT_VERSION
#	pkg-r/inst/lib/shiny/shinychat.css
#	pkg-r/inst/lib/shiny/shinychat.css.map
#	pkg-r/inst/lib/shiny/shinychat.js
#	pkg-r/inst/lib/shiny/shinychat.js.map
@cpsievert
cpsievert requested a review from gadenbuie August 10, 2026 23:17
@cpsievert
cpsievert marked this pull request as ready for review August 10, 2026 23:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gadenbuie gadenbuie Awaiting requested review from gadenbuie

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@cpsievert