Skip to content

fix(skill-creator): make eval-viewer HTML escaping attribute-safe#1395

Open
griffithsbs wants to merge 1 commit into
anthropics:mainfrom
griffithsbs:fix/skill-creator-eval-viewer-html-escaping
Open

fix(skill-creator): make eval-viewer HTML escaping attribute-safe#1395
griffithsbs wants to merge 1 commit into
anthropics:mainfrom
griffithsbs:fix/skill-creator-eval-viewer-html-escaping

Conversation

@griffithsbs

Copy link
Copy Markdown

Fixes #1394.

Problem

viewer.html assembles HTML by string concatenation and assigns it with
innerHTML in renderGrades and renderBenchmark, relying on escapeHtml()
at each interpolation. Two gaps let attacker-influenced data (grader "evidence"
and assertion text can quote a skill-under-test's output; eval and config names
are user-authored) reach the DOM as markup:

  • escapeHtml() was not attribute-safe. It used the textContent
    innerHTML round-trip, which escapes <, >, & but not quotes — yet it is
    used inside a title="..." attribute in the per-assertion table. Evidence
    containing a double quote breaks out of the attribute and can attach an event
    handler (e.g. x" onmouseover="alert(document.domain)).
  • Several externally-sourced strings were interpolated raw: the config label
    in the per-eval breakdown rows (while the same value is escaped in the header),
    and metadata.timestamp / metadata.evals_run / metadata.runs_per_configuration
    / the delta.* cells.

Fix

  • Rewrite escapeHtml() to escape &, <, >, ", ' explicitly, so it is
    safe in both element-text and attribute contexts.
  • Route the remaining raw externally-sourced string interpolations through it, so
    every externally-sourced string rendered via innerHTML is escaped and only
    locally-computed numbers are interpolated directly.

The escaped forms render identically for normal data, so there is no visible
change.

Scope / related

This is the display path. The embed pathgenerate_review.py injecting
EMBEDDED_DATA into the inline <script> without escaping </, which lets a
</script> in skill output break out of the script element (#1075) — is fixed
separately in #1393. The two together harden the viewer end to end; they touch
different files and do not conflict. The xlsx preview relies on SheetJS's own
cell-text escaping and is left as-is.

renderGrades and renderBenchmark build HTML by string concatenation and
assign it via innerHTML, relying on escapeHtml at each interpolation.
Two gaps let attacker-influenced data (grader "evidence" and assertion
text can quote a skill-under-test's output; eval and config names are
user-authored) reach the DOM as markup:

- escapeHtml used the textContent->innerHTML idiom, which escapes
  < > & but not quotes, yet it is used inside a title="..." attribute
  in the per-assertion table. Evidence containing a double quote breaks
  out of the attribute and can attach an event handler. Escape
  & < > " ' explicitly so the helper is safe in attributes and text.
- Several externally-sourced strings were interpolated raw
  (configLabel, metadata.timestamp/evals_run/runs_per_configuration,
  delta.*). Route them through escapeHtml too, so every externally
  sourced string rendered via innerHTML is escaped and only
  locally-computed numbers are interpolated directly.

The escaped forms render identically for normal data, so there is no
visible change.
@Echolonius

Copy link
Copy Markdown

+1 to this. I independently hit the same escapeHtml attribute gap (filed #1398 a day after this, now closed as a duplicate of it — sorry for the noise). One data point in support: I verified the textContentinnerHTML round-trip leaves " and ' unescaped, which is exploitable via the title="..." interpolations in renderBenchmark; the regex replacement here closes that, and this PR additionally escapes the raw metadata.timestamp/evals_run/configLabel/delta interpolations mine didn't touch. This version is the more complete fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

skill-creator: eval-viewer escapeHtml is not attribute-safe and is applied inconsistently (display-path XSS)

2 participants