Skip to content

Add HTML-in-Canvas APIs #11588

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
foolip wants to merge 27 commits into
base: main
Choose a base branch
from
Open

Add HTML-in-Canvas APIs #11588

Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
e181de1
Stub out canvas.drawElement()
foolip Aug 21, 2025
8634ebe
Fix typ: y->h
foolip Aug 28, 2025
afc4a4d
Modernize the style somewhat
foolip Aug 28, 2025
f151c83
Replace handwavy check with real check and drop "update the rendering"
foolip Aug 28, 2025
74d67ae
Add the layoutsubtree attribute
foolip Aug 29, 2025
f7d320a
Rename to drawHTMLElement()
foolip Aug 29, 2025
3cad6d4
Fix the layoutsubtree UA style sheet
foolip Sep 2, 2025
4367aac
Rename to drawElementImage()
foolip Sep 16, 2025
84ec8e6
Define sensitive information
foolip Sep 17, 2025
f351c3d
Expose children to AT
foolip Sep 17, 2025
aed994e
grammar
foolip Sep 17, 2025
8c3ce46
Use containment to make children containing blocks
foolip Sep 17, 2025
ff066c9
Define setHitTestRegions()
foolip Sep 17, 2025
5e3c3d0
Merge remote-tracking branch 'origin/main' into foolip/html-in-canvas
foolip Oct 7, 2025
f8f119a
Wrap algorithms
foolip Oct 7, 2025
202adae
Fix typo and reserved word
foolip Oct 7, 2025
be22310
Add a note about not using CTM
foolip Oct 7, 2025
9866034
Fix reference to hit test regions
foolip Oct 7, 2025
4e559a9
Skip non-positive width/height regions
foolip Oct 7, 2025
4838669
Use CSS border box
foolip Oct 21, 2025
9fa39e9
Replace hit testing API with a simpler model
foolip Nov 11, 2025
f79ed95
Add a hit testing example
foolip Nov 11, 2025
0a243be
Add getElementTransform() and related algorithms
foolip Dec 2, 2025
1f71b9a
Update drawElementImage() to return matrix
foolip Dec 4, 2025
5aeb03b
Express the transformations in a more approachable way
foolip Dec 4, 2025
951bdee
Address progers feedback
foolip Dec 5, 2025
ba5c560
<data> -> <code>
foolip Dec 5, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
283 changes: 277 additions & 6 deletions source
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4033,6 +4033,14 @@ a.setAttribute('href', 'https://example.com/'); // change the content attribute
<li>The <dfn data-x-href="https://drafts.csswg.org/css-text/#propdef-word-spacing">'word-spacing'</dfn> property</li>
</ul>

<p>The following features are defined in <cite>CSS Transforms</cite>: <ref>CSSTRANSFORMS</ref></p>

<ul class="brief">
<li>The <dfn data-x-href="https://drafts.csswg.org/css-transforms/#transform-property">'transform'</dfn> property</li>
<li>The <dfn data-x-href="https://drafts.csswg.org/css-transforms/#transform-origin-property">'transform-origin'</dfn> property</li>
<li>The <dfn data-x-href="https://drafts.csswg.org/css-transforms/#transformation-matrix">transformation matrix</dfn></li>
</ul>

<p>The following features are defined in <cite>CSS Writing Modes</cite>: <ref>CSSWM</ref></p>

<ul class="brief">
Expand Down Expand Up @@ -68316,6 +68324,7 @@ dictionary <dfn dictionary>AssignedNodesOptions</dfn> {
<dd><span>Global attributes</span></dd>
<dd><code data-x="attr-canvas-width">width</code></dd>
<dd><code data-x="attr-canvas-height">height</code></dd>
<dd><code data-x="attr-canvas-layoutsubtree">layoutsubtree</code></dd>
<dt><span
data-x="concept-element-accessibility-considerations">Accessibility considerations</span>:</dt>
<dd><a href="https://w3c.github.io/html-aria/#el-canvas">For authors</a>.</dd>
Expand All @@ -68336,6 +68345,9 @@ interface <dfn interface>HTMLCanvasElement</dfn> : <span>HTMLElement</span> {
USVString <span data-x="dom-canvas-toDataURL">toDataURL</span>(optional DOMString type = "image/png", optional any quality);
undefined <span data-x="dom-canvas-toBlob">toBlob</span>(<span>BlobCallback</span> _callback, optional DOMString type = "image/png", optional any quality);
<span>OffscreenCanvas</span> <span data-x="dom-canvas-transferControlToOffscreen">transferControlToOffscreen</span>();

[<span>CEReactions</span>] attribute boolean <span data-x="dom-canvas-layoutSubtree">layoutSubtree</span>;
DOMMatrix <span data-x="dom-canvas-getElementTransform">getElementTransform</span>(Element element, DOMMatrix drawTransform);
};

callback <dfn callback>BlobCallback</dfn> = undefined (<span>Blob</span>? blob);</code></pre>
Expand Down Expand Up @@ -68424,6 +68436,12 @@ callback <dfn callback>BlobCallback</dfn> = undefined (<span>Blob</span>? blob);
<p>The user agent must use a square pixel density consisting of one pixel of image data per
coordinate space unit for the bitmaps of a <code>canvas</code> and its rendering contexts.</p>

<p>The <dfn element-attr for="canvas"><code
data-x="attr-canvas-layoutsubtree">layoutsubtree</code></dfn> attribute is a <span>boolean
attribute</span>. If present, <span data-x="concept-tree-child">children</span> of the
<code>canvas</code> element are laid out, so that they can be drawn using <code
data-x="dom-context-2d-drawElementImage">drawElementImage()</code>.</p>

<p class="note">A <code>canvas</code> element can be sized arbitrarily by a style sheet, its
bitmap is then subject to the <span>'object-fit'</span> CSS property.</p>

Expand Down Expand Up @@ -68534,10 +68552,37 @@ callback <dfn callback>BlobCallback</dfn> = undefined (<span>Blob</span>? blob);
</div>

<div algorithm>
<p>The <dfn attribute for="HTMLCanvasElement"><code data-x="dom-canvas-width">width</code></dfn>
and <dfn attribute for="HTMLCanvasElement"><code data-x="dom-canvas-height">height</code></dfn>
IDL attributes must <span>reflect</span> the respective content attributes of the same name, with
the same defaults.</p>
<p>The <dfn attribute for="HTMLCanvasElement"><code data-x="dom-canvas-width">width</code></dfn>,
<dfn attribute for="HTMLCanvasElement"><code data-x="dom-canvas-height">height</code></dfn>, and
<dfn attribute for="HTMLCanvasElement"><code
data-x="dom-canvas-layoutSubtree">layoutSubtree</code></dfn> IDL attributes must
<span>reflect</span> the respective content attributes of the same name, with the same
defaults.</p>
</div>

<div algorithm>
<p><code>HTMLCanvasElement</code>'s <dfn method for="HTMLCanvasElement"><code
data-x="dom-canvas-getElementTransform">getElementTransform(<var>element</var>,
<var>drawTransform</var>)</code></dfn> method steps are to <span>get element transform</span>
with <span>this</span>, <var>element</var>, and <var>drawTransform</var>.</p>
</div>

<div algorithm>
<p>To <dfn>get element transform</dfn> with an <code>HTMLCanvasElement</code> <var>canvas</var>,
an element <var>element</var>, and a transformation matrix <var>T<sub>draw</sub></var>:</p>

<ol>
<li><p>Let <var>T<sub>origin</sub></var> be a translation matrix corresponding to the
<span>computed value</span> of the <span>'transform-origin'</span> property of
<var>element</var>.</p></li>

<li><p>Let <var>T<sub>scale</sub></var> be a scaling matrix corresponding to the scale difference
between <span data-x="'px'">CSS pixels</span> and <var>canvas</var>'s backing store's
pixels.</p></li>

<li><p>Return a <code>DOMMatrix</code> initialized to
<var>T<sub>origin</sub></var><sup>-1</sup>&sdot;<var>T<sub>scale</sub></var><sup>-1</sup>&sdot;<var>T<sub>draw</sub></var>&sdot;<var>T<sub>scale</sub></var>&sdot;<var>T<sub>origin</sub></var>.</p></li>
</ol>
</div>

</div>
Expand Down Expand Up @@ -68933,6 +68978,7 @@ interface <dfn interface>CanvasRenderingContext2D</dfn> {
<span>CanvasRenderingContext2D</span> includes <span>CanvasDrawPath</span>;
<span>CanvasRenderingContext2D</span> includes <span>CanvasUserInterface</span>;
<span>CanvasRenderingContext2D</span> includes <span>CanvasText</span>;
<span>CanvasRenderingContext2D</span> includes <span>CanvasDrawElementImage</span>;
<span>CanvasRenderingContext2D</span> includes <span>CanvasDrawImage</span>;
<span>CanvasRenderingContext2D</span> includes <span>CanvasImageData</span>;
<span>CanvasRenderingContext2D</span> includes <span>CanvasPathDrawingStyles</span>;
Expand Down Expand Up @@ -69052,6 +69098,12 @@ interface mixin <dfn interface>CanvasText</dfn> {
<span>TextMetrics</span> <span data-x="dom-context-2d-measureText">measureText</span>(DOMString text);
};

interface mixin <dfn interface>CanvasDrawElementImage</dfn> {
// drawing elements
DOMMatrix <span data-x="dom-context-2d-drawElementImage">drawElementImage</span>(<span>Element</span> element, unrestricted double dx, unrestricted double dy);
DOMMatrix <span data-x="dom-context-2d-drawElementImage">drawElementImage</span>(<span>Element</span> element, unrestricted double dx, unrestricted double dy, unrestricted double dw, unrestricted double dh);
};

interface mixin <dfn interface>CanvasDrawImage</dfn> {
// drawing images
undefined <span data-x="dom-context-2d-drawImage">drawImage</span>(<span>CanvasImageSource</span> image, unrestricted double dx, unrestricted double dy);
Expand Down Expand Up @@ -73525,6 +73577,206 @@ try {

</div>


<h6>Drawing elements</h6>

<dl class="domintro">
<dt><code data-x=""><var>transform</var> = <var>context</var>.<span subdfn data-x="dom-context-2d-drawElementImage">drawElementImage</span>(<var>element</var>, <var>dx</var>, <var>dy</var>)</code></dt>
<dt><code data-x=""><var>transform</var> = <var>context</var>.<span data-x="dom-context-2d-drawElementImage">drawElementImage</span>(<var>element</var>, <var>dx</var>, <var>dy</var>, <var>dw</var>, <var>dh</var>)</code></dt>

<dd>
<p>Draws the given element onto the canvas. Throws a <code>TypeError</code> if the element isn't
a descendant of the canvas. Returns a transform that can be applied to the
<span>'transform'</span> property on element to align its location for hit testing with its
drawn location.</p>
</dd>
</dl>

<div w-nodev>

<p>Objects that implement the <code>CanvasDrawElementImage</code> interface have the <dfn method
for="CanvasDrawElementImage"><code
data-x="dom-context-2d-drawElementImage">drawElementImage()</code></dfn> method to draw
elements.</p>

<div algorithm>
<p>The <code data-x="dom-context-2d-drawElementImage">drawElementImage(element, x, y)</code> method,
when invoked, must <span>draw an element</span> with <span>this</span>, <var>element</var>,
<var>dx</var>, <var>dy</var>.</p>
</div>

<div algorithm>
<p>The <code data-x="dom-context-2d-drawElementImage">drawElementImage(element, x, y, w, h)</code>
method, when invoked, must <span>draw an element</span> with <span>this</span>,
<var>element</var>, <var>dx</var>, <var>dy</var>, <var>dw</var>, and <var>dh</var>.</p>
</div>

<div algorithm>
<p>To <dfn>draw an element</dfn>, with a <code>CanvasRenderingContext2D</code> <var>context</var>,
an element <var>element</var>, numbers <var>dx</var> and <var>dy</var>, and optional numbers
<var>dw</var> and <var>dh</var>:</p>

<ol>
<li><p>If <var>dx</var> or <var>dy</var> are infinite or NaN, then return.</p></li>

<li><p>If <var>dw</var> and <var>dh</var> are given and either are infinite or NaN, then
return.</p></li>

<li><p>Let <var>canvas</var> be the <code>canvas</code> element to which <var>context</var> is
bound.</p></li>

<li><p>If <var>element</var>'s <span>parent</span> is not <var>canvas</var>, then throw a
<code>TypeError</code>.</p></li>

<li><p>If <var>canvas</var> does not have a <code
data-x="attr-canvas-layoutsubtree">layoutsubtree</code> attribute specified, then throw a
<code>TypeError</code>.</p></li>

<li><p>Let <var>borderBox</var> be <var>element</var>'s <span>border box</span>.</p></li>

<li><p>Let <var>layoutWidth</var> and <var>layoutHeight</var> be the width and height of
<var>borderBox</var>, respectively.</p></li>

<li><p>If not given, <var>dw</var> and <var>dh</var> must default to the <var>layoutWidth</var>
and <var>layoutHeight</var>, respectively.</p></li>

<li><p>If either <var>dw</var> or <var>dh</var> are zero, then return.</p></li>

<li>
<p>Paint <var>element</var> to the specified rectangular area, ignoring <var>element</var>'s
<span>transformation matrix</span> and without using any <span>sensitive information</span>.
Instead, either paint nothing or use static information that is the same for all users.</p>

<p class="note">There is no requirement to synchronously rasterize into the backing store at
this point. As an optimization, implementations are free to instead store the information
required to rasterize later, as long as the difference is not observable.</p>
</li>

<li><p>Let <var>T<sub>draw</sub></var> be a copy of the <span>current transformation
matrix</span>.</p></li>

<li><p>Translate <var>T<sub>draw</sub></var> by (<var>dx</var>, <var>dy</var>).</p></li>

<li><p>Scale <var>T<sub>draw</sub></var> by (<var>dw</var> / <var>layoutWidth</var>,
<var>dh</var> / <var>layoutHeight</var>).</p></li>

<li><p>Return the result of <span>get element transform</span> with <var>context</var>'s
<code>canvas</code> element, <var>element</var>, and <var>T<sub>draw</sub></var>.</p></li>
</ol>
</div>

<p>When drawing elements to a <code>canvas</code>, no information should be used that isn't
otherwise observable to author code. Such <dfn export>sensitive information</dfn> includes
but isn't limited to:</p>

<ul>
<li><p><span>CORS-cross-origin</span> data</p></li>

<li><p>system colors, themes, or preferences</p></li>

<li><p>spelling and grammar markers</p></li>

<li><p>search text (find-in-page) and text-fragment (fragment url) markers</p></li>

<li><p>visited link information</p></li>

<li><p>form autofill information</p></li>
</ul>

</div>


<h6>Hit testing</h6>


<p>Descendants of a <code>canvas</code> element with the <code
data-x="attr-canvas-layoutsubtree">layoutsubtree</code> attribute specified participate in hit
testing.</p>


<div class="example" id="hitTestingExample">

<p>This example paints two elements with hit testing. Since all children are laid out at the
top left, the 'transform' CSS property is used to make their position for hit testing match
the drawn order. In order redraw when needed, for example if text is selected, a <code
data-x="">ResizeObserver</code> with the <code data-x="">fireOnEveryPaint</code> parameter is
used.</p>

<pre><code class="html">&lt;!doctype html>
&lt;meta charset="utf-8" />
&lt;title>Hit testing in canvas&lt;/title>

&lt;style>
canvas {
width: 720px;
height: 405px;
}
canvas > div {
background: white;
width: 300px;
height: 325px;
padding: 1em;
box-sizing: border-box;
}
canvas img {
max-width: 100%;
}
&lt;/style>

&lt;canvas layoutsubtree>
&lt;div id="left">
&lt;h1>Left side&lt;/h1>
&lt;p>Interactive content like &lt;a href="">links&lt;/a> work.
A long paragraph eventually breaks.&lt;/p>
&lt;/div>
&lt;div id="right">
&lt;h1>Right side&lt;/h1>
&lt;p>You wouldn't download a wolf:&lt;/p>
&lt;img src="wolf.jpg">&lt;/img>
&lt;/div>
&lt;/canvas>

&lt;script>
const canvas = document.querySelector('canvas');
const leftElm = document.getElementById('left');
const rightElm = document.getElementById('right');

const ctx = canvas.getContext('2d');
Copy link

@progers progers Dec 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I worry this example may be considered an anti-pattern because the purple area isn't hit testable. We could use canvas 2d drawing apis to draw an interesting path that includes both boxes and is painted as the background and which is hard to do with regular css?

Copy link
Member Author

@foolip foolip Dec 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, something like a peanut shape behind both elements would be hard to do with CSS even if we have border shaping, I'm guessing. I haven't done this just yet, but let me know if peanuts or peas in a pod or something along those lines makes sense to you.

Copy link

@progers progers Dec 6, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WDYT of this example? This draws a simple pie chart and uses drawElementImage for the labels.

<!doctype html>
<style>
  canvas { width: 250px; height: 250px; }
  canvas > div { text-align: center; max-width: 40%;}
</style>
<canvas layoutsubtree="true" role="list" aria-label="Example Pie Chart">
  <div role="listitem" tabindex="0" data-val="0.45" data-color="tomato">
    <b>45%</b><br>Apple - a crowd favorite.
  </div>
  <div role="listitem" tabindex="0" data-val="0.35" data-color="cornflowerblue">
    <b>35%</b><br>Blueberry - great for summer.
  </div>
  <div role="listitem" tabindex="0" data-val="0.20" data-color="gold">
    <b>20%</b><br>Durian - our newest flavor.
  </div>
</canvas>

<script>
  const canvas = document.querySelector('canvas');
  const ctx = canvas.getContext('2d');

  function drawPieChart() {
    // 1. Center the coordinate system.
    const radius = Math.min(canvas.width, canvas.height) / 2;
    ctx.translate(radius, radius);

    let angle = 0;
    for (const label of canvas.children) {
      const slice = Number(label.dataset.val) * Math.PI * 2;

      // 2. Draw the wedge.
      ctx.fillStyle = label.dataset.color;
      ctx.beginPath();
      ctx.moveTo(0, 0);
      ctx.arc(0, 0, radius, angle, angle + slice);
      ctx.fill();

      // 3. Draw the label element, and update its transform.
      const mid = angle + slice / 2;
      const label_width = label.offsetWidth * devicePixelRatio;
      const label_height = label.offsetHeight * devicePixelRatio;
      const x = Math.cos(mid) * radius * 0.60 - label_width / 2;
      const y = Math.sin(mid) * radius * 0.60 - label_height / 2;
      let transform = ctx.drawElementImage(label, x, y);
      label.style.transform = transform;

      angle += slice;
    }
  }

  new ResizeObserver(([entry]) => {
    // Size the canvas to the device pixel content box.
    const box = entry.devicePixelContentBoxSize[0];
    canvas.width = box.inlineSize;
    canvas.height = box.blockSize;

    ctx.reset();

    drawPieChart();
  }).observe(canvas, {box: ['device-pixel-content-box'], fireOnEveryPaint: true});
</script>

function draw() {
ctx.reset();

ctx.fillStyle = '#e0e0dd';
ctx.fillRect(0, 0, canvas.width, canvas.height);

let x = 40;
let y = 40;
let transform = ctx.drawElementImage(leftElm, x * devicePixelRatio, y * devicePixelRatio);
leftElm.style.transform = transform;

// purple circle goes in between the elements
ctx.fillStyle = 'purple';
ctx.beginPath();
ctx.arc(canvas.width / 2, canvas.height - 40, 140, 0, 2 * Math.PI);
ctx.fill();

x+= 340;
transform = ctx.drawElementImage(rightElm, x * devicePixelRatio, y * devicePixelRatio);
rightElm.style.transform = transform;
}

onload = () => {
const resizeObserver = new ResizeObserver(([entry]) => {
canvas.width = entry.devicePixelContentBoxSize[0].inlineSize;
canvas.height = entry.devicePixelContentBoxSize[0].blockSize;
draw();
});
resizeObserver.observe(canvas, {box: ['device-pixel-content-box'],
fireOnEveryPaint: true});
}
&lt;/script></code></pre>
</div>


<h6>Drawing images</h6>

<p>Objects that implement the <code>CanvasDrawImage</code> interface have the <dfn method
Expand Down Expand Up @@ -146568,8 +146820,18 @@ legend[align=right i] {
<p>A <code>canvas</code> element that <span>represents</span> <span>embedded content</span> is
<span>expected</span> to be treated as a <span>replaced element</span>; the contents of such
elements are the element's bitmap, if any, or else a <span>transparent black</span> bitmap with
the same <span>natural dimensions</span> as the element. Other <code>canvas</code> elements are
<span>expected</span> to be treated as ordinary elements in the rendering model.</p>
the same <span>natural dimensions</span> as the element. A <code>canvas</code> element that
<span>represents</span> <span>embedded content</span> and has a <code
data-x="attr-canvas-layoutsubtree">layoutsubtree</code> attribute specified is additionally
<span>expected</span> to be treated as 👋replaced element with subtree layout👋, where children are
laid out but not rendered. Additionally, children are exposed to assistive technologies (ATs).</p>

<p class="XXX">This needs to be reconciled with the definition of <span
data-x="replaced element">replaced elements</span>, which says that the content of replaced
elements is not considered in the CSS formatting model.</p>

<p>Other <code>canvas</code> elements are <span>expected</span> to be treated as ordinary elements
in the rendering model.</p>

<p>An <code>object</code> element that <span>represents</span> an image, plugin, or its
<span>content navigable</span> is <span>expected</span> to be treated as a <span>replaced
Expand Down Expand Up @@ -146619,6 +146881,7 @@ legend[align=right i] {

<pre><code class="css">@namespace "http://www.w3.org/1999/xhtml";

canvas[layoutsubtree] > * { isolation: isolate !important; contain: paint !important; }
iframe { border: 2px inset; }
<span id="video-object-fit">video { object-fit: contain; }</span></code></pre>

Expand Down Expand Up @@ -153012,6 +153275,11 @@ interface <dfn interface>External</dfn> {
<code data-x="attr-track-label">track</code>
<td> User-visible label
<td> <a href="#attribute-text">Text</a>
<tr>
<th> <code data-x="">layoutsubtree</code>
<td> <code data-x="attr-canvas-layoutsubtree">canvas</code>
<td> Whether to layout descendants
<td> <span>Boolean attribute</span>
<tr>
<th> <code data-x="">lang</code>
<td> <span data-x="attr-lang">HTML elements</span>
Expand Down Expand Up @@ -155264,6 +155532,9 @@ INSERT INTERFACES HERE
<dt id="refsCSSTEXT">[CSSTEXT]</dt>
<dd><cite><a href="https://drafts.csswg.org/css-text/">CSS Text</a></cite>, E. Etemad, K. Ishii. W3C.</dd>

<dt id="refsCSSTRANSFORMS">[CSSTRANSFORMS]</dt>
<dd><cite><a href="https://drafts.csswg.org/css-transforms/">CSS Transforms</a></cite>, L. Baron, S. Fraser, D. Jackson, T. O'Connor, D. Schulze. W3C.</dd>

<dt id="refsCSSVALUES">[CSSVALUES]</dt>
<dd><cite><a href="https://drafts.csswg.org/css-values/">CSS3 Values and Units</a></cite>, H. Lie, T. Atkins, E. Etemad. W3C.</dd>

Expand Down