>`
+- Handles indentation for multiline ERB-comment
+- Handles spaces between do-arguments and ERB-tags
+
+## [0.10.4] - 2023-08-28
+
+- Avoid grouping single tags
+- Handle multiline ERB-comments
+
+## [0.10.3] - 2023-08-27
+
+## Fixes
+
+- Allows parsing ERB-tags with if, else and end in the same tag
+
+```erb
+<%= if true
+ what
+end %>
+```
+
+This opens the possibility for formatting all if-statements with SyntaxTree properly
+and removes the fix where any if-statement was force to one line.
+
+## [0.10.2] - 2023-08-22
+
+### Fixes
+
+- Handles formatting empty documents and removing leading new-linews in files with content.
+- Removes trailing whitespace from char data if it is the last element in a document, block or group.
+
+## [0.10.1] - 2023-08-20
+
+### Added
+
+- Allow `DOCTYPE` to be after other tags, to work with e.g. ERB-tags on first line.
+
+## [0.10.0] - 2023-08-20
+
+- Changes how whitespace and newlines are handled.
+- Supports syntax like:
+
+```erb
+<%= part %> / <%= total %> (<%= percentage %>%)
+```
+
+## [0.9.5] - 2023-07-02
+
+- Fixes ruby comment in ERB-tag included VoidStatement
+ Example:
+
+```erb
+<% # this is a comment %>
+```
+
+Output:
+
+```diff
+-<%
+-
+- # this is a comment
+-%>
++<% # this is a comment %>
+```
+
+- Updates versions in Bundler
+
+## [0.9.4] - 2023-07-01
+
+- Inline even more empty HTML-tags
+
+```diff
+
+-
++>
+```
+
+## [0.9.3] - 2023-06-30
+
+- Print empty html-tags on one line if possible
+
+## [0.9.2] - 2023-06-30
+
+- Handle whitespace in HTML-strings using ERB-tags
+
+## [0.9.1] - 2023-06-28
+
+- Handle formatting of multi-line ERB-tags with more than one statement.
+
+## [0.9.0] - 2023-06-22
### Added
-- 🎉 Initial release! 🎉
+- 🎉 First version based on syntax_tree-xml 🎉.
+- Can format a lot of .html.erb-syntax and works as a plugin to syntax_tree.
+- This is still early and there are a lot of different weird syntaxes out there.
-[unreleased]: https://github.com/ruby-syntax-tree/syntax_tree-xml/compare/v0.1.0...HEAD
-[0.1.0]: https://github.com/ruby-syntax-tree/syntax_tree-xml/compare/c34baa...v0.1.0
+[unreleased]: https://github.com/davidwessman/syntax_tree-erb/compare/v0.10.5...HEAD
+[0.10.5]: https://github.com/davidwessman/syntax_tree-erb/compare/v0.10.4...v0.10.5
+[0.10.4]: https://github.com/davidwessman/syntax_tree-erb/compare/v0.10.3...v0.10.4
+[0.10.3]: https://github.com/davidwessman/syntax_tree-erb/compare/v0.10.2...v0.10.3
+[0.10.2]: https://github.com/davidwessman/syntax_tree-erb/compare/v0.10.1...v0.10.2
+[0.10.1]: https://github.com/davidwessman/syntax_tree-erb/compare/v0.10.0...v0.10.1
+[0.10.0]: https://github.com/davidwessman/syntax_tree-erb/compare/v0.9.5...v0.10.0
+[0.9.5]: https://github.com/davidwessman/syntax_tree-erb/compare/v0.9.4...v0.9.5
+[0.9.4]: https://github.com/davidwessman/syntax_tree-erb/compare/v0.9.3...v0.9.4
+[0.9.3]: https://github.com/davidwessman/syntax_tree-erb/compare/v0.9.2...v0.9.3
+[0.9.2]: https://github.com/davidwessman/syntax_tree-erb/compare/v0.9.1...v0.9.2
+[0.9.1]: https://github.com/davidwessman/syntax_tree-erb/compare/v0.9.0...v0.9.1
+[0.9.0]: https://github.com/davidwessman/syntax_tree-erb/compare/419727a73af94057ca0980733e69ac8b4d52fdf4...v0.9.0
diff --git a/Gemfile.lock b/Gemfile.lock
index 8dcfc3c..75c8f6c 100644
--- a/Gemfile.lock
+++ b/Gemfile.lock
@@ -1,37 +1,39 @@
PATH
remote: .
specs:
- syntax_tree-xml (0.1.0)
- prettier_print (>= 1.0.2)
- syntax_tree (>= 4.0.1)
+ w_syntax_tree-erb (0.11.0)
+ prettier_print (~> 1.2, >= 1.2.0)
+ syntax_tree (~> 6.1, >= 6.1.1)
GEM
remote: https://rubygems.org/
specs:
docile (1.4.0)
- minitest (5.18.0)
+ minitest (5.20.0)
prettier_print (1.2.1)
- rake (13.0.6)
+ rake (13.1.0)
simplecov (0.22.0)
docile (~> 1.1)
simplecov-html (~> 0.11)
simplecov_json_formatter (~> 0.1)
simplecov-html (0.12.3)
simplecov_json_formatter (0.1.4)
- syntax_tree (6.1.1)
+ syntax_tree (6.2.0)
prettier_print (>= 1.2.0)
PLATFORMS
arm64-darwin-21
+ arm64-darwin-23
x86_64-darwin-21
+ x86_64-darwin-22
x86_64-linux
DEPENDENCIES
- bundler
- minitest
- rake
- simplecov
- syntax_tree-xml!
+ bundler (~> 2)
+ minitest (~> 5)
+ rake (~> 13)
+ simplecov (~> 0.22)
+ w_syntax_tree-erb!
BUNDLED WITH
- 2.3.6
+ 2.4.1
diff --git a/README.md b/README.md
index ca86216..420075d 100644
--- a/README.md
+++ b/README.md
@@ -1,75 +1,106 @@
-# SyntaxTree::XML
+# SyntaxTree::ERB
-[](https://github.com/ruby-syntax-tree/syntax_tree-xml/actions/workflows/main.yml)
-[](https://rubygems.org/gems/syntax_tree-xml)
+[](https://github.com/davidwessman/syntax_tree-erb/actions/workflows/main.yml)
-[Syntax Tree](https://github.com/ruby-syntax-tree/syntax_tree) support for XML.
+[Syntax Tree](https://github.com/ruby-syntax-tree/syntax_tree) support for ERB.
+
+Currently handles
+
+- ERB
+ - Tags with and without output
+ - Tags inside strings
+ - `if`, `elsif`, `else` and `unless` statements
+ - blocks
+ - comments
+ - Formatting of the ruby-code is done by `syntax_tree`
+- HTML
+ - Tags with attributes
+ - Tags with and without closing tags
+ - Comments
+- Vue
+ - Attributes, events and slots using `:`, `@` and `#` respectively
+- Text output
+
+## Unhandled cases
+
+- Please add to this pinned issue (https://github.com/davidwessman/syntax_tree-erb/issues/28) or create a separate issue if you encounter formatting or parsing errors.
## Installation
Add this line to your application's Gemfile:
```ruby
-gem "syntax_tree-xml"
+gem "w_syntax_tree-erb", "~> 0.10", require: false
```
-And then execute:
-
- $ bundle install
-
-Or install it yourself as:
-
- $ gem install syntax_tree-xml
+> I added the `w_` prefix to avoid conflicts if there will ever be an official `syntax_tree-erb` gem.
## Usage
+```sh
+bundle exec stree --plugins=erb "./**/*.html.erb"
+```
+
From code:
```ruby
-require "syntax_tree/xml"
+require "syntax_tree/erb"
+
+pp SyntaxTree::ERB.parse(source) # print out the AST
+puts SyntaxTree::ERB.format(source) # format the AST
+```
-pp SyntaxTree::XML.parse(source) # print out the AST
-puts SyntaxTree::XML.format(source) # format the AST
+## List all parsing errors
+
+In order to get a list of all parsing errors (which needs to be fixed before the formatting works), this script can be used:
+
+```ruby
+#!/bin/ruby
+
+require "syntax_tree/erb"
+
+failures = []
+
+Dir
+ .glob("./app/**/*.html.erb")
+ .each do |file|
+ puts("Processing #{file}")
+ begin
+ source = SyntaxTree::ERB.read(file)
+ SyntaxTree::ERB.parse(source)
+ SyntaxTree::ERB.format(source)
+ rescue => exception
+ failures << { file: file, message: exception.message }
+ end
+ end
+
+puts failures
```
-From the CLI:
+## Development
+
+Install `husky`:
```sh
-$ stree ast --plugins=xml file.xml
-(document
- (misc "\n"),
- (element
- (opening_tag "<", "message", ">"),
- (char_data "\n" + " "),
- (element (opening_tag "<", "hello", ">"), (char_data "Hello"), (closing_tag "")),
- (char_data "\n" + " "),
- (element (opening_tag "<", "world", ">"), (char_data "World"), (closing_tag "")),
- (char_data "\n"),
- (closing_tag "")
- )
-)
+npm i -g husky
```
-or
+Setup linting:
```sh
-$ stree format --plugins=xml file.xml
-
- Hello
- World
-
+npm run prepare
```
-or
+Install dependencies and run tests:
```sh
-$ stree write --plugins=xml file.xml
-file.xml 1ms
+bundle
+bundle exec rake
```
## Contributing
-Bug reports and pull requests are welcome on GitHub at https://github.com/ruby-syntax-tree/syntax_tree-xml.
+Bug reports and pull requests are welcome on GitHub at https://github.com/davidwessman/syntax_tree-erb.
## License
diff --git a/check_erb_parse.rb b/check_erb_parse.rb
new file mode 100644
index 0000000..36a7901
--- /dev/null
+++ b/check_erb_parse.rb
@@ -0,0 +1,20 @@
+#!/bin/ruby
+
+require "syntax_tree/erb"
+
+failures = []
+
+Dir
+ .glob("./app/**/*.html.erb")
+ .each do |file|
+ puts("Processing #{file}")
+ begin
+ source = SyntaxTree::ERB.read(file)
+ SyntaxTree::ERB.parse(source)
+ SyntaxTree::ERB.format(source)
+ rescue => exception
+ failures << { file: file, message: exception.message }
+ end
+ end
+
+puts failures
diff --git a/lib/syntax_tree/xml.rb b/lib/syntax_tree/erb.rb
similarity index 50%
rename from lib/syntax_tree/xml.rb
rename to lib/syntax_tree/erb.rb
index 8f24808..822a1fb 100644
--- a/lib/syntax_tree/xml.rb
+++ b/lib/syntax_tree/erb.rb
@@ -3,16 +3,17 @@
require "prettier_print"
require "syntax_tree"
-require_relative "xml/nodes"
-require_relative "xml/parser"
-require_relative "xml/visitor"
+require_relative "erb/nodes"
+require_relative "erb/parser"
+require_relative "erb/visitor"
-require_relative "xml/format"
-require_relative "xml/pretty_print"
+require_relative "erb/format"
+require_relative "erb/pretty_print"
module SyntaxTree
- module XML
- def self.format(source, maxwidth = 80)
+ module ERB
+ MAX_WIDTH = 80
+ def self.format(source, maxwidth = MAX_WIDTH, options: nil)
PrettierPrint.format(+"", maxwidth) { |q| parse(source).format(q) }
end
@@ -25,5 +26,6 @@ def self.read(filepath)
end
end
- register_handler(".xml", XML)
+ register_handler(".html.erb", ERB)
+ register_handler(".erb", ERB)
end
diff --git a/lib/syntax_tree/erb/format.rb b/lib/syntax_tree/erb/format.rb
new file mode 100644
index 0000000..3e2d7ac
--- /dev/null
+++ b/lib/syntax_tree/erb/format.rb
@@ -0,0 +1,378 @@
+# frozen_string_literal: true
+
+module SyntaxTree
+ module ERB
+ class Format < Visitor
+ attr_reader :q
+
+ def initialize(q)
+ @q = q
+ end
+
+ # Visit a Token node.
+ def visit_token(node)
+ if %i[text whitespace].include?(node.type)
+ q.text(node.value)
+ else
+ q.text(node.value.strip)
+ end
+ end
+
+ # Visit a Document node.
+ def visit_document(node)
+ child_nodes = node.child_nodes.sort_by(&:location)
+
+ handle_child_nodes(child_nodes)
+
+ q.breakable(force: true)
+ end
+
+ def visit_block(node)
+ visit(node.opening)
+
+ breakable = breakable_inside(node)
+ if node.elements.any?
+ q.indent do
+ q.breakable if breakable
+ handle_child_nodes(node.elements)
+ end
+ end
+
+ if node.closing
+ q.breakable("") if breakable
+ visit(node.closing)
+ end
+ end
+
+ def visit_html_groupable(node, group)
+ if node.elements.size == 0
+ visit(node.opening)
+ visit(node.closing)
+ else
+ visit(node.opening)
+
+ with_break = breakable_inside(node)
+ q.indent do
+ if with_break
+ group ? q.breakable("") : q.breakable
+ end
+ handle_child_nodes(node.elements)
+ end
+
+ if with_break
+ group ? q.breakable("") : q.breakable
+ end
+ visit(node.closing)
+ end
+ end
+
+ def visit_html(node)
+ # Make sure to group the tags together if there is no child nodes.
+ if node.elements.size == 0 ||
+ node.elements.any? { |node|
+ node.is_a?(SyntaxTree::ERB::CharData)
+ } ||
+ (
+ node.elements.size == 1 &&
+ node.elements.first.is_a?(SyntaxTree::ERB::ErbNode)
+ )
+ q.group { visit_html_groupable(node, true) }
+ else
+ visit_html_groupable(node, false)
+ end
+ end
+
+ def visit_erb_block(node)
+ visit_block(node)
+ end
+
+ def visit_erb_if(node)
+ visit_block(node)
+ end
+
+ def visit_erb_elsif(node)
+ visit_block(node)
+ end
+
+ def visit_erb_else(node)
+ visit_block(node)
+ end
+
+ def visit_erb_case(node)
+ visit_block(node)
+ end
+
+ def visit_erb_case_when(node)
+ visit_block(node)
+ end
+
+ # Visit an ErbNode node.
+ def visit_erb(node)
+ visit(node.opening_tag)
+
+ if node.keyword
+ q.text(" ")
+ visit(node.keyword)
+ end
+
+ node.content.blank? ? q.text(" ") : visit(node.content)
+
+ visit(node.closing_tag)
+ end
+
+ def visit_erb_do_close(node)
+ closing = node.closing.value.end_with?("-%>") ? "-%>" : "%>"
+ q.text(node.closing.value.gsub(closing, "").rstrip)
+ q.text(" ")
+ q.text(closing)
+ end
+
+ def visit_erb_close(node)
+ visit(node.closing)
+ end
+
+ # Visit an ErbEnd node.
+ def visit_erb_end(node)
+ visit(node.opening_tag)
+ q.text(" ")
+ visit(node.keyword)
+ q.text(" ")
+ visit(node.closing_tag)
+ end
+
+ def visit_erb_content(node)
+ # Reject all VoidStmt to avoid empty lines
+ nodes =
+ (node.value&.statements&.child_nodes || []).reject do |node|
+ node.is_a?(SyntaxTree::VoidStmt)
+ end
+
+ if nodes.size == 1
+ q.text(" ")
+ format_statement(nodes.first)
+ q.text(" ")
+ elsif nodes.size > 1
+ q.indent do
+ q.breakable("")
+ q.seplist(nodes, -> { q.breakable("") }) do |child_node|
+ format_statement(child_node)
+ end
+ end
+
+ q.breakable
+ end
+ end
+
+ def format_statement(statement)
+ formatter =
+ SyntaxTree::Formatter.new("", [], SyntaxTree::ERB::MAX_WIDTH)
+
+ formatter.format(statement)
+ formatter.flush
+ rows = formatter.output.join.split("\n")
+
+ output_rows(rows)
+ end
+
+ def output_rows(rows)
+ if rows.size > 1
+ q.seplist(rows, -> { q.breakable("") }) { |row| q.text(row) }
+ elsif rows.size == 1
+ q.text(rows.first)
+ end
+ end
+
+ # Visit an HtmlNode::OpeningTag node.
+ def visit_opening_tag(node)
+ q.group do
+ visit(node.opening)
+ visit(node.name)
+
+ if node.attributes.any?
+ q.indent do
+ q.breakable
+ q.seplist(node.attributes, -> { q.breakable }) do |child_node|
+ visit(child_node)
+ end
+ end
+
+ # Only add breakable if we have attributes
+ q.breakable(node.closing.value == "/>" ? " " : "")
+ elsif node.closing.value == "/>"
+ # Need a space before end-tag for self-closing
+ q.text(" ")
+ end
+
+ # If element is a valid void element, but not currently self-closing
+ # format to be self-closing
+ q.text(" /") if node.is_void_element? and node.closing.value == ">"
+
+ visit(node.closing)
+ end
+ end
+
+ # Visit an HtmlNode::ClosingTag node.
+ def visit_closing_tag(node)
+ q.group do
+ visit(node.opening)
+ visit(node.name)
+ visit(node.closing)
+ end
+ end
+
+ # Visit an Attribute node.
+ def visit_attribute(node)
+ q.group do
+ visit(node.key)
+ visit(node.equals)
+ visit(node.value)
+ end
+ end
+
+ # Visit a HtmlString node.
+ def visit_html_string(node)
+ q.group do
+ q.text("\"")
+ q.seplist(node.contents, -> { "" }) { |child_node| visit(child_node) }
+ q.text("\"")
+ end
+ end
+
+ def visit_html_comment(node)
+ visit(node.token)
+ end
+
+ def visit_erb_comment(node)
+ q.seplist(node.token.value.split("\n"), -> { q.breakable }) do |line|
+ q.text(line.lstrip)
+ end
+ end
+
+ # Visit a CharData node.
+ def visit_char_data(node)
+ return if node.value.value.strip.empty?
+
+ q.text(node.value.value)
+ end
+
+ def visit_new_line(node)
+ q.breakable(force: :skip_parent_break)
+ q.breakable(force: :skip_parent_break) if node.count > 1
+ end
+
+ # Visit a Doctype node.
+ def visit_doctype(node)
+ q.group do
+ visit(node.opening)
+ q.text(" ")
+ visit(node.name)
+
+ visit(node.closing)
+ end
+ end
+
+ private
+
+ def breakable_inside(node)
+ if node.is_a?(SyntaxTree::ERB::HtmlNode)
+ node.elements.first.class != SyntaxTree::ERB::CharData ||
+ node_new_line_count(node.opening) > 0
+ elsif node.is_a?(SyntaxTree::ERB::Block)
+ true
+ end
+ end
+
+ def breakable_between(node, next_node)
+ new_lines = node_new_line_count(node)
+
+ if new_lines == 1
+ q.breakable
+ elsif new_lines > 1
+ q.breakable
+ q.breakable(force: :skip_parent_break)
+ elsif next_node && !node.is_a?(SyntaxTree::ERB::CharData) &&
+ !next_node.is_a?(SyntaxTree::ERB::CharData)
+ q.breakable
+ end
+ end
+
+ def breakable_between_group(node, next_node)
+ new_lines = node_new_line_count(node)
+
+ if new_lines == 1
+ q.breakable(force: true)
+ elsif new_lines > 1
+ q.breakable(force: true)
+ q.breakable(force: true)
+ elsif next_node && !node.is_a?(SyntaxTree::ERB::CharData) &&
+ !next_node.is_a?(SyntaxTree::ERB::CharData)
+ q.breakable("")
+ end
+ end
+
+ def node_new_line_count(node)
+ node.respond_to?(:new_line) ? node.new_line&.count || 0 : 0
+ end
+
+ def handle_child_nodes(child_nodes)
+ group = []
+
+ if child_nodes.size == 1
+ visit(child_nodes.first.without_new_line)
+ return
+ end
+
+ child_nodes.each_with_index do |child_node, index|
+ is_last = index == child_nodes.size - 1
+
+ # Last element should not have new lines
+ node = is_last ? child_node.without_new_line : child_node
+
+ if node_should_group(node)
+ group << node
+ next
+ end
+
+ # Render all group elements before the current node
+ handle_group(group, break_after: true)
+ group = []
+
+ # Render the current node
+ visit(node)
+ next_node = child_nodes[index + 1]
+
+ breakable_between(node, next_node)
+ end
+
+ # Handle group if we have any nodes left
+ handle_group(group, break_after: false)
+ end
+
+ def handle_group(nodes, break_after:)
+ if nodes.size == 1
+ handle_group_nodes(nodes)
+ elsif nodes.size > 1
+ q.group { handle_group_nodes(nodes) }
+ else
+ return
+ end
+
+ breakable_between_group(nodes.last, nil) if break_after
+ end
+
+ def handle_group_nodes(nodes)
+ nodes.each_with_index do |node, group_index|
+ visit(node)
+ next_node = nodes[group_index + 1]
+ next if next_node.nil?
+ breakable_between_group(node, next_node)
+ end
+ end
+
+ def node_should_group(node)
+ node.is_a?(SyntaxTree::ERB::CharData) ||
+ node.is_a?(SyntaxTree::ERB::ErbNode)
+ end
+ end
+ end
+end
diff --git a/lib/syntax_tree/erb/nodes.rb b/lib/syntax_tree/erb/nodes.rb
new file mode 100644
index 0000000..a644843
--- /dev/null
+++ b/lib/syntax_tree/erb/nodes.rb
@@ -0,0 +1,721 @@
+# frozen_string_literal: true
+
+module SyntaxTree
+ module ERB
+ # A Location represents a position for a node in the source file.
+ class Location
+ attr_reader :start_char, :end_char, :start_line, :end_line
+
+ def initialize(start_char:, end_char:, start_line:, end_line:)
+ @start_char = start_char
+ @end_char = end_char
+ @start_line = start_line
+ @end_line = end_line
+ end
+
+ def deconstruct_keys(keys)
+ {
+ start_char: start_char,
+ end_char: end_char,
+ start_line: start_line,
+ end_line: end_line
+ }
+ end
+
+ def to(other)
+ Location.new(
+ start_char: start_char,
+ start_line: start_line,
+ end_char: other.end_char,
+ end_line: other.end_line
+ )
+ end
+
+ def <=>(other)
+ start_char <=> other.start_char
+ end
+
+ def to_s
+ if start_line == end_line
+ "line #{start_line}, char #{start_char}..#{end_char}"
+ else
+ "line #{start_line},char #{start_char} to line #{end_line}, char #{end_char}"
+ end
+ end
+ end
+
+ # A parent node that contains a bit of shared functionality.
+ class Node
+ def format(q)
+ Format.new(q).visit(self)
+ end
+
+ def pretty_print(q)
+ PrettyPrint.new(q).visit(self)
+ end
+
+ def without_new_line
+ self
+ end
+
+ def skip?
+ false
+ end
+ end
+
+ # A Token is any kind of lexical token from the source. It has a type, a
+ # value which is a subset of the source, and an index where it starts in
+ # the source.
+ class Token < Node
+ attr_reader :type, :value, :location
+
+ def initialize(type:, value:, location:)
+ @type = type
+ @value = value
+ @location = location
+ end
+
+ def accept(visitor)
+ visitor.visit_token(self)
+ end
+
+ def child_nodes
+ []
+ end
+
+ alias deconstruct child_nodes
+
+ def deconstruct_keys(keys)
+ { type: type, value: value, location: location }
+ end
+ end
+
+ # The Document node is the top of the syntax tree.
+ # It contains any number of:
+ # - Text
+ # - HtmlNode
+ # - ErbNodes
+ class Document < Node
+ attr_reader :elements, :location
+
+ def initialize(elements:, location:)
+ @elements = elements
+ @location = location
+ end
+
+ def accept(visitor)
+ visitor.visit_document(self)
+ end
+
+ def child_nodes
+ [*elements].compact
+ end
+
+ alias deconstruct child_nodes
+
+ def deconstruct_keys(keys)
+ { elements: elements, location: location }
+ end
+ end
+
+ # This is a base class for a Node that can also hold an appended
+ # new line.
+ class Element < Node
+ attr_reader(:new_line, :location)
+
+ def initialize(new_line:, location:)
+ @new_line = new_line
+ @location = location
+ end
+
+ def without_new_line
+ self.class.new(**deconstruct_keys([]).merge(new_line: nil))
+ end
+
+ def deconstruct_keys(keys)
+ { new_line: new_line, location: location }
+ end
+ end
+
+ # This is a base class for a block that contains:
+ # - an opening
+ # - optional elements
+ # - optional closing
+ class Block < Node
+ attr_reader(:opening, :elements, :closing, :location)
+ def initialize(opening:, location:, elements: nil, closing: nil)
+ @opening = opening
+ @elements = elements || []
+ @closing = closing
+ @location = location
+ end
+
+ def accept(visitor)
+ visitor.visit_block(self)
+ end
+
+ def child_nodes
+ [opening, *elements, closing].compact
+ end
+
+ def new_line
+ closing.new_line if closing.respond_to?(:new_line)
+ end
+
+ def without_new_line
+ self.class.new(
+ **deconstruct_keys([]).merge(closing: closing&.without_new_line)
+ )
+ end
+
+ alias deconstruct child_nodes
+
+ def deconstruct_keys(keys)
+ {
+ opening: opening,
+ elements: elements,
+ closing: closing,
+ location: location
+ }
+ end
+ end
+
+ # An element is a child of the document. It contains an opening tag, any
+ # optional content within the tag, and a closing tag. It can also
+ # potentially contain an opening tag that self-closes, in which case the
+ # content and closing tag will be nil.
+ class HtmlNode < Block
+ # These elements do not require a closing tag
+ # https://developer.mozilla.org/en-US/docs/Glossary/Void_element
+ HTML_VOID_ELEMENTS = %w[
+ area
+ base
+ br
+ col
+ embed
+ hr
+ img
+ input
+ link
+ meta
+ param
+ source
+ track
+ wbr
+ ]
+
+ # The opening tag of an element. It contains the opening character (<),
+ # the name of the element, any optional attributes, and the closing
+ # token (either > or />).
+ class OpeningTag < Element
+ attr_reader :opening, :name, :attributes, :closing
+
+ def initialize(
+ opening:,
+ name:,
+ attributes:,
+ closing:,
+ new_line:,
+ location:
+ )
+ super(new_line: new_line, location: location)
+ @opening = opening
+ @name = name
+ @attributes = attributes
+ @closing = closing
+ end
+
+ def accept(visitor)
+ visitor.visit_opening_tag(self)
+ end
+
+ def child_nodes
+ [opening, name, *attributes, closing]
+ end
+
+ def is_void_element?
+ HTML_VOID_ELEMENTS.include?(name.value)
+ end
+
+ alias deconstruct child_nodes
+
+ def deconstruct_keys(keys)
+ super.merge(
+ opening: opening,
+ name: name,
+ attributes: attributes,
+ closing: closing
+ )
+ end
+ end
+
+ # The closing tag of an element. It contains the opening character (<),
+ # the name of the element, and the closing character (>).
+ class ClosingTag < Element
+ attr_reader :opening, :name, :closing
+
+ def initialize(opening:, name:, closing:, location:, new_line:)
+ super(new_line: new_line, location: location)
+ @opening = opening
+ @name = name
+ @closing = closing
+ end
+
+ def accept(visitor)
+ visitor.visit_closing_tag(self)
+ end
+
+ def child_nodes
+ [opening, name, closing]
+ end
+
+ alias deconstruct child_nodes
+
+ def deconstruct_keys(keys)
+ super.merge(opening: opening, name: name, closing: closing)
+ end
+ end
+
+ def is_void_element?
+ false
+ end
+
+ def without_new_line
+ self.class.new(
+ **deconstruct_keys([]).merge(
+ opening: closing.nil? ? opening.without_new_line : opening,
+ closing: closing&.without_new_line
+ )
+ )
+ end
+
+ # The HTML-closing tag is responsible for new lines after the node.
+ def new_line
+ closing.nil? ? opening.new_line : closing&.new_line
+ end
+
+ def accept(visitor)
+ visitor.visit_html(self)
+ end
+ end
+
+ class ErbNode < Element
+ attr_reader :opening_tag, :keyword, :content, :closing_tag
+
+ def initialize(
+ opening_tag:,
+ keyword:,
+ content:,
+ closing_tag:,
+ new_line:,
+ location:
+ )
+ super(new_line: new_line, location: location)
+ @opening_tag = opening_tag
+ # prune whitespace from keyword
+ @keyword =
+ if keyword
+ Token.new(
+ type: keyword.type,
+ value: keyword.value.strip,
+ location: keyword.location
+ )
+ end
+
+ @content = prepare_content(content)
+ @closing_tag = closing_tag
+ end
+
+ def accept(visitor)
+ visitor.visit_erb(self)
+ end
+
+ def child_nodes
+ [opening_tag, keyword, content, closing_tag].compact
+ end
+
+ def new_line
+ closing_tag&.new_line
+ end
+
+ def without_new_line
+ self.class.new(
+ **deconstruct_keys([]).merge(
+ closing_tag: closing_tag.without_new_line
+ )
+ )
+ end
+
+ alias deconstruct child_nodes
+
+ def deconstruct_keys(keys)
+ super.merge(
+ opening_tag: opening_tag,
+ keyword: keyword,
+ content: content,
+ closing_tag: closing_tag
+ )
+ end
+
+ private
+
+ def prepare_content(content)
+ if content.is_a?(ErbContent)
+ content
+ else
+ # Set content to nil if it is empty
+ content ||= []
+
+ ErbContent.new(value: content)
+ end
+ rescue SyntaxTree::Parser::ParseError
+ # Try to add the keyword to see if it parses
+ result = ErbContent.new(value: [keyword, *content])
+ @keyword = nil
+ result
+ end
+ end
+
+ class ErbBlock < Block
+ def initialize(opening:, location:, elements: nil, closing: nil)
+ super(
+ opening: opening,
+ location: location,
+ elements: elements,
+ closing: closing
+ )
+ end
+
+ def accept(visitor)
+ visitor.visit_erb_block(self)
+ end
+ end
+
+ class ErbClose < Element
+ attr_reader :closing
+
+ def initialize(closing:, new_line:, location:)
+ super(new_line: new_line, location: location)
+ @closing = closing
+ end
+
+ def accept(visitor)
+ visitor.visit_erb_close(self)
+ end
+
+ def child_nodes
+ []
+ end
+
+ alias deconstruct child_nodes
+
+ def deconstruct_keys(keys)
+ super.merge(closing: closing)
+ end
+ end
+
+ class ErbDoClose < ErbClose
+ def accept(visitor)
+ visitor.visit_erb_do_close(self)
+ end
+ end
+
+ class ErbControl < Block
+ end
+
+ class ErbIf < ErbControl
+ # opening: ErbNode
+ # elements: [[HtmlNode | ErbNode | CharDataNode]]
+ # closing: [nil | ErbElsif | ErbElse]
+ def accept(visitor)
+ visitor.visit_erb_if(self)
+ end
+ end
+
+ class ErbUnless < ErbIf
+ # opening: ErbNode
+ # elements: [[HtmlNode | ErbNode | CharDataNode]]
+ # closing: [nil | ErbElsif | ErbElse]
+ def accept(visitor)
+ visitor.visit_erb_if(self)
+ end
+ end
+
+ class ErbElsif < ErbIf
+ def accept(visitor)
+ visitor.visit_erb_if(self)
+ end
+ end
+
+ class ErbElse < ErbIf
+ def accept(visitor)
+ visitor.visit_erb_else(self)
+ end
+ end
+
+ class ErbEnd < ErbNode
+ def accept(visitor)
+ visitor.visit_erb_end(self)
+ end
+
+ def child_nodes
+ []
+ end
+
+ alias deconstruct child_nodes
+ end
+
+ class ErbCase < ErbControl
+ # opening: ErbNode
+ # elements: [[HtmlNode | ErbNode | CharDataNode]]
+ # closing: [nil | ErbCaseWhen | ErbElse | ErbEnd]
+ def accept(visitor)
+ visitor.visit_erb_case(self)
+ end
+ end
+
+ class ErbCaseWhen < ErbControl
+ # opening: ErbNode
+ # elements: [[HtmlNode | ErbNode | CharDataNode]]
+ # closing: [nil | ErbCaseWhen | ErbElse | ErbEnd]
+ def accept(visitor)
+ visitor.visit_erb_case_when(self)
+ end
+ end
+
+ class ErbContent < Node
+ attr_reader(:value)
+
+ def initialize(value:)
+ if value.is_a?(Array)
+ value =
+ value.map { |token| token.is_a?(Token) ? token.value : token }.join
+ end
+ @value = SyntaxTree.parse(value.strip)
+ end
+
+ def blank?
+ value.nil? ||
+ value
+ .statements
+ .child_nodes
+ .reject { |node| node.is_a?(SyntaxTree::VoidStmt) }
+ .empty?
+ end
+
+ def accept(visitor)
+ visitor.visit_erb_content(self)
+ end
+
+ def child_nodes
+ [@value].compact
+ end
+
+ alias deconstruct child_nodes
+
+ def deconstruct_keys(keys)
+ { value: value }
+ end
+ end
+
+ # An HtmlAttribute is a key-value pair within a tag. It contains the key, the
+ # equals sign, and the value.
+ class HtmlAttribute < Node
+ attr_reader :key, :equals, :value, :location
+
+ def initialize(key:, equals:, value:, location:)
+ @key = key
+ @equals = equals
+ @value = value
+ @location = location
+ end
+
+ def accept(visitor)
+ visitor.visit_attribute(self)
+ end
+
+ def child_nodes
+ [key, equals, value]
+ end
+
+ alias deconstruct child_nodes
+
+ def deconstruct_keys(keys)
+ { key: key, equals: equals, value: value, location: location }
+ end
+ end
+
+ # A HtmlString can include ERB-tags
+ class HtmlString < Node
+ attr_reader :opening, :contents, :closing, :location
+
+ def initialize(opening:, contents:, closing:, location:)
+ @opening = opening
+ @contents = contents
+ @closing = closing
+ @location = location
+ end
+
+ def accept(visitor)
+ visitor.visit_html_string(self)
+ end
+
+ def child_nodes
+ [*contents]
+ end
+
+ alias deconstruct child_nodes
+
+ def deconstruct_keys(keys)
+ {
+ opening: opening,
+ contents: contents,
+ closing: closing,
+ location: location
+ }
+ end
+ end
+
+ class HtmlComment < Element
+ attr_reader :token
+
+ def initialize(token:, new_line:, location:)
+ super(new_line: new_line, location: location)
+ @token = token
+ end
+
+ def accept(visitor)
+ visitor.visit_html_comment(self)
+ end
+
+ def child_nodes
+ []
+ end
+
+ alias deconstruct child_nodes
+
+ def deconstruct_keys(keys)
+ super.merge(token: token)
+ end
+ end
+
+ class ErbComment < Element
+ attr_reader :token
+
+ def initialize(token:, new_line:, location:)
+ super(new_line: new_line, location: location)
+ @token = token
+ end
+
+ def accept(visitor)
+ visitor.visit_erb_comment(self)
+ end
+
+ def child_nodes
+ []
+ end
+
+ alias deconstruct child_nodes
+
+ def deconstruct_keys(keys)
+ super.merge(token: token)
+ end
+ end
+
+ # A CharData contains either plain text or whitespace within an element.
+ # It wraps a single token value.
+ class CharData < Element
+ attr_reader :value
+
+ def initialize(value:, new_line:, location:)
+ super(new_line: new_line, location: location)
+ @value = value
+ end
+
+ def accept(visitor)
+ visitor.visit_char_data(self)
+ end
+
+ def child_nodes
+ [value]
+ end
+
+ alias deconstruct child_nodes
+
+ def deconstruct_keys(keys)
+ super.merge(value: value)
+ end
+
+ def skip?
+ value.value.strip.empty?
+ end
+
+ # Also remove trailing whitespace
+ def without_new_line
+ self.class.new(
+ **deconstruct_keys([]).merge(
+ new_line: nil,
+ value:
+ Token.new(
+ type: value.type,
+ location: value.location,
+ value: value.value.rstrip
+ )
+ )
+ )
+ end
+ end
+
+ class NewLine < Node
+ attr_reader :count, :location
+
+ def initialize(location:, count:)
+ @location = location
+ @count = count
+ end
+
+ def accept(visitor)
+ visitor.visit_new_line(self)
+ end
+
+ def child_nodes
+ []
+ end
+
+ alias deconstruct child_nodes
+
+ def deconstruct_keys(keys)
+ { location: location, count: count }
+ end
+ end
+
+ # A document type declaration is a special kind of tag that specifies the
+ # type of the document. It contains an opening declaration, the name of
+ # the document type, an optional external identifier, and a closing of the
+ # tag.
+ class Doctype < Element
+ attr_reader :opening, :name, :closing
+
+ def initialize(opening:, name:, closing:, new_line:, location:)
+ super(new_line: new_line, location: location)
+ @opening = opening
+ @name = name
+ @closing = closing
+ end
+
+ def accept(visitor)
+ visitor.visit_doctype(self)
+ end
+
+ def child_nodes
+ [opening, name, closing].compact
+ end
+
+ alias deconstruct child_nodes
+
+ def deconstruct_keys(keys)
+ super.merge(opening: opening, name: name, closing: closing)
+ end
+ end
+ end
+end
diff --git a/lib/syntax_tree/erb/parser.rb b/lib/syntax_tree/erb/parser.rb
new file mode 100644
index 0000000..48d43b2
--- /dev/null
+++ b/lib/syntax_tree/erb/parser.rb
@@ -0,0 +1,843 @@
+# frozen_string_literal: true
+
+module SyntaxTree
+ module ERB
+ class Parser
+ # This is the parent class of any kind of errors that will be raised by
+ # the parser.
+ class ParseError < StandardError
+ end
+
+ # This error occurs when a certain token is expected in a certain place
+ # but is not found. Sometimes this is handled internally because some
+ # elements are optional. Other times it is not and it is raised to end the
+ # parsing process.
+ class MissingTokenError < ParseError
+ end
+
+ attr_reader :source, :tokens
+
+ def initialize(source)
+ @source = source
+ @tokens = make_tokens
+ @found_doctype = false
+ end
+
+ def parse
+ elements = many { parse_any_tag }
+
+ location =
+ elements.first.location.to(elements.last.location) if elements.any?
+
+ Document.new(elements: elements, location: location)
+ end
+
+ def debug_tokens
+ @tokens.each do |key, value, index, line|
+ puts("#{key} #{value.inspect} #{index} #{line}")
+ end
+ end
+
+ private
+
+ def parse_any_tag
+ loop do
+ tag =
+ atleast do
+ maybe { parse_doctype } || maybe { parse_html_comment } ||
+ maybe { parse_erb_tag } || maybe { parse_erb_comment } ||
+ maybe { parse_html_element } || maybe { parse_new_line } ||
+ maybe { parse_chardata }
+ end
+
+ if tag.is_a?(Doctype)
+ if @found_doctype
+ raise(ParseError, "Only one doctype element is allowed")
+ else
+ @found_doctype = true
+ end
+ end
+
+ # Ignore new lines in beginning of document
+ next if tag.is_a?(NewLine)
+
+ # Allow skipping empty CharData
+ return tag unless tag.skip?
+ end
+ end
+
+ def make_tokens
+ Enumerator.new do |enum|
+ index = 0
+ line = 1
+ state = %i[outside]
+
+ while index < source.length
+ case state.last
+ in :outside
+ case source[index..]
+ when /\A\n{2,}/
+ # two or more newlines should be ONE blank line
+ enum.yield :blank_line, $&, index, line
+ line += $&.count("\n")
+ when /\A\n/
+ # newlines
+ enum.yield :new_line, $&, index, line
+ line += 1
+ when /\A/m
+ # comments
+ #
+ enum.yield :html_comment, $&, index, line
+ line += $&.count("\n")
+ when /\A/
+ # An ERB-comment
+ # <%# this is an ERB comment %>
+ enum.yield :erb_comment, $&, index, line
+ when /\A<%={1,2}/, /\A<%-/, /\A<%/
+ # the beginning of an ERB tag
+ # <%
+ # <%=, <%==
+ enum.yield :erb_open, $&, index, line
+ state << :erb_start
+ line += $&.count("\n")
+ when %r{\A/
+ enum.yield :erb_do_close, $&, index, line
+ state.pop
+ when /\A-?%>/
+ enum.yield :erb_close, $&, index, line
+ state.pop
+ when /\A[\p{L}\w]*\b/
+ # Split by word boundary while parsing the code
+ # This allows us to separate what_to_do vs do
+ enum.yield :erb_code, $&, index, line
+ else
+ enum.yield :erb_code, source[index], index, line
+ index += 1
+ next
+ end
+ in :string_single_quote
+ case source[index..]
+ when /\A(?: |\t|\n|\r\n)+/m
+ # whitespace
+ enum.yield :whitespace, $&, index, line
+ line += $&.count("\n")
+ when /\A\'/
+ # the end of a quoted string
+ enum.yield :string_close_single_quote, $&, index, line
+ state.pop
+ when /\A<%[=]?/
+ # the beginning of an ERB tag
+ # <%
+ enum.yield :erb_open, $&, index, line
+ state << :erb_start
+ when /\A[^<']+/
+ # plain text content
+ # abc
+ enum.yield :text, $&, index, line
+ else
+ raise ParseError,
+ "Unexpected character in string at #{index}: #{source[index]}"
+ end
+ in :string_double_quote
+ case source[index..]
+ when /\A(?: |\t|\n|\r\n)+/m
+ # whitespace
+ enum.yield :whitespace, $&, index, line
+ line += $&.count("\n")
+ when /\A\"/
+ # the end of a quoted string
+ enum.yield :string_close_double_quote, $&, index, line
+ state.pop
+ when /\A<%[=]?/
+ # the beginning of an ERB tag
+ # <%
+ enum.yield :erb_open, $&, index, line
+ state << :erb_start
+ when /\A[^<"]+/
+ # plain text content
+ # abc
+ enum.yield :text, $&, index, line
+ else
+ raise ParseError,
+ "Unexpected character in string at #{index}: #{source[index]}"
+ end
+ in :inside
+ case source[index..]
+ when /\A[ \t\r\n]+/
+ # whitespace
+ line += $&.count("\n")
+ when /\A-?%>/
+ # the end of an ERB tag
+ # -%> or %>
+ enum.yield :erb_close, $&, index, line
+ state.pop
+ when /\A>/
+ # the end of a tag
+ # >
+ enum.yield :close, $&, index, line
+ state.pop
+ when /\A\?>/
+ # the end of a tag
+ # ?>
+ enum.yield :special_close, $&, index, line
+ state.pop
+ when %r{\A/>}
+ # the end of a self-closing tag
+ enum.yield :slash_close, $&, index, line
+ state.pop
+ when %r{\A/}
+ # a forward slash
+ # /
+ enum.yield :slash, $&, index, line
+ when /\A=/
+ # an equals sign
+ # =
+ enum.yield :equals, $&, index, line
+ when /\A[@#]*[:\w\.\-\_]+\b/
+ # a name for an element or an attribute
+ # strong, vue-component-kebab, VueComponentPascal
+ # abc, #abc, @abc, :abc
+ enum.yield :name, $&, index, line
+ when /\A<%/
+ # the beginning of an ERB tag
+ # <%
+ enum.yield :erb_open, $&, index, line
+ state << :erb_start
+ when /\A"/
+ # the beginning of a string
+ enum.yield :string_open_double_quote, $&, index, line
+ state << :string_double_quote
+ when /\A'/
+ # the beginning of a string
+ enum.yield :string_open_single_quote, $&, index, line
+ state << :string_single_quote
+ else
+ raise ParseError,
+ "Unexpected character at #{index}: #{source[index]}"
+ end
+ end
+
+ index += $&.length
+ end
+
+ enum.yield :EOF, nil, index, line
+ end
+ end
+
+ # If the next token in the list of tokens matches the expected type, then
+ # we're going to create a new Token, advance the token enumerator, and
+ # return the new Token. Otherwise we're going to raise a
+ # MissingTokenError.
+ def consume(expected)
+ type, value, index, line = tokens.peek
+
+ if expected != type
+ raise MissingTokenError, "expected #{expected} got #{type}"
+ end
+
+ tokens.next
+
+ Token.new(
+ type: type,
+ value: value,
+ location:
+ Location.new(
+ start_char: index,
+ end_char: index + value.length,
+ start_line: line,
+ end_line: line + value.count("\n")
+ )
+ )
+ end
+
+ # We're going to yield to the block which should attempt to consume some
+ # number of tokens. If any of them are missing, then we're going to return
+ # nil from this block.
+ def maybe
+ yield
+ rescue MissingTokenError
+ end
+
+ # We're going to attempt to parse everything by yielding to the block. If
+ # nothing is returned by the block, then we're going to raise an error.
+ # Otherwise we'll return the value returned by the block.
+ def atleast
+ result = yield
+ raise MissingTokenError if result.nil?
+ result
+ end
+
+ # We're going to attempt to parse with the block many times. We'll stop
+ # parsing once we get an error back from the block.
+ def many
+ items = []
+
+ loop do
+ begin
+ items << yield
+ rescue MissingTokenError
+ break
+ end
+ end
+
+ items
+ end
+
+ def parse_until_erb(classes:)
+ items = []
+
+ loop do
+ result = parse_any_tag
+ items << result
+ break if classes.any? { |cls| result.is_a?(cls) }
+ end
+
+ items
+ end
+
+ def parse_html_opening_tag
+ opening = consume(:open)
+ name = consume(:name)
+
+ if name.value =~ /\A[@:#]/
+ raise ParseError, "Invalid html-tag name #{name}"
+ end
+
+ attributes =
+ many do
+ atleast do
+ maybe { parse_erb_tag } || maybe { parse_html_attribute }
+ end
+ end
+
+ closing =
+ atleast do
+ maybe { consume(:close) } || maybe { consume(:slash_close) }
+ end
+
+ new_line = maybe { parse_new_line }
+
+ # Parse any whitespace after new lines
+ maybe { consume(:whitespace) }
+
+ HtmlNode::OpeningTag.new(
+ opening: opening,
+ name: name,
+ attributes: attributes,
+ closing: closing,
+ location: opening.location.to(closing.location),
+ new_line: new_line
+ )
+ end
+
+ def parse_html_closing
+ opening = consume(:slash_open)
+ name = consume(:name)
+ closing = consume(:close)
+
+ new_line = maybe { parse_new_line }
+
+ HtmlNode::ClosingTag.new(
+ opening: opening,
+ name: name,
+ closing: closing,
+ location: opening.location.to(closing.location),
+ new_line: new_line
+ )
+ end
+
+ def parse_html_element
+ opening = parse_html_opening_tag
+
+ if opening.closing.value == "/>"
+ HtmlNode.new(opening: opening, location: opening.location)
+ elsif opening.is_void_element?
+ HtmlNode.new(opening: opening, location: opening.location)
+ else
+ elements = many { parse_any_tag }
+ closing = maybe { parse_html_closing }
+
+ if closing.nil?
+ raise(
+ ParseError,
+ "Missing closing tag for <#{opening.name.value}> at #{opening.location}"
+ )
+ end
+
+ if closing.name.value != opening.name.value
+ raise(
+ ParseError,
+ "Expected closing tag for <#{opening.name.value}> but got <#{closing.name.value}> at #{closing.location}"
+ )
+ end
+
+ HtmlNode.new(
+ opening: opening,
+ elements: elements,
+ closing: closing,
+ location: opening.location.to(closing.location)
+ )
+ end
+ end
+
+ def parse_erb_case(erb_node)
+ elements =
+ maybe { parse_until_erb(classes: [ErbCaseWhen, ErbElse, ErbEnd]) } ||
+ []
+
+ erb_tag = elements.pop
+
+ unless erb_tag.is_a?(ErbCaseWhen) || erb_tag.is_a?(ErbElse) ||
+ erb_tag.is_a?(ErbEnd)
+ raise(
+ ParseError,
+ "Found no matching erb-tag to the if-tag at #{erb_node.location}"
+ )
+ end
+
+ case erb_node.keyword.type
+ when :erb_case
+ ErbCase.new(
+ opening: erb_node,
+ elements: elements,
+ closing: erb_tag,
+ location: erb_node.location.to(erb_tag.location)
+ )
+ when :erb_when
+ ErbCaseWhen.new(
+ opening: erb_node,
+ elements: elements,
+ closing: erb_tag,
+ location: erb_node.location.to(erb_tag.location)
+ )
+ else
+ raise(
+ ParseError,
+ "Found no matching when- or else-tag to the case-tag at #{erb_node.location}"
+ )
+ end
+ end
+
+ def parse_erb_if(erb_node)
+ # Skip any leading whitespace
+ maybe { consume(:whitespace) }
+
+ elements =
+ maybe { parse_until_erb(classes: [ErbElsif, ErbElse, ErbEnd]) } || []
+
+ erb_tag = elements.pop
+
+ unless erb_tag.is_a?(ErbControl) || erb_tag.is_a?(ErbEnd)
+ raise(
+ ParseError,
+ "Found no matching erb-tag to the if-tag at #{erb_node.location}"
+ )
+ end
+
+ case erb_node.keyword.type
+ when :erb_if
+ ErbIf.new(
+ opening: erb_node,
+ elements: elements,
+ closing: erb_tag,
+ location: erb_node.location.to(erb_tag.location)
+ )
+ when :erb_unless
+ ErbUnless.new(
+ opening: erb_node,
+ elements: elements,
+ closing: erb_tag,
+ location: erb_node.location.to(erb_tag.location)
+ )
+ when :erb_elsif
+ ErbElsif.new(
+ opening: erb_node,
+ elements: elements,
+ closing: erb_tag,
+ location: erb_node.location.to(erb_tag.location)
+ )
+ else
+ raise(
+ ParseError,
+ "Found no matching elsif- or else-tag to the if-tag at #{erb_node.location}"
+ )
+ end
+ end
+
+ def parse_erb_else(erb_node)
+ elements = maybe { parse_until_erb(classes: [ErbEnd]) } || []
+
+ erb_end = elements.pop
+
+ unless erb_end.is_a?(ErbEnd)
+ raise(
+ ParseError,
+ "Found no matching end-tag for the else-tag at #{erb_node.location}"
+ )
+ end
+
+ ErbElse.new(
+ opening: erb_node,
+ elements: elements,
+ closing: erb_end,
+ location: erb_node.location.to(erb_end.location)
+ )
+ end
+
+ def parse_erb_end(erb_node)
+ new_line = maybe { parse_new_line }
+
+ ErbEnd.new(
+ opening_tag: erb_node.opening_tag,
+ keyword: erb_node.keyword,
+ content: nil,
+ closing_tag: erb_node.closing_tag,
+ new_line: new_line,
+ location: erb_node.location
+ )
+ end
+
+ def parse_erb_tag
+ opening_tag = consume(:erb_open)
+ keyword =
+ maybe { consume(:erb_if) } || maybe { consume(:erb_unless) } ||
+ maybe { consume(:erb_elsif) } || maybe { consume(:erb_else) } ||
+ maybe { consume(:erb_end) } || maybe { consume(:erb_case) } ||
+ maybe { consume(:erb_when) }
+
+ content = parse_until_erb_close
+ closing_tag = content.pop
+
+ if !closing_tag.is_a?(ErbClose)
+ raise(
+ ParseError,
+ "Found no matching closing tag for the erb-tag at #{opening_tag.location}"
+ )
+ end
+
+ new_line = maybe { parse_new_line }
+
+ erb_node =
+ ErbNode.new(
+ opening_tag: opening_tag,
+ keyword: keyword,
+ content: content,
+ closing_tag: closing_tag,
+ new_line: new_line,
+ location: opening_tag.location.to(closing_tag.location)
+ )
+
+ case erb_node.keyword&.type
+ when :erb_if, :erb_unless, :erb_elsif
+ parse_erb_if(erb_node)
+ when :erb_case, :erb_when
+ parse_erb_case(erb_node)
+ when :erb_else
+ parse_erb_else(erb_node)
+ when :erb_end
+ parse_erb_end(erb_node)
+ else
+ if closing_tag.is_a?(ErbDoClose)
+ elements = maybe { parse_until_erb(classes: [ErbEnd]) } || []
+ erb_end = elements.pop
+
+ unless erb_end.is_a?(ErbEnd)
+ raise(
+ ParseError,
+ "Found no matching end-tag for the do-tag at #{erb_node.location}"
+ )
+ end
+
+ ErbBlock.new(
+ opening: erb_node,
+ elements: elements,
+ closing: erb_end,
+ location: erb_node.location.to(erb_end.location)
+ )
+ else
+ erb_node
+ end
+ end
+ rescue MissingTokenError => error
+ # If we have parsed tokens that we cannot process after we parsed <%, we should throw a ParseError
+ # and not let it be handled by a `maybe`.
+ if opening_tag
+ raise(
+ ParseError,
+ "Could not parse ERB-tag at #{opening_tag.location}"
+ )
+ else
+ raise(error)
+ end
+ end
+
+ def parse_until_erb_close
+ items = []
+
+ loop do
+ result =
+ atleast do
+ maybe { parse_erb_do_close } || maybe { parse_erb_close } ||
+ maybe { consume(:erb_code) }
+ end
+
+ items << result
+
+ break if result.is_a?(ErbClose)
+ end
+
+ items
+ end
+
+ # This method is called at the end of most tags, it fixes:
+ # 1. Parsing any new lines after the tag
+ # 2. Parsing any whitespace after the new lines
+ # The whitespace is just consumed
+ def parse_new_line
+ line_break =
+ atleast do
+ maybe { consume(:blank_line) } || maybe { consume(:new_line) }
+ end
+
+ maybe { consume(:whitespace) }
+
+ NewLine.new(
+ location: line_break.location,
+ count: line_break.value.count("\n")
+ )
+ end
+
+ def parse_erb_close
+ closing = consume(:erb_close)
+
+ new_line = maybe { parse_new_line }
+
+ ErbClose.new(
+ location: closing.location,
+ new_line: new_line,
+ closing: closing
+ )
+ end
+
+ def parse_erb_do_close
+ closing = consume(:erb_do_close)
+
+ new_line = maybe { parse_new_line }
+
+ ErbDoClose.new(
+ location: closing.location,
+ new_line: new_line,
+ closing: closing
+ )
+ end
+
+ def parse_html_string
+ opening =
+ maybe { consume(:string_open_double_quote) } ||
+ maybe { consume(:string_open_single_quote) }
+
+ if opening.nil?
+ value = consume(:name)
+
+ return(
+ HtmlString.new(
+ opening: nil,
+ contents: [value],
+ closing: nil,
+ location: value.location
+ )
+ )
+ end
+
+ contents =
+ many do
+ atleast do
+ maybe { consume(:text) } || maybe { consume(:whitespace) } ||
+ maybe { parse_erb_tag }
+ end
+ end
+
+ closing =
+ if opening.type == :string_open_double_quote
+ consume(:string_close_double_quote)
+ else
+ consume(:string_close_single_quote)
+ end
+
+ HtmlString.new(
+ opening: opening,
+ contents: contents,
+ closing: closing,
+ location: opening.location.to(closing.location)
+ )
+ end
+
+ def parse_html_attribute
+ key = consume(:name)
+ equals = maybe { consume(:equals) }
+
+ if equals.nil?
+ HtmlAttribute.new(
+ key: key,
+ equals: nil,
+ value: nil,
+ location: key.location
+ )
+ else
+ value = parse_html_string
+
+ HtmlAttribute.new(
+ key: key,
+ equals: equals,
+ value: value,
+ location: key.location.to(value.location)
+ )
+ end
+ end
+
+ def parse_chardata
+ values =
+ many do
+ atleast do
+ maybe { consume(:string_open_double_quote) } ||
+ maybe { consume(:string_open_single_quote) } ||
+ maybe { consume(:string_close_double_quote) } ||
+ maybe { consume(:string_close_single_quote) } ||
+ maybe { consume(:text) } || maybe { consume(:whitespace) }
+ end
+ end
+
+ token =
+ if values.size > 1
+ Token.new(
+ type: :text,
+ value: values.map(&:value).join(""),
+ location: values.first.location.to(values.last.location)
+ )
+ else
+ values.first
+ end
+
+ new_line = maybe { parse_new_line }
+
+ if token&.value
+ CharData.new(
+ value: token,
+ location: token.location,
+ new_line: new_line
+ )
+ end
+ end
+
+ def parse_doctype
+ opening = consume(:doctype)
+ name = consume(:name)
+ closing = consume(:close)
+
+ new_line = maybe { parse_new_line }
+
+ Doctype.new(
+ opening: opening,
+ name: name,
+ closing: closing,
+ new_line: new_line,
+ location: opening.location.to(closing.location)
+ )
+ end
+
+ def parse_html_comment
+ comment = consume(:html_comment)
+
+ new_line = maybe { parse_new_line }
+
+ HtmlComment.new(
+ token: comment,
+ new_line: new_line,
+ location: comment.location
+ )
+ end
+
+ def parse_erb_comment
+ comment = consume(:erb_comment)
+
+ new_line = maybe { parse_new_line }
+
+ ErbComment.new(
+ token: comment,
+ new_line: new_line,
+ location: comment.location
+ )
+ end
+ end
+ end
+end
diff --git a/lib/syntax_tree/erb/pretty_print.rb b/lib/syntax_tree/erb/pretty_print.rb
new file mode 100644
index 0000000..869d1bc
--- /dev/null
+++ b/lib/syntax_tree/erb/pretty_print.rb
@@ -0,0 +1,222 @@
+# frozen_string_literal: true
+
+module SyntaxTree
+ module ERB
+ class PrettyPrint < Visitor
+ attr_reader :q
+
+ def initialize(q)
+ @q = q
+ end
+
+ # Visit a Token node.
+ def visit_token(node)
+ q.pp(node.value)
+ end
+
+ # Visit a Document node.
+ def visit_document(node)
+ visit_node("document", node)
+ end
+
+ def visit_block(node)
+ visit_node(node.class.to_s, node)
+ end
+
+ # Visit an HtmlNode.
+ def visit_html(node)
+ visit_node("html", node)
+ end
+
+ # Visit an HtmlNode::OpeningTag node.
+ def visit_opening_tag(node)
+ visit_node("opening_tag", node)
+ end
+
+ # Visit an HtmlNode::ClosingTag node.
+ def visit_closing_tag(node)
+ visit_node("closing_tag", node)
+ end
+
+ # Visit an ErbNode node.
+ def visit_erb(node)
+ q.group do
+ q.text("(erb")
+ q.nest(2) do
+ q.breakable
+ visit(node.opening_tag)
+ if node.keyword
+ q.breakable
+ visit(node.keyword)
+ end
+ if node.content
+ q.breakable
+ q.text("content")
+ end
+
+ q.breakable
+ visit(node.closing_tag)
+ end
+ q.breakable("")
+ q.text(")")
+ end
+ end
+
+ def visit_erb_block(node)
+ q.group do
+ q.text("(erb_block")
+ q.nest(2) do
+ q.breakable
+ q.seplist(node.elements) { |child_node| visit(child_node) }
+ end
+ q.breakable
+ visit(node.closing)
+ q.breakable("")
+ q.text(")")
+ end
+ end
+
+ def visit_erb_if(node, key = "erb_if")
+ q.group do
+ q.text("(")
+ visit(node.opening)
+ q.nest(2) do
+ q.breakable()
+ q.seplist(node.child_nodes) { |child_node| visit(child_node) }
+ end
+ q.breakable
+ visit(node.closing)
+ q.breakable("")
+ q.text(")")
+ end
+ end
+
+ def visit_erb_elsif(node)
+ visit_erb_if(node, "erb_elsif")
+ end
+
+ def visit_erb_else(node)
+ visit_erb_if(node, "erb_else")
+ end
+
+ def visit_erb_case(node)
+ visit_erb_if(node, "erb_case")
+ end
+
+ def visit_erb_case_when(node)
+ visit_erb_if(node, "erb_when")
+ end
+
+ def visit_erb_end(node)
+ q.pp("(erb_end)")
+ end
+
+ # Visit an ErbContent node.
+ def visit_erb_content(node)
+ q.pp(node.value)
+ end
+
+ # Visit an Attribute node.
+ def visit_attribute(node)
+ visit_node("attribute", node)
+ end
+
+ # Visit a HtmlString node.
+ def visit_html_string(node)
+ visit_node("html_string", node)
+ end
+
+ # Visit a CharData node.
+ def visit_char_data(node)
+ visit_node("char_data", node)
+ end
+
+ def visit_new_line(node)
+ node.count > 1 ? q.text("(new_line blank)") : q.text("(new_line)")
+ end
+
+ def visit_erb_close(node)
+ visit(node.closing)
+ end
+
+ def visit_erb_do_close(node)
+ visit_node("erb_do_close", node)
+ end
+
+ # Visit a Doctype node.
+ def visit_doctype(node)
+ visit_node("doctype", node)
+ end
+
+ def visit_html_comment(node)
+ visit_node("html_comment", node)
+ end
+
+ def visit_erb_comment(node)
+ visit_node("erb_comment", node)
+ end
+
+ private
+
+ # A generic visit node function for how we pretty print nodes.
+ def visit_node(type, node)
+ q.group do
+ q.text("(#{type}")
+ q.nest(2) do
+ q.breakable
+ q.seplist(node.child_nodes) { |child_node| visit(child_node) }
+ end
+ q.breakable("")
+ q.text(")")
+ end
+ end
+
+ def comments(node)
+ return if node.comments.empty?
+
+ q.breakable
+ q.group(2, "(", ")") do
+ q.seplist(node.comments) { |comment| q.pp(comment) }
+ end
+ end
+
+ def field(_name, value)
+ q.breakable
+ q.pp(value)
+ end
+
+ def list(_name, values)
+ q.breakable
+ q.group(2, "(", ")") { q.seplist(values) { |value| q.pp(value) } }
+ end
+
+ def node(_node, type)
+ q.group(2, "(", ")") do
+ q.text(type)
+ yield
+ end
+ end
+
+ def pairs(_name, values)
+ q.group(2, "(", ")") do
+ q.seplist(values) do |(key, value)|
+ q.pp(key)
+
+ if value
+ q.text("=")
+ q.group(2) do
+ q.breakable("")
+ q.pp(value)
+ end
+ end
+ end
+ end
+ end
+
+ def text(_name, value)
+ q.breakable
+ q.text(value)
+ end
+ end
+ end
+end
diff --git a/lib/syntax_tree/xml/version.rb b/lib/syntax_tree/erb/version.rb
similarity index 62%
rename from lib/syntax_tree/xml/version.rb
rename to lib/syntax_tree/erb/version.rb
index 26871f5..6e53718 100644
--- a/lib/syntax_tree/xml/version.rb
+++ b/lib/syntax_tree/erb/version.rb
@@ -1,7 +1,7 @@
# frozen_string_literal: true
module SyntaxTree
- module XML
- VERSION = "0.1.0"
+ module ERB
+ VERSION = "0.11.0"
end
end
diff --git a/lib/syntax_tree/xml/visitor.rb b/lib/syntax_tree/erb/visitor.rb
similarity index 62%
rename from lib/syntax_tree/xml/visitor.rb
rename to lib/syntax_tree/erb/visitor.rb
index cbf0006..6603f91 100644
--- a/lib/syntax_tree/xml/visitor.rb
+++ b/lib/syntax_tree/erb/visitor.rb
@@ -1,16 +1,18 @@
# frozen_string_literal: true
module SyntaxTree
- module XML
+ module ERB
# Provides a visitor interface for visiting certain nodes. It's used
# internally to implement formatting and pretty-printing. It could also be
# used externally to visit a subset of nodes that are relevant to a certain
# task.
- class Visitor
+ class Visitor < SyntaxTree::Visitor
def visit(node)
node&.accept(self)
end
+ alias visit_statements visit_child_nodes
+
private
def visit_all(nodes)
@@ -27,35 +29,26 @@ def visit_child_nodes(node)
# Visit a Document node.
alias visit_document visit_child_nodes
- # Visit a Prolog node.
- alias visit_prolog visit_child_nodes
-
- # Visit a Doctype node.
- alias visit_doctype visit_child_nodes
-
- # Visit an ExternalID node.
- alias visit_external_id visit_child_nodes
+ # Visit an Html node.
+ alias visit_html visit_child_nodes
- # Visit an Element node.
- alias visit_element visit_child_nodes
-
- # Visit an Element::OpeningTag node.
+ # Visit an HtmlNode::OpeningTag node.
alias visit_opening_tag visit_child_nodes
- # Visit an Element::ClosingTag node.
+ # Visit an HtmlNode::ClosingTag node.
alias visit_closing_tag visit_child_nodes
- # Visit a Reference node.
- alias visit_reference visit_child_nodes
-
# Visit an Attribute node.
alias visit_attribute visit_child_nodes
# Visit a CharData node.
alias visit_char_data visit_child_nodes
- # Visit a Misc node.
- alias visit_misc visit_child_nodes
+ # Visit an ErbNode node.
+ alias visit_erb visit_child_nodes
+
+ # Visit a HtmlString node.
+ alias visit_html_string visit_child_nodes
end
end
end
diff --git a/lib/syntax_tree/xml/format.rb b/lib/syntax_tree/xml/format.rb
deleted file mode 100644
index 401b455..0000000
--- a/lib/syntax_tree/xml/format.rb
+++ /dev/null
@@ -1,238 +0,0 @@
-# frozen_string_literal: true
-
-module SyntaxTree
- module XML
- class Format < Visitor
- attr_reader :q
-
- def initialize(q)
- @q = q
- end
-
- # Visit a Token node.
- def visit_token(node)
- q.text(node.value.strip)
- end
-
- # Visit a Document node.
- def visit_document(node)
- child_nodes =
- node
- .child_nodes
- .select do |child_node|
- case child_node
- in Misc[value: Token[type: :whitespace]]
- false
- else
- true
- end
- end
- .sort_by(&:location)
-
- q.seplist(child_nodes, -> { q.breakable(force: true) }) do |child_node|
- visit(child_node)
- end
-
- q.breakable(force: true)
- end
-
- # Visit a Prolog node.
- def visit_prolog(node)
- q.group do
- visit(node.opening)
-
- if node.attributes.any?
- q.indent do
- q.breakable
- q.seplist(node.attributes, -> { q.breakable }) do |child_node|
- visit(child_node)
- end
- end
- end
-
- q.breakable("")
- visit(node.closing)
- end
- end
-
- # Visit a Doctype node.
- def visit_doctype(node)
- q.group do
- visit(node.opening)
- q.text(" ")
- visit(node.name)
-
- if node.external_id
- q.text(" ")
- visit(node.external_id)
- end
-
- visit(node.closing)
- end
- end
-
- # Visit an ExternalID node.
- def visit_external_id(node)
- q.group do
- q.group do
- visit(node.type)
-
- if node.public_id
- q.indent do
- q.breakable
- visit(node.public_id)
- end
- end
- end
-
- q.indent do
- q.breakable
- visit(node.system_id)
- end
- end
- end
-
- # Visit an Element node.
- def visit_element(node)
- inner_nodes =
- node.content&.select do |child_node|
- case child_node
- in CharData[value: Token[type: :whitespace]]
- false
- in CharData[value: Token[value:]] if value.strip.empty?
- false
- else
- true
- end
- end
-
- case inner_nodes
- in nil
- visit(node.opening_tag)
- in []
- visit(
- Element::OpeningTag.new(
- opening: node.opening_tag.opening,
- name: node.opening_tag.name,
- attributes: node.opening_tag.attributes,
- closing:
- Token.new(
- type: :close,
- value: "/>",
- location: node.opening_tag.closing.location
- ),
- location: node.opening_tag.location
- )
- )
- in [CharData[value: Token[type: :text, value:]]]
- q.group do
- visit(node.opening_tag)
- q.indent do
- q.breakable("")
- format_text(q, value)
- end
-
- q.breakable("")
- visit(node.closing_tag)
- end
- else
- q.group do
- visit(node.opening_tag)
- q.indent do
- q.breakable("")
-
- inner_nodes.each_with_index do |child_node, index|
- if index != 0
- q.breakable(force: true)
-
- end_line = inner_nodes[index - 1].location.end_line
- start_line = child_node.location.start_line
- q.breakable(force: true) if (start_line - end_line) >= 2
- end
-
- case child_node
- in CharData[value: Token[type: :text, value:]]
- format_text(q, value)
- else
- visit(child_node)
- end
- end
- end
-
- q.breakable(force: true)
- visit(node.closing_tag)
- end
- end
- end
-
- # Visit an Element::OpeningTag node.
- def visit_opening_tag(node)
- q.group do
- visit(node.opening)
- visit(node.name)
-
- if node.attributes.any?
- q.indent do
- q.breakable
- q.seplist(node.attributes, -> { q.breakable }) do |child_node|
- visit(child_node)
- end
- end
- end
-
- q.breakable(node.closing.value == "/>" ? " " : "")
- visit(node.closing)
- end
- end
-
- # Visit an Element::ClosingTag node.
- def visit_closing_tag(node)
- q.group do
- visit(node.opening)
- visit(node.name)
- visit(node.closing)
- end
- end
-
- # Visit a Reference node.
- def visit_reference(node)
- visit(node.value)
- end
-
- # Visit an Attribute node.
- def visit_attribute(node)
- q.group do
- visit(node.key)
- visit(node.equals)
- visit(node.value)
- end
- end
-
- # Visit a CharData node.
- def visit_char_data(node)
- lines = node.value.value.strip.split("\n")
-
- q.seplist(lines, -> { q.breakable(indent: false) }) do |line|
- q.text(line)
- end
- end
-
- # Visit a Misc node.
- def visit_misc(node)
- visit(node.value)
- end
-
- private
-
- # Format a text by splitting nicely at newlines and spaces.
- def format_text(q, value)
- sep_line = -> { q.breakable(force: true, indent: false) }
- sep_word = -> { q.group { q.breakable } }
-
- q.seplist(value.strip.split("\n"), sep_line) do |line|
- q.seplist(line.split(/\b(?: +)\b/), sep_word) { |word| q.text(word) }
- end
- end
- end
- end
-end
diff --git a/lib/syntax_tree/xml/nodes.rb b/lib/syntax_tree/xml/nodes.rb
deleted file mode 100644
index 104a65b..0000000
--- a/lib/syntax_tree/xml/nodes.rb
+++ /dev/null
@@ -1,413 +0,0 @@
-# frozen_string_literal: true
-
-module SyntaxTree
- module XML
- # A Location represents a position for a node in the source file.
- class Location
- attr_reader :start_char, :end_char, :start_line, :end_line
-
- def initialize(start_char:, end_char:, start_line:, end_line:)
- @start_char = start_char
- @end_char = end_char
- @start_line = start_line
- @end_line = end_line
- end
-
- def deconstruct_keys(keys)
- {
- start_char: start_char,
- end_char: end_char,
- start_line: start_line,
- end_line: end_line
- }
- end
-
- def to(other)
- Location.new(
- start_char: start_char,
- start_line: start_line,
- end_char: other.end_char,
- end_line: other.end_line
- )
- end
-
- def <=>(other)
- start_char <=> other.start_char
- end
- end
-
- # A parent node that contains a bit of shared functionality.
- class Node
- def format(q)
- Format.new(q).visit(self)
- end
-
- def pretty_print(q)
- PrettyPrint.new(q).visit(self)
- end
- end
-
- # A Token is any kind of lexical token from the source. It has a type, a
- # value which is a subset of the source, and an index where it starts in
- # the source.
- class Token < Node
- attr_reader :type, :value, :location
-
- def initialize(type:, value:, location:)
- @type = type
- @value = value
- @location = location
- end
-
- def accept(visitor)
- visitor.visit_token(self)
- end
-
- def child_nodes
- []
- end
-
- alias deconstruct child_nodes
-
- def deconstruct_keys(keys)
- { type: type, value: value, location: location }
- end
- end
-
- # The Document node is the top of the syntax tree. It contains an optional
- # prolog, an optional doctype declaration, any number of optional
- # miscellenous elements like comments, whitespace, or processing
- # instructions, and a root element.
- class Document < Node
- attr_reader :prolog, :miscs, :doctype, :element, :location
-
- def initialize(prolog:, miscs:, doctype:, element:, location:)
- @prolog = prolog
- @miscs = miscs
- @doctype = doctype
- @element = element
- @location = location
- end
-
- def accept(visitor)
- visitor.visit_document(self)
- end
-
- def child_nodes
- [prolog, *miscs, doctype, element].compact
- end
-
- alias deconstruct child_nodes
-
- def deconstruct_keys(keys)
- {
- prolog: prolog,
- miscs: miscs,
- doctype: doctype,
- element: element,
- location: location
- }
- end
- end
-
- # The prolog to the document includes an XML declaration which opens the
- # tag, any number of attributes, and a closing of the tag.
- class Prolog < Node
- attr_reader :opening, :attributes, :closing, :location
-
- def initialize(opening:, attributes:, closing:, location:)
- @opening = opening
- @attributes = attributes
- @closing = closing
- @location = location
- end
-
- def accept(visitor)
- visitor.visit_prolog(self)
- end
-
- def child_nodes
- [opening, *attributes, closing]
- end
-
- alias deconstruct child_nodes
-
- def deconstruct_keys(keys)
- {
- opening: opening,
- attributes: attributes,
- closing: closing,
- location: location
- }
- end
- end
-
- # A document type declaration is a special kind of tag that specifies the
- # type of the document. It contains an opening declaration, the name of
- # the document type, an optional external identifier, and a closing of the
- # tag.
- class DocType < Node
- attr_reader :opening, :name, :external_id, :closing, :location
-
- def initialize(opening:, name:, external_id:, closing:, location:)
- @opening = opening
- @name = name
- @external_id = external_id
- @closing = closing
- @location = location
- end
-
- def accept(visitor)
- visitor.visit_doctype(self)
- end
-
- def child_nodes
- [opening, name, external_id, closing].compact
- end
-
- alias deconstruct child_nodes
-
- def deconstruct_keys(keys)
- {
- opening: opening,
- name: name,
- external_id: external_id,
- closing: closing,
- location: location
- }
- end
- end
-
- # An external ID is a child of a document type declaration. It represents
- # the location where the external identifier is located. It contains a
- # type (either system or public), an optional public id literal, and the
- # system literal.
- class ExternalID < Node
- attr_reader :type, :public_id, :system_id, :location
-
- def initialize(type:, public_id:, system_id:, location:)
- @type = type
- @public_id = public_id
- @system_id = system_id
- end
-
- def accept(visitor)
- visitor.visit_external_id(self)
- end
-
- def child_nodes
- [type, public_id, system_id].compact
- end
-
- alias deconstruct child_nodes
-
- def deconstruct_keys(keys)
- {
- type: type,
- public_id: public_id,
- system_id: system_id,
- location: location
- }
- end
- end
-
- # An element is a child of the document. It contains an opening tag, any
- # optional content within the tag, and a closing tag. It can also
- # potentially contain an opening tag that self-closes, in which case the
- # content and closing tag will be nil.
- class Element < Node
- # The opening tag of an element. It contains the opening character (<),
- # the name of the element, any optional attributes, and the closing
- # token (either > or />).
- class OpeningTag < Node
- attr_reader :opening, :name, :attributes, :closing, :location
-
- def initialize(opening:, name:, attributes:, closing:, location:)
- @opening = opening
- @name = name
- @attributes = attributes
- @closing = closing
- @location = location
- end
-
- def accept(visitor)
- visitor.visit_opening_tag(self)
- end
-
- def child_nodes
- [opening, name, *attributes, closing]
- end
-
- alias deconstruct child_nodes
-
- def deconstruct_keys(keys)
- {
- opening: opening,
- name: name,
- attributes: attributes,
- closing: closing,
- location: location
- }
- end
- end
-
- # The closing tag of an element. It contains the opening character (<),
- # the name of the element, and the closing character (>).
- class ClosingTag < Node
- attr_reader :opening, :name, :closing, :location
-
- def initialize(opening:, name:, closing:, location:)
- @opening = opening
- @name = name
- @closing = closing
- @location = location
- end
-
- def accept(visitor)
- visitor.visit_closing_tag(self)
- end
-
- def child_nodes
- [opening, name, closing]
- end
-
- alias deconstruct child_nodes
-
- def deconstruct_keys(keys)
- { opening: opening, name: name, closing: closing, location: location }
- end
- end
-
- attr_reader :opening_tag, :content, :closing_tag, :location
-
- def initialize(opening_tag:, content:, closing_tag:, location:)
- @opening_tag = opening_tag
- @content = content
- @closing_tag = closing_tag
- @location = location
- end
-
- def accept(visitor)
- visitor.visit_element(self)
- end
-
- def child_nodes
- [opening_tag, *content, closing_tag].compact
- end
-
- alias deconstruct child_nodes
-
- def deconstruct_keys(keys)
- {
- opening_tag: opening_tag,
- content: content,
- closing_tag: closing_tag,
- location: location
- }
- end
- end
-
- # A Reference is either a character or entity reference. It contains a
- # single value that is the token it contains.
- class Reference < Node
- attr_reader :value, :location
-
- def initialize(value:, location:)
- @value = value
- @location = location
- end
-
- def accept(visitor)
- visitor.visit_reference(self)
- end
-
- def child_nodes
- [value]
- end
-
- alias deconstruct child_nodes
-
- def deconstruct_keys(keys)
- { value: value, location: location }
- end
- end
-
- # An Attribute is a key-value pair within a tag. It contains the key, the
- # equals sign, and the value.
- class Attribute < Node
- attr_reader :key, :equals, :value, :location
-
- def initialize(key:, equals:, value:, location:)
- @key = key
- @equals = equals
- @value = value
- @location = location
- end
-
- def accept(visitor)
- visitor.visit_attribute(self)
- end
-
- def child_nodes
- [key, equals, value]
- end
-
- alias deconstruct child_nodes
-
- def deconstruct_keys(keys)
- { key: key, equals: equals, value: value, location: location }
- end
- end
-
- # A CharData contains either plain text or whitespace within an element.
- # It wraps a single token value.
- class CharData < Node
- attr_reader :value, :location
-
- def initialize(value:, location:)
- @value = value
- @location = location
- end
-
- def accept(visitor)
- visitor.visit_char_data(self)
- end
-
- def child_nodes
- [value]
- end
-
- alias deconstruct child_nodes
-
- def deconstruct_keys(keys)
- { value: value, location: location }
- end
- end
-
- # A Misc is a catch-all for miscellaneous content outside the root tag of
- # the XML document. It contains a single token which can be either a
- # comment, a processing instruction, or whitespace.
- class Misc < Node
- attr_reader :value, :location
-
- def initialize(value:, location:)
- @value = value
- @location = location
- end
-
- def accept(visitor)
- visitor.visit_misc(self)
- end
-
- def child_nodes
- [value]
- end
-
- alias deconstruct child_nodes
-
- def deconstruct_keys(keys)
- { value: value, location: location }
- end
- end
- end
-end
diff --git a/lib/syntax_tree/xml/parser.rb b/lib/syntax_tree/xml/parser.rb
deleted file mode 100644
index cfc0d72..0000000
--- a/lib/syntax_tree/xml/parser.rb
+++ /dev/null
@@ -1,384 +0,0 @@
-# frozen_string_literal: true
-
-module SyntaxTree
- module XML
- class Parser
- NAME_START =
- "[:a-zA-Z_\u{2070}-\u{218F}\u{2C00}-\u{2FEF}\u{3001}-\u{D7FF}\u{F900}-\u{FDCF}\u{FDF0}-\u{FFFD}]"
-
- NAME_CHAR =
- "[#{NAME_START}-\\.\\d\u{00B7}\u{0300}-\u{036F}\u{203F}-\u{2040}]"
-
- NAME = "#{NAME_START}(?:#{NAME_CHAR})*"
-
- # This is the parent class of any kind of errors that will be raised by
- # the parser.
- class ParseError < StandardError
- end
-
- # This error occurs when a certain token is expected in a certain place
- # but is not found. Sometimes this is handled internally because some
- # elements are optional. Other times it is not and it is raised to end the
- # parsing process.
- class MissingTokenError < ParseError
- end
-
- attr_reader :source, :tokens
-
- def initialize(source)
- @source = source
- @tokens = make_tokens
- end
-
- def parse
- parse_document
- end
-
- private
-
- def make_tokens
- Enumerator.new do |enum|
- index = 0
- line = 1
- state = %i[outside]
-
- while index < source.length
- case state.last
- in :outside
- case source[index..]
- when /\A(?: |\t|\n|\r\n)+/m
- # whitespace
- enum.yield :whitespace, $&, index, line
- line += $&.count("\n")
- when /\A/m
- # comments
- #
- enum.yield :comment, $&, index, line
- line += $&.count("\n")
- when /\A/m
- # character data tags
- # Welcome!]]>
- enum.yield :cdata, $&, index, line
- line += $&.count("\n")
- when /\A/
- # document type definition tags
- #
- enum.yield :dtd, $&, index, line
- when /\A<\?xml[ \t\r\n]/
- # xml declaration opening
- # /
- # a processing instruction
- #
- enum.yield :processing_instruction, $&, index, line
- when /\A/
- # the end of a tag
- # >
- enum.yield :close, $&, index, line
- state.pop
- when /\A\?>/
- # the end of a tag
- # ?>
- enum.yield :special_close, $&, index, line
- state.pop
- when %r{\A/>}
- # the end of a self-closing tag
- enum.yield :slash_close, $&, index, line
- state.pop
- when %r{\A/}
- # a forward slash
- # /
- enum.yield :slash, $&, index, line
- when /\A=/
- # an equals sign
- # =
- enum.yield :equals, $&, index, line
- when /\A(?:"[^<"]*"|'[<^']*')/
- # a quoted string
- # "abc"
- enum.yield :string, $&, index, line
- when /\A#{NAME}/
- # a name
- # abc
- enum.yield :name, $&, index, line
- else
- raise ParseError,
- "Unexpected character at #{index}: #{source[index]}"
- end
- end
-
- index += $&.length
- end
-
- enum.yield :EOF, nil, index, line
- end
- end
-
- # If the next token in the list of tokens matches the expected type, then
- # we're going to create a new Token, advance the token enumerator, and
- # return the new Token. Otherwise we're going to raise a
- # MissingTokenError.
- def consume(expected)
- type, value, index, line = tokens.peek
-
- if expected != type
- raise MissingTokenError, "expected #{expected} got #{type}"
- end
-
- tokens.next
-
- Token.new(
- type: type,
- value: value,
- location:
- Location.new(
- start_char: index,
- end_char: index + value.length,
- start_line: line,
- end_line: line + value.count("\n")
- )
- )
- end
-
- # We're going to yield to the block which should attempt to consume some
- # number of tokens. If any of them are missing, then we're going to return
- # nil from this block.
- def maybe
- yield
- rescue MissingTokenError
- end
-
- # We're going to attempt to parse everything by yielding to the block. If
- # nothing is returned by the block, then we're going to raise an error.
- # Otherwise we'll return the value returned by the block.
- def atleast
- result = yield
- raise MissingTokenError if result.nil?
- result
- end
-
- # We're going to attempt to parse with the block many times. We'll stop
- # parsing once we get an error back from the block.
- def many
- items = []
-
- loop do
- begin
- items << yield
- rescue MissingTokenError
- break
- end
- end
-
- items
- end
-
- def parse_document
- prolog = maybe { parse_prolog }
- miscs = many { parse_misc }
-
- doctype = maybe { parse_doctype }
- miscs += many { parse_misc }
-
- element = parse_element
- miscs += many { parse_misc }
-
- parts = [prolog, *miscs, doctype, element].compact
-
- Document.new(
- prolog: prolog,
- miscs: miscs,
- doctype: doctype,
- element: element,
- location: parts.first.location.to(parts.last.location)
- )
- end
-
- def parse_prolog
- opening = consume(:xml_decl)
- attributes = many { parse_attribute }
- closing = consume(:special_close)
-
- Prolog.new(
- opening: opening,
- attributes: attributes,
- closing: closing,
- location: opening.location.to(closing.location)
- )
- end
-
- def parse_doctype
- opening = consume(:doctype)
- name = consume(:name)
- external_id = maybe { parse_external_id }
- closing = consume(:close)
-
- DocType.new(
- opening: opening,
- name: name,
- external_id: external_id,
- closing: closing,
- location: opening.location.to(closing.location)
- )
- end
-
- def parse_external_id
- type = consume(:name)
- public_id = consume(:string) if type.value == "PUBLIC"
- system_id = consume(:string)
-
- ExternalID.new(
- type: type,
- public_id: public_id,
- system_id: system_id,
- location: type.location.to(system_id.location)
- )
- end
-
- def parse_content
- many do
- atleast do
- maybe { parse_element } || maybe { parse_chardata } ||
- maybe { parse_reference } || maybe { consume(:cdata) } ||
- maybe { consume(:processing_instruction) } ||
- maybe { consume(:comment) }
- end
- end
- end
-
- def parse_opening_tag
- opening = consume(:open)
- name = consume(:name)
- attributes = many { parse_attribute }
- closing =
- atleast do
- maybe { consume(:close) } || maybe { consume(:slash_close) }
- end
-
- Element::OpeningTag.new(
- opening: opening,
- name: name,
- attributes: attributes,
- closing: closing,
- location: opening.location.to(closing.location)
- )
- end
-
- def parse_closing_tag
- opening = consume(:slash_open)
- name = consume(:name)
- closing = consume(:close)
-
- Element::ClosingTag.new(
- opening: opening,
- name: name,
- closing: closing,
- location: opening.location.to(closing.location)
- )
- end
-
- def parse_element
- opening_tag = parse_opening_tag
-
- if opening_tag.closing.value == ">"
- content = parse_content
- closing_tag = parse_closing_tag
-
- Element.new(
- opening_tag: opening_tag,
- content: content,
- closing_tag: closing_tag,
- location: opening_tag.location.to(closing_tag.location)
- )
- else
- Element.new(
- opening_tag: opening_tag,
- content: nil,
- closing_tag: nil,
- location: opening_tag.location
- )
- end
- end
-
- def parse_reference
- value =
- atleast do
- maybe { consume(:entity_reference) } ||
- maybe { consume(:character_reference) }
- end
-
- Reference.new(value: value, location: value.location)
- end
-
- def parse_attribute
- key = consume(:name)
- equals = consume(:equals)
- value = consume(:string)
-
- Attribute.new(
- key: key,
- equals: equals,
- value: value,
- location: key.location.to(value.location)
- )
- end
-
- def parse_chardata
- value =
- atleast { maybe { consume(:text) } || maybe { consume(:whitespace) } }
-
- CharData.new(value: value, location: value.location)
- end
-
- def parse_misc
- value =
- atleast do
- maybe { consume(:comment) } ||
- maybe { consume(:processing_instruction) } ||
- maybe { consume(:whitespace) }
- end
-
- Misc.new(value: value, location: value.location)
- end
- end
- end
-end
diff --git a/lib/syntax_tree/xml/pretty_print.rb b/lib/syntax_tree/xml/pretty_print.rb
deleted file mode 100644
index 22428f1..0000000
--- a/lib/syntax_tree/xml/pretty_print.rb
+++ /dev/null
@@ -1,88 +0,0 @@
-# frozen_string_literal: true
-
-module SyntaxTree
- module XML
- class PrettyPrint < Visitor
- attr_reader :q
-
- def initialize(q)
- @q = q
- end
-
- # Visit a Token node.
- def visit_token(node)
- q.pp(node.value)
- end
-
- # Visit a Document node.
- def visit_document(node)
- visit_node("document", node)
- end
-
- # Visit a Prolog node.
- def visit_prolog(node)
- visit_node("prolog", node)
- end
-
- # Visit a Doctype node.
- def visit_doctype(node)
- visit_node("doctype", node)
- end
-
- # Visit an ExternalID node.
- def visit_external_id(node)
- visit_node("external_id", node)
- end
-
- # Visit an Element node.
- def visit_element(node)
- visit_node("element", node)
- end
-
- # Visit an Element::OpeningTag node.
- def visit_opening_tag(node)
- visit_node("opening_tag", node)
- end
-
- # Visit an Element::ClosingTag node.
- def visit_closing_tag(node)
- visit_node("closing_tag", node)
- end
-
- # Visit a Reference node.
- def visit_reference(node)
- visit_node("reference", node)
- end
-
- # Visit an Attribute node.
- def visit_attribute(node)
- visit_node("attribute", node)
- end
-
- # Visit a CharData node.
- def visit_char_data(node)
- visit_node("char_data", node)
- end
-
- # Visit a Misc node.
- def visit_misc(node)
- visit_node("misc", node)
- end
-
- private
-
- # A generic visit node function for how we pretty print nodes.
- def visit_node(type, node)
- q.group do
- q.text("(#{type}")
- q.nest(2) do
- q.breakable
- q.seplist(node.child_nodes) { |child_node| visit(child_node) }
- end
- q.breakable("")
- q.text(")")
- end
- end
- end
- end
-end
diff --git a/package.json b/package.json
new file mode 100644
index 0000000..7651411
--- /dev/null
+++ b/package.json
@@ -0,0 +1,10 @@
+{
+ "scripts": {
+ "prepare": "husky install"
+ },
+ "lint-staged": {
+ "lib/**/*.rb": [
+ "bundle exec stree write"
+ ]
+ }
+}
diff --git a/syntax_tree-erb.gemspec b/syntax_tree-erb.gemspec
new file mode 100644
index 0000000..afa3017
--- /dev/null
+++ b/syntax_tree-erb.gemspec
@@ -0,0 +1,33 @@
+# frozen_string_literal: true
+
+require_relative "lib/syntax_tree/erb/version"
+
+Gem::Specification.new do |spec|
+ spec.name = "w_syntax_tree-erb"
+ spec.version = SyntaxTree::ERB::VERSION
+ spec.authors = ["Kevin Newton", "David Wessman"]
+ spec.email = %w[kddnewton@gmail.com david@wessman.co]
+
+ spec.summary = "Syntax Tree support for ERB"
+ spec.homepage = "https://github.com/davidwessman/syntax_tree-erb"
+ spec.license = "MIT"
+ spec.metadata = { "rubygems_mfa_required" => "true" }
+
+ spec.files =
+ Dir.chdir(__dir__) do
+ `git ls-files -z`.split("\x0")
+ .reject { |f| f.match(%r{^(test|spec|features)/}) }
+ end
+
+ spec.bindir = "exe"
+ spec.executables = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
+ spec.require_paths = %w[lib]
+
+ spec.add_runtime_dependency "prettier_print", "~> 1.2", ">= 1.2.0"
+ spec.add_runtime_dependency "syntax_tree", "~> 6.1", ">= 6.1.1"
+
+ spec.add_development_dependency "bundler", "~> 2"
+ spec.add_development_dependency "minitest", "~> 5"
+ spec.add_development_dependency "rake", "~> 13"
+ spec.add_development_dependency "simplecov", "~> 0.22"
+end
diff --git a/syntax_tree-xml.gemspec b/syntax_tree-xml.gemspec
deleted file mode 100644
index 41adc46..0000000
--- a/syntax_tree-xml.gemspec
+++ /dev/null
@@ -1,33 +0,0 @@
-# frozen_string_literal: true
-
-require_relative "lib/syntax_tree/xml/version"
-
-Gem::Specification.new do |spec|
- spec.name = "syntax_tree-xml"
- spec.version = SyntaxTree::XML::VERSION
- spec.authors = ["Kevin Newton"]
- spec.email = ["kddnewton@gmail.com"]
-
- spec.summary = "Syntax Tree support for XML"
- spec.homepage = "https://github.com/ruby-syntax-tree/syntax_tree-xml"
- spec.license = "MIT"
- spec.metadata = { "rubygems_mfa_required" => "true" }
-
- spec.files = Dir.chdir(__dir__) do
- `git ls-files -z`.split("\x0").reject do |f|
- f.match(%r{^(test|spec|features)/})
- end
- end
-
- spec.bindir = "exe"
- spec.executables = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
- spec.require_paths = %w[lib]
-
- spec.add_dependency "prettier_print", ">= 1.0.2"
- spec.add_dependency "syntax_tree", ">= 4.0.1"
-
- spec.add_development_dependency "bundler"
- spec.add_development_dependency "minitest"
- spec.add_development_dependency "rake"
- spec.add_development_dependency "simplecov"
-end
diff --git a/test/erb_test.rb b/test/erb_test.rb
new file mode 100644
index 0000000..85a4f8d
--- /dev/null
+++ b/test/erb_test.rb
@@ -0,0 +1,170 @@
+# frozen_string_literal: true
+
+require "test_helper"
+
+module SyntaxTree
+ class ErbTest < TestCase
+ def test_empty_file
+ parsed = ERB.parse("")
+ assert_instance_of(SyntaxTree::ERB::Document, parsed)
+ assert_empty(parsed.elements)
+ assert_nil(parsed.location)
+ end
+
+ def test_missing_erb_end_tag
+ assert_raises(SyntaxTree::ERB::Parser::ParseError) do
+ ERB.parse("<% if no_end_tag %>")
+ end
+ end
+
+ def test_missing_erb_block_end_tag
+ assert_raises(SyntaxTree::ERB::Parser::ParseError) do
+ ERB.parse("<% no_end_tag do %>")
+ end
+ end
+
+ def test_missing_erb_case_end_tag
+ assert_raises(SyntaxTree::ERB::Parser::ParseError) do
+ ERB.parse("<% case variabel %>\n<% when 1>\n Hello\n")
+ end
+ end
+
+ def test_erb_code_with_non_ascii
+ parsed = ERB.parse("<% \"Påäööööö\" %>")
+ assert_equal(1, parsed.elements.size)
+ assert_instance_of(SyntaxTree::ERB::ErbNode, parsed.elements.first)
+ end
+
+ def test_if_and_end_in_same_output_tag_short
+ source = "<%= if true\n what\nend %>"
+ expected = "<%= what if true %>\n"
+
+ assert_formatting(source, expected)
+ end
+
+ def test_if_and_end_in_same_tag
+ source =
+ "Hello\n<% if true then this elsif false then that else maybe end %>\n
Hey "
+ expected =
+ "Hello\n<% if true\n this\nelsif false\n that\nelse\n maybe\nend %>\n
Hey \n"
+
+ assert_formatting(source, expected)
+ end
+
+ def test_erb_inside_html_tag
+ source = "
>
"
+ expected = "
>
\n"
+
+ assert_formatting(source, expected)
+ end
+
+ def test_long_if_statement
+ source =
+ "<%=number_to_percentage(@reports&.first&.stability*100,precision: 1) if @reports&.first&.other&.stronger&.longer %>"
+ expected =
+ "<%= if @reports&.first&.other&.stronger&.longer\n number_to_percentage(@reports&.first&.stability * 100, precision: 1)\nend %>\n"
+
+ assert_formatting(source, expected)
+ end
+
+ def test_erb_else_if_statement
+ source =
+ "<%if this%>\n
A \n<%elsif that%>\n
B \n<%else%>\n
C \n<%end%>"
+ expected =
+ "<% if this %>\n
A \n<% elsif that %>\n
B \n<% else %>\n
C \n<% end %>\n"
+
+ assert_formatting(source, expected)
+ end
+
+ def test_long_ternary
+ source =
+ "<%= number_to_percentage(@reports&.first&.stability * 100, precision: @reports&.first&.stability ? 'Stable' : 'Unstable') %>"
+ expected =
+ "<%= number_to_percentage(\n @reports&.first&.stability * 100,\n precision: @reports&.first&.stability ? \"Stable\" : \"Unstable\"\n) %>\n"
+
+ assert_formatting(source, expected)
+ end
+
+ def test_text_erb_text
+ source =
+ "
This is some text <%= variable %> and the special value after
"
+ expected =
+ "
This is some text <%= variable %> and the special value after
\n"
+
+ assert_formatting(source, expected)
+ end
+
+ def test_erb_with_comment
+ source = "<%= what # This is a comment %>\n"
+
+ assert_formatting(source, source)
+ end
+
+ def test_erb_only_ruby_comment
+ source = "<% # This should be written on one line %>\n"
+
+ assert_formatting(source, source)
+ end
+
+ def test_erb_comment
+ source = "<%# This should be written on one line %>\n"
+
+ assert_formatting(source, source)
+ end
+
+ def test_erb_multiline_comment
+ source =
+ "<%#\n This is the first\n This is the second\n This is the third %>"
+ expected =
+ "<%#\nThis is the first\nThis is the second\nThis is the third %>\n"
+
+ assert_formatting(source, expected)
+ end
+
+ def test_erb_ternary_as_argument_without_parentheses
+ source =
+ "<%= f.submit( f.object.id.present? ? t('buttons.titles.save'):t('buttons.titles.create')) %>"
+ expected =
+ "<%= f.submit(\n f.object.id.present? ? t(\"buttons.titles.save\") : t(\"buttons.titles.create\")\n) %>\n"
+
+ assert_formatting(source, expected)
+ end
+
+ def test_erb_whitespace
+ source =
+ "<%= 1 %>,<%= 2 %>What\n<%= link_to(url) do %>
Very long link Very long link Very long link Very long link <% end %>"
+ expected =
+ "<%= 1 %>,<%= 2 %>What\n<%= link_to(url) do %>\n
Very long link Very long link Very long link Very long link \n<% end %>\n"
+
+ assert_formatting(source, expected)
+ end
+
+ def test_erb_block_do_arguments
+ source = "<%= link_to(url) do |link, other_arg|%>Whaaaaaaat<% end %>"
+ expected =
+ "<%= link_to(url) do |link, other_arg| %>\n Whaaaaaaat\n<% end %>\n"
+
+ assert_formatting(source, expected)
+ end
+
+ def test_erb_newline
+ source = "<%= what if this %>\n
hej "
+ expected = "<%= what if this %>\n
hej \n"
+
+ assert_formatting(source, expected)
+ end
+
+ def test_erb_group_blank_line
+ source = "<%= hello %>\n<%= heya %>\n\n<%# breaks the group %>\n"
+
+ assert_formatting(source, source)
+ end
+
+ def test_erb_empty_first_line
+ source = "\n\n<%= what %>\n"
+ expected = "<%= what %>\n"
+
+ assert_formatting(source, expected)
+ end
+ end
+end
diff --git a/test/fixture/block_formatted.html.erb b/test/fixture/block_formatted.html.erb
new file mode 100644
index 0000000..bed1d7f
--- /dev/null
+++ b/test/fixture/block_formatted.html.erb
@@ -0,0 +1,17 @@
+<%= form_with(url: format_path) do |form| %>
+
+ <%= form.label(:name, "Name") %>
+ <%= form.text_field(:name, class: "form-control") %>
+
+
+ <%= form.submit(
+ "Very very very very very long text",
+ class: "btn btn-primary btn btn-primary btn btn-primary btn btn-primary"
+ ) %>
+<% end %>
+
+<%= link_to(dont_replace, what_to_do, class: "do |what,bad|") do |hello| %>
+
Should allow to use the word do in the code
+<% end %>
+
+<%= this_is_not_a_do_block_do %>
diff --git a/test/fixture/block_unformatted.html.erb b/test/fixture/block_unformatted.html.erb
new file mode 100644
index 0000000..bfb0796
--- /dev/null
+++ b/test/fixture/block_unformatted.html.erb
@@ -0,0 +1,24 @@
+<%= form_with(url: format_path) do |form| %>
+
+ <%=form.label(:name, "Name")%>
+ <%=form.text_field(:name, class: "form-control")%>
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <%= form.submit("Very very very very very long text", class: "btn btn-primary btn btn-primary btn btn-primary btn btn-primary") %>
+<% end %>
+
+<%= link_to(dont_replace, what_to_do, class: 'do |what,bad|') do |hello| %>
Should allow to use the word do in the code
+ <% end %>
+
+<%= this_is_not_a_do_block_do %>
diff --git a/test/fixture/case_formatted.html.erb b/test/fixture/case_formatted.html.erb
new file mode 100644
index 0000000..3db1640
--- /dev/null
+++ b/test/fixture/case_formatted.html.erb
@@ -0,0 +1,10 @@
+<% case variable %>
+<% when 1 %>
+ This is the first case.
+<% when 2 %>
+ This is the second case.
+<% when 3 %>
+ This is the third case.
+<% else %>
+ This is the default case.
+<% end %>
diff --git a/test/fixture/case_unformatted.html.erb b/test/fixture/case_unformatted.html.erb
new file mode 100644
index 0000000..cdb95c0
--- /dev/null
+++ b/test/fixture/case_unformatted.html.erb
@@ -0,0 +1,7 @@
+<%case variable%><%when 1 %>This is the first case.<% when 2 %>
+This is the second case.
+<% when 3%>
+ This is the third case.
+<% else %>
+This is the default case.
+<% end%>
diff --git a/test/fixture/erb_syntax_formatted.html.erb b/test/fixture/erb_syntax_formatted.html.erb
new file mode 100644
index 0000000..835750a
--- /dev/null
+++ b/test/fixture/erb_syntax_formatted.html.erb
@@ -0,0 +1,49 @@
+<% this = "avoids line break after expression" -%>
+<%# This is an ERB-comment https://stackoverflow.com/a/25626629 this answer describes ERB and erubis syntax%>
+<%== rails_raw_output %>
+<%- "this only works in ERB not erubis" %>
+<% # This should be written on one line %>
+<%#
+This is a comment
+It can be mutiline
+Treat it as a comment
+%>
+
+<% if this -%>
+ <%= form.submit -%>
+<% elsif that -%>
+ <%= form.submit -%>
+<% else -%>
+ <%= form.submit -%>
+<% end -%>
+
+<%- if this %>
+ <%= form.submit -%>
+<%- elsif that %>
+ <%= form.submit -%>
+<%- else %>
+ <%= form.submit -%>
+<%- end %>
+
+<%= link_to(link, text) do -%>
+
Cool
+<%- end %>
+
+<%= t(
+ ".verified_at",
+ at:
+ (
+ if @repository.github_status_at
+ l(@repository.github_status_at, format: :long)
+ else
+ "?"
+ end
+ )
+) %>
+
+<%
+ assign_b ||= "b"
+ assign_c ||= "c"
+%>
+
+
mt-<%= 5 * 5 %>">
diff --git a/test/fixture/erb_syntax_unformatted.html.erb b/test/fixture/erb_syntax_unformatted.html.erb
new file mode 100644
index 0000000..4763ae7
--- /dev/null
+++ b/test/fixture/erb_syntax_unformatted.html.erb
@@ -0,0 +1,50 @@
+<% this = "avoids line break after expression"-%>
+<%# This is an ERB-comment https://stackoverflow.com/a/25626629 this answer describes ERB and erubis syntax%>
+<%== rails_raw_output%>
+<%-"this only works in ERB not erubis"%>
+<% # This should be written on one line %>
+<%#
+ This is a comment
+ It can be mutiline
+ Treat it as a comment
+%>
+
+<% if this -%>
+ <%= form.submit -%>
+<% elsif that -%>
+ <%= form.submit -%>
+<% else -%>
+ <%= form.submit -%>
+<% end -%>
+
+<%- if this %>
+ <%= form.submit -%>
+<%- elsif that %>
+ <%= form.submit -%>
+<%- else %>
+ <%= form.submit -%>
+<%- end %>
+
+<%= link_to(link, text) do -%>
+
Cool
+<%- end %>
+
+
+<%= t(
+ ".verified_at",
+ at:
+ (
+ if @repository.github_status_at
+ l(@repository.github_status_at, format: :long)
+ else
+ "?"
+ end
+ )
+) %>
+
+<%
+ assign_b ||= "b"
+ assign_c ||= "c"
+%>
+
+
mt-<%=5 * 5%>">
diff --git a/test/fixture/formatted.xml b/test/fixture/formatted.xml
deleted file mode 100644
index 3e68413..0000000
--- a/test/fixture/formatted.xml
+++ /dev/null
@@ -1,69 +0,0 @@
-
-
-
-
-
- Style inheritance and the use element
-
- &
- 〹
-
- bar
-
-
-
-
-
-
-
-
- - 1
- - 2
- - 3
-
-
-
-
-
-
-
-
-
-
-
- Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed at est eget
- enim consectetur accumsan. Aliquam pretium sodales ipsum quis dignissim. Sed
- id sem vel diam luctus fringilla. Aliquam quis egestas magna. Curabitur
- molestie lorem et odio porta, et molestie libero laoreet. Morbi rhoncus
- sagittis cursus. Nullam vehicula pretium consequat. Praesent porta ante at
- posuere sollicitudin. Nullam commodo tempor arcu, at condimentum neque
- elementum ut.
-
- content
-
-
- even more
-
-
-
diff --git a/test/fixture/if_statements_formatted.html.erb b/test/fixture/if_statements_formatted.html.erb
new file mode 100644
index 0000000..2e7a9d3
--- /dev/null
+++ b/test/fixture/if_statements_formatted.html.erb
@@ -0,0 +1,26 @@
+<% if this %>
+
that
+<% elsif that %>
+
this
+ <% if nested_this %>
+
this
+ <% end %>
+<% else %>
+
else
+<% end %>
+<%= what if this %>
+
+ <% unless what %>
+ Ja
+ <% elsif allowed? %>
+ Nej
+ <% else %>
+ Kanske
+ <% end %>
+
+
+<%= if @model
+ @model.name
+else
+ t("views.more_than_80_characters_long_row.categories.shared.version.default")
+end %>
diff --git a/test/fixture/if_statements_unformatted.html.erb b/test/fixture/if_statements_unformatted.html.erb
new file mode 100644
index 0000000..e0d28b4
--- /dev/null
+++ b/test/fixture/if_statements_unformatted.html.erb
@@ -0,0 +1,8 @@
+<%if this%>
that <%elsif that %>
this
+<% if nested_this %>
this <%end%><%else%>
else <%end %>
+<%= what if this %>
+
<% unless what %> Ja<% elsif allowed? %>
+ Nej<% else %>Kanske
+ <% end %>
+
+<%= @model ? @model.name : t("views.more_than_80_characters_long_row.categories.shared.version.default") %>
diff --git a/test/fixture/javascript_frameworks_formatted.html.erb b/test/fixture/javascript_frameworks_formatted.html.erb
new file mode 100644
index 0000000..df68dc7
--- /dev/null
+++ b/test/fixture/javascript_frameworks_formatted.html.erb
@@ -0,0 +1,32 @@
+
+
+ "
+ boolean
+ :value="['a', 'b']"
+ :long-variable-name="data.item.javascript.code"
+ >
+
+
+ Hello
+
+
+
+
+
+
+
+ Expand
+
+ Hello
+
+
diff --git a/test/fixture/javascript_frameworks_unformatted.html.erb b/test/fixture/javascript_frameworks_unformatted.html.erb
new file mode 100644
index 0000000..33ae765
--- /dev/null
+++ b/test/fixture/javascript_frameworks_unformatted.html.erb
@@ -0,0 +1,10 @@
+
+
" boolean :value="['a', 'b']" :long-variable-name="data.item.javascript.code">
+
+Hello
+
+
+
+
+
+Expand Hello
diff --git a/test/fixture/layout_formatted.html.erb b/test/fixture/layout_formatted.html.erb
new file mode 100644
index 0000000..1607c7c
--- /dev/null
+++ b/test/fixture/layout_formatted.html.erb
@@ -0,0 +1,45 @@
+
+
+
+
+
+
+
<%= full_title(t("general.title"), yield(:title)) %>
+ <%= stylesheet_link_tag(
+ "application",
+ media: "all",
+ "data-turbolinks-track": "reload"
+ ) %>
+ <%= javascript_include_tag(
+ "application",
+ "data-turbolinks-track": "reload",
+ defer: true
+ ) %>
+ <%= csrf_meta_tags %>
+ <%= csp_meta_tag %>
+ <%= render("application/favicon") %>
+
+
+
+
+ <%= render("header") %>
+
+
+ <%= yield(:sidebar) %>
+
+ <%= render("application/heading") %>
+ <%= render("flashes") %>
+ <%= yield %>
+
+
+
+
+
+
+
diff --git a/test/fixture/layout_unformatted.html.erb b/test/fixture/layout_unformatted.html.erb
new file mode 100644
index 0000000..3176d5a
--- /dev/null
+++ b/test/fixture/layout_unformatted.html.erb
@@ -0,0 +1,33 @@
+
+
+
+
+
+
+
<%= full_title(t('general.title'), yield(:title)) %>
+ <%= stylesheet_link_tag('application', media: 'all', 'data-turbolinks-track': 'reload') %>
+ <%= javascript_include_tag('application', 'data-turbolinks-track': 'reload', defer: true) %>
+ <%= csrf_meta_tags %>
+ <%= csp_meta_tag %>
+ <%= render('application/favicon') %>
+
+
+
+
+ <%= render('header') %>
+
+
+ <%= yield(:sidebar) %>
+
+ <%= render('application/heading') %>
+ <%= render('flashes') %>
+ <%= yield %>
+
+
+
+
+ <%= render('footer') %>
+
+
+
+
diff --git a/test/fixture/nested_html_formatted.html.erb b/test/fixture/nested_html_formatted.html.erb
new file mode 100644
index 0000000..1322953
--- /dev/null
+++ b/test/fixture/nested_html_formatted.html.erb
@@ -0,0 +1,5 @@
+
+
+ <%= t(".title") + " " + t(".description") + " " + t(".pretty_long") %>'This is a single quote'
+ "This is a double quote"
+
diff --git a/test/fixture/nested_html_unformatted.html.erb b/test/fixture/nested_html_unformatted.html.erb
new file mode 100644
index 0000000..ce02151
--- /dev/null
+++ b/test/fixture/nested_html_unformatted.html.erb
@@ -0,0 +1 @@
+
<%= t(".title") + " " + t(".description") + " " + t(".pretty_long") %>'This is a single quote' "This is a double quote"
diff --git a/test/fixture/unformatted.xml b/test/fixture/unformatted.xml
deleted file mode 100644
index 596d7df..0000000
--- a/test/fixture/unformatted.xml
+++ /dev/null
@@ -1,63 +0,0 @@
-
-
-
-
-
- Style inheritance and the use element
-
- & 〹
-
- bar
-
-
-
-
-
-
-
-
- - 1
- - 2
- - 3
-
-
-
-
-
-
-
-
-
- < ignored />
-
-
-
- Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed at est eget enim consectetur accumsan. Aliquam pretium sodales ipsum quis dignissim. Sed id sem vel diam luctus fringilla. Aliquam quis egestas magna. Curabitur molestie lorem et odio porta, et molestie libero laoreet. Morbi rhoncus sagittis cursus. Nullam vehicula pretium consequat. Praesent porta ante at posuere sollicitudin. Nullam commodo tempor arcu, at condimentum neque elementum ut.
-
-
- content
-
-
-
-
- even more
-
-
-
diff --git a/test/formatting_test.rb b/test/formatting_test.rb
new file mode 100644
index 0000000..e0b69b5
--- /dev/null
+++ b/test/formatting_test.rb
@@ -0,0 +1,62 @@
+# frozen_string_literal: true
+
+require "test_helper"
+
+module SyntaxTree
+ class FormattingTest < Minitest::Test
+ def test_block
+ assert_formatting("block")
+ end
+
+ def test_erb_syntax
+ assert_formatting("erb_syntax")
+ end
+
+ def test_nested_html
+ assert_formatting("nested_html")
+ end
+
+ def test_if_statements
+ assert_formatting("if_statements")
+ end
+
+ def test_javascript_frameworks
+ assert_formatting("javascript_frameworks")
+ end
+
+ def test_case_statements
+ assert_formatting("case")
+ end
+
+ def test_layout
+ assert_formatting("layout")
+ end
+
+ private
+
+ def assert_formatting(name)
+ directory = File.expand_path("fixture", __dir__)
+ unformatted_file = File.join(directory, "#{name}_unformatted.html.erb")
+ formatted_file = File.join(directory, "#{name}_formatted.html.erb")
+ source = SyntaxTree::ERB.read(unformatted_file)
+
+ expected = SyntaxTree::ERB.read(formatted_file)
+ formatted = SyntaxTree::ERB.format(source)
+
+ if (expected != formatted)
+ puts("Failed to format #{name}, see ./tmp/#{name}_failed.html.erb")
+ Dir.mkdir("./tmp") unless Dir.exist?("./tmp")
+ File.write("./tmp/#{name}_failed.html.erb", formatted)
+ end
+
+ assert_equal(formatted, expected)
+
+ formatted_twice = SyntaxTree::ERB.format(formatted)
+ assert_equal(formatted_twice, expected)
+
+ # Check that pretty_print works
+ output = SyntaxTree::ERB.parse(expected).pretty_inspect
+ refute_predicate(output, :empty?)
+ end
+ end
+end
diff --git a/test/html_test.rb b/test/html_test.rb
new file mode 100644
index 0000000..7fff590
--- /dev/null
+++ b/test/html_test.rb
@@ -0,0 +1,216 @@
+# frozen_string_literal: true
+
+require "test_helper"
+
+module SyntaxTree
+ class HtmlTest < TestCase
+ def test_html_missing_end_tag
+ assert_raises(SyntaxTree::ERB::Parser::ParseError) do
+ ERB.parse("
Hello World")
+ end
+ end
+
+ def test_html_incorrect_end_tag
+ assert_raises(SyntaxTree::ERB::Parser::ParseError) do
+ ERB.parse("Hello World")
+ end
+ end
+
+ def test_html_unmatched_double_quote
+ assert_raises(SyntaxTree::ERB::Parser::ParseError) do
+ ERB.parse(" Hello World
")
+ end
+ end
+
+ def test_html_unmatched_single_quote
+ assert_raises(SyntaxTree::ERB::Parser::ParseError) do
+ ERB.parse("Hello World
")
+ end
+ end
+
+ def test_empty_file
+ source = ""
+ assert_formatting(source, "\n")
+ end
+
+ def test_html_doctype
+ parsed = ERB.parse("")
+ assert_instance_of(SyntaxTree::ERB::Doctype, parsed.elements.first)
+
+ parsed = ERB.parse("")
+ assert_instance_of(SyntaxTree::ERB::Doctype, parsed.elements.first)
+
+ # Allow doctype to not be the first element
+ parsed = ERB.parse("<% theme = \"general\" %> ")
+ assert_equal(2, parsed.elements.size)
+ assert_equal(
+ [SyntaxTree::ERB::ErbNode, SyntaxTree::ERB::Doctype],
+ parsed.elements.map(&:class)
+ )
+
+ # Do not allow multiple doctype elements
+ assert_raises(SyntaxTree::ERB::Parser::ParseError) do
+ ERB.parse("\n\n")
+ end
+ end
+
+ def test_html_comment
+ source = "\n"
+ parsed = ERB.parse(source)
+ elements = parsed.elements
+ assert_equal([SyntaxTree::ERB::HtmlComment], elements.map(&:class))
+
+ assert_formatting(source, source)
+ end
+
+ def test_html_within_quotes
+ source =
+ "This is our text \"<%= @object.quote %> \"
"
+ parsed = ERB.parse(source)
+ elements = parsed.elements
+
+ assert_equal(1, elements.size)
+ assert_instance_of(SyntaxTree::ERB::HtmlNode, elements.first)
+ elements = elements.first.elements
+
+ assert_equal("This is our text \"", elements.first.value.value)
+ assert_equal("\"", elements.last.value.value)
+ end
+
+ def test_html_tag_names
+ assert_raises(SyntaxTree::ERB::Parser::ParseError) do
+ ERB.parse("<@br />")
+ end
+ assert_raises(SyntaxTree::ERB::Parser::ParseError) do
+ ERB.parse("<:br />")
+ end
+ assert_raises(SyntaxTree::ERB::Parser::ParseError) do
+ ERB.parse("<#br />")
+ end
+ end
+
+ def test_html_attribute_without_quotes
+ source = "
Hello World
"
+ parsed = ERB.parse(source)
+ elements = parsed.elements
+
+ assert_equal(1, elements.size)
+ assert_instance_of(SyntaxTree::ERB::HtmlNode, elements.first)
+ assert_equal(1, elements.first.opening.attributes.size)
+
+ attribute = elements.first.opening.attributes.first
+ assert_equal("class", attribute.key.value)
+ assert_equal("card", attribute.value.contents.first.value)
+
+ expected = "
Hello World
\n"
+ assert_formatting(source, expected)
+ end
+
+ def test_empty_component_without_attributes
+ source = "
\n \n"
+ expected = "
\n"
+
+ assert_formatting(source, expected)
+ end
+
+ def test_empty_component_with_attributes
+ source =
+ "
\n "
+ expected =
+ "
\n"
+ assert_formatting(source, expected)
+ end
+
+ def test_keep_lines_with_text_in_block
+ source = "Hello <%= @football_team_membership.user %>, "
+ expected = "Hello <%= @football_team_membership.user %>, \n"
+
+ assert_formatting(source, expected)
+ end
+
+ def test_keep_lines_with_text_in_block_in_document
+ source = "Hello Name !"
+ expected = "Hello Name !\n"
+ assert_formatting(source, expected)
+ end
+
+ def test_keep_lines_with_nested_html
+ source = "Hello Name !
"
+ expected = "Hello Name !
\n"
+ assert_formatting(source, expected)
+ end
+
+ def test_newlines
+ source = "Hello\n\n\n\nGoodbye!\n"
+ expected = "Hello\n\nGoodbye!\n"
+
+ assert_formatting(source, expected)
+ end
+
+ def test_indentation
+ source =
+ "\n"
+
+ expected = "\n"
+
+ assert_formatting(source, expected)
+ end
+
+ def test_append_newlines
+ source = "\nWhat\n
"
+ parsed = ERB.parse(source)
+
+ assert_equal(1, parsed.elements.size)
+ html = parsed.elements.first
+
+ refute_nil(html.opening.new_line)
+ refute_nil(html.elements.first.new_line)
+ assert_nil(html.closing.new_line)
+
+ assert_formatting(source, "What
\n")
+ assert_formatting("What
", "What
\n")
+ end
+
+ def test_self_closing_with_blank_line
+ source =
+ "Test \n"
+
+ assert_formatting(source, source)
+ end
+
+ def test_tag_with_leading_and_trailing_spaces
+ source = " What
"
+ expected = "What
\n"
+ assert_formatting(source, expected)
+ end
+
+ def test_tag_with_leading_and_trailing_spaces_erb
+ source = " <%=user.name%>
"
+ expected = "<%= user.name %>
\n"
+ assert_formatting(source, expected)
+ end
+
+ def test_breakable_on_char_data_white_space
+ source =
+ "You have been removed as a user from <%= @company.title %> by <%= @administrator.name %>."
+ expected =
+ "You have been removed as a user from \n <%= @company.title %>\n by <%= @administrator.name %>.\n"
+
+ assert_formatting(source, expected)
+ end
+
+ def test_self_closing_group
+ source = "