Skip to content

Commit 05deec6

Browse files
committed
Change bin/prism lex, add bin/prism lex_compat
* Current `bin/prism lex` becomes `bin/prism lex_compat` * Change `bin/prism lex` to only output prism tokens I sometimes find myself wanting to look at tokens but on the cli it only compares against other sources. Then I check the code and see that that `VERBOSE=1` does something but the output isn't very readable and doesn't fit on my terminal screen. The new output looks like this: ``` $ bin/prism lex -e "foo(1, BAR, baz, 'bat')" IDENTIFIER (1,0)-(1,3) "foo" PARENTHESIS_LEFT (1,3)-(1,4) "(" INTEGER (1,4)-(1,5) "1" COMMA (1,5)-(1,6) "," CONSTANT (1,7)-(1,10) "BAR" COMMA (1,10)-(1,11) "," IDENTIFIER (1,12)-(1,15) "baz" COMMA (1,15)-(1,16) "," STRING_BEGIN (1,17)-(1,18) "'" STRING_CONTENT (1,18)-(1,21) "bat" STRING_END (1,21)-(1,22) "'" PARENTHESIS_RIGHT (1,22)-(1,23) ")" EOF (1,23)-(1,23) "" ```
1 parent 2ecd49d commit 05deec6

File tree

1 file changed

+17
-0
lines changed

1 file changed

+17
-0
lines changed

bin/prism

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ module Prism
1515
when "encoding" then encoding(argv)
1616
when "error" then error(argv)
1717
when "lex" then lex(argv)
18+
when "lex_compat" then lex_compat(argv)
1819
when "locals" then locals(argv)
1920
when "parse" then parse(argv)
2021
when "parser" then parser(argv)
@@ -31,6 +32,7 @@ module Prism
3132
bin/prism encoding [encoding]
3233
bin/prism error [name] [source]
3334
bin/prism lex [source]
35+
bin/prism lex_compat [source]
3436
bin/prism locals [source]
3537
bin/prism parse [source]
3638
bin/prism parser [source]
@@ -195,6 +197,21 @@ module Prism
195197
# bin/prism lex [source]
196198
def lex(argv)
197199
source, filepath = read_source(argv)
200+
prism = Prism.lex(source, filepath: filepath)
201+
max_token_type_length = prism.value.max_by { |token,| token.type.length }[0].type.length
202+
203+
prism.value.each do |token,|
204+
loc = token.location
205+
puts(format(
206+
"%-#{max_token_type_length + 1}s(%s,%s)-(%s,%s) %s",
207+
token.type, loc.start_line, loc.start_column, loc.end_line, loc.end_column, token.value.inspect
208+
))
209+
end
210+
end
211+
212+
# bin/prism lex_compat [source]
213+
def lex_compat(argv)
214+
source, filepath = read_source(argv)
198215

199216
ripper_value =
200217
begin

0 commit comments

Comments
 (0)