Syntax errors are challenging to parse

[aaronjensen]

I'm not sure if this is a prism concern or a Ruby concern, but the syntax errors that Ruby prints are relatively challenging to parse for something like flycheck (flycheck/flycheck#2111). It's easy enough if we only want to highlight the very first syntax error, but if we would like to highlight all of the lines that there are errors along with the message it would require a multi-line regular expression (and counting spaces to ascertain the column). For a human they look great, so my concern is only about machine parsing.

$ echo "do\ndo" | ruby -wc
/redacted-path/ruby: -:1: syntax errors found (SyntaxError)
> 1 | do
    | ^~ unexpected 'do', ignoring it
> 2 | do
    | ^~ unexpected 'do', ignoring it

I'm curious if there could be a mode where Ruby printed the syntax errors each on their own line, similar to how warnings are printed:

$ echo "1 +x 2 do\ndo" | ruby -wc
-:1: warning: '+' after local variable or literal is interpreted as binary operator even though it seems like unary operator
-:1: warning: possibly useless use of + in void context
-:1: warning: possibly useless use of a literal in void context
/redacted-path/ruby: -:1: syntax errors found (SyntaxError)
> 1 | 1 +x 2 do
    |      ^ unexpected integer, expecting end-of-input
    |        ^~ unexpected 'do', ignoring it
    |        ^~ unexpected 'do', expecting end-of-input
> 2 | do
    | ^~ unexpected 'do', ignoring it

It's also unclear why the interpreter path is included in the syntax error output but not for the warnings. It's a seemingly odd inconsistency, but perhaps there is a reason for it.

If I should post this message to https://bugs.ruby-lang.org/ instead, please let me know. Thank you!

[bbatsov]

It's also unclear why the interpreter path is included in the syntax error output but not for the warnings. It's a seemingly odd inconsistency, but perhaps there is a reason for it.

I'm curious about the reasoning behind this as well. Especially, given that it wasn't like this before Ruby 3.4. It seems to me it'd be nice to either have detailed diagnostics for both warnings and errors or not for both of them.

I agree with @aaronjensen that ideally there should be a mode for humans (detailed diagnostics) and something more compact for editor lint plugins and similar tools, that is easier to parse.

[kddnewton]

Hey there. I definitely see what you're saying here, we really shouldn't be parsing formatted syntax errors, that's difficult.

From Prism's perspective (this repo specifically) all we do is bundle up the errors and warnings in the same kind of objects. In C, it's a linked list of objects that contain location information, a type, and a message. So from here, I think this is about what you would expect from the parser.

From CRuby's perspective, things get a little weirder. Warnings are gathered up and printed out one line at a time, and errors are grouped together into a singular syntax error and raised all as one. I've raised issues before (https://bugs.ruby-lang.org/issues/20024) about trying to get more detailed error messaging information, but haven't gotten too far.

Let me back up and ask your specific use case. Is a human or a computer running this code? Is this editor feedback? Are you in a place where you could manually parse the source Prism.parse(...) or is this basically parsing stderr?

In the simplest case, it might be easiest to add an environment variable that's like PRISM_ERRORS_PLAIN or something like that.

[aaronjensen]

The thing that led me here was attempting to add a computer-based parsing of Ruby's errors in a lint error displaying plugin for Emacs called Flycheck. It invokes Ruby with -wc. A different checker could potentially be written that invokes Prism directly, but I'm not sure how we'd do that and also get the Ruby warnings. I suppose we could syntax check first with Prism and skip the warning checker if the syntax check fails.

It does seem like from the Ruby issue you linked that you've considered this, so that's good. I'm not quite sure why the format changed so much in Ruby with the inclusion of the ruby path. Seems like that may have been a mistake. Not sure if that could be corrected.

What I'm hearing though is that it may be a lift at this point for Ruby to emit more than one syntax error, so I guess consider this me expressing interest in that functionality for Ruby. If it would be helpful for me to chime in on that Ruby issue, I'd be happy to.