Meta-issue: Tree restructuring #278
Description
In order to better enable a bunch of functionality in Syntax Tree and to take advantage of YARP, we need to restructure our tree. This is a meta-issue that is meant to track the overall status of each node being changed.
Issues that are tracked in this issue should target the structure branch so that we can continue to release from main in case there are any bugs.
| YARP | Syntax Tree | Issue |
|---|---|---|
| AliasNode | AliasNode | #348 |
| AlternationPatternNode | Binary | #349 |
| AndNode | Binary | #350 |
| ArgumentsNode | Args | #351 |
| ArrayNode | ArrayLiteral, MRHS, QSymbols, QWords, Symbols, Words | #352 |
| ArrayPatternNode | AryPtn | #353 |
| AssocNode | Assoc | |
| AssocSplatNode | AssocSplat | |
| BeginNode | Begin, BodyStmt | |
| BlockArgumentNode | ArgBlock | |
| BlockNode | BlockNode | |
| BlockParameterNode | BlockArg | |
| BlockParametersNode | BlockVar, LambdaVar | |
| BreakNode | Break | |
| CallNode | ARef, ARefField, CallNode, Command, CommandCall, Field, Not, Unary, VCall | #355 |
| CapturePatternNode | Binary | #354 |
| CaseNode | Case | |
| ClassNode | ClassDeclaration | |
| ClassVariableReadNode | CVar | |
| ClassVariableWriteNode | Assign, VarField | |
| ConstantPathNode | ConstPathRef, TopConstRef | |
| ConstantPathWriteNode | ConstPathField, TopConstField, VarField | |
| ConstantReadNode | Const, ConstRef | |
| DefNode | DefNode | |
| DefinedNode | Defined | |
| ElseNode | Else | |
| EnsureNode | Ensure | |
| FalseNode | Kw, VarRef | |
| FindPatternNode | FndPtn | |
| FloatNode | FloatLiteral | |
| ForNode | For | |
| ForwardingArgumentsNode | ArgsForward | |
| ForwardingParameterNode | ArgsForward | |
| ForwardingSuperNode | ZSuper | |
| GlobalVariableReadNode | Backref, GVar | |
| GlobalVariableWriteNode | Assign, VarField | |
| HashNode | BareAssocHash, HashLiteral | |
| HashPatternNode | HshPtn | |
| HeredocNode | Heredoc | |
| IfNode | Elsif, IfNode | |
| ImaginaryNode | Imaginary | |
| InNode | In | |
| InstanceVariableReadNode | IVar | |
| InstanceVariableWriteNode | Assign, VarField | |
| IntegerNode | Int | |
| InterpolatedRegularExpressionNode | RegexpLiteral | |
| InterpolatedStringNode | StringLiteral, Word | |
| InterpolatedSymbolNode | DynaSymbol, Word | |
| InterpolatedXStringNode | XStringLiteral | |
| KeywordParameterNode | Label | |
| KeywordRestParameterNode | KwRestParam | |
| LambdaNode | Lambda | |
| LocalVariableReadNode | VarRef | |
| LocalVariableWriteNode | VarField | |
| MatchPredicateNode | RAssign | |
| MatchRequiredNode | RAssign | |
| MissingNode | ||
| ModuleNode | ModuleDeclaration | |
| MultiWriteNode | MAssign, MLHS | |
| NextNode | Next | |
| NilNode | Kw, VarRef | |
| NoKeywordsParameterNode | :nil | |
| OperatorAndAssignmentNode | OpAssign | |
| OperatorAssignmentNode | OpAssign | |
| OperatorOrAssignmentNode | OpAssign | |
| OptionalParameterNode | Ident | |
| OrNode | Binary | |
| ParametersNode | Params | |
| ParenthesesNode | Paren | |
| PinnedExpressionNode | PinnedBegin | |
| PinnedVariableNode | PinnedVarRef | |
| PostExecutionNode | ENDBlock | |
| PreExecutionNode | BEGINBlock | |
| ProgramNode | Program | |
| RangeNode | RangeNode | |
| RationalNode | RationalLiteral | |
| RedoNode | Redo | |
| RegularExpressionNode | RegexpLiteral | |
| RequiredDestructuredParameterNode | MLHSParen | |
| RequiredParameterNode | Ident | |
| RescueModifierNode | RescueMod | |
| RescueNode | Rescue, RescueEx | |
| RestParameterNode | RestParam | |
| RetryNode | Retry | |
| ReturnNode | ReturnNode | |
| Scope | ||
| SelfNode | Kw, VarRef | |
| SingletonClassNode | SClass | |
| SourceEncodingNode | Kw, VarRef | |
| SourceFileNode | Kw, VarRef | |
| SourceLineNode | Kw, VarRef | |
| SplatNode | ArgStar | |
| StatementsNode | Statements | |
| StringConcatNode | StringConcat | |
| StringInterpolatedNode | StringEmbExpr | |
| StringNode | CHAR, StringLiteral, Word | |
| SuperNode | Super | |
| SymbolNode | Label, SymbolLiteral, Word | |
| TernaryNode | IfOp | |
| TrueNode | Kw, VarRef | |
| UndefNode | Undef | |
| UnlessNode | UnlessNode | |
| UntilNode | UntilNode | |
| WhenNode | When | |
| WhileNode | WhileNode | |
| XStringNode | XStringLiteral | |
| YieldNode | YieldNode |
These nodes are seen as comments and attached to the parse result. For that we should create the same kind of structure instead of adding them to the AST.
CommentEmbDoc,EmbExprBeg,EmbExprEndEndContent
The rest of the nodes in the AST can be removed in favor of just tracking source ranges. This is what YARP and the whitequark/parser gem does and should hopefully cut down on our allocations. These nodes are:
ArgParenBacktickCommaEmbVarExcessedCommaHeredocBegHeredocEndIdentLBraceLBracketLParenLabelEndMethodAddBlockOpPeriodQSymbolsBegQWordsBegRBraceRBracketRParenRegexpBegRegexpContentRegexpEndStringContentStringDVarSymBegSymbolContentSymbolsBegTLamBegTLambdaTStringBegTStringContentTStringEndVoidStmtWordsBegXString