The approach taken by IntelliJ is two-fold:
- pinning: when a certain token is reached in a parse rule, commit to that rule even if subsequent tokens do not match the remaining rule
- recovery: when committed to a failed rule*, discard tokens until you reach some sentinel token (or arbitrary logic) which indicates a point from which you can recover.
The error recovery can be nested so you can recover from top-level declarations, as Evan described, or you can recover inside expressions, i.e. the incomplete List literal [ 1, 2,
Pinning happens when you see something that unequivocally indicates what the user intends. So if I’m writing an expression in Elm and I type [, we can be pretty sure that the user is trying to write a List literal and we commit to parsing it as a list literal at this point even though there is no matching ].
Recovery happens when you need to drop garbage tokens so that the parser can resume. If I write [1, 2, 3xyz ], the parser will fail when we reach 3xyz (because it’s not a valid expression). Assuming that we pinned (committed to parsing the list literal) when we saw the [, we can recover by dropping tokens until we consume the closing ].
Footnote *: this is a bit of a simplification. I think recovery in IntelliJ/GrammarKit used to work this way (where it would only happen when a rule failed but was pinned), but now I think the recovery is always done, even when a rule succeeds normally.