Commit Graph

3 Commits

Author SHA1 Message Date
Martin Atkins
755fe38505 zclsyntax: peeker pretends single-line comments are newlines
When we're skipping comments but retaining newlines, we need to do some
slight-of-hand because single-line comment tokens contain the newline
that terminates them (for simpler handling of lead doc comments) but our
parsing can be newline-sensitive.

To allow for this, as a special case we transform single-line comment
tokens into newlines when in this situation, thus allowing parser code
to just worry about the newlines and skip over the comments.
2017-06-09 07:19:25 -07:00
Martin Atkins
14d256467a zclsyntax: peeker methods to get next and previous token ranges 2017-05-29 19:01:37 -07:00
Martin Atkins
23cb83ef3e zclsyntax: "peeker" helper
This will make it easier for the parser to walk through the token sequence
with one token of lookahead, maintaining state of where we've got to and
also allowing us to switch into and out of the newline-sensitive mode
as we transition to and from newline-sensitive constructs.
2017-05-29 18:09:58 -07:00