Commit Graph

5 Commits

Author SHA1 Message Date
Martin Atkins
18e45ec05c zclsyntax: scanner support of string templates
This requires some extra state-keeping because we allow templates to be
nested inside templates. This takes us outside of the world of regular
languages, but we accept that here because it makes things easier to
deal with down the line in the parser.

The methodology is to keep track of how many braces are open at a given
time and then, when a nested template interpolation begins, record the
current brace level. Then, when a closing brace is encountered, if its
nesting level is at the top of the stack then we pop off the stack and
return to "main" parsing mode.

Ragel's existing idea of calling and returning from machines is important
here too. As this currently stands this is not actually needed, but once
heredocs are in play we will have two possible places to return to at
the end of an interpolation sequence, so the state return stack maintained
by Ragel will determine whether to return to string mode or heredoc mode.
2017-05-28 15:33:01 -07:00
Martin Atkins
5898b36695 zclsyntax: scan single-character tokens that represent themselves
For convenience we use the rune values of these tokens as their token
enum values, so we can handle them all via a single rule.
2017-05-28 09:34:20 -07:00
Martin Atkins
53944335f1 zclsyntax: identifiers in the scanner 2017-05-28 09:16:53 -07:00
Martin Atkins
6362354c87 zclsyntax: scanning of numeric literals 2017-05-28 08:56:43 -07:00
Martin Atkins
187d7b8045 zclsyntax: re-organize and simplify the scanner 2017-05-28 08:38:13 -07:00