Martin Atkins
5898b36695
zclsyntax: scan single-character tokens that represent themselves
...
For convenience we use the rune values of these tokens as their token
enum values, so we can handle them all via a single rule.
2017-05-28 09:34:20 -07:00
Martin Atkins
53944335f1
zclsyntax: identifiers in the scanner
2017-05-28 09:16:53 -07:00
Martin Atkins
6362354c87
zclsyntax: scanning of numeric literals
2017-05-28 08:56:43 -07:00
Martin Atkins
187d7b8045
zclsyntax: re-organize and simplify the scanner
2017-05-28 08:38:13 -07:00
Martin Atkins
76c0ca70f0
zclsyntax: scanner to return whole token slice at once
...
On reflection, it seems easier to maintain the necessary state we need
by doing all of the scanning in a single pass, since we can then just
use local variables within the scanner function.
2017-05-28 07:11:24 -07:00
