Martin Atkins
377e20b7b8
json: AST node types
2017-05-15 08:34:35 -07:00
Martin Atkins
838a0332e6
json: token peeker
...
This will help our hand-written recursive-descent parser walk the tokens
with one token of lookahead.
2017-05-15 08:34:00 -07:00
Martin Atkins
45d426b99c
json: Generate EOF tokens marking the end of scanning
...
This helps the parser include a good position when it needs to generate
"unexpected EOF" diagnostics.
2017-05-15 07:15:53 -07:00
Martin Atkins
b5ce4360cd
json: initial scanner implementation
...
Currently lacking correct support for unicode text segmentation into
grapheme clusters, so it miscounts "Column" in positions. This will be
addressed later.
2017-05-15 07:02:13 -07:00
