Commit Graph

5 Commits

Author SHA1 Message Date
Martin Atkins
42b18cc7c6 json: count whole grapheme clusters as columns for position purposes
This means we can actually point at a column in the console without it
getting misaligned by multi-byte UTF-8 sequences and Unicode combining
characters.
2017-05-27 17:10:51 -07:00
Martin Atkins
4bbfa6d6ab json: recovery behavior for the parser
When using the parser to do static analysis and editor integrations, it's
still useful to be able to get the incomplete AST resulting from a parse
error.

Callers that intend to use the returned value to take real actions (as
opposed to just analysis) must check diags.HasError() to determine if
the returned file can be considered valid.
2017-05-20 13:32:12 -07:00
Martin Atkins
8dfc3c4bbe json: fix overly-greedy keyword scanning
Logic error in the scanner caused it to always consume the remainder of
the string.
2017-05-16 07:23:33 -07:00
Martin Atkins
45d426b99c json: Generate EOF tokens marking the end of scanning
This helps the parser include a good position when it needs to generate
"unexpected EOF" diagnostics.
2017-05-15 07:15:53 -07:00
Martin Atkins
b5ce4360cd json: initial scanner implementation
Currently lacking correct support for unicode text segmentation into
grapheme clusters, so it miscounts "Column" in positions. This will be
addressed later.
2017-05-15 07:02:13 -07:00