Previously we were detecting the exactly-one-part case at parse time and
skipping the TemplateExpr node in the AST, but this was problematic
because we then misaligned the source range for the expression to the
_content_ of the quotes, rather than including the quotes themselves.
As well as producing confusing diagnostics, this also caused problems for
zclwrite since it relies on source ranges to map our AST back onto the
source tokens it came from.
This allows tooling to get a string describing the context of a particular
offset into the file. This is used, for example, to provide context
above the source code snippets in console-printed diagnostic messages.
There are certain tokens that are _never_ valid, so we might as well
catch them early in the Lex... functions rather than having to handle
them in many different contexts within the parser.
Unfortunately for now when such errors occur they tend to be echoed by
more confusing errors coming from the parser, but we'll accept that for
now.
This applies both two the whole of bare expressions and to any nested
expressions within parentheses. The latter means that an attribute value
can span over multiple lines if it's wrapped in parens:
foo = (
1 + 2 + 3 + 4 + 5
)
These are the top-level interface to parsing configuration files in the
native zcl syntax, similarly to the existing methods for parsing other
syntaxes.
The parser has some recovery heuristics but they will not always achieve
the best result. To prevent unsuccessful recovery from causing a cascade
of confusing follow-on errors, we'll track in the parser whether recovery
has been attempted and then the specific sub-parsers can use this to
skip certain "unexpected token type" errors in recovery situations,
assuming instead that an earlier error will cover the problem and that
we just want to bail out quickly.
Mirroring the Lex... triplet of functions, callers here can choose to
parse in one of three modes:
- ParseConfig is the main entry point and parses what we might consider
a whole config file.
- ParseExpression parses a sequence as an isolated expression, which
may be useful in implementing a REPL to inspect configuration or
application state.
- ParseTemplate parses a sequence as an isolated template, which may be
useful in parsing external files as templates, for situations where the
input is too large to be conveniently embedded within configuration.
This LexConfig, LexExpression and LexTemplate set of functions allow
outside callers to use the scanner in isolation, skipping the parser.
This may be useful for use-cases such as syntax highlighting, separate
parsers (such as the one in zclwrite), and so forth. Most callers should
use the parser (once implemented) though, to get a semantic AST.
