We previously weren't returning appropriate Expression and EvalContext
references inside many of the diagnostics for ForExpr.
First, it was using the top-level expression instead of one of the nested
expressions in many cases. Secondly, it was using the given context
rather than the child context when talking about expressions that get
evaluated once per iteration.
As a result of this reporting we must now produce a new EvalContext for
each iteration, rather than sharing and mutating as we did before, but
in retrospect that's less likely to cause other confusing bugs anyway,
since we don't generally expect EvalContexts to be mutated.
In practice this should never arise because the index operator only works
for lists and maps and they use string keys, but we'll guard against this
anyway and return a placeholder for other values so that the output
doesn't grow unreadably long in that case.
If a diagnostic has an associated Expression and EvalContext then we can
look up the values of any variables referenced in the expression and show
them in the diagnostics message as additional context.
This is particularly useful when dealing with situations where a given
expression is evaluated multiple times with different variables, such as
in a 'for' expression, since each evaluation may produce a different set
of diagnostics.
If a diagnostic occurs while we're evaluating an expression, we'll now
include a reference to that expression in the diagnostic object. We
previously added the corresponding EvalContext here too, and so with these
together it is now possible for a diagnostic renderer to see not only
what was in scope when the problem occurred but also what parts of that
scope the expression was relying on (via method Expression.Variables).
When building tools around HCL configuration files it is useful to be
able to ask what is present at a given position in a file. This set of
new helper functions provide a best-effort implementation of this for
the native syntax only.
It cannot be supported for JSON syntax with these signatures because the
JSON syntax is ambiguous and thus can't be interpreted without a schema
for each structural level. In practice this is not a big loss because
JSON files will usually be generated rather than hand-written anyway, and
so doing automatic analysis and transformation of them would not be
useful: the program that generated the file must be updated instead.
When we're evaluating expressions, we may end up evaluating the same
source-level expression a number of times in different contexts, such as
in a 'for' expression, where each one may produce a different set of
diagnostic messages.
Now we'll attach the EvalContext to each expression diagnostic so that
a diagnostic renderer can potentially show additional information to help
distinguish the different iterations in rendered diagnostics.
During implementation of HCL in other applications, it became clear that
the overloading of the word "attribute" to mean both a key/value pair in
a body and an element within an object value creates confusion.
It's too late to change that in the HCL Go API now, but here we at least
update the diagnostic messages. The new convention is that a key/value
pair within a block is now called an "argument", while an element of an
object is still called an "attribute".
It is unfortunate that the Go-facing API still uses the word "attribute"
for both, but the user experience is the most important thing and in
practice many applications will treat block arguments as one way to set
the attributes of some object anyway, and in that case arguments can be
thought of as the subset of attributes of an object whose values come
from that object's associated block.
This also includes a few other minor terminology tweaks in the diagnostic
messages the reflect how our lexicon has evolved during development and
authoring of user-facing documentation.
This will allow for use-cases such as renaming a variable (changing the
content of the first token) and replacing variable references with
constant values that they evaluate to for debug purposes.
This is for the core HCL syntax, so it doesn't include any
application-specific keyword highlighting, etc.
The structural, expression, and template languages are separated into
different grammar definitions so that they can be used independently, but
they embed each other as needed to complete the language.
This is just a first pass, really. There are probably some bugs here, and
also some missing features.
Previously this implementation was doing only one level of recursion in
its walk, which gave the appearance of working until the
transform/container-type specs (DefaultSpec, TransformSpec, ...) were
introduced, creating the possibility of "same body children" being more
than one level away from the initial spec.
It's still correct to only process the schema and content once, because
ImpliedSchema is already collecting all of the requirements from the
"same body children", and so our content object will include everything
that the nested specs should need to analyze needed variables.
This is another heuristic because the "[" syntax is also the tuple
constructor start marker, but this takes care of the common cases of
indexing keywords and bracketed expressions.
This fixes#29.
We automatically convert from set to list in many other situations, so for
consistency we should accept sets here too and just treat them as
unordered sequences.
This closes#30.
Due to the special handling of the anonymous symbol employed to evaluate
a splat expression, we need to employ a lock on that symbol so that it's
safe for concurrent evaluation.
As before, it's not safe to concurrently evaluate the same expression in
the same context, but it is now safe to do so as long as all concurrent
evaluations have a _distinct_ EvalContext.
This fixes#28.
Previously it was not implementing the two optional interfaces required
for this, and so decoding would fail for any AttrSpec or block spec nested
inside.
Now it passes through attribute requirements from both the primary and
default, and passes block requirements only from the primary, thus
allowing either fallback between two attributes, fallback from an
attribute to a constant, or fallback from a block to a constant. Other
permutations are also possible, but not very important.
This change splits out the formatting of simple single-line lists. A list
is considered "simple" if all of its elements are on one line, all
elements are literals (except heredoc) and there are no line comments.
As an exception, a heredoc string is allowed when it is the only element
in the list.
This fixes an issue with a single-line list with one element and a line
comment. The formatter used to pull the closing bracket on the same line
(after the comment), causing parse errors.
When there are multiple cartridge returns at the end of the line, the regular expression will consider n-1 of them to be part of the string. Later, the last `\r` is removed. That may mean that a line that did previously *not* terminate a heredoc string may now terminate it, changing the meaning of the HCL file.
This (invalid) Unicode codepoint is used by the printer package to fix up
the indentation of generated files. If this codepoint is present in the
input, the package gets confused and removes more than it should,
producing unparsable output.
Previously we were only counting a \n as starting a new line, so input
using \r\n endings would get treated as one long line for source-range
purposes.
Now we also consider \r\n to be a newline marker, resetting the column
count to zero and incrementing the line just as we would do for a single
\n. This is made easier because the unicode definition of "grapheme
cluster" considers \r\n to be a single character, so we don't need to
do anything special in order to match it.
Previously, due to how heredoc scanning was implemented, the closing
marker for a heredoc would consume the newline that terminated it. This
was problematic in any context that is newline-sensitive, because it
would cause us to skip the TokenNewline that might terminate e.g. an
attribute definition:
foo = <<EOT
hello
EOT
bar = "hello"
Previously the "foo" attribute would fail to parse properly due to trying
to consume the "bar" definition as part of its expression.
Now we synthetically split the marker token into two parts: the marker
itself and the newline that follows it. This means that using a heredoc
in any context where newlines are sensitive will involuntarily introduce
a newline, but that seems consistent with user expectation based on how
heredocs seem to be used "in the wild".
