Lately we've made some changes that have affected the behavior of the
specsuite tests, causing them to fail. Much of this was caused by changes
to the harness itself (based on hcldec), although one break in particular
here was an intentional change to the implementation of modulo in
upstream cty to make it produce a more accurate result when used with
a fractional divisor.
HCL uses a number of upstream libraries that implement algorithms defined
in Unicode. This commit is updating those libraries all to versions that
have Unicode 13 support.
The main implication of this for HCL directly is that when it returns
column numbers in source locations it will count characters using the
Unicode 13 definition of "character", which includes various new
multi-codeunit characters added in Unicode 13.
These new version dependencies will also make Unicode 13 support available
for other functionality that HCL callers might use, such as the stdlib
functions in upstream cty, even though HCL itself does not directly use
those.
This adjusts prior behavior that would error to now allow keys that
are marked; those marks are removed in evaluation in order to give
a valid string for the map key, but the resulting object or tuple
is marked as a whole with whatever mark the key contained, ensuring
marks are maintained.
Using marked values in a for expression conditional expression would
previously panic due to calling `.False()` on the result of the
expression.
This commit makes two changes:
- Unmark the conditional expression value before testing if it is false;
- Merge any marks from the conditional into the resulting marks for the
for expression.
When evaluating an HCL expression attempting
to use a marked value as an object key,
return an error rather than falling through
to the cty panic. The error style mimics similar
errors in the area.
Functions which accept multiple parameters can be called with the
expansion operator, `...`. When doing so, we must unmark the expanded
argument value before transforming it into a collection of function
arguments. To ensure that any marks applied to the collection are
preserved, we transfer the collection marks to the individual elements
as we build the argument list.
So far the expression parentheses syntax has been handled entirely in the
parser and has been totally invisible in the AST. That's fine for typical
expression evaluation, but over the years it's led to a few quirky
behaviors in less common situations where we've assumed that all
expressions are covered by the AST itself or by the source ranges that the
AST captures.
In particular, hclwrite assumes that all expressions will have source
ranges that cover their tokens, and it generates an incorrect physical
syntax tree when the AST doesn't uphold that.
After resisting through a few other similar bugs, this commit finally
introduces an explicit AST node for parentheses, which makes the
parentheses explicit in the AST and captures the larger source range that
includes the TokenOParen and the TokenCParen.
This means that parentheses will now be visible as a distinct node when
walking the AST, as reflected in the updated tests here. That may cause
downstream applications that traverse the tree to exhibit different
behaviors but we're not considering that as a "breaking change" because
the Walk function doesn't make any guarantees about the specific AST
shape.
The rules for the splat operator call for it to return an empty tuple
when its operand is null, but this rule was previously being
overridden by another rule that a value whose type is unknown
causes the operator to return an unknown value of unknown
type.
This was initially reported and discussed in Terraform, under
hashicorp/terraform#26746.
