This is a super-invasive update since the "zcl" package in particular
is referenced all over.
There are probably still a few zcl references hanging around in comments,
etc but this takes care of most of it.
The main "zcl" package requires a bit more care because of how many
callers it has and because of its two subpackages, so we'll take care
of that one separately.
This package was a prototype of wrapping the HCL/HIL API in the new zcl
API as a compatibility strategy. This avenue wasn't chosen in the end, so
we'll remove this to avoid confusion as we rename everything else in this
repository to be called "hcl" now.
This package implements a language extension that allows configuration
authors to include the content of another file into a body, using syntax
like this:
include {
path = "./foo.zcl"
}
This is implemented as a transform.Transformer so that it can be used
as part of a transform chain when decoding nested block structures to
allow includes at any arbitrary point.
This capability is not built into the language because certain
applications will offer higher-level constructs for connecting multiple
separate config files, which may e.g. have a separate evaluation scope
for each file, etc.
This utility is intended to support the extension packages that are
siblings of this package, along with third-party extensions, by providing
a way to transform bodies in arbitrary ways.
The "Deep" function then provides a means to apply a particular transform
recursively to a nested block tree, allowing a particular extension to
be supported at arbitrary nesting levels.
This functionality is provided in terms of the standard zcl.Body
interface, so that transform results can be used with any code that
operates generically on bodies. This includes the zcldec and gozcl
packages, so files with extensions can still be decoded in the usual
way.
This package provides helper function that looks in a given body for
blocks that define functions, returning a function map suitable for use
in a zcl.EvalContext.
These will provide additional language features that are implemented in
terms of the basic zcl functionality, so that applications can opt-in to
some more-dynamic behavior if desired.
The general pattern here will be to provide a function that
partially-decodes a given zcl.Body to look for certain block types and
then returns its result along with a zcl.Body representing the remaining,
as-yet-unprocessed content.
Terraform interprets HIL variables in such a way that it allows numeric
attribute names which then get interpreted as numeric indices into a
list. This is used to work around the fact that the splat expressions
don't work for the index operator.
zcl has "full splats" that _do_ support the index operator, but to allow
old Terraform configs to be processed by zcl we'll accept this special
case within attribute-only-splats only.
For the moment this is a special exception made by this specific
implementation of zcl rather than part of the spec, since it's
specifically a pragmatic Terraform migration strategy, but it might get
upgraded to full spec status later if we end up needing to support it
in other host languages.
This requires the scanner to be a little more picky about the ending
of numeric literals, so that they won't absorb the trailing period after
the number in foo.*.baz.1.baz . This is okay because the spec doesn't
allow trailing periods anyway, and this is not actually a change in
final behavior because the parser was already catching this situation
and rejecting it at a later point.
While this does create some ambiguity with arithmetic on variables, like
a-b, this is permitted by HCL and so we'll permit it for zcl too, at the
expense of requiring spaces to be used around minus signs for correct
interpretation.
This applies the simple native syntax reformatting function to one or
more files. It does not support JSON or any other syntax.
Calling applications might provide their own versions of this that e.g.
can format an entire directory by matching on filename patterns, but
this serves as an example and a utility for single files.
This uses a subset of the expression syntax to allow representations of
traversals. The use-case for this is in applications that need to support
references to objects using familiar syntax without actually evaluating
those objects or dealing with the complexity of arbitrary expression
syntax.
This further alters the object-construction mode so that rather than
producing a flat object it instead produces an object of tuples. All
items that produce the same key are grouped together under the same
key in the result, allowing projections that flip the orientation of
a key/value sequence where the new keys are not necessarily unique.
Rather than setting an "Unwrap" field on the existing TemplateExpr, we'll
instead use a separate, simpler AST node.
There is very little in common between these two cases, so overloading the
TemplateExpr node doesn't buy much. A separate node is needed here,
rather than just returning the wrapped node directly, to give us somewhere
to capture the full source range extent of the wrapping template, whereas
the inner expression only captures the range of itself. This is important
both for good diagnostics and for transforming zclsyntax AST into
zclwrite AST.
Previously it was !{, but in real examples this looked confusing since
an exclamation point after a word looks (to humans) like literal
punctuation rather than syntax.
% is not ideal either since it's also the marker traditionally used for
printf, but has the advantage that programmers are already primed for it
to be syntax.
Since the native syntax requires that attribute names and block types
be identifiers, there is no way for "//" to be a valid symbol in a body.
Therefore we can special-case it here as an ignored property name that
users can then use for comments.
This special handling intentionally does not apply to objects representing
zcl object expressions, since it _is_ valid to have non-identifier keys
there and so "//" may be a legitimate object key for some applications.
Previously we tried to do the whole template parse in one pass. This was
adequate for dealing with literals and interpolations because they
create a flat structure, but to parse the template control sequences we
need to be able to deal with nested template sequences.
As a building block towards this, we first do a pass of extracting the
template-level "tokens": literals, interpolations, control sequences.
We then pass over that sequence of tokens and parse it, which is then
simplified because the larger template atoms have already been produced.
Attribute-only splat expressions cannot have other splats nested inside,
since we're only interested in supporting how these behaved for HIL when
running inside Hashicorp Terraform. More complex cases should be dealt
with using either full splats (bracketed *) or "for" expressions.
Attribute-only splat expressions, indicated by using the star symbol as
if it were an attribute, are inherited from HIL and are a limited sort of
splat that only works for walking through attributes in each item of its
source.
Later zcl will also get full splat expressions, whose syntax is using
the star symbol as if it were an _index_, which will generalize to
supporting _all_ postfix traversal operations, including indexing and
nested splats.
If we can't find a function in the given EvalContext, we must traverse
the context chain until either we run out of contexts or we find a
matching function.
At present there is no reason for a non-root context to have any
functions, so this will always traverse to the root. This may change in
future if we introduce constructs that define local functions.
it's acceptable to have "Variables" set to nil in an EvalContext, if
a particular scope has no variables at all. If _no_ contexts in the
chain have non-nil variables, this is considered to mean that variables
are not allowed at all, which produces a different error message.
This syntax func(arg...) allows the final argument to be a sequence-typed
value that then expands to be one argument for each element of the
value.
This allows applications to define variadic functions where that's
user-friendly while still allowing users to pass tuples to those functions
in situations where the args are chosen dynamically.
The purpose of the Variables function is to tell a calling application
what symbols need to be present in the _root_ scope, so it would be
unhelpful to include child scope traversals. Child scopes are populated
by the nodes that create them, and are thus not interesting to the
calling application (for this purpose, at least).
The ForExpr is essentially a list/map comprehension, allowing projecting
one expression into another. From a syntactic standpoint it's the most
complex structure we've dealt with so far, with many separate parts.
The tests introduced here are not exhaustive but illustrate that the
basic mechanism is working.
Martin Atkins