ext/customdecode: Custom expression decoding extension

Most of the time, the standard expression decoding built in to HCL is
sufficient. Sometimes though, it's useful to be able to customize the
decoding of certain arguments where the application intends to use them
in a very specific way, such as in static analysis.

This extension is an approximate analog of gohcl's support for decoding
into an hcl.Expression, allowing hcldec-based applications and
applications with custom functions to similarly capture and manipulate
the physical expressions used in arguments, rather than their values.

This includes one example use-case: the typeexpr extension now includes
a cty.Function called ConvertFunc that takes a type expression as its
second argument. A type expression is not evaluatable in the usual sense,
but thanks to cty capsule types we _can_ produce a cty.Value from one
and then make use of it inside the function implementation, without
exposing this custom type to the broader language:

    convert(["foo"], set(string))

This mechanism is intentionally restricted only to "argument-like"
locations where there is a specific type we are attempting to decode into.
For now, that's hcldec AttrSpec/BlockAttrsSpec -- analogous to gohcl
decoding into hcl.Expression -- and in arguments to functions.
This commit is contained in:
Martin Atkins 2019-12-13 08:52:25 -08:00
parent afbe52484f
commit d8ae04bc78
10 changed files with 952 additions and 18 deletions

209
ext/customdecode/README.md Normal file
View File

@ -0,0 +1,209 @@
# HCL Custom Static Decoding Extension
This HCL extension provides a mechanism for defining arguments in an HCL-based
language whose values are derived using custom decoding rules against the
HCL expression syntax, overriding the usual behavior of normal expression
evaluation.
"Arguments", for the purpose of this extension, currently includes the
following two contexts:
* For applications using `hcldec` for dynamic decoding, a `hcldec.AttrSpec`
or `hcldec.BlockAttrsSpec` can be given a special type constraint that
opts in to custom decoding behavior for the attribute(s) that are selected
by that specification.
* When working with the HCL native expression syntax, a function given in
the `hcl.EvalContext` during evaluation can have parameters with special
type constraints that opt in to custom decoding behavior for the argument
expression associated with that parameter in any call.
The above use-cases are rather abstract, so we'll consider a motivating
real-world example: sometimes we (language designers) need to allow users
to specify type constraints directly in the language itself, such as in
[Terraform's Input Variables](https://www.terraform.io/docs/configuration/variables.html).
Terraform's `variable` blocks include an argument called `type` which takes
a type constraint given using HCL expression building-blocks as defined by
[the HCL `typeexpr` extension](../typeexpr/README.md).
A "type constraint expression" of that sort is not an expression intended to
be evaluated in the usual way. Instead, the physical expression is
deconstructed using [the static analysis operations](../../spec.md#static-analysis)
to produce a `cty.Type` as the result, rather than a `cty.Value`.
The purpose of this Custom Static Decoding Extension, then, is to provide a
bridge to allow that sort of custom decoding to be used via mechanisms that
normally deal in `cty.Value`, such as `hcldec` and native syntax function
calls as listed above.
(Note: [`gohcl`](https://pkg.go.dev/github.com/hashicorp/hcl/v2/gohcl) has
its own mechanism to support this use case, exploiting the fact that it is
working directly with "normal" Go types. Decoding into a struct field of
type `hcl.Expression` obtains the expression directly without evaluating it
first. The Custom Static Decoding Extension is not necessary for that `gohcl`
technique. You can also implement custom decoding by working directly with
the lowest-level HCL API, which separates extraction of and evaluation of
expressions into two steps.)
## Custom Decoding Types
This extension relies on a convention implemented in terms of
[_Capsule Types_ in the underlying `cty` type system](https://github.com/zclconf/go-cty/blob/master/docs/types.md#capsule-types). `cty` allows a capsule type to carry arbitrary
extension metadata values as an aid to creating higher-level abstractions like
this extension.
A custom argument decoding mode, then, is implemented by creating a new `cty`
capsule type that implements the `ExtensionData` custom operation to return
a decoding function when requested. For example:
```go
var keywordType cty.Type
keywordType = cty.CapsuleWithOps("keyword", reflect.TypeOf(""), &cty.CapsuleOps{
ExtensionData: func(key interface{}) interface{} {
switch key {
case customdecode.CustomExpressionDecoder:
return customdecode.CustomExpressionDecoderFunc(
func(expr hcl.Expression, ctx *hcl.EvalContext) (cty.Value, hcl.Diagnostics) {
var diags hcl.Diagnostics
kw := hcl.ExprAsKeyword(expr)
if kw == "" {
diags = append(diags, &hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid keyword",
Detail: "A keyword is required",
Subject: expr.Range().Ptr(),
})
return cty.UnkownVal(keywordType), diags
}
return cty.CapsuleVal(keywordType, &kw)
},
)
default:
return nil
}
},
})
```
The boilerplate here is a bit fussy, but the important part for our purposes
is the `case customdecode.CustomExpressionDecoder:` clause, which uses
a custom extension key type defined in this package to recognize when a
component implementing this extension is checking to see if a target type
has a custom decode implementation.
In the above case we've defined a type that decodes expressions as static
keywords, so a keyword like `foo` would decode as an encapsulated `"foo"`
string, while any other sort of expression like `"baz"` or `1 + 1` would
return an error.
We could then use `keywordType` as a type constraint either for a function
parameter or a `hcldec` attribute specification, which would require the
argument for that function parameter or the expression for the matching
attributes to be a static keyword, rather than an arbitrary expression.
For example, in a `hcldec.AttrSpec`:
```go
keywordSpec := &hcldec.AttrSpec{
Name: "keyword",
Type: keywordType,
}
```
The above would accept input like the following and would set its result to
a `cty.Value` of `keywordType`, after decoding:
```hcl
keyword = foo
```
## The Expression and Expression Closure `cty` types
Building on the above, this package also includes two capsule types that use
the above mechanism to allow calling applications to capture expressions
directly and thus defer analysis to a later step, after initial decoding.
The `customdecode.ExpressionType` type encapsulates an `hcl.Expression` alone,
for situations like our type constraint expression example above where it's
the static structure of the expression we want to inspect, and thus any
variables and functions defined in the evaluation context are irrelevant.
The `customdecode.ExpressionClosureType` type encapsulates a
`*customdecode.ExpressionClosure` value, which binds the given expression to
the `hcl.EvalContext` it was asked to evaluate against and thus allows the
receiver of that result to later perform normal evaluation of the expression
with all the same variables and functions that would've been available to it
naturally.
Both of these types can be used as type constraints either for `hcldec`
attribute specifications or for function arguments. Here's an example of
`ExpressionClosureType` to implement a function that can evaluate
an expression with some additional variables defined locally, which we'll
call the `with(...)` function:
```go
var WithFunc = function.New(&function.Spec{
Params: []function.Parameter{
{
Name: "variables",
Type: cty.DynamicPseudoType,
},
{
Name: "expression",
Type: customdecode.ExpressionClosureType,
},
},
Type: func(args []cty.Value) (cty.Type, error) {
varsVal := args[0]
exprVal := args[1]
if !varsVal.Type().IsObjectType() {
return cty.NilVal, function.NewArgErrorf(0, "must be an object defining local variables")
}
if !varsVal.IsKnown() {
// We can't predict our result type until the variables object
// is known.
return cty.DynamicPseudoType, nil
}
vars := varsVal.AsValueMap()
closure := customdecode.ExpressionClosureFromVal(exprVal)
result, err := evalWithLocals(vars, closure)
if err != nil {
return cty.NilVal, err
}
return result.Type(), nil
},
Impl: func(args []cty.Value, retType cty.Type) (cty.Value, error) {
varsVal := args[0]
exprVal := args[1]
vars := varsVal.AsValueMap()
closure := customdecode.ExpressionClosureFromVal(exprVal)
return evalWithLocals(vars, closure)
},
})
func evalWithLocals(locals map[string]cty.Value, closure *customdecode.ExpressionClosure) (cty.Value, error) {
childCtx := closure.EvalContext.NewChild()
childCtx.Variables = locals
val, diags := closure.Expression.Value(childCtx)
if diags.HasErrors() {
return cty.NilVal, function.NewArgErrorf(1, "couldn't evaluate expression: %s", diags.Error())
}
return val, nil
}
```
If the above function were placed into an `hcl.EvalContext` as `with`, it
could be used in a native syntax call to that function as follows:
```hcl
foo = with({name = "Cory"}, "${greeting}, ${name}!")
```
The above assumes a variable in the main context called `greeting`, to which
the `with` function adds `name` before evaluating the expression given in
its second argument. This makes that second argument context-sensitive -- it
would behave differently if the user wrote the same thing somewhere else -- so
this capability should be used with care to make sure it doesn't cause confusion
for the end-users of your language.
There are some other examples of this capability to evaluate expressions in
unusual ways in the `tryfunc` directory that is a sibling of this one.

View File

@ -0,0 +1,56 @@
// Package customdecode contains a HCL extension that allows, in certain
// contexts, expression evaluation to be overridden by custom static analysis.
//
// This mechanism is only supported in certain specific contexts where
// expressions are decoded with a specific target type in mind. For more
// information, see the documentation on CustomExpressionDecoder.
package customdecode
import (
"github.com/hashicorp/hcl/v2"
"github.com/zclconf/go-cty/cty"
)
type customDecoderImpl int
// CustomExpressionDecoder is a value intended to be used as a cty capsule
// type ExtensionData key for capsule types whose values are to be obtained
// by static analysis of an expression rather than normal evaluation of that
// expression.
//
// When a cooperating capsule type is asked for ExtensionData with this key,
// it must return a non-nil CustomExpressionDecoderFunc value.
//
// This mechanism is not universally supported; instead, it's handled in a few
// specific places where expressions are evaluated with the intent of producing
// a cty.Value of a type given by the calling application.
//
// Specifically, this currently works for type constraints given in
// hcldec.AttrSpec and hcldec.BlockAttrsSpec, and it works for arguments to
// function calls in the HCL native syntax. HCL extensions implemented outside
// of the main HCL module may also implement this; consult their own
// documentation for details.
const CustomExpressionDecoder = customDecoderImpl(1)
// CustomExpressionDecoderFunc is the type of value that must be returned by
// a capsule type handling the key CustomExpressionDecoder in its ExtensionData
// implementation.
//
// If no error diagnostics are returned, the result value MUST be of the
// capsule type that the decoder function was derived from. If the returned
// error diagnostics prevent producing a value at all, return cty.NilVal.
type CustomExpressionDecoderFunc func(expr hcl.Expression, ctx *hcl.EvalContext) (cty.Value, hcl.Diagnostics)
// CustomExpressionDecoderForType takes any cty type and returns its
// custom expression decoder implementation if it has one. If it is not a
// capsule type or it does not implement a custom expression decoder, this
// function returns nil.
func CustomExpressionDecoderForType(ty cty.Type) CustomExpressionDecoderFunc {
if !ty.IsCapsuleType() {
return nil
}
if fn, ok := ty.CapsuleExtensionData(CustomExpressionDecoder).(CustomExpressionDecoderFunc); ok {
return fn
}
return nil
}

View File

@ -0,0 +1,146 @@
package customdecode
import (
"fmt"
"reflect"
"github.com/hashicorp/hcl/v2"
"github.com/zclconf/go-cty/cty"
)
// ExpressionType is a cty capsule type that carries hcl.Expression values.
//
// This type implements custom decoding in the most general way possible: it
// just captures whatever expression is given to it, with no further processing
// whatsoever. It could therefore be useful in situations where an application
// must defer processing of the expression content until a later step.
//
// ExpressionType only captures the expression, not the evaluation context it
// was destined to be evaluated in. That means this type can be fine for
// situations where the recipient of the value only intends to do static
// analysis, but ExpressionClosureType is more appropriate in situations where
// the recipient will eventually evaluate the given expression.
var ExpressionType cty.Type
// ExpressionVal returns a new cty value of type ExpressionType, wrapping the
// given expression.
func ExpressionVal(expr hcl.Expression) cty.Value {
return cty.CapsuleVal(ExpressionType, &expr)
}
// ExpressionFromVal returns the expression encapsulated in the given value, or
// panics if the value is not a known value of ExpressionType.
func ExpressionFromVal(v cty.Value) hcl.Expression {
if !v.Type().Equals(ExpressionType) {
panic("value is not of ExpressionType")
}
ptr := v.EncapsulatedValue().(*hcl.Expression)
return *ptr
}
// ExpressionClosureType is a cty capsule type that carries hcl.Expression
// values along with their original evaluation contexts.
//
// This is similar to ExpressionType except that during custom decoding it
// also captures the hcl.EvalContext that was provided, allowing callers to
// evaluate the expression later in the same context where it would originally
// have been evaluated, or a context derived from that one.
var ExpressionClosureType cty.Type
// ExpressionClosure is the type encapsulated in ExpressionClosureType
type ExpressionClosure struct {
Expression hcl.Expression
EvalContext *hcl.EvalContext
}
// ExpressionClosureVal returns a new cty value of type ExpressionClosureType,
// wrapping the given expression closure.
func ExpressionClosureVal(closure *ExpressionClosure) cty.Value {
return cty.CapsuleVal(ExpressionClosureType, closure)
}
// Value evaluates the closure's expression using the closure's EvalContext,
// returning the result.
func (c *ExpressionClosure) Value() (cty.Value, hcl.Diagnostics) {
return c.Expression.Value(c.EvalContext)
}
// ExpressionClosureFromVal returns the expression closure encapsulated in the
// given value, or panics if the value is not a known value of
// ExpressionClosureType.
//
// The caller MUST NOT modify the returned closure or the EvalContext inside
// it. To derive a new EvalContext, either create a child context or make
// a copy.
func ExpressionClosureFromVal(v cty.Value) *ExpressionClosure {
if !v.Type().Equals(ExpressionClosureType) {
panic("value is not of ExpressionClosureType")
}
return v.EncapsulatedValue().(*ExpressionClosure)
}
func init() {
// Getting hold of a reflect.Type for hcl.Expression is a bit tricky because
// it's an interface type, but we can do it with some indirection.
goExpressionType := reflect.TypeOf((*hcl.Expression)(nil)).Elem()
ExpressionType = cty.CapsuleWithOps("expression", goExpressionType, &cty.CapsuleOps{
ExtensionData: func(key interface{}) interface{} {
switch key {
case CustomExpressionDecoder:
return CustomExpressionDecoderFunc(
func(expr hcl.Expression, ctx *hcl.EvalContext) (cty.Value, hcl.Diagnostics) {
return ExpressionVal(expr), nil
},
)
default:
return nil
}
},
TypeGoString: func(_ reflect.Type) string {
return "customdecode.ExpressionType"
},
GoString: func(raw interface{}) string {
exprPtr := raw.(*hcl.Expression)
return fmt.Sprintf("customdecode.ExpressionVal(%#v)", *exprPtr)
},
RawEquals: func(a, b interface{}) bool {
aPtr := a.(*hcl.Expression)
bPtr := b.(*hcl.Expression)
return reflect.DeepEqual(*aPtr, *bPtr)
},
})
ExpressionClosureType = cty.CapsuleWithOps("expression closure", reflect.TypeOf(ExpressionClosure{}), &cty.CapsuleOps{
ExtensionData: func(key interface{}) interface{} {
switch key {
case CustomExpressionDecoder:
return CustomExpressionDecoderFunc(
func(expr hcl.Expression, ctx *hcl.EvalContext) (cty.Value, hcl.Diagnostics) {
return ExpressionClosureVal(&ExpressionClosure{
Expression: expr,
EvalContext: ctx,
}), nil
},
)
default:
return nil
}
},
TypeGoString: func(_ reflect.Type) string {
return "customdecode.ExpressionClosureType"
},
GoString: func(raw interface{}) string {
closure := raw.(*ExpressionClosure)
return fmt.Sprintf("customdecode.ExpressionClosureVal(%#v)", closure)
},
RawEquals: func(a, b interface{}) bool {
closureA := a.(*ExpressionClosure)
closureB := b.(*ExpressionClosure)
// The expression itself compares by deep equality, but EvalContexts
// conventionally compare by pointer identity, so we'll comply
// with both conventions here by testing them separately.
return closureA.EvalContext == closureB.EvalContext &&
reflect.DeepEqual(closureA.Expression, closureB.Expression)
},
})
}

View File

@ -65,3 +65,71 @@ type checking it will be one that has identifiers as its attributes; object
types with weird attributes generally show up only from arbitrary object types with weird attributes generally show up only from arbitrary object
constructors in configuration files, which are usually treated either as maps constructors in configuration files, which are usually treated either as maps
or as the dynamic pseudo-type. or as the dynamic pseudo-type.
## Type Constraints as Values
Along with defining a convention for writing down types using HCL expression
constructs, this package also includes a mechanism for representing types as
values that can be used as data within an HCL-based language.
`typeexpr.TypeConstraintType` is a
[`cty` capsule type](https://github.com/zclconf/go-cty/blob/master/docs/types.md#capsule-types)
that encapsulates `cty.Type` values. You can construct such a value directly
using the `TypeConstraintVal` function:
```go
tyVal := typeexpr.TypeConstraintVal(cty.String)
// We can unpack the type from a value using TypeConstraintFromVal
ty := typeExpr.TypeConstraintFromVal(tyVal)
```
However, the primary purpose of `typeexpr.TypeConstraintType` is to be
specified as the type constraint for an argument, in which case it serves
as a signal for HCL to treat the argument expression as a type constraint
expression as defined above, rather than as a normal value expression.
"An argument" in the above in practice means the following two locations:
* As the type constraint for a parameter of a cty function that will be
used in an `hcl.EvalContext`. In that case, function calls in the HCL
native expression syntax will require the argument to be valid type constraint
expression syntax and the function implementation will receive a
`TypeConstraintType` value as the argument value for that parameter.
* As the type constraint for a `hcldec.AttrSpec` or `hcldec.BlockAttrsSpec`
when decoding an HCL body using `hcldec`. In that case, the attributes
with that type constraint will be required to be valid type constraint
expression syntax and the result will be a `TypeConstraintType` value.
Note that the special handling of these arguments means that an argument
marked in this way must use the type constraint syntax directly. It is not
valid to pass in a value of `TypeConstraintType` that has been obtained
dynamically via some other expression result.
`TypeConstraintType` is provided with the intent of using it internally within
application code when incorporating type constraint expression syntax into
an HCL-based language, not to be used for dynamic "programming with types". A
calling application could support programming with types by defining its _own_
capsule type, but that is not the purpose of `TypeConstraintType`.
## The "convert" `cty` Function
Building on the `TypeConstraintType` described in the previous section, this
package also provides `typeexpr.ConvertFunc` which is a cty function that
can be placed into a `cty.EvalContext` (conventionally named "convert") in
order to provide a general type conversion function in an HCL-based language:
```hcl
foo = convert("true", bool)
```
The second parameter uses the mechanism described in the previous section to
require its argument to be a type constraint expression rather than a value
expression. In doing so, it allows converting with any type constraint that
can be expressed in this package's type constraint syntax. In the above example,
the `foo` argument would receive a boolean true, or `cty.True` in `cty` terms.
The target type constraint must always be provided statically using inline
type constraint syntax. There is no way to _dynamically_ select a type
constraint using this function.

118
ext/typeexpr/type_type.go Normal file
View File

@ -0,0 +1,118 @@
package typeexpr
import (
"fmt"
"reflect"
"github.com/hashicorp/hcl/v2"
"github.com/hashicorp/hcl/v2/ext/customdecode"
"github.com/zclconf/go-cty/cty"
"github.com/zclconf/go-cty/cty/convert"
"github.com/zclconf/go-cty/cty/function"
)
// TypeConstraintType is a cty capsule type that allows cty type constraints to
// be used as values.
//
// If TypeConstraintType is used in a context supporting the
// customdecode.CustomExpressionDecoder extension then it will implement
// expression decoding using the TypeConstraint function, thus allowing
// type expressions to be used in contexts where value expressions might
// normally be expected, such as in arguments to function calls.
var TypeConstraintType cty.Type
// TypeConstraintVal constructs a cty.Value whose type is
// TypeConstraintType.
func TypeConstraintVal(ty cty.Type) cty.Value {
return cty.CapsuleVal(TypeConstraintType, &ty)
}
// TypeConstraintFromVal extracts the type from a cty.Value of
// TypeConstraintType that was previously constructed using TypeConstraintVal.
//
// If the given value isn't a known, non-null value of TypeConstraintType
// then this function will panic.
func TypeConstraintFromVal(v cty.Value) cty.Type {
if !v.Type().Equals(TypeConstraintType) {
panic("value is not of TypeConstraintType")
}
ptr := v.EncapsulatedValue().(*cty.Type)
return *ptr
}
// ConvertFunc is a cty function that implements type conversions.
//
// Its signature is as follows:
// convert(value, type_constraint)
//
// ...where type_constraint is a type constraint expression as defined by
// typeexpr.TypeConstraint.
//
// It relies on HCL's customdecode extension and so it's not suitable for use
// in non-HCL contexts or if you are using a HCL syntax implementation that
// does not support customdecode for function arguments. However, it _is_
// supported for function calls in the HCL native expression syntax.
var ConvertFunc function.Function
func init() {
TypeConstraintType = cty.CapsuleWithOps("type constraint", reflect.TypeOf(cty.Type{}), &cty.CapsuleOps{
ExtensionData: func(key interface{}) interface{} {
switch key {
case customdecode.CustomExpressionDecoder:
return customdecode.CustomExpressionDecoderFunc(
func(expr hcl.Expression, ctx *hcl.EvalContext) (cty.Value, hcl.Diagnostics) {
ty, diags := TypeConstraint(expr)
if diags.HasErrors() {
return cty.NilVal, diags
}
return TypeConstraintVal(ty), nil
},
)
default:
return nil
}
},
TypeGoString: func(_ reflect.Type) string {
return "typeexpr.TypeConstraintType"
},
GoString: func(raw interface{}) string {
tyPtr := raw.(*cty.Type)
return fmt.Sprintf("typeexpr.TypeConstraintVal(%#v)", *tyPtr)
},
RawEquals: func(a, b interface{}) bool {
aPtr := a.(*cty.Type)
bPtr := b.(*cty.Type)
return (*aPtr).Equals(*bPtr)
},
})
ConvertFunc = function.New(&function.Spec{
Params: []function.Parameter{
{
Name: "value",
Type: cty.DynamicPseudoType,
AllowNull: true,
AllowDynamicType: true,
},
{
Name: "type",
Type: TypeConstraintType,
},
},
Type: func(args []cty.Value) (cty.Type, error) {
wantTypePtr := args[1].EncapsulatedValue().(*cty.Type)
got, err := convert.Convert(args[0], *wantTypePtr)
if err != nil {
return cty.NilType, function.NewArgError(0, err)
}
return got.Type(), nil
},
Impl: func(args []cty.Value, retType cty.Type) (cty.Value, error) {
v, err := convert.Convert(args[0], retType)
if err != nil {
return cty.NilVal, function.NewArgError(0, err)
}
return v, nil
},
})
}

View File

@ -0,0 +1,118 @@
package typeexpr
import (
"fmt"
"testing"
"github.com/zclconf/go-cty/cty"
)
func TestTypeConstraintType(t *testing.T) {
tyVal1 := TypeConstraintVal(cty.String)
tyVal2 := TypeConstraintVal(cty.String)
tyVal3 := TypeConstraintVal(cty.Number)
if !tyVal1.RawEquals(tyVal2) {
t.Errorf("tyVal1 not equal to tyVal2\ntyVal1: %#v\ntyVal2: %#v", tyVal1, tyVal2)
}
if tyVal1.RawEquals(tyVal3) {
t.Errorf("tyVal1 equal to tyVal2, but should not be\ntyVal1: %#v\ntyVal3: %#v", tyVal1, tyVal3)
}
if got, want := TypeConstraintFromVal(tyVal1), cty.String; !got.Equals(want) {
t.Errorf("wrong type extracted from tyVal1\ngot: %#v\nwant: %#v", got, want)
}
if got, want := TypeConstraintFromVal(tyVal3), cty.Number; !got.Equals(want) {
t.Errorf("wrong type extracted from tyVal3\ngot: %#v\nwant: %#v", got, want)
}
}
func TestConvertFunc(t *testing.T) {
// This is testing the convert function directly, skipping over the HCL
// parsing and evaluation steps that would normally lead there. There is
// another test in the "integrationtest" package called TestTypeConvertFunc
// that exercises the full path to this function via the hclsyntax parser.
tests := []struct {
val, ty cty.Value
want cty.Value
wantErr string
}{
// The goal here is not an exhaustive set of conversions, since that's
// already covered in cty/convert, but rather exercising different
// permutations of success and failure to make sure the function
// handles all of the results in a reasonable way.
{
cty.StringVal("hello"),
TypeConstraintVal(cty.String),
cty.StringVal("hello"),
``,
},
{
cty.True,
TypeConstraintVal(cty.String),
cty.StringVal("true"),
``,
},
{
cty.StringVal("hello"),
TypeConstraintVal(cty.Bool),
cty.NilVal,
`a bool is required`,
},
{
cty.UnknownVal(cty.Bool),
TypeConstraintVal(cty.Bool),
cty.UnknownVal(cty.Bool),
``,
},
{
cty.DynamicVal,
TypeConstraintVal(cty.Bool),
cty.UnknownVal(cty.Bool),
``,
},
{
cty.NullVal(cty.Bool),
TypeConstraintVal(cty.Bool),
cty.NullVal(cty.Bool),
``,
},
{
cty.NullVal(cty.DynamicPseudoType),
TypeConstraintVal(cty.Bool),
cty.NullVal(cty.Bool),
``,
},
{
cty.StringVal("hello").Mark(1),
TypeConstraintVal(cty.String),
cty.StringVal("hello").Mark(1),
``,
},
}
for _, test := range tests {
t.Run(fmt.Sprintf("%#v to %#v", test.val, test.ty), func(t *testing.T) {
got, err := ConvertFunc.Call([]cty.Value{test.val, test.ty})
if err != nil {
if test.wantErr != "" {
if got, want := err.Error(), test.wantErr; got != want {
t.Errorf("wrong error\ngot: %s\nwant: %s", got, want)
}
} else {
t.Errorf("unexpected error\ngot: %s\nwant: <nil>", err)
}
return
}
if test.wantErr != "" {
t.Errorf("wrong error\ngot: <nil>\nwant: %s", test.wantErr)
}
if !test.want.RawEquals(got) {
t.Errorf("wrong result\ngot: %#v\nwant: %#v", got, test.want)
}
})
}
}

View File

@ -6,6 +6,7 @@ import (
"sort" "sort"
"github.com/hashicorp/hcl/v2" "github.com/hashicorp/hcl/v2"
"github.com/hashicorp/hcl/v2/ext/customdecode"
"github.com/zclconf/go-cty/cty" "github.com/zclconf/go-cty/cty"
"github.com/zclconf/go-cty/cty/convert" "github.com/zclconf/go-cty/cty/convert"
"github.com/zclconf/go-cty/cty/function" "github.com/zclconf/go-cty/cty/function"
@ -193,6 +194,14 @@ func (s *AttrSpec) decode(content *hcl.BodyContent, blockLabels []blockLabel, ct
return cty.NullVal(s.Type), nil return cty.NullVal(s.Type), nil
} }
if decodeFn := customdecode.CustomExpressionDecoderForType(s.Type); decodeFn != nil {
v, diags := decodeFn(attr.Expr, ctx)
if v == cty.NilVal {
v = cty.UnknownVal(s.Type)
}
return v, diags
}
val, diags := attr.Expr.Value(ctx) val, diags := attr.Expr.Value(ctx)
convVal, err := convert.Convert(val, s.Type) convVal, err := convert.Convert(val, s.Type)
@ -1223,6 +1232,16 @@ func (s *BlockAttrsSpec) decode(content *hcl.BodyContent, blockLabels []blockLab
vals := make(map[string]cty.Value, len(attrs)) vals := make(map[string]cty.Value, len(attrs))
for name, attr := range attrs { for name, attr := range attrs {
if decodeFn := customdecode.CustomExpressionDecoderForType(s.ElementType); decodeFn != nil {
attrVal, attrDiags := decodeFn(attr.Expr, ctx)
diags = append(diags, attrDiags...)
if attrVal == cty.NilVal {
attrVal = cty.UnknownVal(s.ElementType)
}
vals[name] = attrVal
continue
}
attrVal, attrDiags := attr.Expr.Value(ctx) attrVal, attrDiags := attr.Expr.Value(ctx)
diags = append(diags, attrDiags...) diags = append(diags, attrDiags...)

View File

@ -5,6 +5,7 @@ import (
"sync" "sync"
"github.com/hashicorp/hcl/v2" "github.com/hashicorp/hcl/v2"
"github.com/hashicorp/hcl/v2/ext/customdecode"
"github.com/zclconf/go-cty/cty" "github.com/zclconf/go-cty/cty"
"github.com/zclconf/go-cty/cty/convert" "github.com/zclconf/go-cty/cty/convert"
"github.com/zclconf/go-cty/cty/function" "github.com/zclconf/go-cty/cty/function"
@ -350,26 +351,38 @@ func (e *FunctionCallExpr) Value(ctx *hcl.EvalContext) (cty.Value, hcl.Diagnosti
param = varParam param = varParam
} }
val, argDiags := argExpr.Value(ctx) var val cty.Value
if len(argDiags) > 0 { if decodeFn := customdecode.CustomExpressionDecoderForType(param.Type); decodeFn != nil {
var argDiags hcl.Diagnostics
val, argDiags = decodeFn(argExpr, ctx)
diags = append(diags, argDiags...) diags = append(diags, argDiags...)
} if val == cty.NilVal {
val = cty.UnknownVal(param.Type)
}
} else {
var argDiags hcl.Diagnostics
val, argDiags = argExpr.Value(ctx)
if len(argDiags) > 0 {
diags = append(diags, argDiags...)
}
// Try to convert our value to the parameter type // Try to convert our value to the parameter type
val, err := convert.Convert(val, param.Type) var err error
if err != nil { val, err = convert.Convert(val, param.Type)
diags = append(diags, &hcl.Diagnostic{ if err != nil {
Severity: hcl.DiagError, diags = append(diags, &hcl.Diagnostic{
Summary: "Invalid function argument", Severity: hcl.DiagError,
Detail: fmt.Sprintf( Summary: "Invalid function argument",
"Invalid value for %q parameter: %s.", Detail: fmt.Sprintf(
param.Name, err, "Invalid value for %q parameter: %s.",
), param.Name, err,
Subject: argExpr.StartRange().Ptr(), ),
Context: e.Range().Ptr(), Subject: argExpr.StartRange().Ptr(),
Expression: argExpr, Context: e.Range().Ptr(),
EvalContext: ctx, Expression: argExpr,
}) EvalContext: ctx,
})
}
} }
argVals[i] = val argVals[i] = val

View File

@ -0,0 +1,56 @@
package integrationtest
import (
"testing"
"github.com/hashicorp/hcl/v2"
"github.com/hashicorp/hcl/v2/ext/typeexpr"
"github.com/hashicorp/hcl/v2/hclsyntax"
"github.com/zclconf/go-cty/cty"
"github.com/zclconf/go-cty/cty/function"
)
// TestTypeConvertFunc is an integration test of all of the layers involved
// in making the type conversion function from ext/typeexpr work.
//
// This requires co-operation between the hclsyntax package, the ext/typeexpr
// package, and the underlying cty functionality in order to work correctly.
//
// There are unit tests for the function implementation itself in the
// ext/typeexpr package, so this test is focused on making sure the function
// is given the opportunity to decode the second argument as a type expression
// when the function is called from HCL native syntax.
func TestTypeConvertFunc(t *testing.T) {
// The convert function is special because it takes a type expression
// rather than a value expression as its second argument. In this case,
// we're asking it to convert a tuple into a list of strings:
const exprSrc = `convert(["hello"], list(string))`
// It achieves this by marking that second argument as being of a custom
// type (a "capsule type", in cty terminology) that has a special
// annotation which hclsyntax.FunctionCallExpr understands as allowing
// the type to handle the analysis of the unevaluated expression, instead
// of evaluating it as normal.
//
// To see more details of how this works, look at the definitions of
// typexpr.TypeConstraintType and typeexpr.ConvertFunc, and at the
// implementation of hclsyntax.FunctionCallExpr.Value.
expr, diags := hclsyntax.ParseExpression([]byte(exprSrc), "", hcl.Pos{Line: 1, Column: 1})
if diags.HasErrors() {
t.Fatalf("unexpected problems: %s", diags.Error())
}
ctx := &hcl.EvalContext{
Functions: map[string]function.Function{
"convert": typeexpr.ConvertFunc,
},
}
got, diags := expr.Value(ctx)
if diags.HasErrors() {
t.Fatalf("unexpected problems: %s", diags.Error())
}
want := cty.ListVal([]cty.Value{cty.StringVal("hello")})
if !want.RawEquals(got) {
t.Errorf("wrong result\ngot: %#v\nwant: %#v", got, want)
}
}

View File

@ -0,0 +1,131 @@
package integrationtest
import (
"testing"
"github.com/google/go-cmp/cmp"
"github.com/google/go-cmp/cmp/cmpopts"
"github.com/hashicorp/hcl/v2"
"github.com/hashicorp/hcl/v2/ext/customdecode"
"github.com/hashicorp/hcl/v2/hcldec"
"github.com/hashicorp/hcl/v2/hclsyntax"
"github.com/zclconf/go-cty/cty"
)
// TestHCLDecDecodeToExpr tests both hcldec's support for types with custom
// expression decoding rules and the two expression capsule types implemented
// in ext/customdecode. This mechanism requires cooperation between those
// two components and cty in order to work, so it's helpful to exercise it in
// an integration test.
func TestHCLDecDecodeToExpr(t *testing.T) {
// Here we're going to capture the structure of two simple expressions
// without immediately evaluating them.
const input = `
a = foo
b = foo
c = "hello"
`
// We'll capture "a" directly as an expression, losing its evaluation
// context but retaining its structure. We'll capture "b" as a
// customdecode.ExpressionClosure, which gives us both the expression
// itself and the evaluation context it was originally evaluated in.
// We also have "c" here just to make sure we can still decode into a
// "normal" type via standard expression evaluation.
f, diags := hclsyntax.ParseConfig([]byte(input), "", hcl.Pos{Line: 1, Column: 1})
if diags.HasErrors() {
t.Fatalf("unexpected problems: %s", diags.Error())
}
spec := hcldec.ObjectSpec{
"a": &hcldec.AttrSpec{
Name: "a",
Type: customdecode.ExpressionType,
Required: true,
},
"b": &hcldec.AttrSpec{
Name: "b",
Type: customdecode.ExpressionClosureType,
Required: true,
},
"c": &hcldec.AttrSpec{
Name: "c",
Type: cty.String,
Required: true,
},
}
ctx := &hcl.EvalContext{
Variables: map[string]cty.Value{
"foo": cty.StringVal("foo value"),
},
}
objVal, diags := hcldec.Decode(f.Body, spec, ctx)
if diags.HasErrors() {
t.Fatalf("unexpected problems: %s", diags.Error())
}
aVal := objVal.GetAttr("a")
bVal := objVal.GetAttr("b")
cVal := objVal.GetAttr("c")
if got, want := aVal.Type(), customdecode.ExpressionType; !got.Equals(want) {
t.Fatalf("wrong type for 'a'\ngot: %#v\nwant: %#v", got, want)
}
if got, want := bVal.Type(), customdecode.ExpressionClosureType; !got.Equals(want) {
t.Fatalf("wrong type for 'b'\ngot: %#v\nwant: %#v", got, want)
}
if got, want := cVal.Type(), cty.String; !got.Equals(want) {
t.Fatalf("wrong type for 'c'\ngot: %#v\nwant: %#v", got, want)
}
gotAExpr := customdecode.ExpressionFromVal(aVal)
wantAExpr := &hclsyntax.ScopeTraversalExpr{
Traversal: hcl.Traversal{
hcl.TraverseRoot{
Name: "foo",
SrcRange: hcl.Range{
Start: hcl.Pos{Line: 2, Column: 5, Byte: 5},
End: hcl.Pos{Line: 2, Column: 8, Byte: 8},
},
},
},
SrcRange: hcl.Range{
Start: hcl.Pos{Line: 2, Column: 5, Byte: 5},
End: hcl.Pos{Line: 2, Column: 8, Byte: 8},
},
}
if diff := cmp.Diff(wantAExpr, gotAExpr, cmpopts.IgnoreUnexported(hcl.TraverseRoot{})); diff != "" {
t.Errorf("wrong expression for a\n%s", diff)
}
bClosure := customdecode.ExpressionClosureFromVal(bVal)
gotBVal, diags := bClosure.Value()
wantBVal := cty.StringVal("foo value")
if diags.HasErrors() {
t.Fatalf("unexpected problems: %s", diags.Error())
}
if got, want := gotBVal, wantBVal; !want.RawEquals(got) {
t.Errorf("wrong 'b' result\ngot: %#v\nwant: %#v", got, want)
}
if got, want := cVal, cty.StringVal("hello"); !want.RawEquals(got) {
t.Errorf("wrong 'c'\ngot: %#v\nwant: %#v", got, want)
}
// One additional "trick" we can do with the expression closure is to
// evaluate the expression in a _derived_ EvalContext, rather than the
// captured one. This could be useful for introducing additional local
// variables/functions in a particular context, for example.
deriveCtx := bClosure.EvalContext.NewChild()
deriveCtx.Variables = map[string]cty.Value{
"foo": cty.StringVal("overridden foo value"),
}
gotBVal2, diags := bClosure.Expression.Value(deriveCtx)
wantBVal2 := cty.StringVal("overridden foo value")
if diags.HasErrors() {
t.Fatalf("unexpected problems: %s", diags.Error())
}
if got, want := gotBVal2, wantBVal2; !want.RawEquals(got) {
t.Errorf("wrong 'b' result with derived EvalContext\ngot: %#v\nwant: %#v", got, want)
}
}