ext/customdecode: Custom expression decoding extension

Most of the time, the standard expression decoding built in to HCL is
sufficient. Sometimes though, it's useful to be able to customize the
decoding of certain arguments where the application intends to use them
in a very specific way, such as in static analysis.

This extension is an approximate analog of gohcl's support for decoding
into an hcl.Expression, allowing hcldec-based applications and
applications with custom functions to similarly capture and manipulate
the physical expressions used in arguments, rather than their values.

This includes one example use-case: the typeexpr extension now includes
a cty.Function called ConvertFunc that takes a type expression as its
second argument. A type expression is not evaluatable in the usual sense,
but thanks to cty capsule types we _can_ produce a cty.Value from one
and then make use of it inside the function implementation, without
exposing this custom type to the broader language:

    convert(["foo"], set(string))

This mechanism is intentionally restricted only to "argument-like"
locations where there is a specific type we are attempting to decode into.
For now, that's hcldec AttrSpec/BlockAttrsSpec -- analogous to gohcl
decoding into hcl.Expression -- and in arguments to functions.

ext/customdecode/README.md

@@ -0,0 +1,209 @@
+# HCL Custom Static Decoding Extension
+
+This HCL extension provides a mechanism for defining arguments in an HCL-based
+language whose values are derived using custom decoding rules against the
+HCL expression syntax, overriding the usual behavior of normal expression
+evaluation.
+
+"Arguments", for the purpose of this extension, currently includes the
+following two contexts:
+
+* For applications using `hcldec` for dynamic decoding, a `hcldec.AttrSpec`
+  or `hcldec.BlockAttrsSpec` can be given a special type constraint that
+  opts in to custom decoding behavior for the attribute(s) that are selected
+  by that specification.
+
+* When working with the HCL native expression syntax, a function given in
+  the `hcl.EvalContext` during evaluation can have parameters with special
+  type constraints that opt in to custom decoding behavior for the argument
+  expression associated with that parameter in any call.
+
+The above use-cases are rather abstract, so we'll consider a motivating
+real-world example: sometimes we (language designers) need to allow users
+to specify type constraints directly in the language itself, such as in
+[Terraform's Input Variables](https://www.terraform.io/docs/configuration/variables.html).
+Terraform's `variable` blocks include an argument called `type` which takes
+a type constraint given using HCL expression building-blocks as defined by
+[the HCL `typeexpr` extension](../typeexpr/README.md).
+
+A "type constraint expression" of that sort is not an expression intended to
+be evaluated in the usual way. Instead, the physical expression is
+deconstructed using [the static analysis operations](../../spec.md#static-analysis)
+to produce a `cty.Type` as the result, rather than a `cty.Value`.
+
+The purpose of this Custom Static Decoding Extension, then, is to provide a
+bridge to allow that sort of custom decoding to be used via mechanisms that
+normally deal in `cty.Value`, such as `hcldec` and native syntax function
+calls as listed above.
+
+(Note: [`gohcl`](https://pkg.go.dev/github.com/hashicorp/hcl/v2/gohcl) has
+its own mechanism to support this use case, exploiting the fact that it is
+working directly with "normal" Go types. Decoding into a struct field of
+type `hcl.Expression` obtains the expression directly without evaluating it
+first. The Custom Static Decoding Extension is not necessary for that `gohcl`
+technique. You can also implement custom decoding by working directly with
+the lowest-level HCL API, which separates extraction of and evaluation of
+expressions into two steps.)
+
+## Custom Decoding Types
+
+This extension relies on a convention implemented in terms of
+[_Capsule Types_ in the underlying `cty` type system](https://github.com/zclconf/go-cty/blob/master/docs/types.md#capsule-types). `cty` allows a capsule type to carry arbitrary
+extension metadata values as an aid to creating higher-level abstractions like
+this extension.
+
+A custom argument decoding mode, then, is implemented by creating a new `cty`
+capsule type that implements the `ExtensionData` custom operation to return
+a decoding function when requested. For example:
+
+```go
+var keywordType cty.Type
+keywordType = cty.CapsuleWithOps("keyword", reflect.TypeOf(""), &cty.CapsuleOps{
+    ExtensionData: func(key interface{}) interface{} {
+        switch key {
+        case customdecode.CustomExpressionDecoder:
+            return customdecode.CustomExpressionDecoderFunc(
+                func(expr hcl.Expression, ctx *hcl.EvalContext) (cty.Value, hcl.Diagnostics) {
+                    var diags hcl.Diagnostics
+                    kw := hcl.ExprAsKeyword(expr)
+                    if kw == "" {
+                        diags = append(diags, &hcl.Diagnostic{
+                            Severity: hcl.DiagError,
+                            Summary:  "Invalid keyword",
+                            Detail:   "A keyword is required",
+                            Subject:  expr.Range().Ptr(),
+                        })
+                        return cty.UnkownVal(keywordType), diags
+                    }
+                    return cty.CapsuleVal(keywordType, &kw)
+                },
+            )
+        default:
+            return nil
+        }
+    },
+})
+```
+
+The boilerplate here is a bit fussy, but the important part for our purposes
+is the `case customdecode.CustomExpressionDecoder:` clause, which uses
+a custom extension key type defined in this package to recognize when a
+component implementing this extension is checking to see if a target type
+has a custom decode implementation.
+
+In the above case we've defined a type that decodes expressions as static
+keywords, so a keyword like `foo` would decode as an encapsulated `"foo"`
+string, while any other sort of expression like `"baz"` or `1 + 1` would
+return an error.
+
+We could then use `keywordType` as a type constraint either for a function
+parameter or a `hcldec` attribute specification, which would require the
+argument for that function parameter or the expression for the matching
+attributes to be a static keyword, rather than an arbitrary expression.
+For example, in a `hcldec.AttrSpec`:
+
+```go
+keywordSpec := &hcldec.AttrSpec{
+    Name: "keyword",
+    Type: keywordType,
+}
+```
+
+The above would accept input like the following and would set its result to
+a `cty.Value` of `keywordType`, after decoding:
+
+```hcl
+keyword = foo
+```
+
+## The Expression and Expression Closure `cty` types
+
+Building on the above, this package also includes two capsule types that use
+the above mechanism to allow calling applications to capture expressions
+directly and thus defer analysis to a later step, after initial decoding.
+
+The `customdecode.ExpressionType` type encapsulates an `hcl.Expression` alone,
+for situations like our type constraint expression example above where it's
+the static structure of the expression we want to inspect, and thus any
+variables and functions defined in the evaluation context are irrelevant.
+
+The `customdecode.ExpressionClosureType` type encapsulates a
+`*customdecode.ExpressionClosure` value, which binds the given expression to
+the `hcl.EvalContext` it was asked to evaluate against and thus allows the
+receiver of that result to later perform normal evaluation of the expression
+with all the same variables and functions that would've been available to it
+naturally.
+
+Both of these types can be used as type constraints either for `hcldec`
+attribute specifications or for function arguments. Here's an example of
+`ExpressionClosureType` to implement a function that can evaluate
+an expression with some additional variables defined locally, which we'll
+call the `with(...)` function:
+
+```go
+var WithFunc = function.New(&function.Spec{
+    Params: []function.Parameter{
+        {
+            Name: "variables",
+            Type: cty.DynamicPseudoType,
+        },
+        {
+            Name: "expression",
+            Type: customdecode.ExpressionClosureType,
+        },
+    },
+    Type: func(args []cty.Value) (cty.Type, error) {
+        varsVal := args[0]
+        exprVal := args[1]
+        if !varsVal.Type().IsObjectType() {
+            return cty.NilVal, function.NewArgErrorf(0, "must be an object defining local variables")
+        }
+        if !varsVal.IsKnown() {
+            // We can't predict our result type until the variables object
+            // is known.
+            return cty.DynamicPseudoType, nil
+        }
+        vars := varsVal.AsValueMap()
+        closure := customdecode.ExpressionClosureFromVal(exprVal)
+        result, err := evalWithLocals(vars, closure)
+        if err != nil {
+            return cty.NilVal, err
+        }
+        return result.Type(), nil
+    },
+    Impl: func(args []cty.Value, retType cty.Type) (cty.Value, error) {
+        varsVal := args[0]
+        exprVal := args[1]
+        vars := varsVal.AsValueMap()
+        closure := customdecode.ExpressionClosureFromVal(exprVal)
+        return evalWithLocals(vars, closure)
+    },
+})
+
+func evalWithLocals(locals map[string]cty.Value, closure *customdecode.ExpressionClosure) (cty.Value, error) {
+    childCtx := closure.EvalContext.NewChild()
+    childCtx.Variables = locals
+    val, diags := closure.Expression.Value(childCtx)
+    if diags.HasErrors() {
+        return cty.NilVal, function.NewArgErrorf(1, "couldn't evaluate expression: %s", diags.Error())
+    }
+    return val, nil
+}
+```
+
+If the above function were placed into an `hcl.EvalContext` as `with`, it
+could be used in a native syntax call to that function as follows:
+
+```hcl
+  foo = with({name = "Cory"}, "${greeting}, ${name}!")
+```
+
+The above assumes a variable in the main context called `greeting`, to which
+the `with` function adds `name` before evaluating the expression given in
+its second argument. This makes that second argument context-sensitive -- it
+would behave differently if the user wrote the same thing somewhere else -- so
+this capability should be used with care to make sure it doesn't cause confusion
+for the end-users of your language.
+
+There are some other examples of this capability to evaluate expressions in
+unusual ways in the `tryfunc` directory that is a sibling of this one.

ext/customdecode/customdecode.go

@@ -0,0 +1,56 @@
+// Package customdecode contains a HCL extension that allows, in certain
+// contexts, expression evaluation to be overridden by custom static analysis.
+//
+// This mechanism is only supported in certain specific contexts where
+// expressions are decoded with a specific target type in mind. For more
+// information, see the documentation on CustomExpressionDecoder.
+package customdecode
+
+import (
+	"github.com/hashicorp/hcl/v2"
+	"github.com/zclconf/go-cty/cty"
+)
+
+type customDecoderImpl int
+
+// CustomExpressionDecoder is a value intended to be used as a cty capsule
+// type ExtensionData key for capsule types whose values are to be obtained
+// by static analysis of an expression rather than normal evaluation of that
+// expression.
+//
+// When a cooperating capsule type is asked for ExtensionData with this key,
+// it must return a non-nil CustomExpressionDecoderFunc value.
+//
+// This mechanism is not universally supported; instead, it's handled in a few
+// specific places where expressions are evaluated with the intent of producing
+// a cty.Value of a type given by the calling application.
+//
+// Specifically, this currently works for type constraints given in
+// hcldec.AttrSpec and hcldec.BlockAttrsSpec, and it works for arguments to
+// function calls in the HCL native syntax. HCL extensions implemented outside
+// of the main HCL module may also implement this; consult their own
+// documentation for details.
+const CustomExpressionDecoder = customDecoderImpl(1)
+
+// CustomExpressionDecoderFunc is the type of value that must be returned by
+// a capsule type handling the key CustomExpressionDecoder in its ExtensionData
+// implementation.
+//
+// If no error diagnostics are returned, the result value MUST be of the
+// capsule type that the decoder function was derived from. If the returned
+// error diagnostics prevent producing a value at all, return cty.NilVal.
+type CustomExpressionDecoderFunc func(expr hcl.Expression, ctx *hcl.EvalContext) (cty.Value, hcl.Diagnostics)
+
+// CustomExpressionDecoderForType takes any cty type and returns its
+// custom expression decoder implementation if it has one. If it is not a
+// capsule type or it does not implement a custom expression decoder, this
+// function returns nil.
+func CustomExpressionDecoderForType(ty cty.Type) CustomExpressionDecoderFunc {
+	if !ty.IsCapsuleType() {
+		return nil
+	}
+	if fn, ok := ty.CapsuleExtensionData(CustomExpressionDecoder).(CustomExpressionDecoderFunc); ok {
+		return fn
+	}
+	return nil
+}

ext/customdecode/expression_type.go

@@ -0,0 +1,146 @@
+package customdecode
+
+import (
+	"fmt"
+	"reflect"
+
+	"github.com/hashicorp/hcl/v2"
+	"github.com/zclconf/go-cty/cty"
+)
+
+// ExpressionType is a cty capsule type that carries hcl.Expression values.
+//
+// This type implements custom decoding in the most general way possible: it
+// just captures whatever expression is given to it, with no further processing
+// whatsoever. It could therefore be useful in situations where an application
+// must defer processing of the expression content until a later step.
+//
+// ExpressionType only captures the expression, not the evaluation context it
+// was destined to be evaluated in. That means this type can be fine for
+// situations where the recipient of the value only intends to do static
+// analysis, but ExpressionClosureType is more appropriate in situations where
+// the recipient will eventually evaluate the given expression.
+var ExpressionType cty.Type
+
+// ExpressionVal returns a new cty value of type ExpressionType, wrapping the
+// given expression.
+func ExpressionVal(expr hcl.Expression) cty.Value {
+	return cty.CapsuleVal(ExpressionType, &expr)
+}
+
+// ExpressionFromVal returns the expression encapsulated in the given value, or
+// panics if the value is not a known value of ExpressionType.
+func ExpressionFromVal(v cty.Value) hcl.Expression {
+	if !v.Type().Equals(ExpressionType) {
+		panic("value is not of ExpressionType")
+	}
+	ptr := v.EncapsulatedValue().(*hcl.Expression)
+	return *ptr
+}
+
+// ExpressionClosureType is a cty capsule type that carries hcl.Expression
+// values along with their original evaluation contexts.
+//
+// This is similar to ExpressionType except that during custom decoding it
+// also captures the hcl.EvalContext that was provided, allowing callers to
+// evaluate the expression later in the same context where it would originally
+// have been evaluated, or a context derived from that one.
+var ExpressionClosureType cty.Type
+
+// ExpressionClosure is the type encapsulated in ExpressionClosureType
+type ExpressionClosure struct {
+	Expression  hcl.Expression
+	EvalContext *hcl.EvalContext
+}
+
+// ExpressionClosureVal returns a new cty value of type ExpressionClosureType,
+// wrapping the given expression closure.
+func ExpressionClosureVal(closure *ExpressionClosure) cty.Value {
+	return cty.CapsuleVal(ExpressionClosureType, closure)
+}
+
+// Value evaluates the closure's expression using the closure's EvalContext,
+// returning the result.
+func (c *ExpressionClosure) Value() (cty.Value, hcl.Diagnostics) {
+	return c.Expression.Value(c.EvalContext)
+}
+
+// ExpressionClosureFromVal returns the expression closure encapsulated in the
+// given value, or panics if the value is not a known value of
+// ExpressionClosureType.
+//
+// The caller MUST NOT modify the returned closure or the EvalContext inside
+// it. To derive a new EvalContext, either create a child context or make
+// a copy.
+func ExpressionClosureFromVal(v cty.Value) *ExpressionClosure {
+	if !v.Type().Equals(ExpressionClosureType) {
+		panic("value is not of ExpressionClosureType")
+	}
+	return v.EncapsulatedValue().(*ExpressionClosure)
+}
+
+func init() {
+	// Getting hold of a reflect.Type for hcl.Expression is a bit tricky because
+	// it's an interface type, but we can do it with some indirection.
+	goExpressionType := reflect.TypeOf((*hcl.Expression)(nil)).Elem()
+
+	ExpressionType = cty.CapsuleWithOps("expression", goExpressionType, &cty.CapsuleOps{
+		ExtensionData: func(key interface{}) interface{} {
+			switch key {
+			case CustomExpressionDecoder:
+				return CustomExpressionDecoderFunc(
+					func(expr hcl.Expression, ctx *hcl.EvalContext) (cty.Value, hcl.Diagnostics) {
+						return ExpressionVal(expr), nil
+					},
+				)
+			default:
+				return nil
+			}
+		},
+		TypeGoString: func(_ reflect.Type) string {
+			return "customdecode.ExpressionType"
+		},
+		GoString: func(raw interface{}) string {
+			exprPtr := raw.(*hcl.Expression)
+			return fmt.Sprintf("customdecode.ExpressionVal(%#v)", *exprPtr)
+		},
+		RawEquals: func(a, b interface{}) bool {
+			aPtr := a.(*hcl.Expression)
+			bPtr := b.(*hcl.Expression)
+			return reflect.DeepEqual(*aPtr, *bPtr)
+		},
+	})
+	ExpressionClosureType = cty.CapsuleWithOps("expression closure", reflect.TypeOf(ExpressionClosure{}), &cty.CapsuleOps{
+		ExtensionData: func(key interface{}) interface{} {
+			switch key {
+			case CustomExpressionDecoder:
+				return CustomExpressionDecoderFunc(
+					func(expr hcl.Expression, ctx *hcl.EvalContext) (cty.Value, hcl.Diagnostics) {
+						return ExpressionClosureVal(&ExpressionClosure{
+							Expression:  expr,
+							EvalContext: ctx,
+						}), nil
+					},
+				)
+			default:
+				return nil
+			}
+		},
+		TypeGoString: func(_ reflect.Type) string {
+			return "customdecode.ExpressionClosureType"
+		},
+		GoString: func(raw interface{}) string {
+			closure := raw.(*ExpressionClosure)
+			return fmt.Sprintf("customdecode.ExpressionClosureVal(%#v)", closure)
+		},
+		RawEquals: func(a, b interface{}) bool {
+			closureA := a.(*ExpressionClosure)
+			closureB := b.(*ExpressionClosure)
+			// The expression itself compares by deep equality, but EvalContexts
+			// conventionally compare by pointer identity, so we'll comply
+			// with both conventions here by testing them separately.
+			return closureA.EvalContext == closureB.EvalContext &&
+				reflect.DeepEqual(closureA.Expression, closureB.Expression)
+		},
+	})
+}

ext/typeexpr/README.md

@@ -65,3 +65,71 @@ type checking it will be one that has identifiers as its attributes; object
 types with weird attributes generally show up only from arbitrary object
 constructors in configuration files, which are usually treated either as maps
 or as the dynamic pseudo-type.
+
+## Type Constraints as Values
+
+Along with defining a convention for writing down types using HCL expression
+constructs, this package also includes a mechanism for representing types as
+values that can be used as data within an HCL-based language.
+
+`typeexpr.TypeConstraintType` is a
+[`cty` capsule type](https://github.com/zclconf/go-cty/blob/master/docs/types.md#capsule-types)
+that encapsulates `cty.Type` values. You can construct such a value directly
+using the `TypeConstraintVal` function:
+
+```go
+tyVal := typeexpr.TypeConstraintVal(cty.String)
+
+// We can unpack the type from a value using TypeConstraintFromVal
+ty := typeExpr.TypeConstraintFromVal(tyVal)
+```
+
+However, the primary purpose of `typeexpr.TypeConstraintType` is to be
+specified as the type constraint for an argument, in which case it serves
+as a signal for HCL to treat the argument expression as a type constraint
+expression as defined above, rather than as a normal value expression.
+
+"An argument" in the above in practice means the following two locations:
+
+* As the type constraint for a parameter of a cty function that will be
+  used in an `hcl.EvalContext`. In that case, function calls in the HCL
+  native expression syntax will require the argument to be valid type constraint
+  expression syntax and the function implementation will receive a
+  `TypeConstraintType` value as the argument value for that parameter.
+
+* As the type constraint for a `hcldec.AttrSpec` or `hcldec.BlockAttrsSpec`
+  when decoding an HCL body using `hcldec`. In that case, the attributes
+  with that type constraint will be required to be valid type constraint
+  expression syntax and the result will be a `TypeConstraintType` value.
+
+Note that the special handling of these arguments means that an argument
+marked in this way must use the type constraint syntax directly. It is not
+valid to pass in a value of `TypeConstraintType` that has been obtained
+dynamically via some other expression result.
+
+`TypeConstraintType` is provided with the intent of using it internally within
+application code when incorporating type constraint expression syntax into
+an HCL-based language, not to be used for dynamic "programming with types". A
+calling application could support programming with types by defining its _own_
+capsule type, but that is not the purpose of `TypeConstraintType`.
+
+## The "convert" `cty` Function
+
+Building on the `TypeConstraintType` described in the previous section, this
+package also provides `typeexpr.ConvertFunc` which is a cty function that
+can be placed into a `cty.EvalContext` (conventionally named "convert") in
+order to provide a general type conversion function in an HCL-based language:
+
+```hcl
+  foo = convert("true", bool)
+```
+
+The second parameter uses the mechanism described in the previous section to
+require its argument to be a type constraint expression rather than a value
+expression. In doing so, it allows converting with any type constraint that
+can be expressed in this package's type constraint syntax. In the above example,
+the `foo` argument would receive a boolean true, or `cty.True` in `cty` terms.
+
+The target type constraint must always be provided statically using inline
+type constraint syntax. There is no way to _dynamically_ select a type
+constraint using this function.

ext/typeexpr/type_type.go

@@ -0,0 +1,118 @@
+package typeexpr
+
+import (
+	"fmt"
+	"reflect"
+
+	"github.com/hashicorp/hcl/v2"
+	"github.com/hashicorp/hcl/v2/ext/customdecode"
+	"github.com/zclconf/go-cty/cty"
+	"github.com/zclconf/go-cty/cty/convert"
+	"github.com/zclconf/go-cty/cty/function"
+)
+
+// TypeConstraintType is a cty capsule type that allows cty type constraints to
+// be used as values.
+//
+// If TypeConstraintType is used in a context supporting the
+// customdecode.CustomExpressionDecoder extension then it will implement
+// expression decoding using the TypeConstraint function, thus allowing
+// type expressions to be used in contexts where value expressions might
+// normally be expected, such as in arguments to function calls.
+var TypeConstraintType cty.Type
+
+// TypeConstraintVal constructs a cty.Value whose type is
+// TypeConstraintType.
+func TypeConstraintVal(ty cty.Type) cty.Value {
+	return cty.CapsuleVal(TypeConstraintType, &ty)
+}
+
+// TypeConstraintFromVal extracts the type from a cty.Value of
+// TypeConstraintType that was previously constructed using TypeConstraintVal.
+//
+// If the given value isn't a known, non-null value of TypeConstraintType
+// then this function will panic.
+func TypeConstraintFromVal(v cty.Value) cty.Type {
+	if !v.Type().Equals(TypeConstraintType) {
+		panic("value is not of TypeConstraintType")
+	}
+	ptr := v.EncapsulatedValue().(*cty.Type)
+	return *ptr
+}
+
+// ConvertFunc is a cty function that implements type conversions.
+//
+// Its signature is as follows:
+//     convert(value, type_constraint)
+//
+// ...where type_constraint is a type constraint expression as defined by
+// typeexpr.TypeConstraint.
+//
+// It relies on HCL's customdecode extension and so it's not suitable for use
+// in non-HCL contexts or if you are using a HCL syntax implementation that
+// does not support customdecode for function arguments. However, it _is_
+// supported for function calls in the HCL native expression syntax.
+var ConvertFunc function.Function
+
+func init() {
+	TypeConstraintType = cty.CapsuleWithOps("type constraint", reflect.TypeOf(cty.Type{}), &cty.CapsuleOps{
+		ExtensionData: func(key interface{}) interface{} {
+			switch key {
+			case customdecode.CustomExpressionDecoder:
+				return customdecode.CustomExpressionDecoderFunc(
+					func(expr hcl.Expression, ctx *hcl.EvalContext) (cty.Value, hcl.Diagnostics) {
+						ty, diags := TypeConstraint(expr)
+						if diags.HasErrors() {
+							return cty.NilVal, diags
+						}
+						return TypeConstraintVal(ty), nil
+					},
+				)
+			default:
+				return nil
+			}
+		},
+		TypeGoString: func(_ reflect.Type) string {
+			return "typeexpr.TypeConstraintType"
+		},
+		GoString: func(raw interface{}) string {
+			tyPtr := raw.(*cty.Type)
+			return fmt.Sprintf("typeexpr.TypeConstraintVal(%#v)", *tyPtr)
+		},
+		RawEquals: func(a, b interface{}) bool {
+			aPtr := a.(*cty.Type)
+			bPtr := b.(*cty.Type)
+			return (*aPtr).Equals(*bPtr)
+		},
+	})
+
+	ConvertFunc = function.New(&function.Spec{
+		Params: []function.Parameter{
+			{
+				Name:             "value",
+				Type:             cty.DynamicPseudoType,
+				AllowNull:        true,
+				AllowDynamicType: true,
+			},
+			{
+				Name: "type",
+				Type: TypeConstraintType,
+			},
+		},
+		Type: func(args []cty.Value) (cty.Type, error) {
+			wantTypePtr := args[1].EncapsulatedValue().(*cty.Type)
+			got, err := convert.Convert(args[0], *wantTypePtr)
+			if err != nil {
+				return cty.NilType, function.NewArgError(0, err)
+			}
+			return got.Type(), nil
+		},
+		Impl: func(args []cty.Value, retType cty.Type) (cty.Value, error) {
+			v, err := convert.Convert(args[0], retType)
+			if err != nil {
+				return cty.NilVal, function.NewArgError(0, err)
+			}
+			return v, nil
+		},
+	})
+}

ext/typeexpr/type_type_test.go

@@ -0,0 +1,118 @@
+package typeexpr
+
+import (
+	"fmt"
+	"testing"
+
+	"github.com/zclconf/go-cty/cty"
+)
+
+func TestTypeConstraintType(t *testing.T) {
+	tyVal1 := TypeConstraintVal(cty.String)
+	tyVal2 := TypeConstraintVal(cty.String)
+	tyVal3 := TypeConstraintVal(cty.Number)
+
+	if !tyVal1.RawEquals(tyVal2) {
+		t.Errorf("tyVal1 not equal to tyVal2\ntyVal1: %#v\ntyVal2: %#v", tyVal1, tyVal2)
+	}
+	if tyVal1.RawEquals(tyVal3) {
+		t.Errorf("tyVal1 equal to tyVal2, but should not be\ntyVal1: %#v\ntyVal3: %#v", tyVal1, tyVal3)
+	}
+
+	if got, want := TypeConstraintFromVal(tyVal1), cty.String; !got.Equals(want) {
+		t.Errorf("wrong type extracted from tyVal1\ngot:  %#v\nwant: %#v", got, want)
+	}
+	if got, want := TypeConstraintFromVal(tyVal3), cty.Number; !got.Equals(want) {
+		t.Errorf("wrong type extracted from tyVal3\ngot:  %#v\nwant: %#v", got, want)
+	}
+}
+
+func TestConvertFunc(t *testing.T) {
+	// This is testing the convert function directly, skipping over the HCL
+	// parsing and evaluation steps that would normally lead there. There is
+	// another test in the "integrationtest" package called TestTypeConvertFunc
+	// that exercises the full path to this function via the hclsyntax parser.
+
+	tests := []struct {
+		val, ty cty.Value
+		want    cty.Value
+		wantErr string
+	}{
+		// The goal here is not an exhaustive set of conversions, since that's
+		// already covered in cty/convert, but rather exercising different
+		// permutations of success and failure to make sure the function
+		// handles all of the results in a reasonable way.
+		{
+			cty.StringVal("hello"),
+			TypeConstraintVal(cty.String),
+			cty.StringVal("hello"),
+			``,
+		},
+		{
+			cty.True,
+			TypeConstraintVal(cty.String),
+			cty.StringVal("true"),
+			``,
+		},
+		{
+			cty.StringVal("hello"),
+			TypeConstraintVal(cty.Bool),
+			cty.NilVal,
+			`a bool is required`,
+		},
+		{
+			cty.UnknownVal(cty.Bool),
+			TypeConstraintVal(cty.Bool),
+			cty.UnknownVal(cty.Bool),
+			``,
+		},
+		{
+			cty.DynamicVal,
+			TypeConstraintVal(cty.Bool),
+			cty.UnknownVal(cty.Bool),
+			``,
+		},
+		{
+			cty.NullVal(cty.Bool),
+			TypeConstraintVal(cty.Bool),
+			cty.NullVal(cty.Bool),
+			``,
+		},
+		{
+			cty.NullVal(cty.DynamicPseudoType),
+			TypeConstraintVal(cty.Bool),
+			cty.NullVal(cty.Bool),
+			``,
+		},
+		{
+			cty.StringVal("hello").Mark(1),
+			TypeConstraintVal(cty.String),
+			cty.StringVal("hello").Mark(1),
+			``,
+		},
+	}
+
+	for _, test := range tests {
+		t.Run(fmt.Sprintf("%#v to %#v", test.val, test.ty), func(t *testing.T) {
+			got, err := ConvertFunc.Call([]cty.Value{test.val, test.ty})
+
+			if err != nil {
+				if test.wantErr != "" {
+					if got, want := err.Error(), test.wantErr; got != want {
+						t.Errorf("wrong error\ngot:  %s\nwant: %s", got, want)
+					}
+				} else {
+					t.Errorf("unexpected error\ngot:  %s\nwant: <nil>", err)
+				}
+				return
+			}
+			if test.wantErr != "" {
+				t.Errorf("wrong error\ngot:  <nil>\nwant: %s", test.wantErr)
+			}
+
+			if !test.want.RawEquals(got) {
+				t.Errorf("wrong result\ngot:  %#v\nwant: %#v", got, test.want)
+			}
+		})
+	}
+}

hcldec/spec.go

@@ -6,6 +6,7 @@ import (
 	"sort"
 
 	"github.com/hashicorp/hcl/v2"
+	"github.com/hashicorp/hcl/v2/ext/customdecode"
 	"github.com/zclconf/go-cty/cty"
 	"github.com/zclconf/go-cty/cty/convert"
 	"github.com/zclconf/go-cty/cty/function"
@@ -193,6 +194,14 @@ func (s *AttrSpec) decode(content *hcl.BodyContent, blockLabels []blockLabel, ct
 		return cty.NullVal(s.Type), nil
 	}
 
+	if decodeFn := customdecode.CustomExpressionDecoderForType(s.Type); decodeFn != nil {
+		v, diags := decodeFn(attr.Expr, ctx)
+		if v == cty.NilVal {
+			v = cty.UnknownVal(s.Type)
+		}
+		return v, diags
+	}
+
 	val, diags := attr.Expr.Value(ctx)
 
 	convVal, err := convert.Convert(val, s.Type)
@@ -1223,6 +1232,16 @@ func (s *BlockAttrsSpec) decode(content *hcl.BodyContent, blockLabels []blockLab
 
 	vals := make(map[string]cty.Value, len(attrs))
 	for name, attr := range attrs {
+		if decodeFn := customdecode.CustomExpressionDecoderForType(s.ElementType); decodeFn != nil {
+			attrVal, attrDiags := decodeFn(attr.Expr, ctx)
+			diags = append(diags, attrDiags...)
+			if attrVal == cty.NilVal {
+				attrVal = cty.UnknownVal(s.ElementType)
+			}
+			vals[name] = attrVal
+			continue
+		}
+
 		attrVal, attrDiags := attr.Expr.Value(ctx)
 		diags = append(diags, attrDiags...)
 

hclsyntax/expression.go

@@ -5,6 +5,7 @@ import (
 	"sync"
 
 	"github.com/hashicorp/hcl/v2"
+	"github.com/hashicorp/hcl/v2/ext/customdecode"
 	"github.com/zclconf/go-cty/cty"
 	"github.com/zclconf/go-cty/cty/convert"
 	"github.com/zclconf/go-cty/cty/function"
@@ -350,26 +351,38 @@ func (e *FunctionCallExpr) Value(ctx *hcl.EvalContext) (cty.Value, hcl.Diagnosti
 			param = varParam
 		}
 
-		val, argDiags := argExpr.Value(ctx)
+		var val cty.Value
-		if len(argDiags) > 0 {
+		if decodeFn := customdecode.CustomExpressionDecoderForType(param.Type); decodeFn != nil {
+			var argDiags hcl.Diagnostics
+			val, argDiags = decodeFn(argExpr, ctx)
 			diags = append(diags, argDiags...)
-		}
+			if val == cty.NilVal {
+				val = cty.UnknownVal(param.Type)
+			}
+		} else {
+			var argDiags hcl.Diagnostics
+			val, argDiags = argExpr.Value(ctx)
+			if len(argDiags) > 0 {
+				diags = append(diags, argDiags...)
+			}
 
-		// Try to convert our value to the parameter type
+			// Try to convert our value to the parameter type
-		val, err := convert.Convert(val, param.Type)
+			var err error
-		if err != nil {
+			val, err = convert.Convert(val, param.Type)
-			diags = append(diags, &hcl.Diagnostic{
+			if err != nil {
-				Severity: hcl.DiagError,
+				diags = append(diags, &hcl.Diagnostic{
-				Summary:  "Invalid function argument",
+					Severity: hcl.DiagError,
-				Detail: fmt.Sprintf(
+					Summary:  "Invalid function argument",
-					"Invalid value for %q parameter: %s.",
+					Detail: fmt.Sprintf(
-					param.Name, err,
+						"Invalid value for %q parameter: %s.",
-				),
+						param.Name, err,
-				Subject:     argExpr.StartRange().Ptr(),
+					),
-				Context:     e.Range().Ptr(),
+					Subject:     argExpr.StartRange().Ptr(),
-				Expression:  argExpr,
+					Context:     e.Range().Ptr(),
-				EvalContext: ctx,
+					Expression:  argExpr,
-			})
+					EvalContext: ctx,
+				})
+			}
 		}
 
 		argVals[i] = val

integrationtest/convertfunc_test.go

@@ -0,0 +1,56 @@
+package integrationtest
+
+import (
+	"testing"
+
+	"github.com/hashicorp/hcl/v2"
+	"github.com/hashicorp/hcl/v2/ext/typeexpr"
+	"github.com/hashicorp/hcl/v2/hclsyntax"
+	"github.com/zclconf/go-cty/cty"
+	"github.com/zclconf/go-cty/cty/function"
+)
+
+// TestTypeConvertFunc is an integration test of all of the layers involved
+// in making the type conversion function from ext/typeexpr work.
+//
+// This requires co-operation between the hclsyntax package, the ext/typeexpr
+// package, and the underlying cty functionality in order to work correctly.
+//
+// There are unit tests for the function implementation itself in the
+// ext/typeexpr package, so this test is focused on making sure the function
+// is given the opportunity to decode the second argument as a type expression
+// when the function is called from HCL native syntax.
+func TestTypeConvertFunc(t *testing.T) {
+	// The convert function is special because it takes a type expression
+	// rather than a value expression as its second argument. In this case,
+	// we're asking it to convert a tuple into a list of strings:
+	const exprSrc = `convert(["hello"], list(string))`
+	// It achieves this by marking that second argument as being of a custom
+	// type (a "capsule type", in cty terminology) that has a special
+	// annotation which hclsyntax.FunctionCallExpr understands as allowing
+	// the type to handle the analysis of the unevaluated expression, instead
+	// of evaluating it as normal.
+	//
+	// To see more details of how this works, look at the definitions of
+	// typexpr.TypeConstraintType and typeexpr.ConvertFunc, and at the
+	// implementation of hclsyntax.FunctionCallExpr.Value.
+
+	expr, diags := hclsyntax.ParseExpression([]byte(exprSrc), "", hcl.Pos{Line: 1, Column: 1})
+	if diags.HasErrors() {
+		t.Fatalf("unexpected problems: %s", diags.Error())
+	}
+
+	ctx := &hcl.EvalContext{
+		Functions: map[string]function.Function{
+			"convert": typeexpr.ConvertFunc,
+		},
+	}
+	got, diags := expr.Value(ctx)
+	if diags.HasErrors() {
+		t.Fatalf("unexpected problems: %s", diags.Error())
+	}
+	want := cty.ListVal([]cty.Value{cty.StringVal("hello")})
+	if !want.RawEquals(got) {
+		t.Errorf("wrong result\ngot:  %#v\nwant: %#v", got, want)
+	}
+}

integrationtest/hcldec_into_expr_test.go

@@ -0,0 +1,131 @@
+package integrationtest
+
+import (
+	"testing"
+
+	"github.com/google/go-cmp/cmp"
+	"github.com/google/go-cmp/cmp/cmpopts"
+	"github.com/hashicorp/hcl/v2"
+	"github.com/hashicorp/hcl/v2/ext/customdecode"
+	"github.com/hashicorp/hcl/v2/hcldec"
+	"github.com/hashicorp/hcl/v2/hclsyntax"
+	"github.com/zclconf/go-cty/cty"
+)
+
+// TestHCLDecDecodeToExpr tests both hcldec's support for types with custom
+// expression decoding rules and the two expression capsule types implemented
+// in ext/customdecode. This mechanism requires cooperation between those
+// two components and cty in order to work, so it's helpful to exercise it in
+// an integration test.
+func TestHCLDecDecodeToExpr(t *testing.T) {
+	// Here we're going to capture the structure of two simple expressions
+	// without immediately evaluating them.
+	const input = `
+a = foo
+b = foo
+c = "hello"
+`
+	// We'll capture "a" directly as an expression, losing its evaluation
+	// context but retaining its structure. We'll capture "b" as a
+	// customdecode.ExpressionClosure, which gives us both the expression
+	// itself and the evaluation context it was originally evaluated in.
+	// We also have "c" here just to make sure we can still decode into a
+	// "normal" type via standard expression evaluation.
+
+	f, diags := hclsyntax.ParseConfig([]byte(input), "", hcl.Pos{Line: 1, Column: 1})
+	if diags.HasErrors() {
+		t.Fatalf("unexpected problems: %s", diags.Error())
+	}
+
+	spec := hcldec.ObjectSpec{
+		"a": &hcldec.AttrSpec{
+			Name:     "a",
+			Type:     customdecode.ExpressionType,
+			Required: true,
+		},
+		"b": &hcldec.AttrSpec{
+			Name:     "b",
+			Type:     customdecode.ExpressionClosureType,
+			Required: true,
+		},
+		"c": &hcldec.AttrSpec{
+			Name:     "c",
+			Type:     cty.String,
+			Required: true,
+		},
+	}
+	ctx := &hcl.EvalContext{
+		Variables: map[string]cty.Value{
+			"foo": cty.StringVal("foo value"),
+		},
+	}
+	objVal, diags := hcldec.Decode(f.Body, spec, ctx)
+	if diags.HasErrors() {
+		t.Fatalf("unexpected problems: %s", diags.Error())
+	}
+
+	aVal := objVal.GetAttr("a")
+	bVal := objVal.GetAttr("b")
+	cVal := objVal.GetAttr("c")
+
+	if got, want := aVal.Type(), customdecode.ExpressionType; !got.Equals(want) {
+		t.Fatalf("wrong type for 'a'\ngot:  %#v\nwant: %#v", got, want)
+	}
+	if got, want := bVal.Type(), customdecode.ExpressionClosureType; !got.Equals(want) {
+		t.Fatalf("wrong type for 'b'\ngot:  %#v\nwant: %#v", got, want)
+	}
+	if got, want := cVal.Type(), cty.String; !got.Equals(want) {
+		t.Fatalf("wrong type for 'c'\ngot:  %#v\nwant: %#v", got, want)
+	}
+
+	gotAExpr := customdecode.ExpressionFromVal(aVal)
+	wantAExpr := &hclsyntax.ScopeTraversalExpr{
+		Traversal: hcl.Traversal{
+			hcl.TraverseRoot{
+				Name: "foo",
+				SrcRange: hcl.Range{
+					Start: hcl.Pos{Line: 2, Column: 5, Byte: 5},
+					End:   hcl.Pos{Line: 2, Column: 8, Byte: 8},
+				},
+			},
+		},
+		SrcRange: hcl.Range{
+			Start: hcl.Pos{Line: 2, Column: 5, Byte: 5},
+			End:   hcl.Pos{Line: 2, Column: 8, Byte: 8},
+		},
+	}
+	if diff := cmp.Diff(wantAExpr, gotAExpr, cmpopts.IgnoreUnexported(hcl.TraverseRoot{})); diff != "" {
+		t.Errorf("wrong expression for a\n%s", diff)
+	}
+
+	bClosure := customdecode.ExpressionClosureFromVal(bVal)
+	gotBVal, diags := bClosure.Value()
+	wantBVal := cty.StringVal("foo value")
+	if diags.HasErrors() {
+		t.Fatalf("unexpected problems: %s", diags.Error())
+	}
+	if got, want := gotBVal, wantBVal; !want.RawEquals(got) {
+		t.Errorf("wrong 'b' result\ngot:  %#v\nwant: %#v", got, want)
+	}
+
+	if got, want := cVal, cty.StringVal("hello"); !want.RawEquals(got) {
+		t.Errorf("wrong 'c'\ngot:  %#v\nwant: %#v", got, want)
+	}
+
+	// One additional "trick" we can do with the expression closure is to
+	// evaluate the expression in a _derived_ EvalContext, rather than the
+	// captured one. This could be useful for introducing additional local
+	// variables/functions in a particular context, for example.
+	deriveCtx := bClosure.EvalContext.NewChild()
+	deriveCtx.Variables = map[string]cty.Value{
+		"foo": cty.StringVal("overridden foo value"),
+	}
+	gotBVal2, diags := bClosure.Expression.Value(deriveCtx)
+	wantBVal2 := cty.StringVal("overridden foo value")
+	if diags.HasErrors() {
+		t.Fatalf("unexpected problems: %s", diags.Error())
+	}
+	if got, want := gotBVal2, wantBVal2; !want.RawEquals(got) {
+		t.Errorf("wrong 'b' result with derived EvalContext\ngot:  %#v\nwant: %#v", got, want)
+	}
+}