CEL-Py API¶
Details of the CEL-Python implementation and the API to the various components.
Pure Python implementation of CEL.
Visible interface to CEL. This exposes the Environment
,
the Evaluator
run-time, and the celtypes
module
with Python types wrapped to be CEL compatible.
Example¶
Here’s an example with some details:
>>> import celpy
# A list of type names and class bindings used to create an environment.
>>> types = []
>>> env = celpy.Environment(types)
# Parse the code to create the CEL AST.
>>> ast = env.compile("355. / 113.")
# Use the AST and any overriding functions to create an executable program.
>>> functions = {}
>>> prgm = env.program(ast, functions)
# Variable bindings.
>>> activation = {}
# Final evaluation.
>>> try:
... result = prgm.evaluate(activation)
... error = None
... except CELEvalError as ex:
... result = None
... error = ex.args[0]
>>> result
DoubleType(3.14159...)
Another Example¶
See https://github.com/google/cel-go/blob/master/examples/simple_test.go
The model Go we’re sticking close to:
d := cel.Declarations(decls.NewVar("name", decls.String))
env, err := cel.NewEnv(d)
if err != nil {
log.Fatalf("environment creation error: %v\n", err)
}
ast, iss := env.Compile(`"Hello world! I'm " + name + "."`)
// Check iss for compilation errors.
if iss.Err() != nil {
log.Fatalln(iss.Err())
}
prg, err := env.Program(ast)
if err != nil {
log.Fatalln(err)
}
out, _, err := prg.Eval(map[string]interface{}{
"name": "CEL",
})
if err != nil {
log.Fatalln(err)
}
fmt.Println(out)
// Output:Hello world! I'm CEL.
Here’s the Pythonic approach, using concept patterned after the Go implementation:
>>> from celpy import *
>>> decls = {"name": celtypes.StringType}
>>> env = Environment(annotations=decls)
>>> ast = env.compile('"Hello world! I\'m " + name + "."')
>>> out = env.program(ast).evaluate({"name": "CEL"})
>>> print(out)
Hello world! I'm CEL.
- class celpy.__init__.Runner(environment: Environment, ast: Tree, functions: Dict[str, Callable[[...], BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType]]] | None = None)[source]¶
Abstract runner.
Given an AST, this can evaluate the AST in the context of a specific activation with any override function definitions.
- __init__(environment: Environment, ast: Tree, functions: Dict[str, Callable[[...], BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType]]] | None = None) None [source]¶
- new_activation(context: Mapping[str, BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType] | NameContainer]) Activation [source]¶
Builds the working activation from the environmental defaults.
- evaluate(activation: Mapping[str, BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType] | NameContainer]) BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType [source]¶
- class celpy.__init__.InterpretedRunner(environment: Environment, ast: Tree, functions: Dict[str, Callable[[...], BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType]]] | None = None)[source]¶
Pure AST expression evaluator. Uses
evaluation.Evaluator
class.Given an AST, this evauates the AST in the context of a specific activation.
The returned value will be a celtypes type.
Generally, this should raise an
CELEvalError
for most kinds of ordinary problems. It may raise anCELUnsupportedError
for future features.- evaluate(context: Mapping[str, BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType] | NameContainer]) BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType [source]¶
- class celpy.__init__.CompiledRunner(environment: Environment, ast: Tree, functions: Dict[str, Callable[[...], BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType]]] | None = None)[source]¶
Python compiled expression evaluator. Uses Python byte code and
eval()
.Given an AST, this evaluates the AST in the context of a specific activation.
Transform the AST into Python, uses
compile()
to create a code object. Useseval()
to evaluate.- __init__(environment: Environment, ast: Tree, functions: Dict[str, Callable[[...], BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType]]] | None = None) None [source]¶
- evaluate(activation: Mapping[str, BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType] | NameContainer]) BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType [source]¶
- class celpy.__init__.Int32Value(value: Any = 0)[source]¶
- static __new__(cls: Type[Int32Value], value: Any = 0) Int32Value [source]¶
TODO: Check range. This seems to matter for protobuf.
- class celpy.__init__.Environment(package: str | None = None, annotations: Dict[str, Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType] | Callable[[...], BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType] | Type[FunctionType]] | None = None, runner_class: Type[Runner] | None = None)[source]¶
Compiles CEL text to create an Expression object.
From the Go implementation, there are things to work with the type annotations:
type adapters registry make other native types available for CEL.
type providers registry make ProtoBuf types available for CEL.
- __init__(package: str | None = None, annotations: Dict[str, Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType] | Callable[[...], BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType] | Type[FunctionType]] | None = None, runner_class: Type[Runner] | None = None) None [source]¶
Create a new environment.
This also increases the default recursion limit to handle the defined minimums for CEL.
- Parameters:
package – An optional package name used to resolve names in an Activation
annotations –
Names with type annotations. There are two flavors of names provided here.
Variable names based on :py:mod:
celtypes
Function names, using
typing.Callable
.
runner_class – the class of Runner to use, either InterpretedRunner or CompiledRunner
- program(expr: Tree, functions: Dict[str, Callable[[...], BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType]]] | None = None) Runner [source]¶
Transforms the AST into an executable runner.
- activation() Activation [source]¶
Returns a base activation
__main__
¶
Pure Python implementation of CEL.
This provides a few jq-like, bc-like, and shell expr-like features.
jq
uses.
to refer the current document. By setting a package name of"jq"
and placing the JSON object in the package, we achieve similar syntax.bc
offers complex function definitions and other programming support. CEL can only evaluate a few bc-like expressions.This does everything
expr
does, but the syntax is slightly different. The output of comparisons – by default – is boolean, whereexpr
is an integer 1 or 0. Use-f 'd'
to see decimal output instead of Boolean text values.This does some of what
test
does, without a lot of the sophisticated file system data gathering. Use-b
to set the exit status code from a Boolean result.
TODO: This can also have a REPL, as well as process CSV files.
SYNOPSIS¶
python -m celpy [--arg name:type=value ...] [--null-input] expr
Options:
- –arg:
Provides argument names, types and optional values. If the value is not provided, the name is expected to be an environment variable, and the value of the environment variable is converted and used.
- –null-input:
Normally, JSON documents are read from stdin in ndjson format. If no JSON documents are provided, the
--null-input
option skips trying to read from stdin.- expr:
A CEL expression to evaluate.
JSON documents are read from stdin in NDJSON format (http://jsonlines.org/, http://ndjson.org/). For each JSON document, the expression is evaluated with the document in a default package. This allows .name to pick items from the document.
By default, the output is JSON serialized. This means strings will be JSON-ified and have quotes.
If a --format
option is provided, this is applied to the resulting object; this can be
used to strip quotes, or limit precision on double objects, or convert numbers to hexadecimal.
Arguments, Types, and Namespaces¶
CEL objects rely on the celtypes definitions.
Because of the close association between CEL and protobuf, some well-known protobuf types are also supported.
Further, type providers can be bound to CEL. This means an extended CEL
may have additional types beyond those defined by the Activation
class.
- celpy.__main__.arg_type_value(text: str) Tuple[str, Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType] | Callable[[...], BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType] | Type[FunctionType], BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType] [source]¶
Decompose
-a name:type=value
argument into a useful triple.Also accept
-a name:type
. This will findname
in the environment and convert to the requested type.Also accepts
-a name
. This will findname
in the environment and treat it as a string.Currently, names do not reflect package naming. An environment can be a package, and the activation can include variables that are also part of the package. This is not supported via the CLI.
Types can be celtypes class names or TYPE_NAME or PROTOBUF_TYPE
TYPE_NAME : "int64_value" | "null_value" | "uint64_value" | "double_value" | "bool_value" | "string_value" | "bytes_value" | "number_value" PROTOBUF_TYPE : "single_int64" | "single_int32" | "single_uint64" | "single_uint32" | "single_sint64" | "single_sint32" | "single_fixed64" | "single_fixed32" | "single_sfixed32" | "single_sfixed64" | "single_float" | "single_double" | "single_bool" | "single_string" | "single_bytes" | "single_duration" | "single_timestamp"
- Parameters:
text – Argument value
- Returns:
Tuple with name, annotation, and resulting object.
- celpy.__main__.get_options(argv: List[str] | None = None) Namespace [source]¶
Parses command-line arguments.
- class celpy.__main__.CEL_REPL(completekey='tab', stdin=None, stdout=None)[source]¶
- prompt = 'CEL> '¶
- intro = 'Enter an expression to have it evaluated.'¶
- logger = <Logger celpy.repl (WARNING)>¶
- cel_eval(text: str) BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType [source]¶
- do_set(args: str) bool [source]¶
Set variable expression
Evaluates the expression, saves the result as the given variable in the current activation.
- do_exit(args: str) bool ¶
Quits from the REPL.
- do_bye(args: str) bool ¶
Quits from the REPL.
- celpy.__main__.process_json_doc(display: Callable[[BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType]], None], prgm: Runner, activation: Dict[str, Any], variable: str, document: str, boolean_to_status: bool = False) int [source]¶
Process a single JSON document. Either one line of an NDJSON stream or the only document in slurp mode. We assign it to the variable “jq”. This variable can be the package name, allowing
.name
) to work. Or. It can be left as a variable, allowingjq
andjq.map(x, x*2)
to work.Returns status code 0 for success, 3 for failure.
- celpy.__main__.main(argv: List[str] | None = None) int [source]¶
Given options from the command-line, execute the CEL expression.
With –null-input option, only –arg and expr matter.
Without –null-input, JSON documents are read from STDIN, following ndjson format.
With the –slurp option, it reads one JSON from stdin, spread over multiple lines.
- If “–json-package” is used, each JSON document becomes a package, and
top-level dictionary keys become valid
.name
expressions. Otherwise, “–json-object” is the default, and each JSON document is assigned to a variable. The default name is “jq” to allow expressions that are similar tojq
but with a “jq” prefix.
celtypes¶
CEL Types: wrappers on Python types to provide CEL semantics.
This can be used by a Python module to work with CEL-friendly values and CEL results.
Examples of distinctions between CEL and Python:
Unlike Python
bool
, CELBoolType
won’t do some math.CEL has
int64
anduint64
subclasses of integer. These have specific ranges and raiseValueError
errors on overflow.
CEL types will raise ValueError
for out-of-range values and TypeError
for operations they refuse.
The evaluation
module can capture these exceptions and turn them into result values.
This can permit the logic operators to quietly silence them via “short-circuiting”.
In the normal course of events, CEL’s evaluator may attempt operations between a
CEL exception result and an instance of one of CEL types.
We rely on this leading to an ordinary Python TypeError
to be raised to propogate
the error. Or. A logic operator may discard the error object.
The evaluation
module extends these types with it’s own CELEvalError
exception.
We try to keep that as a separate concern from the core operator implementations here.
We leverage Python features, which means raising exceptions when there is a problem.
Types¶
See https://github.com/google/cel-go/tree/master/common/types
These are the Go type definitions that are used by CEL:
BoolType
BytesType
DoubleType
DurationType
IntType
ListType
MapType
NullType
StringType
TimestampType
TypeType
UintType
The above types are handled directly byt CEL syntax.
e.g., 42
vs. 42u
vs. "42"
vs. b"42"
vs. 42.
.
We provide matching Python class names for each of these types. The Python type names are subclasses of Python native types, allowing a client to transparently work with CEL results. A Python host should be able to provide values to CEL that will be tolerated.
A type hint of Value
unifies these into a common hint.
The CEL Go implementation also supports protobuf types:
dpb.Duration
tpb.Timestamp
structpb.ListValue
structpb.NullValue
structpb.Struct
structpb.Value
wrapperspb.BoolValue
wrapperspb.BytesValue
wrapperspb.DoubleValue
wrapperspb.FloatValue
wrapperspb.Int32Value
wrapperspb.Int64Value
wrapperspb.StringValue
wrapperspb.UInt32Value
wrapperspb.UInt64Value
These types involve expressions like the following:
google.protobuf.UInt32Value{value: 123u}
In this case, the well-known protobuf name is directly visible as CEL syntax.
There’s a google
package with the needed definitions.
Type Provider¶
A type provider can be bound to the environment, this will support additional types. This appears to be a factory to map names of types to type classes.
Run-time type binding is shown by a CEL expression like the following:
TestAllTypes{single_uint32_wrapper: 432u}
The TestAllTypes
is a protobuf type added to the CEL run-time. The syntax
is defined by this syntax rule:
member_object : member "{" [fieldinits] "}"
The member
is part of a type provider library,
either a standard protobuf definition or an extension. The field inits build
values for the protobuf object.
See https://github.com/google/cel-go/blob/master/test/proto3pb/test_all_types.proto
for the TestAllTypes
protobuf definition that is registered as a type provider.
This expression will describes a Protobuf uint32
object.
Type Adapter¶
So far, it appears that a type adapter wraps existing Go or C++ types with CEL-required methods. This seems like it does not need to be implemented in Python.
Numeric Details¶
Integer division truncates toward zero.
The Go definition of modulus:
// Mod returns the floating-point remainder of x/y.
// The magnitude of the result is less than y and its
// sign agrees with that of x.
https://golang.org/ref/spec#Arithmetic_operators
“Go has the nice property that -a/b == -(a/b).”
x y x / y x % y
5 3 1 2
-5 3 -1 -2
5 -3 -1 2
-5 -3 1 -2
Python definition:
The modulo operator always yields a result
with the same sign as its second operand (or zero);
the absolute value of the result is strictly smaller than
the absolute value of the second operand.
Here’s the essential rule:
x//y * y + x%y == x
However. Python //
truncates toward negative infinity. Go /
truncates toward zero.
To get Go-like behavior, we need to use absolute values and restore the signs later.
x_sign = -1 if x < 0 else +1
go_mod = x_sign * (abs(x) % abs(y))
return go_mod
Timzone Details¶
An implementation may have additional timezone names that must be injected into
the pendulum
processing. (Formerly dateutil.gettz()
.)
For example, there may be the following sequence:
A lowercase match for an alias or an existing timezone.
A titlecase match for an existing timezone.
The fallback, which is a +/-HH:MM string.
- celpy.celtypes.type_matched(method: Callable[[Any, Any], Any]) Callable[[Any, Any], Any] [source]¶
Decorates a method to assure the “other” value has the same type.
- celpy.celtypes.logical_condition(e: BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType, x: BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType, y: BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType) BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType [source]¶
CEL e ? x : y operator. Choose one of x or y. Exceptions in the unchosen expression are ignored.
Example:
2 / 0 > 4 ? 'baz' : 'quux'
is a “division by zero” error.
>>> logical_condition( ... BoolType(True), StringType("this"), StringType("Not That")) StringType('this') >>> logical_condition( ... BoolType(False), StringType("Not This"), StringType("that")) StringType('that')
- celpy.celtypes.logical_and(x: BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType, y: BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType) BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType [source]¶
Native Python has a left-to-right rule. CEL && is commutative with non-Boolean values, including error objects.
- celpy.celtypes.logical_not(x: BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType) BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType [source]¶
Native python not isn’t fully exposed for CEL types.
- celpy.celtypes.logical_or(x: BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType, y: BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType) BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType [source]¶
Native Python has a left-to-right rule: (True or y) is True, (False or y) is y. CEL || is commutative with non-Boolean values, including errors.
(x || false)
isx
, and(false || y)
isy
.Example 1:
false || 1/0 != 0
is a “no matching overload” error.
Example 2:
(2 / 0 > 3 ? false : true) || true
is a “True”
If the operand(s) are not BoolType, we’ll create an TypeError that will become a CELEvalError.
- class celpy.celtypes.BoolType(source: Any)[source]¶
Native Python permits unary operators on Booleans.
For CEL, We need to prevent -false from working.
- class celpy.celtypes.BytesType(source: str | bytes | Iterable[int] | BytesType | StringType, *args: Any, **kwargs: Any)[source]¶
Python’s bytes semantics are close to CEL.
- class celpy.celtypes.DoubleType(source: Any)[source]¶
Native Python permits mixed type comparisons, doing conversions as needed.
For CEL, we need to prevent mixed-type comparisons from working.
TODO: Conversions from string? IntType? UintType? DoubleType?
- static __new__(cls: Type[DoubleType], source: Any) DoubleType [source]¶
- __neg__() DoubleType [source]¶
-self
- __truediv__(other: Any) DoubleType [source]¶
Return self/value.
- __rtruediv__(other: Any) DoubleType [source]¶
Return value/self.
- celpy.celtypes.int64(operator: IntOperator) IntOperator [source]¶
Apply an operation, but assure the value is within the int64 range.
- class celpy.celtypes.IntType(source: Any, *args: Any, **kwargs: Any)[source]¶
A version of int with overflow errors outside int64 range.
features/integer_math.feature:277 “int64_overflow_positive”
>>> IntType(9223372036854775807) + IntType(1) Traceback (most recent call last): ... ValueError: overflow
>>> 2**63 9223372036854775808
features/integer_math.feature:285 “int64_overflow_negative”
>>> -IntType(9223372036854775808) - IntType(1) Traceback (most recent call last): ... ValueError: overflow
>>> IntType(DoubleType(1.9)) IntType(2) >>> IntType(DoubleType(-123.456)) IntType(-123)
- celpy.celtypes.uint64(operator: IntOperator) IntOperator [source]¶
Apply an operation, but assure the value is within the uint64 range.
- class celpy.celtypes.UintType(source: Any, *args: Any, **kwargs: Any)[source]¶
A version of int with overflow errors outside uint64 range.
Alternatives:
Option 1 - Use https://pypi.org/project/fixedint/
Option 2 - use array or struct modules to access an unsigned object.
Test Cases:
features/integer_math.feature:149 “unary_minus_no_overload”
>>> -UintType(42) Traceback (most recent call last): ... TypeError: no such overload
uint64_overflow_positive
>>> UintType(18446744073709551615) + UintType(1) Traceback (most recent call last): ... ValueError: overflow
uint64_overflow_negative
>>> UintType(0) - UintType(1) Traceback (most recent call last): ... ValueError: overflow
>>> - UintType(5) Traceback (most recent call last): ... TypeError: no such overload
- class celpy.celtypes.ListType(iterable=(), /)[source]¶
Native Python implements comparison operations between list objects.
For CEL, we prevent list comparison operators from working.
We provide an
__eq__()
and__ne__()
that gracefully ignore type mismatch problems, calling them not equal.See https://github.com/google/cel-spec/issues/127
An implied logical And means a singleton behaves in a distinct way from a non-singleton list.
- __hash__ = None¶
- __orig_bases__ = (typing.List[typing.Union[ForwardRef('BoolType'), ForwardRef('BytesType'), ForwardRef('DoubleType'), ForwardRef('DurationType'), ForwardRef('IntType'), ForwardRef('ListType'), ForwardRef('MapType'), NoneType, ForwardRef('StringType'), ForwardRef('TimestampType'), ForwardRef('UintType')]],)¶
- __parameters__ = ()¶
- class celpy.celtypes.MapType(items: Mapping[Any, Any] | Sequence[Tuple[Any, Any]] | None = None)[source]¶
Native Python allows mapping updates and any hashable type as a kay.
- CEL prevents mapping updates and has a limited domain of key types.
int, uint, bool, or string keys
We provide an
__eq__()
and__ne__()
that gracefully ignore type mismatch problems for the values, calling them not equal.See https://github.com/google/cel-spec/issues/127
An implied logical And means a singleton behaves in a distinct way from a non-singleton mapping.
- static valid_key_type(key: Any) bool [source]¶
Valid CEL key types. Plus native str for tokens in the source when evaluating
e.f
- __hash__ = None¶
- __orig_bases__ = (typing.Dict[typing.Union[ForwardRef('BoolType'), ForwardRef('BytesType'), ForwardRef('DoubleType'), ForwardRef('DurationType'), ForwardRef('IntType'), ForwardRef('ListType'), ForwardRef('MapType'), NoneType, ForwardRef('StringType'), ForwardRef('TimestampType'), ForwardRef('UintType')], typing.Union[ForwardRef('BoolType'), ForwardRef('BytesType'), ForwardRef('DoubleType'), ForwardRef('DurationType'), ForwardRef('IntType'), ForwardRef('ListType'), ForwardRef('MapType'), NoneType, ForwardRef('StringType'), ForwardRef('TimestampType'), ForwardRef('UintType')]],)¶
- __parameters__ = ()¶
- class celpy.celtypes.NullType[source]¶
Python’s None semantics aren’t quite right for CEL.
- __hash__ = None¶
- class celpy.celtypes.StringType(source: str | bytes | BytesType | StringType, *args: Any, **kwargs: Any)[source]¶
Python’s str semantics are very, very close to CEL.
We rely on the overlap between
"/u270c"
and"/U0001f431"
in CEL and Python.- static __new__(cls: Type[StringType], source: str | bytes | BytesType | StringType, *args: Any, **kwargs: Any) StringType [source]¶
- class celpy.celtypes.TimestampType(source: int | str | datetime, *args: Any, **kwargs: Any)[source]¶
Implements google.protobuf.Timestamp
See https://developers.google.com/protocol-buffers/docs/reference/google.protobuf
Also see https://www.ietf.org/rfc/rfc3339.txt.
The protobuf implementation is an ordered pair of int64 seconds and int32 nanos.
Instead of a Tuple[int, int] we use a wrapper for
datetime.datetime
.From protobuf documentation for making a Timestamp in Python:
now = time.time() seconds = int(now) nanos = int((now - seconds) * 10**9) timestamp = Timestamp(seconds=seconds, nanos=nanos)
Also:
>>> t = TimestampType("2009-02-13T23:31:30Z") >>> repr(t) "TimestampType('2009-02-13T23:31:30Z')" >>> t.timestamp() 1234567890.0 >>> str(t) '2009-02-13T23:31:30Z'
Timezones
Timezones are expressed in the following grammar:
TimeZone = "UTC" | LongTZ | FixedTZ ; LongTZ = ? list available at http://joda-time.sourceforge.net/timezones.html ? ; FixedTZ = ( "+" | "-" ) Digit Digit ":" Digit Digit ; Digit = "0" | "1" | ... | "9" ;
Fixed timezones are explicit hour and minute offsets from UTC. Long timezone names are like Europe/Paris, CET, or US/Central.
The Joda project (https://www.joda.org/joda-time/timezones.html) says “Time zone data is provided by the public IANA time zone database.”
TZ handling and timestamp parsing is doine with the
pendulum
(https://pendulum.eustace.io) project.Additionally, there is a
TZ_ALIASES
mapping available in this class to permit additional timezone names. By default, the mapping is empty, and the only names available are those recognized bypendulum.timezone
.- TZ_ALIASES: Dict[str, str] = {}¶
- static __new__(cls: Type[TimestampType], source: int | str | datetime, *args: Any, **kwargs: Any) TimestampType [source]¶
- __add__(other: Any) TimestampType [source]¶
Timestamp + Duration -> Timestamp
- __radd__(other: Any) TimestampType [source]¶
Duration + Timestamp -> Timestamp
- __sub__(other: TimestampType) DurationType [source]¶
- __sub__(other: DurationType) TimestampType
Return self-value.
- classmethod tz_name_lookup(tz_name: str) tzinfo | None [source]¶
The
dateutil.tz.gettz()
may be extended with additional aliases.
- getDate(tz_name: StringType | None = None) IntType [source]¶
- getDayOfMonth(tz_name: StringType | None = None) IntType [source]¶
- getDayOfWeek(tz_name: StringType | None = None) IntType [source]¶
- getDayOfYear(tz_name: StringType | None = None) IntType [source]¶
- getMonth(tz_name: StringType | None = None) IntType [source]¶
- getFullYear(tz_name: StringType | None = None) IntType [source]¶
- getHours(tz_name: StringType | None = None) IntType [source]¶
- getMilliseconds(tz_name: StringType | None = None) IntType [source]¶
- getMinutes(tz_name: StringType | None = None) IntType [source]¶
- getSeconds(tz_name: StringType | None = None) IntType [source]¶
- class celpy.celtypes.DurationType(seconds: Any, nanos: int = 0, **kwargs: Any)[source]¶
Implements google.protobuf.Duration
https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#duration
The protobuf implementation is an ordered pair of int64 seconds and int32 nanos. Instead of a Tuple[int, int] we use a wrapper for
datetime.timedelta
.The definition once said this:
"type conversion, duration should be end with "s", which stands for seconds"
This is obsolete, however, considering the following issue.
See https://github.com/google/cel-spec/issues/138
This refers to the following implementation detail
// A duration string is a possibly signed sequence of // decimal numbers, each with optional fraction and a unit suffix, // such as "300ms", "-1.5h" or "2h45m". // Valid time units are "ns", "us" (or "µs"), "ms", "s", "m", "h".
The real regex, then is this:
[-+]?([0-9]*(\.[0-9]*)?[a-z]+)+
- MaxSeconds = 315576000000¶
- MinSeconds = -315576000000¶
- NanosecondsPerSecond = 1000000000¶
- scale: Dict[str, float] = {'d': 86400.0, 'h': 3600.0, 'm': 60.0, 'ms': 0.001, 'ns': 1e-09, 's': 1.0, 'us': 1e-06, 'µs': 1e-06}¶
- static __new__(cls: Type[DurationType], seconds: Any, nanos: int = 0, **kwargs: Any) DurationType [source]¶
- __add__(other: Any) DurationType [source]¶
This doesn’t need to handle the rich variety of TimestampType overloadds. This class only needs to handle results of duration + duration. A duration + timestamp is not implemented by the timedelta superclass; it is handled by the datetime superclass that implementes timestamp + duration.
- __radd__(other: Any) DurationType [source]¶
This doesn’t need to handle the rich variety of TimestampType overloadds.
Most cases are handled by TimeStamp.
- class celpy.celtypes.FunctionType[source]¶
We need a concrete Annotation object to describe callables to celpy. We need to describe functions as well as callable objects. The description would tend to shadow
typing.Callable
.An
__isinstance__()
method, for example, may be helpful for run-time type-checking.Superclass for CEL extension functions that are defined at run-time. This permits a formal annotation in the environment construction that creates an intended type for a given name.
This allows for some run-time type checking to see if the actual object binding matches the declared type binding.
Also used to define protobuf classes provided as an annotation.
We could define this as three overloads to cover unary, binary, and tertiary cases.
- __call__(*args: BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType, **kwargs: BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType) BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType [source]¶
Call self as a function.
- class celpy.celtypes.PackageType(items: Mapping[Any, Any] | Sequence[Tuple[Any, Any]] | None = None)[source]¶
A package of message types, usually protobuf.
TODO: This may not be needed.
- __parameters__ = ()¶
- class celpy.celtypes.MessageType(*args: BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType, **fields: BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType)[source]¶
An individual protobuf message definition. A mapping from field name to field value.
See Scenario: “message_literal” in the parse.feature. This is a very deeply-nested message (30? levels), but the navigation to “payload” field seems to create a default value at the top level.
- __init__(*args: BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType, **fields: BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType) None [source]¶
- __parameters__ = ()¶
- class celpy.celtypes.TypeType(value: Any = '')[source]¶
Annotation used to mark protobuf type objects. We map these to CELTypes so that type name testing works.
- type_name_mapping = {'BOOL': <class 'celpy.celtypes.BoolType'>, 'BYTES': <class 'celpy.celtypes.BytesType'>, 'DOUBLE': <class 'celpy.celtypes.DoubleType'>, 'INT32': <class 'celpy.celtypes.IntType'>, 'INT64': <class 'celpy.celtypes.IntType'>, 'STRING': <class 'celpy.celtypes.StringType'>, 'UINT32': <class 'celpy.celtypes.UintType'>, 'UINT64': <class 'celpy.celtypes.UintType'>, 'bool': <class 'celpy.celtypes.BoolType'>, 'bytes': <class 'celpy.celtypes.BytesType'>, 'double': <class 'celpy.celtypes.DoubleType'>, 'google.protobuf.Any': <class 'celpy.celtypes.MessageType'>, 'google.protobuf.DoubleValue': <class 'celpy.celtypes.DoubleType'>, 'google.protobuf.Duration': <class 'celpy.celtypes.DurationType'>, 'google.protobuf.FloatValue': <class 'celpy.celtypes.DoubleType'>, 'google.protobuf.Int32Value': <class 'celpy.celtypes.IntType'>, 'google.protobuf.Int64Value': <class 'celpy.celtypes.IntType'>, 'google.protobuf.Timestamp': <class 'celpy.celtypes.TimestampType'>, 'google.protobuf.UInt32Value': <class 'celpy.celtypes.UintType'>, 'google.protobuf.UInt64Value': <class 'celpy.celtypes.UintType'>, 'google.protobuf.Value': <class 'celpy.celtypes.MessageType'>, 'google.protubuf.Any': <class 'celpy.celtypes.MessageType'>, 'int': <class 'celpy.celtypes.IntType'>, 'list': <class 'celpy.celtypes.ListType'>, 'list_type': <class 'celpy.celtypes.ListType'>, 'map': <class 'celpy.celtypes.MapType'>, 'map_type': <class 'celpy.celtypes.MapType'>, 'null_type': <class 'NoneType'>, 'string': <class 'celpy.celtypes.StringType'>, 'uint': <class 'celpy.celtypes.UintType'>}¶
- __hash__ = None¶
evaluation¶
CEL Interpreter using the AST directly.
The general idea is to map CEL operators to Python operators and push the
real work off to Python objects defined by the celpy.celtypes
module.
CEL operator “+” is implemented by “_+_” function. We map this to operator.add()
.
This will then look for __add__() methods in the various celpy.celtypes.CELType
types.
In order to deal gracefully with missing and incomplete data,
exceptions are turned into first-class Result
objects.
They’re not raised directly, but instead saved as part of the evaluation so that
short-circuit operators can ignore the exceptions.
This means that Python exceptions like TypeError
, IndexError
, and KeyError
are caught and transformed into CELEvalError
objects.
The Resut
type hint is a union of the various values that are encountered
during evaluation. It’s a union of the celpy.celtypes.CELTypes
type and the
CELEvalError
exception.
Important
Debugging
If the os environment variable CEL_TRACE
is set, then detailed tracing of methods is made available.
To see the trace, set the logging level for celpy.Evaluator
to logging.DEBUG
.
- exception celpy.evaluation.CELSyntaxError(arg: Any, line: int | None = None, column: int | None = None)[source]¶
CEL Syntax error – the AST did not have the expected structure.
- exception celpy.evaluation.CELUnsupportedError(arg: Any, line: int, column: int)[source]¶
Feature unsupported by this implementation of CEL.
- exception celpy.evaluation.CELEvalError(*args: Any, tree: Tree | None = None, token: Token | None = None)[source]¶
CEL evaluation problem. This can be saved as a temporary value for later use. This is politely ignored by logic operators to provide commutative short-circuit.
We provide operator-like special methods so an instance of an error returns itself when operated on.
- with_traceback(tb: Any) CELEvalError [source]¶
Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.
- __neg__() CELEvalError [source]¶
- __add__(other: Any) CELEvalError [source]¶
- __sub__(other: Any) CELEvalError [source]¶
- __mul__(other: Any) CELEvalError [source]¶
- __truediv__(other: Any) CELEvalError [source]¶
- __floordiv__(other: Any) CELEvalError [source]¶
- __mod__(other: Any) CELEvalError [source]¶
- __pow__(other: Any) CELEvalError [source]¶
- __radd__(other: Any) CELEvalError [source]¶
- __rsub__(other: Any) CELEvalError [source]¶
- __rmul__(other: Any) CELEvalError [source]¶
- __rtruediv__(other: Any) CELEvalError [source]¶
- __rfloordiv__(other: Any) CELEvalError [source]¶
- __rmod__(other: Any) CELEvalError [source]¶
- __rpow__(other: Any) CELEvalError [source]¶
- __call__(*args: Any) CELEvalError [source]¶
Call self as a function.
- __hash__ = None¶
- celpy.evaluation.eval_error(new_text: str, exc_class: Type[BaseException] | Sequence[Type[BaseException]]) Callable[[TargetFunc], TargetFunc] [source]¶
Wrap a function to transform native Python exceptions to CEL CELEvalError values. Any exception of the given class is replaced with the new CELEvalError object.
- Parameters:
new_text – Text of the exception, e.g., “divide by zero”, “no such overload”) this is the return value if the
CELEvalError
becomes the result.exc_class – A Python exception class to match, e.g. ZeroDivisionError, or a sequence of exception classes (e.g. (ZeroDivisionError, ValueError))
- Returns:
A decorator that can be applied to a function to map Python exceptions to
CELEvalError
instances.
This is used in the
all()
andexists()
macros to silently ignore TypeError exceptions.
- celpy.evaluation.boolean(function: Callable[[...], BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType]) Callable[[...], BoolType] [source]¶
Wraps boolean operators to create CEL BoolType results.
- Parameters:
function – One of the operator.lt, operator.gt, etc. comparison functions
- Returns:
Decorated function with type coercion.
- celpy.evaluation.operator_in(item: BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType], container: BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType]) BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType] [source]¶
CEL contains test; ignores type errors.
During evaluation of
'elem' in [1, 'elem', 2]
, CEL will raise internal exceptions for'elem' == 1
and'elem' == 2
. TheTypeError
exceptions are gracefully ignored.During evaluation of
'elem' in [1u, 'str', 2, b'bytes']
, however, CEL will raise internal exceptions every step of the way, and an exception value is the final result. (NotFalse
from the one non-exceptional comparison.)It would be nice to make use of the following:
eq_test = eval_error("no such overload", TypeError)(lambda x, y: x == y)
It seems like
next(iter(filter(lambda x: eq_test(c, x) for c in container))))
would do it. But. It’s not quite right for the job.There need to be three results, something
filter()
doesn’t handle. These are the chocies:True. There was a item found. Exceptions may or may not have been found.
False. No item found AND no expceptions.
CELEvalError. No item found AND at least one exception.
To an extent this is a little like the
exists()
macro. We can think ofcontainer.contains(item)
ascontainer.exists(r, r == item)
. However, exists() tends to silence exceptions, where this can expost them.
- celpy.evaluation.function_size(container: BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType]) BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType] [source]¶
The size() function applied to a Value. Delegate to Python’s
len()
.(string) -> int string length (bytes) -> int bytes length (list(A)) -> int list size (map(A, B)) -> int map size
For other types, this will raise a Python
TypeError
. (This is captured and becomes anCELEvalError
Result.)
- class celpy.evaluation.Referent(ref_to: Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType] | Callable[[...], BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType] | Type[FunctionType] | None = None)[source]¶
A Name can refer to any of the following things:
Annotations – initially most names are these or a CELFunction that may implement a type. Must be provided as part of the initialization.
NameContainer – some names are these. This is true when the name is not provided as part of the initialization because we discovered the name during type or environment binding.
celpy.celtypes.Value – many annotations also have values. These are provided after Annotations, and require them.
CELEvalError – This seems unlikely, but we include it because it’s possible.
Functions – All of the type conversion functions are names in a NameContainer.
A name can be ambiguous and refer to both a nested
NameContainer
as well as acelpy.celtypes.Value
(usually a MapType instance.)Object
b
has two possible meanings:b.c
is a NameContainer forc
, a string.b
is a mapping, andb.c
is syntax sugar forb['c']
.
The “longest name” rule means that the useful value is the “c” object in the nested
NameContainer
. The syntax sugar interpretation is done in the rare case we can’t find theNameContainer
.>>> nc = NameContainer("c", celpy.celtypes.StringType) >>> b = Referent(celpy.celtypes.MapType) >>> b.value = celpy.celtypes.MapType({"c": "oops"}) >>> b.value == celpy.celtypes.MapType({"c": "oops"}) True >>> b.container = nc >>> b.value == nc True
In effect, this class is
Referent = Union[ Annotation, celpy.celtypes.Value, CELEvalError, CELFunction, ]
- __init__(ref_to: Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType] | Callable[[...], BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType] | Type[FunctionType] | None = None) None [source]¶
- property value: Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType] | Callable[[...], BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType] | Type[FunctionType] | BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Callable[[...], BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType]] | NameContainer¶
The longest-path rule means we prefer
NameContainer
over any locally defined value. Otherwise, we’ll provide a value if there is one. Finally, we’ll provide the annotation if there’s no value. :return:
- class celpy.evaluation.NameContainer(name: str | None = None, ref_to: Referent | None = None, parent: NameContainer | None = None)[source]¶
A namespace that fulfills the CEL name resolution requirement.
Scenario: "qualified_identifier_resolution_unchecked" "namespace resolution should try to find the longest prefix for the evaluator."
NameContainer instances can be chained (via parent) to create a sequence of searchable locations for a name.
Local-most is an Activation with local variables within a macro. These are part of a nested chain of Activations for each macro. Each local activation is a child with a reference to the parent Activation.
Parent of any local Activation is the overall Activation for this CEL evaluation. The overall Activation contains a number of NameContainers:
The global variable bindings.
Bindings of function definitions. This is the default set of functions for CEL plus any add-on functions introduced by C7N.
The run-time annotations from the environment. There are two kinds:
Protobuf message definitions. These are types, really.
Annotations for global variables. The annotations tend to be hidden by the values. They’re in the lookup chain to simplify access to protobuf messages.
The environment also provides the built-in type names and aliases for the
celtypes
package of built-in types.
This means name resolution marches from local-most to remote-most, searching for a binding. The global variable bindings have a local-most value and a more remote annotation. The annotations (i.e. protobuf message types) have only a fairly remote annotation without a value.
Structure.
A NameContainer is a mapping from names to Referents.
A Referent can be one of three things.
A NameContainer further down the path
An Annotation
An Annotation with a value.
Loading Names.
There are several “phases” to building the chain of
NameContainer
instances.The
Activation
creates the initialname : annotation
bindings. Generally, the names are type names, like “int”, bound toceltypes.IntType
. In some cases, the name is a future variable name, “resource”, bound toceltypes.MapType
.The
Activation
creates a secondNameContainer
that has variable names. This has a reference back to the parent to resolve names that are types.
This involves decomposing the paths of names to make a tree of nested
NameContainers
. Upper-level containers don’t (necessarily) have types or values – they’re merelyNameContainer
along the path to the target names.Resolving Names.
See https://github.com/google/cel-spec/blob/master/doc/langdef.md#name-resolution
There are three cases required in the
Evaluator
engine.Variables and Functions. These are
Result_Function
instances: i.e., ordinary values.Name.Name
can be navigation into a protobuf package, whenName
is protobuf package. The idea is to locate the longest possible match.If a.b is a name to be resolved in the context of a protobuf declaration with scope A.B, then resolution is attempted, in order, as A.B.a.b, A.a.b, and finally a.b. To override this behavior, one can use .a.b; this name will only be attempted to be resolved in the root scope, i.e. as a.b.
Name.Name
can be syntactic sugar for indexing into a mapping whenName
is a value ofMapType
or aMessageType
. It’s evaluated as if it wasName["Name"]
. This is a fall-back plan if the previous resolution failed.
The longest chain of nested packages should be resolved first. This will happen when each name is a
NameContainer
object containing otherNameContainer
objects.The chain of evaluations for
IDENT . IDENT . IDENT
is (in effect)member_dot(member_dot(primary(IDENT), IDENT), IDENT)
This makes the
member_dot
processing left associative.The
primary(IDENT)
resolves to a CEL object of some kind. Once theprimary(IDENT)
has been resolved, it establishes a context for subsequentmember_dot
methods.If this is a
MapType
or aMessageType
with an object, thenmember_dot
will pluck out a field value and return this.If this is a
NameContainer
or aPackageType
then themember_dot
will pluck out a sub-package orEnumType
orMessageType
and return the type object instead of a value. At some point amember_object
production will build an object from the type.
The evaluator’s
ident_value()
method resolves the identifier into theReferent
.Acceptance Test Case
We have two names
a.b -> NameContainer in which c = “yeah”. (i.e., a.b.c : “yeah”)
a.b -> Mapping with {“c”: “oops”}.
This means any given name can have as many as three meanings:
Primarily as a NameContainer. This resolves name.name.name to find the longest namespace possible.
Secondarily as a Mapping. This will be a fallback when name.name.name is really syntactic sugar for name.name[‘name’].
Finally as a type annotation.
- ident_pat = re.compile('[_a-zA-Z][_a-zA-Z0-9]*')¶
- extended_name_path = re.compile('^\\.?[_a-zA-Z][_a-zA-Z0-9]*(?:\\.[_a-zA-Z][_a-zA-Z0-9]*)*$')¶
- logger = <Logger celpy.NameContainer (WARNING)>¶
- __init__(name: str | None = None, ref_to: Referent | None = None, parent: NameContainer | None = None) None [source]¶
- load_annotations(names: Mapping[str, Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType] | Callable[[...], BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType] | Type[FunctionType]]) None [source]¶
Used by an
Activation
to build a container used to resolve long path names into nested NameContainers. Sets annotations for all supplied identifiers.{"name1.name2": annotation}
becomes two things:nc2 = NameContainer({“name2” : Referent(annotation)})
nc1 = NameContainer({“name1” : Referent(nc2)})
- Parameters:
names – A dictionary of {“name1.name1….”: Referent, …} items.
- load_values(values: Mapping[str, BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType] | NameContainer]) None [source]¶
Update annotations with actual values.
- exception NotFound[source]¶
Raised locally when a name is not found in the middle of package search. We can’t return
None
from find_name because that’s a valid value.
- static dict_find_name(some_dict: Dict[str, Referent], path: List[str]) BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType] [source]¶
Extension to navgiate into mappings, messages, and packages.
- Parameters:
some_dict – An instance of a MapType, MessageType, or PackageType.
path – names to follow into the structure.
- Returns:
Value found down inside the structure.
- find_name(path: List[str]) NameContainer | BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType] [source]¶
Find the name by searching down through nested packages or raise NotFound. This is a kind of in-order tree walk of contained packages.
- parent_iter() Iterator[NameContainer] [source]¶
Yield this NameContainer and all of its parents to create a flat list.
- resolve_name(package: str | None, name: str) Referent [source]¶
Search with less and less package prefix until we find the thing.
Resolution works as follows. If a.b is a name to be resolved in the context of a protobuf declaration with scope A.B, then resolution is attempted, in order, as
A.B.a.b. (Search for “a” in paackage “A.B”; the “.b” is handled separately.)
A.a.b. (Search for “a” in paackage “A”; the “.b” is handled separately.)
(finally) a.b. (Search for “a” in paackage None; the “.b” is handled separately.)
To override this behavior, one can use .a.b; this name will only be attempted to be resolved in the root scope, i.e. as a.b.
We Start with the longest package name, a
List[str]
assigned totarget
.Given a target, search through this
NameContainer
and all parents in theparent_iter()
iterable. The first name we find in the parent sequence is the goal. This is because values are first, type annotations are laast.If we can’t find the identifier with given package target, truncate the package name from the end to create a new target and try again. This is a bottom-up look that favors the longest name.
- Parameters:
package – Prefix string “name.name.name”
name – The variable we’re looking for
- Returns:
Name resolution as a Rereferent, often a value, but maybe a package or an annotation.
- clone() NameContainer [source]¶
- __orig_bases__ = (typing.Dict[str, celpy.evaluation.Referent],)¶
- __parameters__ = ()¶
- class celpy.evaluation.Activation(annotations: Mapping[str, Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType] | Callable[[...], BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType] | Type[FunctionType]] | None = None, package: str | None = None, vars: Mapping[str, BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType] | NameContainer] | None = None, parent: Activation | None = None)[source]¶
Namespace with variable bindings and type name (“annotation”) bindings.
Life and Content
An Activation is created by an Environment and contains the annotations (and a package name) from that Environment. Variables are loaded into the activation for evaluation.
A nested Activation is created each time we evaluate a macro.
An Activation contains a
NameContainer
instance to resolve identifers. (This may be a needless distinction and the two classes could, perhaps, be combined.)Chaining/Nesting
Activations can form a chain so locals are checked first. Activations can nest via macro evaluation, creating transient local variables.
``"[2, 4, 6].map(n, n / 2)"``
means nested activations with
n
bound to 2, 4, and 6 respectively. The resulting objects then form a resulting list.This is used by an
Evaluator
as follows:sub_activation: Activation = self.activation.nested_activation() sub_eval: Evaluator = self.sub_eval(sub_activation) sub_eval_partial: Callable[[Value], Value] = sub_eval.partial( tree_for_variable, tree_for_expression) push(celtypes.ListType(map(sub_eval_partial, pop()))
The
localized_eval()
creates a newActivation
and an associatedEvaluator
for this nested activation context. It uses theEvaluator.visit
method to evaluate the given expression for a new object bound to the given variable.Namespace Creation
We expand
{"a.b.c": 42}
to create nested namespaces:{"a": {"b": {"c": 42}}}
.This depends on two syntax rules to define the valid names:
member : primary | member "." IDENT ["(" [exprlist] ")"] primary : ["."] IDENT ["(" [exprlist] ")"]
Ignore the
["(" [exprlist] ")"]
options used for member functions. We have members and primaries, both of which depend on the following lexical rule:IDENT : /[_a-zA-Z][_a-zA-Z0-9]*/
Name expansion is handled in order of length. Here’s why:
Scenario: "qualified_identifier_resolution_unchecked" "namespace resolution should try to find the longest prefix for the evaluator."
Most names start with
IDENT
, but a primary can start with.
.- __init__(annotations: Mapping[str, Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType] | Callable[[...], BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType] | Type[FunctionType]] | None = None, package: str | None = None, vars: Mapping[str, BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType] | NameContainer] | None = None, parent: Activation | None = None) None [source]¶
Create an Activation.
The annotations are loaded first. The variables are loaded second, and placed in front of the annotations in the chain of name resolutions. Values come before annotations.
- Parameters:
annotations – Variables and type annotations. Annotations are loaded first to serve as defaults to create a parent NameContainer.
package – The package name to assume as a prefix for name resolution.
vars – Variables and their values, loaded to update the NameContainer.
parent – A parent activation in the case of macro evaluations.
- clone() Activation [source]¶
Create a clone of this activation with a deep copy of the identifiers.
- nested_activation(annotations: Mapping[str, Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType] | Callable[[...], BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType] | Type[FunctionType]] | None = None, vars: Mapping[str, BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType] | NameContainer] | None = None) Activation [source]¶
Create a nested sub-Activation that chains to the current activation. The sub-activations don’t have the same implied package context,
- Parameters:
annotations – Variable type annotations
vars – Variables with literals to be converted to the desired types.
- Returns:
An
Activation
that chains to this Activation.
- resolve_variable(name: str) BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType] | NameContainer [source]¶
Find the object referred to by the name.
An Activation usually has a chain of NameContainers to be searched.
A variable can refer to an annotation and/or a value and/or a nested container. Most of the time, we want the value attribute of the Referent. This can be a Result (a Union[Value, CelType])
- class celpy.evaluation.FindIdent[source]¶
Locate the ident token at the bottom of an AST.
This is needed to find the bind variable for macros.
It works by doing a “visit” on the entire tree, but saving the details of the
ident
nodes only.- __parameters__ = ()¶
- celpy.evaluation.trace(method: Callable[[Evaluator, Tree], Any]) Callable[[Evaluator, Tree], Any] [source]¶
Decorator to create consistent evaluation trace logging. This is generally applied to the methods matching parse rule names.
This only works for a class with a
level
attribute, likeEvaluator
.
- class celpy.evaluation.Evaluator(ast: Tree, activation: Activation, functions: Sequence[Callable[[...], BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType]]] | Mapping[str, Callable[[...], BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType]]] | None = None)[source]¶
Evaluate an AST in the context of a specific Activation.
See https://github.com/google/cel-go/blob/master/examples/README.md
General Evaluation.
An AST node must call
self.visit_children(tree)
explicitly to build the values for all the children of this node.Exceptions.
To handle
2 / 0 || true
, the||
,&&
, and?:
operators do not trivially evaluate and raise exceptions. They bottle up the exceptions and treat them as a kind of undecided value.Identifiers.
Identifiers have three meanings:
An object. This is either a variable provided in the activation or a function provided when building an execution. Objects also have type annotations.
A type annotation without an object, This is used to build protobuf messages.
A macro name. The
member_dot_arg
construct may have a macro. Plus theident_arg
construct may also have adyn()
orhas()
macro. See below for more.
Other than macros, a name maps to an
Referent
instance. This will have an annotation and – perhaps – an associated object.Names have nested paths.
a.b.c
is a mapping,a
, that contains a mapping,b
, that containsc
.MACROS ARE SPECIAL.
The macros do not all simply visit their children to perform evaluation. There are three cases:
dyn()
does effectively nothing. It visits it’s children, but also provides progressive type resolution through annotation of the AST.has()
attempts to visit the child and does a boolean transformation on the result. This is a macro because it doesn’t raise an exception for a missing member item reference, but instead maps an exception to False. It doesn’t return the value found, for a member item reference; instead, it maps this to True.The various
member.macro()
constructs do NOT visit children. They create a nested evaluation environment for the child variable name and expression.
The
member()
method implements the macro evaluation behavior. It does not always trivially descend into the children. In the case of macros, the member evaluates one child tree in the presence of values from another child tree using specific variable binding in a kind of stack frame.- logger = <Logger celpy.Evaluator (WARNING)>¶
- __init__(ast: Tree, activation: Activation, functions: Sequence[Callable[[...], BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType]]] | Mapping[str, Callable[[...], BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType]]] | None = None) None [source]¶
Create an evaluator for an AST with specific variables and functions.
- Parameters:
ast – The AST to evaluate.
activation – The variable bindings to use.
functions – The functions to use. If nothing is supplied, the default global base_functions are used. Otherwise a ChainMap is created so these local functions override the base functions.
- sub_evaluator(ast: Tree) Evaluator [source]¶
Build an evaluator for a sub-expression in a macro. :param ast: The AST for the expression in the macro. :return: A new Evaluator instance.
- set_activation(values: Mapping[str, BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType] | NameContainer]) Evaluator [source]¶
Chain a new activation using the given Context. This is used for two things:
Bind external variables like command-line arguments or environment variables.
Build local variable(s) for macro evaluation.
- ident_value(name: str, root_scope: bool = False) BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType] | Callable[[...], BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType]] [source]¶
Resolve names in the current activation. This includes variables, functions, the type registry for conversions, and protobuf packages, as well as protobuf types.
We may be limited to root scope, which prevents searching through alternative protobuf package definitions.
- evaluate() BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType [source]¶
Evaluate this AST and return the value or raise an exception.
There are two variant use cases.
External clients want the value or the exception.
Internally, we sometimes want to silence CELEvalError exceptions so that we can apply short-circuit logic and choose a non-exceptional result.
- visit_children(tree: Tree) List[BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType]] [source]¶
Extend the superclass to track nesting and current evaluation context.
- function_eval(name_token: Token, exprlist: Iterable[BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType]] | None = None) BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType] [source]¶
Function evaluation.
Object creation and type conversions.
Other built-in functions like size()
Extension functions
- method_eval(object: BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType], method_ident: Token, exprlist: Iterable[BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType]] | None = None) BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType] [source]¶
Method evaluation. While are (nominally) attached to an object, the only thing actually special is that the object is the first parameter to a function.
- macro_has_eval(exprlist: Tree) BoolType [source]¶
The has(e.f) macro.
https://github.com/google/cel-spec/blob/master/doc/langdef.md#field-selection
If e evaluates to a map, then has(e.f) indicates whether the string f is a key in the map (note that f must syntactically be an identifier).
If e evaluates to a message and f is not a declared field for the message, has(e.f) raises a no_such_field error.
If e evaluates to a protocol buffers version 2 message and f is a defined field:
If f is a repeated field or map field, has(e.f) indicates whether the field is non-empty.
If f is a singular or oneof field, has(e.f) indicates whether the field is set.
If e evaluates to a protocol buffers version 3 message and f is a defined field:
If f is a repeated field or map field, has(e.f) indicates whether the field is non-empty.
If f is a oneof or singular message field, has(e.f) indicates whether the field is set.
If f is some other singular field, has(e.f) indicates whether the field’s value is its default value (zero for numeric fields, false for booleans, empty for strings and bytes).
In all other cases, has(e.f) evaluates to an error.
- expr(tree: Tree) BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType] [source]¶
expr : conditionalor [“?” conditionalor “:” expr]
The default implementation short-circuits and can ignore a CELEvalError in the two alternative sub-expressions. The conditional sub-expression CELEvalError is propogated out as the result.
See https://github.com/google/cel-spec/blob/master/doc/langdef.md#logical-operators
> To get traditional left-to-right short-circuiting evaluation of logical operators, as in C or other languages (also called “McCarthy Evaluation”), the expression e1 && e2 can be rewritten e1 ? e2 : false. Similarly, e1 || e2 can be rewritten e1 ? true : e2.
- conditionalor(tree: Tree) BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType] [source]¶
conditionalor : [conditionalor “||”] conditionaland
The default implementation short-circuits and can ignore an CELEvalError in a sub-expression.
- conditionaland(tree: Tree) BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType] [source]¶
conditionaland : [conditionaland “&&”] relation
The default implementation short-circuits and can ignore an CELEvalError in a sub-expression.
- relation(tree: Tree) BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType] [source]¶
- relation[relation_lt | relation_le | relation_ge | relation_gt
- relation_eq | relation_ne | relation_in] addition
relation_lt : relation “<” relation_le : relation “<=” relation_gt : relation “>” relation_ge : relation “>=” relation_eq : relation “==” relation_ne : relation “!=” relation_in : relation “in”
This could be refactored into separate methods to skip the lookup.
Ideally:
values = self.visit_children(tree) func = functions[op_name_map[tree.data]] result = func(*values)
The AST doesn’t provide a flat list of values, however.
- addition(tree: Tree) BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType] [source]¶
addition : [addition_add | addition_sub] multiplication
addition_add : addition “+” addition_sub : addition “-”
This could be refactored into separate methods to skip the lookup.
Ideally:
values = self.visit_children(tree) func = functions[op_name_map[tree.data]] result = func(*values)
The AST doesn’t provide a flat list of values, however.
- multiplication(tree: Tree) BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType] [source]¶
multiplication : [multiplication_mul | multiplication_div | multiplication_mod] unary
multiplication_mul : multiplication “*” multiplication_div : multiplication “/” multiplication_mod : multiplication “%”
This could be refactored into separate methods to skip the lookup.
Ideally:
values = self.visit_children(tree) func = functions[op_name_map[tree.data]] result = func(*values)
The AST doesn’t provide a flat list of values, however.
- unary(tree: Tree) BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType] [source]¶
unary : [unary_not | unary_neg] member
unary_not : “!” unary_neg : “-”
This should be refactored into separate methods to skip the lookup.
ideally:
values = self.visit_children(tree) func = functions[op_name_map[tree.data]] result = func(*values)
But, values has the structure
[[], right]
- build_macro_eval(child: Tree) Callable[[BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType], Any] [source]¶
Builds macro function.
For example
[1, 2, 3].map(n, n/2)
Builds the function =
lambda n: n/2
.The function will expose exceptions, disabling short-circuit
||
and&&
.The child is a member_dot_arg construct:
[0] is the expression to the left of the ‘.’
[1] is the function, map, to the right of the .
[2] is the arguments in ()’s. Within this, there are two children: a variable and an expression.
- build_ss_macro_eval(child: Tree) Callable[[BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType], Any] [source]¶
Builds macro function for short-circuit logical evaluation ignoring exception values.
For example
[1, 2, 'hello'].exists(n, n >= 2)
Builds the function =
lambda n: n >= 2
.The function will swallow exceptions, enabling short-circuit
||
and&&
.
- build_reduce_macro_eval(child: Tree) Tuple[Callable[[BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType], BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType]], BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType]], Tree] [source]¶
Builds macro function and intiial expression for reduce().
For example
[0, 1, 2].reduce(r, i, 0, r + 2*i+1)
Builds the function =
lambda r, i: r + 2*i+1
and initial value = 0.The child is a member_dot_arg construct:
[0] is the expression to the left of the ‘.’
[1] is the function, reduce, to the right of the .
[2] is the arguments in ()’s. Within this, there are four children: two variables and two expressions.
- member(tree: Tree) BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType] [source]¶
member : member_dot | member_dot_arg | member_item | member_object | primary
member_dot : member “.” IDENT member_dot_arg : member “.” IDENT “(” [exprlist] “)” member_item : member “[” expr “]” member_object : member “{” [fieldinits] “}”
https://github.com/google/cel-spec/blob/master/doc/langdef.md#field-selection
- member_dot(tree: Tree) BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType] [source]¶
member : member_dot | member_dot_arg | member_item | member_object | primary
member_dot : member “.” IDENT member_dot_arg : member “.” IDENT “(” [exprlist] “)” member_item : member “[” expr “]” member_object : member “{” [fieldinits] “}”
https://github.com/google/cel-spec/blob/master/doc/langdef.md#name-resolution
primary
: Variables and Functions: some simple names refer to variables in the execution context, standard functions, or other name bindings provided by the CEL application.member_dot
: Field selection: appending a period and identifier to an expression could indicate that we’re accessing a field within a protocol buffer or map. See below for Field Selection.member_dot
: Protocol buffer package names: a simple or qualified name could represent an absolute or relative name in the protocol buffer package namespace. Package names must be followed by a message type, enum type, or enum constant.member_dot
: Protocol buffer message types, enum types, and enum constants: following an optional protocol buffer package name, a simple or qualified name could refer to a message type, and enum type, or an enum constant in the package’s namespace.
Field Selection. There are four cases.
https://github.com/google/cel-spec/blob/master/doc/langdef.md#field-selection
If e evaluates to a message and f is not declared in this message, the runtime error no_such_field is raised.
If e evaluates to a message and f is declared, but the field is not set, the default value of the field’s type will be produced.
If e evaluates to a map, then e.f is equivalent to e[‘f’].
In all other cases, e.f evaluates to an error.
TODO: implement member “.” IDENT for messages.
- member_dot_arg(tree: Tree) BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType] [source]¶
member : member_dot | member_dot_arg | member_item | member_object | primary
member_dot : member “.” IDENT member_dot_arg : member “.” IDENT “(” [exprlist] “)” member_item : member “[” expr “]” member_object : member “{” [fieldinits] “}”
https://github.com/google/cel-spec/blob/master/doc/langdef.md#field-selection
Method or macro? We Distinguish between these three similar cases.
Macros: https://github.com/google/cel-spec/blob/master/doc/langdef.md#macros
member “.” IDENT “(” [exprlist] “)” – used for string operations
member “.” IDENT “(” “)” – used for a several timestamp operations.
- member_index(tree: Tree) BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType] [source]¶
member : member_dot | member_dot_arg | member_item | member_object | primary
member_dot : member “.” IDENT member_dot_arg : member “.” IDENT “(” [exprlist] “)” member_item : member “[” expr “]” member_object : member “{” [fieldinits] “}”
https://github.com/google/cel-spec/blob/master/doc/langdef.md#field-selection
Locating an item in a Mapping or List
- __abstractmethods__ = frozenset({})¶
- __parameters__ = ()¶
- member_object(tree: Tree) BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType] [source]¶
member : member_dot | member_dot_arg | member_item | member_object | primary
member_dot : member “.” IDENT member_dot_arg : member “.” IDENT “(” [exprlist] “)” member_item : member “[” expr “]” member_object : member “{” [fieldinits] “}”
https://github.com/google/cel-spec/blob/master/doc/langdef.md#field-selection
An object constructor requires a protobyf type, not an object as the “member”.
- primary(tree: Tree) BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType] [source]¶
- primarydot_ident_arg | dot_ident | ident_arg | ident
- paren_expr | list_lit | map_lit | literal
dot_ident_arg : “.” IDENT “(” [exprlist] “)” dot_ident : “.” IDENT ident_arg : IDENT “(” [exprlist] “)” ident : IDENT paren_expr : “(” expr “)” list_lit : “[” [exprlist] “]” map_lit : “{” [mapinits] “}”
TODO: Refactor into separate methods to skip this complex elif chain. top-level
primary()
is similar tomethod()
. Each of the individual rules then works with a tree instead of a child of the primary tree.This includes function-like macros: has() and dyn(). These are special cases and cannot be overridden.
- literal(tree: Tree) BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType] [source]¶
Create a literal from the token at the top of the parse tree.
- exprlist(tree: Tree) BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType] [source]¶
exprlist : expr (“,” expr)*
- fieldinits(tree: Tree) BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType] [source]¶
fieldinits : IDENT “:” expr (“,” IDENT “:” expr)*
The even items, children[0::2] are identifiers, nothing to evaluate. The odd items, childnre[1::2] are expressions.
This creates a mapping, used by the
member_object()
method to create and populate a protobuf object. Duplicate names are an error.
- mapinits(tree: Tree) BoolType | BytesType | DoubleType | DurationType | IntType | ListType | MapType | None | StringType | TimestampType | UintType | CELEvalError | Type[BoolType] | Type[BytesType] | Type[DoubleType] | Type[DurationType] | Type[IntType] | Type[ListType] | Type[MapType] | Callable[[...], None] | Type[StringType] | Type[TimestampType] | Type[TypeType] | Type[UintType] | Type[PackageType] | Type[MessageType] [source]¶
mapinits : expr “:” expr (“,” expr “:” expr)*
Extract the key expr’s and value expr’s to a list of pairs. This raises an exception on a duplicate key.
TODO: Is
{'a': 1, 'b': 2/0}['a']
a meaningful result in CEL? Or is this an error because the entire member is erroneous?
- celpy.evaluation.celstr(token: Token) StringType [source]¶
Evaluate a CEL string literal, expanding escapes to create a Python string.
It may be that built-in
eval()
might work for some of this, but the octal escapes aren’t really viable.- Parameters:
token – CEL token value
- Returns:
str
parser¶
CEL Parser.
See https://github.com/google/cel-spec/blob/master/doc/langdef.md
https://github.com/google/cel-cpp/blob/master/parser/Cel.g4
https://github.com/google/cel-go/blob/master/parser/gen/CEL.g4
Builds a parser from the supplied cel.lark grammar.
Example:
>>> from celpy.celparser import CELParser
>>> p = CELParser()
>>> text2 = 'type(null)'
>>> ast2 = p.parse(text2)
>>> print(ast2.pretty().replace(" "," "))
expr
conditionalor
conditionaland
relation
addition
multiplication
unary
member
primary
ident_arg
type
exprlist
expr
conditionalor
conditionaland
relation
addition
multiplication
unary
member
primary
literal null
- exception celpy.celparser.CELParseError(*args: Any, line: int | None = None, column: int | None = None)[source]¶
- class celpy.celparser.CELParser[source]¶
Wrapper for the CEL parser and the syntax error messages.
- CEL_PARSER: Lark | None = None¶