Development Tools

The development effort is dependent on several parts of the CEL project.

1. The language specification. https://github.com/google/cel-spec/blob/master/doc/langdef.md

2. The test cases. https://github.com/google/cel-spec/tree/master/tests/simple/testdata

The language specification is transformed into a Lark grammar. This is in the src/cel.lark file. This changes very slowly. Any changes must be reflected (manually) by revising the lark version of the EBNF.

The test cases present a more challenging problem.

A tool, gherkinize.py, converts the test cases from Protobuf messages to Gherkin scenarios.

The gherkinize.py Tool

The gherkinize.py tool converts a .textproto test case collection into a Gherkin .feature file. This can be used to update the conformance tests in the features directory.

Synopsis

-o <output>, --output <output>

Where to write the feature file. Generally, it’s helpful to have the .textproto and .feature stems match. The Makefile assures this.

-s, --silent

No console output is produced

-v, --verbose

Verbose debugging output on the console.

source

A source .textproto file. This is often the path to a file in a local download of https://github.com/google/cel-spec/tree/master/tests/simple/testdata.

A URL for the source is not supported.

Description

Convert one .textproto file to a Gherkin .feature file.

Files

source:

A .textproto test case file from the cel-spec repository.

output:

A .feature file with the same stem as the source file is written to the output directory. basic.textproto will create basic.feature.

Examples

The basic.textproto starts like this:

name: "basic"
description: "Basic conformance tests that all implementations should pass."
section {
  name: "self_eval_zeroish"
  description: "Simple self-evaluating forms to zero-ish values."
  test {
    name: "self_eval_int_zero"
    expr: "0"
    value: { int64_value: 0 }
  }
  test {
    name: "self_eval_uint_zero"
    expr: "0u"
    value: { uint64_value: 0 }
  }

The basic.feature file created looks like this:

@conformance
Feature: basic
        Basic conformance tests that all implementations should pass.


# self_eval_zeroish -- Simple self-evaluating forms to zero-ish values.

Scenario: self_eval_zeroish/self_eval_int_zero

    When CEL expression '0' is evaluated
    Then value is celpy.celtypes.IntType(source=0)

Scenario: self_eval_zeroish/self_eval_uint_zero

    When CEL expression '0u' is evaluated
    Then value is celpy.celtypes.UintType(source=0)

The source .textproto files have a “section” heading which doesn’t have a precise parallel in the Gherkin language. The sections become comments in the .feature file, and the section name is used to prefix each feature name.

class gherkinize.Config(path: str)

This class reads in optional configuration for conformance tests. Each scenario is within a feature and a section.

	feature	section	scenario
example	string_ext	ascii_casing	lowerascii_unicode

The value for each scenario can be a string tag (which must begin with @), an array of tags (each of which must begin with @) or a dictionary with a tags key containing an array of tags (each of which… y’know).

For example, each of the following are valid:

[bindings_ext.bind]
bind_nested = "@wip"
boolean_literal = [ "@wip" ]

[bindings_ext.bind.macro_exists]
tags = [ "@wip" ]

In the future, dictionaries with additional features may be supported.

__init__(path: str) → None

tags_for(feature: str, section: str, scenario: str) → list[str]

Get a list of tags for a given scenario.

The features/steps Directory

The features/steps directory has step definition modules that implement the Given, When, and Then clauses.

Provides step definitions for the c7n_interface.feature. This is not part of the CEL language specification.

Provides step definitions for the expr_test_bc.feature, json_query.feature, neither of which are part of the CEL language specificaiton.

Provides step definitions for the features generated by the gherkinize.py tool.

The features/Makefile

This Makefile has the following targets:

 Available commands:
   %.textproto   copies the .textproto from the cel-spec directory
   %.feature     generates Gherkin feature from %.textproto source
   clean         cleans generated feature files
   clean         cleans generated feature files and textproto sources

Currently, the following feature files are built from the CEL specification.

basic.feature
bindings_ext.feature
block_ext.feature
comparisons.feature
conversions.feature
dynamic.feature
encoders_ext.feature
enums.feature
fields.feature
fp_math.feature
integer_math.feature
lists.feature
logic.feature
macros.feature
macros2.feature
math_ext.feature
namespace.feature
optionals.feature
parse.feature
plumbing.feature
proto2.feature
proto2_ext.feature
proto3.feature
string.feature
string_ext.feature
timestamps.feature
type_deduction.feature
unknowns.feature
wrappers.feature

The docs/Makefile

This is a Sphinx Makefile to build documentation. For more information, see https://www.sphinx-doc.org/en/master/index.html

Sphinx v8.1.3
Please use `make target' where target is one of
  html        to make standalone HTML files
  dirhtml     to make HTML files named index.html in directories
  singlehtml  to make a single large HTML file
  pickle      to make pickle files
  json        to make JSON files
  htmlhelp    to make HTML files and an HTML help project
  qthelp      to make HTML files and a qthelp project
  devhelp     to make HTML files and a Devhelp project
  epub        to make an epub
  latex       to make LaTeX files, you can set PAPER=a4 or PAPER=letter
  latexpdf    to make LaTeX and PDF files (default pdflatex)
  latexpdfja  to make LaTeX files and run them through platex/dvipdfmx
  text        to make text files
  man         to make manual pages
  texinfo     to make Texinfo files
  info        to make Texinfo files and run them through makeinfo
  gettext     to make PO message catalogs
  changes     to make an overview of all changed/added/deprecated items
  xml         to make Docutils-native XML files
  pseudoxml   to make pseudoxml-XML files for display purposes
  linkcheck   to check all external links for integrity
  doctest     to run all doctests embedded in the documentation (if enabled)
  coverage    to run coverage check of the documentation (if enabled)
  clean       to remove everything in the build directory

The Project Makefile

A top-level Makefile has a number of phony targets:

 Available commands:
   build         runs uv build to create a distribution kit
   build-all     alias of build, conformance, and docs
 
   test          runs the Python 3.12 test environment to execute a quick test
   test-all      runs the full test suite
   test-<env>    runs tests for any of the available tox environments:
 
     default environments:
     py39                     -> test suite
     py310                    -> test suite
     py311                    -> test suite
     py312                    -> test suite
     py313                    -> test suite
     lint                     -> check types, formatting, and best practices
     tools                    -> conformance suite conversion tools
     
     additional environments:
     wip                      -> work-in-progress test suite
     conformance              -> test suite
     conformance-compiled     -> compiled runner
     conformance-wip          -> work in progress
     conformance-wip-compiled -> work in progress + compiled runner
     coverage                 -> generate coverage reports
 
   conformance   generates conformance tests
   conf-clean    cleans generated conformance tests
 
   docs          generates HTML documentation
   docs-clean    lints documentation sources
   docs-clean    cleans generated HTML documentation
 
   lint          runs code coverage, type hint checking, and other lint checks
                 (alias of test-lint)
 
   format        runs code formatting
   coverage      generates code coverage reports
   clean         cleans all content ignored by git
   clean-all     alias of clean, conformance-clean, and docs-clean
   benchmarks    runs performance benchmarks