feat: add evaluator testkit gherkin features and flag configuration

[aepfli]

Summary

Adds Gherkin feature files and flag configuration for the flagd-api-testkit — a compliance testkit for testing implementations of the flagd-api Evaluator interface (see java-sdk-contrib#1742).

Files added

evaluator/gherkin/ — 10 feature files split by concern

File	What it tests
evaluation.feature	Basic static resolution (Boolean, String, Integer, Float, Object)
zero-values.feature	Zero/falsy/empty values, static and targeted
no-default-variant.feature	Null/undefined defaultVariant edge cases
errors.feature	FLAG_NOT_FOUND, TYPE_MISMATCH error codes
fractional.feature	Fractional bucketing targeting rule
semver.feature	Semantic version comparison operators
string.feature	starts_with / ends_with string operators
targeting.feature	Targeting key resolution
evaluator-refs.feature	$ref shared evaluator reuse
metadata.feature	Per-flag metadata assertions

evaluator/flags/testkit-flags.json

Combined flagd flag configuration covering all scenarios above, including $evaluators for shared targeting rules.

How it's used

The flagd-api-testkit JAR (in java-sdk-contrib) bundles these files via git submodule at build time. Consumers depend only on the JAR — no submodule needed on their side.

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the flagd-api-testkit by introducing a comprehensive suite of Gherkin feature tests and a corresponding flag configuration. The primary goal is to ensure robust compliance testing for implementations of the flagd-api's Evaluator interface, covering a wide array of evaluation scenarios and edge cases. This addition provides a standardized and automated way to validate evaluator behavior, improving the reliability and consistency of flag evaluation across different SDKs.

Highlights

• New Gherkin Feature Files: Ten new Gherkin feature files were added to the evaluator/gherkin/ directory. These files are designed to test various aspects of the flagd-api-testkit's Evaluator interface, covering basic evaluation, zero values, default variant handling, error conditions, fractional bucketing, semantic versioning, string operators, targeting, shared evaluators, and metadata.
• Comprehensive Flag Configuration: A new testkit-flags.json file was introduced in evaluator/flags/. This file contains a combined flagd flag configuration that supports all the scenarios defined in the new Gherkin feature files, including the use of $evaluators for shared targeting rules.
• Testkit Integration: These new test assets are intended to be bundled with the flagd-api-testkit JAR via a git submodule at build time, enabling consumers to depend solely on the JAR without needing the submodule directly.

Changelog

• evaluator/flags/testkit-flags.json
  • Added a new JSON file containing a comprehensive flag configuration for the testkit, including various flag types, targeting rules, and shared evaluators.
• evaluator/gherkin/errors.feature
  • Added a new Gherkin feature file to test evaluator error handling for FLAG_NOT_FOUND and TYPE_MISMATCH conditions.
• evaluator/gherkin/evaluation.feature
  • Added a new Gherkin feature file to validate basic static flag resolution for boolean, string, integer, float, and object types.
• evaluator/gherkin/evaluator-refs.feature
  • Added a new Gherkin feature file to test the correct functionality of shared targeting rules using $ref.
• evaluator/gherkin/fractional.feature
  • Added a new Gherkin feature file to test the fractional bucketing operator, including shorthand notation and shared seed scenarios.
• evaluator/gherkin/metadata.feature
  • Added a new Gherkin feature file to validate flag metadata handling, covering cases where metadata is present or absent.
• evaluator/gherkin/no-default-variant.feature
  • Added a new Gherkin feature file to test the evaluator's behavior when a flag's defaultVariant is null or undefined.
• evaluator/gherkin/semver.feature
  • Added a new Gherkin feature file to test semantic version comparison operators (=, >, <, ~, ^).
• evaluator/gherkin/string.feature
  • Added a new Gherkin feature file to test string comparison operators, specifically starts_with and ends_with.
• evaluator/gherkin/targeting.feature
  • Added a new Gherkin feature file to test context-based targeting rules, including evaluation based on the targeting key.
• evaluator/gherkin/zero-values.feature
  • Added a new Gherkin feature file to validate the correct resolution of zero, falsy, and empty values in both static and targeted flags.

Activity

• No activity has been recorded for this pull request yet.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a comprehensive testkit for the flagd-api Evaluator interface, including a wide range of Gherkin feature files and a corresponding flag configuration JSON. The test scenarios cover basic evaluation, error handling, various operators, and edge cases, which is excellent for ensuring compliance of implementations. The structure is logical and the test cases are well-defined. I have one suggestion to improve the clarity of the flag configuration for fractional targeting, making it more explicit and easier to understand for consumers of this testkit.