best_engine_ai_helper.validate_llm module

validate_llm — Ralph prose-loop gate for LLM text quality.

Validates that the selected text model can detect and fix charter violations in reference fixture texts. Two fixtures are tested:

DASH_EN — an English sentence with an em dash used as a punctuation aside (WRITING.md §5: no punctuation dashes).
SEAM_FR — two French paragraphs joined by “Par ailleurs” over a logic gap (ECRITURE.md: banned machine-tic transition).

The model passes the gate if both fixtures are cleaned within two passes of the prose Ralph loop (i.e., neither the em dash pattern nor “Par ailleurs” survives in the fixed output).

Author

Warith Harchaoui <warith.harchaoui@deraison.ai>

best_engine_ai_helper.validate_llm.validate(llm_chat)[source]

Run the prose-loop gate on both text fixtures.

Uses the prose Ralph loop from ralph.prose_loop to refine each fixture. The result is checked for the specific charter violations that were seeded in the fixture. Both fixtures must pass for the gate to return True.

Parameters:: llm_chat (callable) – The chat function from llm.py, or a compatible mock.
Returns:: True if both fixtures are free of their seeded violations after at most two prose-loop passes; False otherwise.
Return type:: bool

Examples

>>> def stubborn_model(p, **kw):
...     return {"needs_fix": False, "reasons": []}  # claims nothing needs fixing
>>> validate(stubborn_model)  # the seeded violations survive, so the gate fails
False