from ai_infra.eval import SemanticSimilarityEvaluate semantic similarity between output and expected output. Uses ai_infra.Embeddings to compute cosine similarity between the output and expected_output embeddings.
provider: Embedding provider (openai, google, voyage, cohere, huggingface). If None, auto-detects from environment. model: Embedding model name. Uses provider default if not specified. threshold: Minimum similarity score to pass (0.0-1.0). Default: 0.8. embeddings: Pre-configured Embeddings instance. If provided, `provider` and `model` are ignored.
>>> from ai_infra.eval.evaluators import SemanticSimilarity >>> from pydantic_evals import Case, Dataset >>> >>> dataset = Dataset( ... cases=[ ... Case( ... inputs="What is the capital of France?", ... expected_output="Paris is the capital", ... ), ... ], ... evaluators=[SemanticSimilarity(threshold=0.7)], ... )
EvaluationReason with: - value: float (similarity score 0.0-1.0) - reason: Explanation of the score and pass/fail