LLM qualitative evaluations and labeling

[Luca-Blight]

Description

It would be nice to have a place in the platform for this.
Another option would be to allow for integration with a partner that does provide it.

[samuelcolvin]

Yup, we're going on this very thing, see pydantic/pydantic-ai#915 and linked pull request.

[Luca-Blight]

That's awesome to see!

One thing that could be an interesting feature to have, particularly for online performance, is to enable the ability for another model to be set up as evaluator versus using a human.