Moderating a conversation (LLMs and humans)
In this group chat, there is a LLM-based "Mediator" tasked with keeping the conversation on topic, and Mr. Ratburn, an experimenter.
- Go to ⚙️ Settings > Add a Gemini API key.
- Create a new experiment. Add a "Chat" stage.
- Add a "mediator," any LLM agent you'd like to participate in the conversation.
Here, we enable JSON parsing, which allows us to extract an additional
explanation from the API. This is rendered below the mediator's responses in
the screenshot above.
Without JSON parsing, you could do a prompt like:
Chime in if you think that the conversation is not about apples, to steer the conversation to apples. Otherwise, return with an empty string.
You can add as many agents as you'd like. Agents are called after every update to the conversation. If multiple agents would like to respond at the same time, one will be chosen randomly.
When previewing and monitoring an experiment, experimenters have access to an experimenter-only panel on the right.
Use the "manual chat" tab to directly interact with the participants.
Use the "API key" tab to update your Gemini API key.
Use the "LLM config" tab to update your mediator prompt(s); if you click "Save," your old prompt will be overwritten, and the new prompt will be used for all mediator responses moving forward (note that this will update across all your experiments).
NOTE: If the mediator repeatedly fails to respond, it may be due to a JSON parsing error. Try troubleshooting by adjusting the prompt or changing the prompt to non-JSON (where all non-empty string responses become chat messages).
Experimenters can also toggle "debug mode" (top right corner of the top banner) to view mediator response explanations (if JSON is enabled) in chat.
