Skip to content

Latest commit

 

History

History

week-3

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
 
 

Evaluating LLMs

Slides can be found here.

Lab Assignment

  1. Pick a topical area of some importance:
  • legal advice (wills, prenuptial agreements, e.g.)
  • medical or dietary advice
  • mental health and wellbeing
  • relationships
  • financial advice (pensions, crypto, saving for a down payment, gambling)
  • tell a joke on some theme
  1. Pick a contrast pair of languages, sociolects, dialects, or politeness

  2. Pick a pair of LLMs readily accessible to you (variants are a good idea!)

  3. Experiment to see if you are able to construct a pair of equivalent but stylistically different prompts that provoke at least one of the LLMs to deliver stylistically different but topically different responses.

  4. Remember this over the course of this class and retest with new models!

  5. This may become a baseline report for ELOQUENT!