dspy.ai Metrics Project

This project is designed to help you learn and experiment with dspy.ai by adding metrics to a simple reasoning task using different language models (LLMs and SLMs).

Project Overview

We'll create a simple question for a chain of thought response to a Language Model and create metrics for what we expect the response to be.

Files

• cot_slm.py
  Given a mathematical problem, use the Chain of Thought primitive, to reason over the answer. Sends this to a locally running SLM (Small Language Model) using Ollama's Llama3.2-1b and prints the response.

• cot.py
  Given a mathematical problem, use the Chain of Thought primitive, to reason over the answer. Sends this to an OpenAI model (gpt-4o-mini) and prints the response.

Getting Started

1. Clone this repository
2. Create .env file and fill in OPENAI_API_KEY and CLAUDE_API_KEY API keys
3. Run scripts

python llm_chatresponse_openai.py

Resources

• dspy.ai Documentation
• OpenAI API
• Anthropic API
• Ollama