Practical 1: the chat loop

Technique 1: setup and the first request

Problem it solves: talk to the course model safely, without leaking your key.

source ~/.config/csx940/env     # sets OPENAI_BASE_URL and OPENAI_API_KEY
python -m venv .venv
source .venv/bin/activate
pip install litellm

import os
from litellm import completion

response = completion(
    model="openai/qwen3-14b",
    api_base=os.environ["OPENAI_BASE_URL"],   # where the server is
    api_key=os.environ["OPENAI_API_KEY"],     # the credential
    messages=[{"role": "user", "content": "What is the capital of France?"}],
)
print(response.choices[0].message.content)

Read the last line left to right: the list of generated choices (usually one), the first, its message, its text content. The response also carries usage (prompt, completion and total tokens).

• Why a venv? It keeps this project’s packages apart from everything else, so installs don’t break each other.
• Why not put the key in the code? Code gets shared, screenshotted and committed to Git. An environment variable keeps the secret out of the file.

Technique 2: the statelessness experiment

Problem it solves: shows you, by experiment, where memory does and doesn’t live.

1. Send "My favourite module is CS3940."
2. Change the message to "What is my favourite module?" and run again.

The model doesn’t know. The second request contained only the second message. It may say so, or guess (which is why you inspect rather than trust).

Key idea: the model can only use the context your application sends with the current request.

Web chat apps seem to remember because the app resends the conversation every time.

Technique 3: conversation history with roles

Problem it solves: gives the model a memory, by sending the whole conversation each time.

messages = []
messages.append({"role": "user", "content": "My favourite module is CS3940."})
response = completion(model=..., api_base=..., api_key=..., messages=messages)
answer = response.choices[0].message.content
messages.append({"role": "assistant", "content": answer})

Where is the memory? In the messages list. Not in the model, not in LiteLLM.

Why store the model’s replies too? Without them the model sees a user talking to nobody. It can’t resolve “what did you mean by that?”, repeats itself, and loses questions it asked (which the study-assistant challenge needs).

Technique 4: the system message

Problem it solves: sets behaviour for the whole session, separately from what the user types.

messages = [
    {"role": "system", "content": "You are a pirate. Answer every question like a pirate."},
    {"role": "user", "content": "Explain what a Python list is."},
]

Same question, different system message, different style. Try “one sentence”, “explain to a 10-year-old”, “at most 30 words”. The model usually complies but not always: instructions influence behaviour, they don’t guarantee it.

Study-assistant challenge: where should the topic the user picks go? Best in the system message (f"... The topic is {topic}."), because it should hold for the whole session and stays at the top of the context.

Technique 5: the interactive loop

Problem it solves: a real conversation, one turn after another.

messages = []
while True:
    user_message = input("You: ")
    if user_message.strip().lower() == "quit":
        break
    messages.append({"role": "user", "content": user_message})
    response = completion(model=..., api_base=..., api_key=..., messages=messages)
    answer = response.choices[0].message.content
    messages.append({"role": "assistant", "content": answer})
    print("Assistant:", answer)

get input → add to messages → send → receive → store → repeat. The practical’s version uses .lower() only; adding .strip() also catches " Quit ".

In the lecture’s terms: observation = what the user types; memory = the messages list (outside the model); objective = the system prompt; decision = model inference; learning = none; actions = printing text. No tools and no choice of control flow, so it is not yet an agent (see agent vs workflow).

Technique 6: the cost of a growing context

Problem it solves: explains why long chats get slow and expensive, and why week 2 trims the history.

Say the system prompt is \(s\) tokens, each user message \(u\) and each reply \(a\). Request \(t\) carries the system prompt, \(t\) user messages and \(t - 1\) replies:

\[\text{tokens sent at turn } t = s + t\,u + (t-1)\,a\]

Adding up \(T\) turns, with \(\sum_{t=1}^T t = \frac{T(T+1)}{2}\):

\[\text{total} = T s + u\,\frac{T(T+1)}{2} + a\,\frac{T(T-1)}{2}\]

That is quadratic in \(T\): double the conversation and the input tokens roughly quadruple.

The practical’s rules, and why

1. Use the LLM only through the interfaces provided.
2. No unrestricted web access for the LLM.
3. Never automatically run code the LLM wrote. Treat it as untrusted input.
4. No secrets or personal data in prompts.
5. Credentials outside source code.
6. Inspect what the system does; a response is not automatically correct.

These limit what the agent can actually do (design dimension 6), which is the delegation problem handled in practice.

Practice prompt

I'm studying CS5940 Intelligent Agents. The week 1 practical builds a chat program in Python with LiteLLM (model "openai/qwen3-14b", OPENAI_BASE_URL and OPENAI_API_KEY in environment variables). I want practice on THE LLM CHAT LOOP:
- reading response.choices[0].message.content and the usage fields
- why an API call is stateless, and predicting what the model says in a two-run experiment
- building the messages list with system / user / assistant roles, and why the assistant's replies must be stored
- where to put instructions (system message vs first user message)
- writing and debugging the input loop (quit handling, .strip().lower())
- token arithmetic: tokens sent at turn t = s + t*u + (t-1)*a, and the quadratic total over T turns
- why secrets stay in environment variables, why venvs, and why generated code must never be run automatically
- whether this program is an agent or a workflow

Please:
1. Use plain, everyday language and explain any technical term in one short sentence.
2. Give me ONE question at a time, easy first, then harder; include short "what does this code print / what's the bug" questions.
3. Wait for my answer, then mark it and show a model answer (with code where useful).
4. If I get it wrong, explain the mistake and give me a similar question.
5. Every 5 questions, summarise my strengths and what to practise.