Environment properties
Six yes/no questions about the world an agent lives in. The answers tell you how hard the agent is to build and which parts its architecture will need.
Kind: toolkit.
Week: 1 (slides 25 to 28, appendix slides 39 to 44).
Book: Wooldridge p. 25 (the Environments box) and p. 26.
Techniques on this page: the six properties, classifying an environment, reading off what the architecture needs.
The problem it solves
You can’t judge an agent design without knowing its world. A rule-follower that’s perfect for Sudoku is hopeless in traffic. These six properties are the standard way to describe the world, and the lecture’s table (Technique 3) links each “hard” answer to a part the agent must have: memory, planning, fast reaction, models of other agents.
Expect “classify this environment and justify each property” in the quiz and the exam.
Words you’ll hear
| Word | Plain meaning |
|---|---|
| accessible / fully observable | the agent can get complete, accurate, up-to-date information about the state |
| deterministic | each action has one guaranteed effect |
| stochastic / non-deterministic | the same action can lead to different results |
| episodic | decisions come in separate episodes; one doesn’t affect the next |
| sequential | today’s decision changes tomorrow’s situation |
| static | nothing changes except through the agent’s own actions |
| dynamic | other processes change the world while the agent thinks |
| semi-dynamic | the world doesn’t change, but the agent’s score does with time (chess with a clock) |
| discrete | a fixed, finite number of actions and percepts |
Technique 1: the six properties
Problem it solves: a fixed vocabulary for “how hard is this world?”.
| Property | Question to ask | Harder end |
|---|---|---|
| fully vs partially observable | can it see everything relevant, accurately and now? | partially |
| deterministic vs stochastic | does each action have one guaranteed effect? | stochastic |
| episodic vs sequential | are decisions independent of each other? | sequential |
| static vs dynamic | does the world change only when the agent acts? | dynamic |
| discrete vs continuous | finitely many actions and percepts? | continuous |
| single- vs multi-agent | does it share the world with other decision-makers? | multi-agent |
Wooldridge’s summary (p. 25): the hardest environments are inaccessible, non-deterministic, dynamic and continuous. The physical world and the Internet are both.
Episodic, carefully. A part inspector on an assembly line is episodic: judging this part doesn’t affect whether the next one is faulty. Chess and driving are sequential: a move now has consequences later. Episodic is easier because the agent never has to think ahead.
Technique 2: classify an environment
Problem it solves: the exam question itself. Go property by property and give a one-line reason for each.
Worked example: the lecture’s pair (slide 28).
| Property | Sudoku | Flight booking agent |
|---|---|---|
| observable | fully | partially (can’t see other customers or future prices) |
| agents | single | multi (other bookers, airline pricing systems) |
| deterministic? | deterministic | stochastic (the seat may go before you click) |
| episodic? | sequential (each number constrains later ones) | sequential (leg 1 constrains leg 2) |
| static? | static | dynamic (prices change while it decides) |
| discrete? | discrete | discrete (finite flights and actions) |
Worked example: more to practise on.
| Environment | Observable | Deterministic | Episodic | Static | Discrete | Agents |
|---|---|---|---|---|---|---|
| crossword | fully | yes | sequential | static | discrete | single |
| chess with a clock | fully | yes | sequential | semi-dynamic | discrete | multi |
| poker | partially | stochastic | sequential | static | discrete | multi |
| taxi driving | partially | stochastic | sequential | dynamic | continuous | multi |
| image classifier | fully (for its task) | yes | episodic | static | continuous (pixels) | single |
| spam filter | partially | stochastic | episodic | dynamic | discrete | arguably multi (spammers adapt) |
Many answers depend on how you model the world. State your assumption in a phrase: “thermostat: discrete, if we model temperature as too cold / OK”. That’s exactly how the book models it (p. 36).
Multi-agent usually means dynamic. From one agent’s point of view, the others change the world in ways it doesn’t control, which is the definition of dynamic.
Technique 3: from environment to architecture
Problem it solves: connects this page to every later week. Each hard property calls for a particular part.
| If the environment… | the agent may need… | You’ll meet it in |
|---|---|---|
| reveals only part of the state | internal state, memory, beliefs, a world model | formal model (see/next), BDI beliefs |
| has uncertain action outcomes | reasoning under uncertainty, learning | expected utility; later RL weeks |
| has consequences that unfold over time | planning, thinking about long-term effects | STRIPS planning |
| changes while the agent is deciding | fast reaction, continual updating | reconsideration, subsumption |
| contains other decision-makers | models of others, communication, coordination | Part II of the module |
Worked example: an automated stock-trading agent.
Partially observable (other traders’ intentions are hidden), stochastic (prices), sequential (positions carry over), dynamic (the market moves while it thinks), discrete in practice (prices in ticks), multi-agent (other traders).
So it needs: a world model of the market (partial observability), fast reaction (dynamic), and models of other traders (multi-agent).
What to read in the book
| Section | Read? | Why |
|---|---|---|
| Environments box, p. 25 | Read | The slides’ appendix (slides 39 to 44) is this almost word for word |
| p. 26, episodic and real time | Read | The book’s wording for episodic vs non-episodic |
| Russell & Norvig ch. 2 (not your book) | Optional | Where the slides take the property names from, including semi-dynamic |
The book says accessible / inaccessible where the slides say fully / partially observable, and non-deterministic where they say stochastic. Same ideas.
Practice prompt
I'm studying CS5940 Intelligent Agents (Wooldridge, An Introduction to MultiAgent Systems, 2nd ed., p. 25; Russell & Norvig ch. 2). I want practice on ENVIRONMENT PROPERTIES:
- the six properties: fully/partially observable (accessible), deterministic/stochastic, episodic/sequential, static/dynamic (and semi-dynamic), discrete/continuous, single/multi-agent
- classifying a described environment on all six, with a one-line justification each (games, robots, web agents, trading, chatbots, assembly lines...)
- spotting answers that depend on how the environment is modelled, and stating the assumption
- for a given environment, which architecture components it calls for (memory or a world model, planning, fast reaction, models of other agents)
- why multi-agent environments are effectively dynamic, and why episodic is easier than sequential
Please:
1. Use plain, everyday language and explain any technical term in one short sentence.
2. Give me ONE environment or question at a time, easy first, then harder.
3. Wait for my answer, then mark each property and show a model answer with reasons.
4. If I get one wrong, explain why and give me a similar case.
5. Every 5 questions, summarise my strengths and what to practise.