Using AI
Organize the project so an agent can follow it, verify what the agent produces, and keep a record a reader can check.
AI assistance is part of the methods we study. You can use it to explain unfamiliar code, draft a search, critique a design, write and debug scripts, and classify documents. You are responsible for checking the result and for explaining why it belongs in your analysis.
The course follows the approach of Chapter 2 of the required text. Poor results from an AI agent usually come from a disorganized project, not a weak model, so you organize the project first. That same organization becomes your record of what the AI did.
Organize the project: rules, memory, knowledge
Use the Chapter 2 project skeleton as your project folder from Week 1. It keeps three things apart that an agent, or a reader, must not confuse.
| Dimension | The question it answers | Where it lives |
|---|---|---|
| Rules | How should work be done here? | AGENTS.md: what an agent may change, file conventions, what it must verify, and what to do when uncertain |
| Memory | Where does the project stand? | memory/CURRENT.md for direction and next steps, memory/DECISIONS.md for what you decided and why, memory/PROMPTLOG.md for how your prompts changed, and memory/runs/ for dated notes on each run |
| Knowledge | What does the project know? | kb/: concept definitions, data documentation, codebooks, and promptbooks |
Start with the three files Chapter 2 calls the minimal starting point: AGENTS.md, memory/CURRENT.md, and kb/README.md. They are due with Milestone 1. The companion’s Chapter 2 page shows worked examples of the rules file, the current-state file, and a promptlog entry, and includes a recording of the structure steering a real analysis.
Write rules an agent can follow. “Do not modify files in data/raw/” names the files and the restriction. “Be careful with the data” leaves both to the agent.
Verify before you rely on it
Start with a specific task and decide what a correct result would look like before you ask. Then check the output against something you can inspect.
- Summaries and claims. Ask the model to quote the passage behind each point, and check every claim you intend to use against the source.
- Searches and references. Run a suggested search yourself and judge the results. Do not ask a model to supply references; fabricated citations remain common.
- Code. Run generated code on a case small enough to verify by hand, and confirm it does what you intended and not something plausible but different. The companion’s coding tools page lists what to check in generated analysis code.
- Labels. Treat a coding prompt as a measurement instrument. Code a random sample yourself before you look at the model’s labels, then compare and read every disagreement. Report results for each class, and check whether the errors cluster in a group your question cares about.
Stress-test a prompt before you trust it. Chapter 6 has you run a deliberately flawed version of your prompt on the same documents, compare the label distributions, and read the documents that flipped. The companion’s prompting primer gives the protocol and the four ways a reasonable prompt goes wrong. Develop on about 20 documents, stress-test on a few hundred, and only then code the full corpus.
Freeze the first output before you correct it. Save the code, labels, or claim as first produced, check it, and then record what you kept, changed, or rejected. An error is not required for a useful record. If the output survives a meaningful check, describe the check. We evaluate the quality of your reasoning and verification, not the number of errors you claim to have caught.
Keep the record in the project
Your project folder is the record of your AI use. There is no separate log.
- Promptbook, under
kb/codebooks/: the prompt currently in force, with its output format, worked examples, the model name and version, and the settings. It is the machine coder’s codebook. - Promptlog, in
memory/PROMPTLOG.md: each substantive revision of a prompt, the audit finding that led to it, and what had to be rerun. - Decisions, in
memory/DECISIONS.md: what you decided, why, and what you rejected, including changes you made after feedback. - Run notes, under
memory/runs/: which promptbook version each coding run used, and what its validation showed. - Outputs. Save the prompt-and-output pairs themselves. You cannot pin a commercial model the way you pin a package version, so the saved outputs are what a reader can check.
Give each promptbook version an identifier so an earlier output can be traced to the instructions that produced it. Milestone 5 assembles these files into the documentation package of Exercise 10.2, which a classmate then tries to reproduce.
Disclose what a reader can check
End every milestone and reflection with a short disclosure. The final term paper is a milestone, so it carries one too. If you did not use AI, say so. If you did, name the tool and model, what it did, what you checked, and where the record is.
I used [tool and model] for [task]. I checked [evidence or test]. I changed or rejected [output] because [reason]. The prompts, settings, and outputs are in [files].
Chapter 11 sets the standard: a disclosure should make a verifiable record visible, not only assert good faith. Disclosure makes assistance visible. It does not substitute for checking the output.
Keep the interpretation yours
You should be able to explain the question, the data, the main analytical steps, and the limits of every finding you submit. Submitting unverified output, invented references or results, or work you cannot explain does not meet the assignment criteria.
Your reflections analyze your own experience. AI may help you edit wording, but the events, judgments, and learning you describe must be yours. The same holds for your study presentation: read both studies yourself, and check any summary a model gives you against the paper.
Choose tools that fit the data and the task
Do not put personal information, credentials, or restricted material into an AI service without the required authorization. Check the terms of any hosted notebook or model before you upload data to it. Chapter 11 asks a prior question of any corpus: do your uses preserve the norms under which its records were produced?
Consider whether automation is needed at all, whether a simpler method works, and what the choice costs in money, time, computation, and human review. Bias, privacy, environmental costs, and who bears the errors are part of evaluating a method.
No paid AI subscription is required. The companion’s setup guide describes access through your institution, a commercial provider, or a model that runs on your own computer. Whichever you use, record the model name, version, date, and settings. Ask the instructor for an alternative when you need one.