Week 12: Data analysis: prediction, opacity, and review

Design stage: Data analysis. Match a computational method to an epistemic purpose, and work out what its output can and cannot support.

Prepare for class

Read: Chapter 8; prediction, interpretation, and the validity of AI-assisted analysis. The chapters are in Canvas, and the companion page has the materials that go with them.

Reading focus: Read the sections on supervised prediction, on predictive modeling and opacity, and on the validity of AI-assisted analysis. Identify what a held-out test can and cannot tell you.

Bring your analysis so far, and the dictionary and prompt-based labels from Milestone 3 side by side for the same records.

Weekly time budget

Outside class: 360 minutes (6 clock hours) maximum planned work per student. These allocations include all required preparation and submissions; they are planning estimates, not time logs for grading.

Finish one capacity exercise and one purpose exercise, with a baseline or comparison. Draft the AI-assisted validity statement. Simulation and embeddings are optional.

How the weekly cap works. If required work will exceed the budget, save your attempt and contact the instructor to adjust scope or arrange support.

From the book

Exercises 8.1.3 and 8.2.4 fit and interpret a predictive model. Exercise 8.3 is required on either route: it asks what kind of validity you can claim for the AI-assisted parts of your analysis, and ends in a statement you could put in a methods section.

• Exercise 8.1.3: Non-linear prediction. One of your choices for Milestone 4.
• Exercise 8.2.4: Model interpretation and epistemic opacity. One of your choices for Milestone 4.
• Exercise 8.3: Reflecting on AI-assisted validity. Required.
• Exercise 8.1.4: Word embeddings. Optional.
• Exercise 8.2.2: Measuring semantic dimensions. One of your choices for Milestone 4.
• Exercise 8.2.5: Integrative prediction and cross-domain evaluation. One of your choices for Milestone 4.

The book gives each exercise’s steps. What to submit, and when.

On the companion website: Chapter 8 materials · Supervised learning, gently · Computational analysis tools

Monday, March 29: Predict, interpret, and question the labels

12:45–2:00 PM · PV A203

Prediction route: finish Exercise 8.1.3 by comparing the linear baseline with a non-linear model on held-out data, then do Exercise 8.2.4 on feature importance, partial dependence, and opacity. Everyone: do Exercise 8.3 by comparing the model’s labels with your dictionary or hand labels and drafting the validity statement.

Check the reasoning: Check the majority-class baseline before reading any accuracy figure, and fit features on the training data only. High agreement with your own labels does not show that a model understands the concept.

Wednesday, March 31: Exercise review 3: your analysis

12:45–2:00 PM · PV A203

Show your analysis and its baseline or comparison in a short demonstration. A classmate states, from your output alone, what claim it supports and what would change that claim. Close with the instructor’s worked comparison on a shared example.

Check the reasoning: Compare your reviewer’s reading with your purpose statement. Where they differ, either the output or the statement needs to change.

Work to carry forward

Revise after the review and assemble Milestone 4. Note which result you will carry into the artifact, and whose uncertainty you can estimate.

Coming up: Milestone 4: Data analysis (15 points), due Wednesday, April 7.

Exercise review 3, ungraded. Bring: Your purpose statement, the network or predictive analysis with its baseline or comparison, and a draft of the AI-assisted validity statement. Your reviewer states, from your output alone, what claim it supports and what would change that claim. Compare their reading with your purpose statement. This work goes into Milestone 4. How the reviews work.

Assignments and deliverables · Your project · The book and companion website