Week 5: Data management: quality, workflow, and review
Could a classmate rebuild your dataset from your documentation alone?
Design stage: Data management. Examine how a source, its structure, and a standardized workflow shape the evidence you can use, then build and document your own corpus.
Prepare for class
Read: Revisit Chapters 3–4; no new chapter. The chapters are in Canvas, and the companion page has the materials that go with them.
Reading focus: Return to Chapter 3’s section on cheap data and costly validation before you assess your own records. Use Chapter 4’s sections on quality and workflow as the instructions for this week’s two exercises.
Bring your corpus, diagram, and codebook. Pick the fields that matter most to your question; those are the ones to check.
Weekly time budget
Outside class: 360 minutes (6 clock hours) maximum planned work per student. These allocations include all required preparation and submissions; they are planning estimates, not time logs for grading.
| Work | Planned minutes |
|---|---|
| Reading and discussion preparation | 60 |
| Exercises and project work, including troubleshooting | 210 |
| Writing, documentation, and reflection | 90 |
| Total | 360 |
Write the data-quality assessment from your own records and draw the workflow you really followed. The optional screening exercise fits only if the required pieces are done.
How the weekly cap works. If required work will exceed the budget, save your attempt and contact the instructor to adjust scope or arrange support.
From the book
Exercise 4.3 assesses the quality of your own records, and Exercise 4.4 documents the workflow that produced them. Chapter 4’s Reflective Exercises ask whether a classmate could rebuild your dataset from your documentation alone. Wednesday’s review tests exactly that.
- Exercise 4.3: Inspecting data quality in OpenAlex. Required.
- Exercise 4.4: Workflow. Required.
- Exercise 4.3.1: LLM-assisted screening with an agreement check. Optional.
The book gives each exercise’s steps. What to submit, and when.
On the companion website: Chapter 4 materials · Prompting and its failure modes, with a screening template · Data management tools
Monday, February 8: Data quality and workflow
12:45–2:00 PM · PV A203
Do Exercise 4.3 on your own records: look for missing identifiers, incomplete abstracts, and inconsistent names or dates, and decide how to handle each. Then draw the workflow you actually followed for Exercise 4.4 and organize the project directory to match it. The optional Exercise 4.3.1 screens the corpus with a model and checks it against records you screened by hand first.
Check the reasoning: Give one concrete example from your records for each quality problem you report. A problem without an example is a guess, and a handling plan without a count cannot be checked.
Wednesday, February 10: Exercise review 1: your corpus
12:45–2:00 PM · PV A203
Show the path from query to analysis table in a short demonstration. A classmate then follows your workflow documentation without asking you anything. Close with the instructor’s worked example of one corpus and its documentation.
Check the reasoning: Mark the first step your reviewer could not reproduce and fix its documentation now. Keep the earlier version so you can show what changed.
Work to carry forward
Revise the five Chapter 4 deliverables after the review and assemble Milestone 2. Draft Reflection 1 from what the source review and the peer review showed you.
Coming up: Milestone 2: Data management (15 points) and Reflection 1: data and setup (5 points), due Wednesday, February 17.
Exercise review 1, ungraded. Bring: Your retrieved corpus, source review, entity relationship diagram, codebook, data-quality assessment, and workflow diagram, in whatever state they are in. Your reviewer follows your workflow documentation without asking you anything and marks the first step they could not reproduce. This work goes into Milestone 2. How the reviews work.
Assignments and deliverables · Your project · The book and companion website