When AI Gets It Wrong: Replication Prediction Project

A semester-long student project to predict study replication, expose overfitting, and teach responsible skepticism about AI.

Introduction: Build a Project That Tests AI, Not Just Uses It

The fastest way to understand the limits of machine learning is not to memorize theory, but to build a system that sometimes fails in public. This semester-long student project asks one simple, powerful question: can a model predict whether a study will replicate, and if not, why not? That question turns abstract ideas like model evaluation, responsible AI governance, and research integrity into something students can measure, debate, and present.

The angle matters because AI often sounds more certain than it should. When a model outputs a probability, students may mistake confidence for truth, even when the underlying signal is weak, biased, or incomplete. A project on replication prediction teaches a healthier habit: use AI as a decision-support tool, then pressure-test its conclusions with skeptical, documented analysis. That is the exact mindset employers want in analysts, data scientists, research assistants, and product teams.

As grounding context, a recent New York Times report described a major study suggesting that AI is not yet ready to reliably predict when scientific findings will fail to replicate. That theme should be central to the project: the point is not to prove AI can “solve science,” but to show where model limits, data quality issues, and overfitting distort results. If you want more examples of thoughtful technical framing, see our guides on AI in employee training and project-based learning.

Pro tip: The most valuable student projects are not the ones with the highest accuracy. They are the ones that explain failure clearly, document assumptions, and show how the team would improve the pipeline if they had another semester.

Why Replication Prediction Is a Strong Student Project

It teaches the difference between prediction and explanation

Replication prediction is a perfect classroom challenge because the target outcome is messy, real, and meaningfully uncertain. Students can build meta-models that ingest features about studies—sample size, p-values, journal prestige, field, statistical power proxies, author history, and language patterns—and output a probability that the study will replicate. But because the target is noisy and incomplete, students quickly learn that a decent score does not equal causal understanding.

This distinction is crucial for research integrity. In practice, a model might learn that certain fields replicate better, or that larger studies are more robust, without truly understanding the mechanisms behind those patterns. That is why explainability is essential, and why students should compare black-box predictions against interpretable baselines and human judgment.

It mirrors how employers use AI in the real world

Most companies do not use AI to make final decisions in isolation. They use it to rank, triage, or surface risk, then ask humans to review the output. That is the same logic behind a replication predictor: the model is not a judge, but a triage assistant. Students who can show they know how to build, evaluate, and then question a meta-model will stand out in internships and entry-level roles.

This aligns with practical career skills like building a portfolio, justifying a methodology, and explaining tradeoffs. If you are teaching or self-studying, pair this project with dashboard reporting, internal analytics dashboards, and a simple write-up about what the model can and cannot tell you.

It naturally exposes overfitting and data leakage

Because the dataset is multi-dimensional and often small relative to the number of features, the project becomes a real lesson in overfitting. Students will be tempted to add journal-level features, author-level features, and text embeddings all at once. That may improve training accuracy, but it often hurts generalization. The project forces students to see why cross-validation, held-out test sets, time-based splits, and pre-registered feature selection matter.

For more on building disciplined workflows, see maintainer workflows, which illustrates how process discipline improves output quality. The same principle applies here: a clean methodology is not overhead, it is the difference between a credible model and a misleading one.

Project Overview: What Students Build Over a Semester

Phase 1: Define the question and set the scope

Students begin by deciding what “replication” means for the project. A strong scope might focus on one domain, such as psychology, medicine, economics, or machine learning itself, because replication standards and data sources differ across fields. They should define a binary or ordinal label, such as “replicated,” “partially replicated,” or “did not replicate,” and document how the label is sourced.

At this stage, students should write a one-page problem statement and a short ethics note. What data will they use? What harms could happen if the model is interpreted carelessly? What kinds of studies are excluded because the metadata is too sparse? These questions teach data ethics, not just coding.

Phase 2: Gather and clean the data

Students will likely need to combine structured metadata from replication databases with basic text features from abstracts. The goal is not to create a huge AI system; it is to build a small, explainable pipeline. Clean feature engineering might include year of publication, sample size, number of authors, effect size, p-values, journal impact proxies, and basic abstract statistics such as readability or hedging language frequency.

They should also track missingness explicitly. Missing data is not just a nuisance—it is often informative. If some fields or journals have less complete metadata, the model may inadvertently learn proxies for visibility rather than scientific robustness. This is a useful lesson in AI governance: better inputs usually lead to better decisions, but poor inputs can create a polished illusion of certainty.

Phase 3: Build a baseline before fancy models

The first model should be boring on purpose. A majority-class baseline, logistic regression, and a small decision tree are enough to show whether the signal exists at all. Students then compare these against more complex models such as random forests, gradient boosting, or a simple neural network. The point is not to maximize leaderboard performance; it is to understand which features are actually helping.

This is where students learn that responsible AI begins with restraint. If a shallow model performs similarly to a complex one, that is not a failure. It is evidence that the extra complexity may be unnecessary or harmful.

Data Sources, Features, and Label Design

What to include in the dataset

A replication-prediction dataset should include both study-level and publication-level information. Common inputs include sample size, number of experimental conditions, p-values, confidence intervals, citation count, journal type, publication year, preregistration status, open-data status, and whether the paper reports effect sizes clearly. Students can also add text-derived features from titles and abstracts, such as sentiment, uncertainty words, or methodological specificity.

Explain to students that more data is not automatically better. Certain variables may be tempting because they look predictive, but they can encode hidden biases. For example, citation counts are influenced by field popularity and timing, not only study quality. In that sense, the project is similar to learning from tool reviews: usefulness depends on context, not just feature count.

How to define the target label carefully

The label is the heart of the project. A replication label must be consistent, documented, and defensible. If one source defines replication as a statistically significant result in the same direction, while another uses effect size similarity, the model will learn inconsistent targets. Students should choose one labeling schema and record edge cases, such as inconclusive replications or studies with different but related outcomes.

That ambiguity is a feature, not a bug, because it reveals how much research integrity depends on operational definitions. One team could even create two versions of the target and compare model behavior under each definition. That turns a technical assignment into a lesson about scientific language, measurement error, and the road from poster session to publication.

How to prevent leakage in feature design

Students must be careful not to feed the model information that would not be available at prediction time. If the goal is to predict replication from the original publication, then post hoc replication-specific commentary should be excluded. Otherwise, the model may appear brilliant simply because it saw the answer. This is a classic overfitting trap and one of the most important machine learning limits students will encounter.

It is helpful to frame this like planning a project brief: every feature needs a justification. If you want inspiration for structured planning, see developer checklists and targeted outreach style workflows, where the order of operations determines whether outputs are actionable or misleading.

Modeling Choice	Strength	Weakness	Best Use
Majority-class baseline	Simple sanity check	Not predictive	Benchmarking
Logistic regression	Interpretable coefficients	Limited nonlinearity	Feature screening
Decision tree	Easy to explain	High variance	Classroom demonstration
Random forest	Captures interactions	Harder to interpret	Stronger baseline
Gradient boosting	Often strong performance	Can overfit small data	Final comparison

Model Evaluation: How to Tell Whether the AI Is Actually Useful

Accuracy is not enough

In an imbalanced dataset, accuracy can be deceptive. If most studies replicate or most do not, a trivial classifier can look good while learning nothing useful. Students should evaluate precision, recall, F1 score, ROC-AUC, calibration, and confusion matrices. For a replication task, calibration is especially important because the output should be treated as risk, not prophecy.

That is also why students need a clear evaluation narrative. A model that has moderate discrimination but poor calibration could still be useful for ranking studies into “lower confidence” and “higher confidence” buckets. But it would be dangerous if used as a binary gatekeeper. For a broader example of careful performance thinking, compare this with crowd-sourced performance estimates, where raw numbers need interpretation before action.

Use cross-validation and a held-out test set

Students should never evaluate on the same data they used to tune the model. A practical setup is nested cross-validation for model selection plus a final untouched test set for reporting. If the project timeline is short, at minimum, use stratified train-validation-test splits. If publication year matters, consider a time-based split to test whether the model generalizes to newer studies rather than memorizing older publication patterns.

This is where the lesson about overfitting becomes concrete. A model that performs well on random splits but fails on later studies may have learned historical artifacts rather than robust replication signals. That type of failure is exactly what students should document.

Compare machine predictions to human judgment

A strong classroom extension is to ask students or instructors to make blind predictions from a small sample of studies. Then compare human and model performance side by side. Do humans outperform the model on some subdomains? Does the model pick up numerical regularities that people miss? Do both fail for the same reasons?

This comparative approach improves explainability because it reveals whether the model is doing something meaningfully different or merely reproducing intuitions. It also helps students practice humility. Just because a system is automated does not mean it is objective, and just because a human sounds thoughtful does not mean they are accurate.

Where AI Predictions Go Wrong, and Why That Matters

Bias from publication and field effects

AI systems often learn shortcuts. In replication prediction, the model may latch onto field-level patterns, journal prestige, or publication year instead of the deeper properties associated with robustness. That can produce a model that is really a publication classifier in disguise. Students should test this by removing suspicious features and seeing whether performance collapses.

These kinds of shortcuts are common in machine learning and reinforce why skepticism is a skill, not a personality trait. For another example of how context can distort interpretation, read responsible coverage of shocks, where surface events can easily be confused with underlying causes. The same discipline applies here: do not mistake correlation for scientific explanation.

Text models often overread style and underread substance

If students use abstract embeddings or language models, they may find that textual style contributes more than scientific rigor. The model might associate careful hedging language with non-replication, or concise claims with robustness, even when the effect is spurious. That creates a useful class discussion: should we trust a model that is detecting writing style instead of methodological quality?

To explore that question, ask students to run ablations: one model with metadata only, one with text only, and one with both. Then compare which features contribute most to prediction and which cause instability. The goal is not to reject NLP, but to demonstrate that explainability and domain review are necessary guardrails.

Small datasets magnify overfitting

Replication datasets are often small relative to the number of plausible predictors. That makes them fertile ground for overfitting, especially when students try many models and choose the one with the best validation score. To counter this, students should predefine their metrics, keep feature engineering simple, and report uncertainty intervals or repeated cross-validation results.

For students interested in how process discipline improves reliability in other domains, look at operational KPIs and structured targeting workflows. The lesson is the same: disciplined measurement beats improvisation when the stakes are high.

Ethics, Integrity, and Responsible Skepticism

Do not turn the model into a verdict machine

A replication predictor is a support tool, not an arbiter of truth. Students should never present the output as “this study is bad” or “this paper is fake.” Instead, they should phrase results probabilistically: “the model estimates a lower replication likelihood based on available metadata.” That wording matters because the social consequences of misclassification can be serious.

This also helps students practice ethical communication. In the real world, AI tools influence resource allocation, hiring, peer review, and research reputation. A project like this trains future professionals to ask whether the model is appropriate for the decision being made.

Document uncertainty and missing data

Every project report should include a section on uncertainty, unknowns, and missing fields. Students should show how many studies were excluded and why, since exclusion criteria can materially change conclusions. They should also explain whether the label distribution differs across subfields or time periods, which may indicate hidden biases in the sample.

That attention to omissions is part of trustworthy analysis. If a model’s inputs are incomplete, the output should be treated as a hypothesis generator, not a final answer. This is exactly why data ethics belongs in technical education rather than as an afterthought.

Consider governance and reproducibility from day one

Students should version their code, document their dataset sources, and create a reproducible notebook or pipeline. They should also maintain a changelog for feature decisions, because evaluation results often change when a single preprocessing step changes. These habits mirror professional AI governance and prepare students for collaborative work.

For more on policy-aware technical practice, see AI governance frameworks and responsible AI implementation playbooks. Both reinforce the same message: documentation is part of the model, not separate from it.

Recommended Semester Timeline and Deliverables

Weeks 1–3: Question, scope, and data audit

Students define the research question, identify the replication dataset, and write a short data dictionary. They also create a risk log: where could leakage happen, where might labels be inconsistent, and which variables could create ethical issues? This early audit prevents the common mistake of collecting too much data before thinking about the problem.

By the end of this phase, each team should have a project brief and a list of approved features. They should also state what success looks like beyond accuracy: interpretability, calibration, and a defensible failure analysis.

Weeks 4–8: Baselines and first evaluation

Students build baseline models, run cross-validation, and compare metrics. They should present not only average performance but also class-wise errors and calibration plots. If results are weak, that is informative; students can then focus on why the signal is limited rather than rushing to add complexity.

This phase is ideal for a checkpoint presentation. A good presentation explains what the model got right, what it missed, and which assumptions might be driving the results. That presentation skill translates directly into portfolio and interview success.

Weeks 9–13: Error analysis and ablations

Now the team investigates false positives and false negatives. Are low-powered studies being misclassified? Are pre-registered studies ranked correctly? Do features like journal prestige dominate the predictions? They should also perform ablation studies to measure the contribution of each feature group.

This is where students learn to think like investigators rather than just coders. Good analysts do not stop at “the model is 72% accurate.” They ask which 28% failed, why, and what those failures imply about the data and the task.

Weeks 14–15: Final report and portfolio packaging

The final submission should include a report, slides, a reproducible notebook, and a concise README. Students should summarize findings in plain language for a nontechnical audience and include one section titled “What this model cannot tell us.” That section is often the most important one.

If the student wants career value, they should also turn the project into a portfolio case study. Include a methodology diagram, evaluation table, a short explanation of overfitting, and a reflection on ethical use. Employers love projects that show both technical execution and mature judgment.

How to Present the Project in a Resume or Portfolio

Frame it as evidence of judgment, not just coding

Instead of writing “built a machine learning model,” students should describe the problem, methods, and key takeaway. For example: “Built and evaluated meta-models to predict study replication using publication metadata and text features; compared logistic regression, random forest, and gradient boosting; performed error analysis to identify overfitting and leakage risks.” That version sounds more credible because it names the task and the learning.

Students can also quantify responsibly: dataset size, number of features, evaluation method, and final metric range. But they should not inflate results. A portfolio that shows honest limits is often more compelling than one that overstates success.

Include visuals that make uncertainty visible

Useful portfolio visuals include calibration curves, confusion matrices, feature importance charts, and error slices by field or publication year. If possible, add a simple diagram showing the end-to-end workflow from data collection to model evaluation to error review. That makes the project easier for recruiters and faculty to understand quickly.

Think of the portfolio as a communication product. In that sense, it resembles how creators package value in other domains, from creative success stories to visual branding lessons. Clear storytelling turns technical work into memorable evidence.

Describe the ethical lesson explicitly

The best portfolios include a reflection on what the student learned about AI limits. In this project, the likely takeaway is that models can detect weak signals but cannot replace careful reading, replication standards, or domain expertise. That is not a defeat. It is a mature conclusion that employers and graduate programs respect.

Students can even add a short “future work” note: more balanced datasets, better label definitions, domain-specific models, or human-in-the-loop workflows. That shows ambition without pretending the current model is definitive.

FAQ and Troubleshooting

What is a meta-model in this project?

A meta-model is a model trained to predict an outcome about another research artifact—in this case, whether a published study will replicate. Students typically use metadata, text features, and publication signals rather than raw experimental data. The project is useful because it demonstrates how machine learning can summarize patterns, but also how those patterns can be unstable or biased.

How do we know if the model is overfitting?

Look for a large gap between training and test performance, unstable results across cross-validation folds, or sharp drops when a small set of features is removed. If the model performs much better on random splits than on time-based splits, it may be learning historical artifacts instead of robust signals. Overfitting is especially common when the dataset is small and the feature set is large.

What if the dataset is too small for strong results?

That is a valid finding. The student should report that the task may be underpowered, explain which metrics were unstable, and propose a smaller feature set or a narrower domain. In many cases, weak performance is actually the most honest answer, and it teaches more than a polished but misleading score.

Should students use deep learning or an LLM?

Only if it is justified. For a semester project, simpler models are often better because they are easier to evaluate and explain. If a language model is used for feature extraction or comparison, students should clearly document the prompt, embeddings, and validation logic so the work remains reproducible.

How should students present uncertain predictions?

Predictions should be framed probabilistically and paired with caveats about label quality, missingness, and generalization. Students should avoid strong claims like “this study will fail,” and instead say the model identifies a higher or lower estimated replication likelihood. Good communication is part of model evaluation because it determines how the output will be used.

What makes this project valuable for careers?

It proves students can work with messy data, evaluate models honestly, and explain limitations to nontechnical audiences. Those are transferable skills across data science, policy research, product analytics, and AI operations. A candidate who can discuss failure modes clearly often looks more credible than one who only reports a high score.

Conclusion: Teach Students to Trust Carefully, Not Blindly

This semester project works because it combines technical practice with intellectual humility. Students build meta-models, compare baselines, evaluate overfitting, and analyze where AI predictions fall short. They also learn that the absence of perfect prediction is not a flaw in the assignment; it is the lesson. In a field full of exaggerated promises, the ability to say “the model helps, but it is not enough” is a professional advantage.

If you want to extend the project into a broader AI literacy pathway, pair it with our guides on teaching design for patchy attendance, healthy dev rituals, and preparing trainees for changing systems. The deeper lesson is the same across all of them: good tools are useful, but disciplined thinking is what turns tools into outcomes.

For students, teachers, and lifelong learners, this is the kind of project that creates real career value. It can live in a GitHub repo, a report, a poster, or a portfolio page, but its real value is in the judgment it builds. AI will get things wrong. The student who learns how to detect that, explain it, and respond responsibly is the one most likely to succeed.

From Poster Session to Publication: A Beginner’s Roadmap for Physics Students - A practical guide to turning academic work into a publishable story.
A Playbook for Responsible AI Investment - Governance ideas you can adapt for student AI projects.
Maintainer Workflows - Useful process discipline for keeping projects reproducible.
Turning News Shocks into Thoughtful Content - A framework for responsible analysis under uncertainty.
Website KPIs for 2026 - A helpful reminder that good metrics need context and interpretation.

Daniel Mercer

Senior SEO Content Strategist

Senior editor and content strategist. Writing about technology, design, and the future of digital media. Follow along for deep dives into the industry's moving parts.