@misc{chu_mmint-labvera_2026,
	title = {mmint-lab/{VERA}: {VERA}: {Autonomous} {Replication} and {Validation} of {Machine} {Learning} {Experiments} from {Research} {Papers}},
	shorttitle = {mmint-lab/{VERA}},
	url = {https://zenodo.org/records/18165553},
	doi = {10.5281/zenodo.18165553},
	abstract = {v1.0.0 - Initial Release



First public release of VERA (Verification Engine for Reproducible Analysis)


VERA is an autonomous agent that extracts, replicates, and validates machine learning experiments from research papers without manual intervention. Simply provide a PDF, and VERA handles extraction, execution, and comprehensive comparison reporting in {\textasciitilde}35 seconds.



Highlights



100\% extraction accuracy across 4 validated research papers (32 models total)

82\% average Results Match score with paper-reported benchmarks

Zero manual intervention required during execution

Full transparency with reviewable standalone Python scripts ({\textasciitilde}250 lines per paper)

Web dashboard + CLI interface included




Features

Core Capabilities



Autonomous PDF extraction with hybrid rule-based + semantic NLP

UCI dataset auto-resolution (20+ datasets)

14+ ML models (SVM, Random Forest, XGBoost, LightGBM, CatBoost, etc.)

Ensemble model support (e.g., "AdaBoost+LR")

Hyperparameter extraction (30+ patterns)

4-component scoring: EQ, CQ, RM, E2E

Interactive visualizations (heatmaps, delta charts, slopegraphs)


Interfaces



CLI: python -m vera --pdf paper.pdf --output\_dir results/

Web UI: React dashboard with FastAPI backend


Output Artifacts



COMPREHENSIVE\_REPORT.html - Interactive report

replication\_standalone.py - Reviewable code

results.json - Structured benchmarks

blueprint.md - Extraction details

figures/ - Publication-quality visualizations




Quick Start

Requirements



Python 3.11+

4GB RAM

Internet connection


Installation

\# Create environment
conda create -n vera python=3.12 -y
conda activate vera

\# Install dependencies
pip install -r requirements.txt

\# Optional: Install boosting libraries for better RM scores
pip install xgboost lightgbm catboost


Run Demo

\# CLI
python -m vera --pdf test\_papers/HeartDiseaseProj.pdf --output\_dir demo/heart\_disease
open demo/heart\_disease/article/COMPREHENSIVE\_REPORT.html

\# Web Interface
python -m vera web  \# Terminal 1
cd frontend \&\& npm install \&\& npm run dev  \# Terminal 2
\# Open http://localhost:5173




Validation Results

{\textbar} Metric {\textbar} Value {\textbar}
{\textbar}--------{\textbar}-------{\textbar}
{\textbar} Papers Tested {\textbar} 4 {\textbar}
{\textbar} Models Extracted {\textbar} 32 {\textbar}
{\textbar} Extraction Accuracy {\textbar} 100\% {\textbar}
{\textbar} Code Success Rate {\textbar} 100\% {\textbar}
{\textbar} Avg RM Score {\textbar} 82\% {\textbar}
{\textbar} Avg Execution Time {\textbar} {\textasciitilde}35s {\textbar}

Demo Papers Included



Heart Disease (10 models, 82\% RM)

Breast Cancer (12 models, 84\% RM)

Cardiovascular Disease (9 models, 77\% RM)

Diabetes SVM (1 model, 84\% RM)




Known Limitations



UCI datasets only (no Kaggle, HuggingFace, custom data)

Tabular ML only (no PyTorch/TensorFlow deep learning)

Machine-readable PDFs only (no OCR)

Binary classification focus (limited multi-class)




Documentation



README.md - Quick start guide

BLOG.md - Vision and detailed walkthrough

TECHNICAL\_BLOG.md - Technical deep-dive

demo/ - 4 pre-run demo outputs




What's Next

See ROADMAP.md for planned enhancements:



LLM extraction fallback

Deep learning support

OCR for scanned papers

Batch processing

Extended dataset coverage




Authors

Andre Chu and Warren Pettine
Medical Machine Intelligence Lab, Department of Psychiatry, University of Utah

Contact: andre.chu@utah.edu



License

Academic research tool. Not for clinical use.



Acknowledgments

This is a proof-of-concept demonstrating autonomous replication feasibility. It should complement, not replace, human review and expertise.},
	urldate = {2026-01-06},
	publisher = {Zenodo},
	author = {Chu, Andre},
	month = jan,
	year = {2026},
}

{"_id":"wzNFdoXdAEAMsqoFo","bibbaseid":"chu-mmintlabveraveraautonomousreplicationandvalidationofmachinelearningexperimentsfromresearchpapers-2026","author_short":["Chu, A."],"bibdata":{"bibtype":"misc","type":"misc","title":"mmint-lab/VERA: VERA: Autonomous Replication and Validation of Machine Learning Experiments from Research Papers","shorttitle":"mmint-lab/VERA","url":"https://zenodo.org/records/18165553","doi":"10.5281/zenodo.18165553","abstract":"v1.0.0 - Initial Release First public release of VERA (Verification Engine for Reproducible Analysis) VERA is an autonomous agent that extracts, replicates, and validates machine learning experiments from research papers without manual intervention. Simply provide a PDF, and VERA handles extraction, execution, and comprehensive comparison reporting in ~35 seconds. Highlights 100% extraction accuracy across 4 validated research papers (32 models total) 82% average Results Match score with paper-reported benchmarks Zero manual intervention required during execution Full transparency with reviewable standalone Python scripts (~250 lines per paper) Web dashboard + CLI interface included Features Core Capabilities Autonomous PDF extraction with hybrid rule-based + semantic NLP UCI dataset auto-resolution (20+ datasets) 14+ ML models (SVM, Random Forest, XGBoost, LightGBM, CatBoost, etc.) Ensemble model support (e.g., \"AdaBoost+LR\") Hyperparameter extraction (30+ patterns) 4-component scoring: EQ, CQ, RM, E2E Interactive visualizations (heatmaps, delta charts, slopegraphs) Interfaces CLI: python -m vera –pdf paper.pdf –output_dir results/ Web UI: React dashboard with FastAPI backend Output Artifacts COMPREHENSIVE_REPORT.html - Interactive report replication_standalone.py - Reviewable code results.json - Structured benchmarks blueprint.md - Extraction details figures/ - Publication-quality visualizations Quick Start Requirements Python 3.11+ 4GB RAM Internet connection Installation # Create environment conda create -n vera python=3.12 -y conda activate vera # Install dependencies pip install -r requirements.txt # Optional: Install boosting libraries for better RM scores pip install xgboost lightgbm catboost Run Demo # CLI python -m vera –pdf test_papers/HeartDiseaseProj.pdf –output_dir demo/heart_disease open demo/heart_disease/article/COMPREHENSIVE_REPORT.html # Web Interface python -m vera web # Terminal 1 cd frontend && npm install && npm run dev # Terminal 2 # Open http://localhost:5173 Validation Results \\textbar Metric \\textbar Value \\textbar \\textbar——–\\textbar——-\\textbar \\textbar Papers Tested \\textbar 4 \\textbar \\textbar Models Extracted \\textbar 32 \\textbar \\textbar Extraction Accuracy \\textbar 100% \\textbar \\textbar Code Success Rate \\textbar 100% \\textbar \\textbar Avg RM Score \\textbar 82% \\textbar \\textbar Avg Execution Time \\textbar ~35s \\textbar Demo Papers Included Heart Disease (10 models, 82% RM) Breast Cancer (12 models, 84% RM) Cardiovascular Disease (9 models, 77% RM) Diabetes SVM (1 model, 84% RM) Known Limitations UCI datasets only (no Kaggle, HuggingFace, custom data) Tabular ML only (no PyTorch/TensorFlow deep learning) Machine-readable PDFs only (no OCR) Binary classification focus (limited multi-class) Documentation README.md - Quick start guide BLOG.md - Vision and detailed walkthrough TECHNICAL_BLOG.md - Technical deep-dive demo/ - 4 pre-run demo outputs What's Next See ROADMAP.md for planned enhancements: LLM extraction fallback Deep learning support OCR for scanned papers Batch processing Extended dataset coverage Authors Andre Chu and Warren Pettine Medical Machine Intelligence Lab, Department of Psychiatry, University of Utah Contact: andre.chu@utah.edu License Academic research tool. Not for clinical use. Acknowledgments This is a proof-of-concept demonstrating autonomous replication feasibility. It should complement, not replace, human review and expertise.","urldate":"2026-01-06","publisher":"Zenodo","author":[{"propositions":[],"lastnames":["Chu"],"firstnames":["Andre"],"suffixes":[]}],"month":"January","year":"2026","bibtex":"@misc{chu_mmint-labvera_2026,\n\ttitle = {mmint-lab/{VERA}: {VERA}: {Autonomous} {Replication} and {Validation} of {Machine} {Learning} {Experiments} from {Research} {Papers}},\n\tshorttitle = {mmint-lab/{VERA}},\n\turl = {https://zenodo.org/records/18165553},\n\tdoi = {10.5281/zenodo.18165553},\n\tabstract = {v1.0.0 - Initial Release\n\n\n\nFirst public release of VERA (Verification Engine for Reproducible Analysis)\n\n\nVERA is an autonomous agent that extracts, replicates, and validates machine learning experiments from research papers without manual intervention. Simply provide a PDF, and VERA handles extraction, execution, and comprehensive comparison reporting in {\\textasciitilde}35 seconds.\n\n\n\nHighlights\n\n\n\n100\\% extraction accuracy across 4 validated research papers (32 models total)\n\n82\\% average Results Match score with paper-reported benchmarks\n\nZero manual intervention required during execution\n\nFull transparency with reviewable standalone Python scripts ({\\textasciitilde}250 lines per paper)\n\nWeb dashboard + CLI interface included\n\n\n\n\nFeatures\n\nCore Capabilities\n\n\n\nAutonomous PDF extraction with hybrid rule-based + semantic NLP\n\nUCI dataset auto-resolution (20+ datasets)\n\n14+ ML models (SVM, Random Forest, XGBoost, LightGBM, CatBoost, etc.)\n\nEnsemble model support (e.g., \"AdaBoost+LR\")\n\nHyperparameter extraction (30+ patterns)\n\n4-component scoring: EQ, CQ, RM, E2E\n\nInteractive visualizations (heatmaps, delta charts, slopegraphs)\n\n\nInterfaces\n\n\n\nCLI: python -m vera --pdf paper.pdf --output\\_dir results/\n\nWeb UI: React dashboard with FastAPI backend\n\n\nOutput Artifacts\n\n\n\nCOMPREHENSIVE\\_REPORT.html - Interactive report\n\nreplication\\_standalone.py - Reviewable code\n\nresults.json - Structured benchmarks\n\nblueprint.md - Extraction details\n\nfigures/ - Publication-quality visualizations\n\n\n\n\nQuick Start\n\nRequirements\n\n\n\nPython 3.11+\n\n4GB RAM\n\nInternet connection\n\n\nInstallation\n\n\\# Create environment\nconda create -n vera python=3.12 -y\nconda activate vera\n\n\\# Install dependencies\npip install -r requirements.txt\n\n\\# Optional: Install boosting libraries for better RM scores\npip install xgboost lightgbm catboost\n\n\nRun Demo\n\n\\# CLI\npython -m vera --pdf test\\_papers/HeartDiseaseProj.pdf --output\\_dir demo/heart\\_disease\nopen demo/heart\\_disease/article/COMPREHENSIVE\\_REPORT.html\n\n\\# Web Interface\npython -m vera web \\# Terminal 1\ncd frontend \\&\\& npm install \\&\\& npm run dev \\# Terminal 2\n\\# Open http://localhost:5173\n\n\n\n\nValidation Results\n\n{\\textbar} Metric {\\textbar} Value {\\textbar}\n{\\textbar}--------{\\textbar}-------{\\textbar}\n{\\textbar} Papers Tested {\\textbar} 4 {\\textbar}\n{\\textbar} Models Extracted {\\textbar} 32 {\\textbar}\n{\\textbar} Extraction Accuracy {\\textbar} 100\\% {\\textbar}\n{\\textbar} Code Success Rate {\\textbar} 100\\% {\\textbar}\n{\\textbar} Avg RM Score {\\textbar} 82\\% {\\textbar}\n{\\textbar} Avg Execution Time {\\textbar} {\\textasciitilde}35s {\\textbar}\n\nDemo Papers Included\n\n\n\nHeart Disease (10 models, 82\\% RM)\n\nBreast Cancer (12 models, 84\\% RM)\n\nCardiovascular Disease (9 models, 77\\% RM)\n\nDiabetes SVM (1 model, 84\\% RM)\n\n\n\n\nKnown Limitations\n\n\n\nUCI datasets only (no Kaggle, HuggingFace, custom data)\n\nTabular ML only (no PyTorch/TensorFlow deep learning)\n\nMachine-readable PDFs only (no OCR)\n\nBinary classification focus (limited multi-class)\n\n\n\n\nDocumentation\n\n\n\nREADME.md - Quick start guide\n\nBLOG.md - Vision and detailed walkthrough\n\nTECHNICAL\\_BLOG.md - Technical deep-dive\n\ndemo/ - 4 pre-run demo outputs\n\n\n\n\nWhat's Next\n\nSee ROADMAP.md for planned enhancements:\n\n\n\nLLM extraction fallback\n\nDeep learning support\n\nOCR for scanned papers\n\nBatch processing\n\nExtended dataset coverage\n\n\n\n\nAuthors\n\nAndre Chu and Warren Pettine\nMedical Machine Intelligence Lab, Department of Psychiatry, University of Utah\n\nContact: andre.chu@utah.edu\n\n\n\nLicense\n\nAcademic research tool. Not for clinical use.\n\n\n\nAcknowledgments\n\nThis is a proof-of-concept demonstrating autonomous replication feasibility. It should complement, not replace, human review and expertise.},\n\turldate = {2026-01-06},\n\tpublisher = {Zenodo},\n\tauthor = {Chu, Andre},\n\tmonth = jan,\n\tyear = {2026},\n}\n\n","author_short":["Chu, A."],"key":"chu_mmint-labvera_2026","id":"chu_mmint-labvera_2026","bibbaseid":"chu-mmintlabveraveraautonomousreplicationandvalidationofmachinelearningexperimentsfromresearchpapers-2026","role":"author","urls":{"Paper":"https://zenodo.org/records/18165553"},"metadata":{"authorlinks":{}}},"bibtype":"misc","biburl":"https://api.zotero.org/users/1528358/collections/DH4R6ZWA/items?key=L8lztpUgybScRoWSW76OhIE0&format=bibtex&limit=100","dataSources":["hoqThwEepw4WmWudJ"],"keywords":[],"search_terms":["mmint","lab","vera","vera","autonomous","replication","validation","machine","learning","experiments","research","papers","chu"],"title":"mmint-lab/VERA: VERA: Autonomous Replication and Validation of Machine Learning Experiments from Research Papers","year":2026}