MIMIC-IV — a procurement-grade audit under the FDA 21 CFR 11 / HIPAA / §1557 lens.

MIT Lab for Computational Physiology under Roger Mark has maintained the MIMIC series since the 1990s. Multi-decade institutional commitment. Beth Israel Deaconess Medical Center IRB engagement is sustained. Public errata and version history are responsive (MIMIC-III → IV → IV.v2 → IV.v3 transitions are documented). The Johnson et al. authorship line is exceptionally credible. This is the highest maintainer score the audit series has ever produced. The 2-point deduction is reserved entirely for the possibility of a future maintenance transition; nothing in the maintainer behavior itself is below the bar.

Full data dictionary published. Schema documentation for every table. Patient-level admission journey is reconstructable from the relational structure. Lab values, vital signs, medications, procedures, diagnoses (ICD-9-CM and ICD-10-CM), notes (where available), and ICU events are all present. Missing-data patterns are reported per table in the documentation. The 4-point deduction reflects the inherent incompleteness of any real-world clinical record: some patients have ED-only encounters with no structured ICU data; some admissions lack discharge summaries; some chart events have value-but-not-unit fields.

CSV (gzipped) is the canonical distribution; Parquet versions are published for direct DuckDB and Polars loading. Schema is documented with foreign-key relationships across tables. PostgreSQL and BigQuery setup scripts are maintained by the MIT LCP team. Loads cleanly via pandas, polars, dask, R's data.table. The 5-point deduction is for the absence of a published OMOP CDM mapping in the canonical release (third-party OMOP mappings exist but are not maintainer-published) — relevant for procurement workflows that have standardized on OMOP for cross-dataset comparability.

Nature Scientific Data paper (2023). Detailed cohort-derivation code in mit-lcp/mimic-code repo. Discussion forum with archived maintainer responses. Per-table documentation on PhysioNet. Datasheet-for-datasets-style coverage is implicit rather than explicit (the Nature paper covers most fields but doesn't use the Gebru et al. datasheet template). 5-point deduction reserved for the absence of a formal datasheet — minor, since the equivalent content exists in the paper.

BIDMC IRB approved a waiver of informed consent under HIPAA §164.512(i) for the deidentified data and §164.514(b)(2) Safe Harbor for the release. Documented in the methodology paper. MIT IRB endorsed the secondary distribution. The credentialed-access model (CITI training + signed DUA + verified credentialed user status) operationalizes ongoing consent-equivalent oversight: each researcher attests to the use case, agrees to non-redistribution and non-re-identification, and is auditable by PhysioNet. This is the procurement-grade framework other healthcare datasets should be measured against. 5-point deduction reflects the inherent asymmetry of any waiver-of-consent framework — patients cannot opt out retroactively even if they could in principle have been asked at the time of care.

Source-of-truth is the underlying electronic health record itself — lab results, billing codes, medication orders, vitals as recorded by clinical staff at point of care. Validation is in-context (the record is the ground truth). Residual noise is documented: transcription errors in nursing flowsheet entries, occasional duplicate medication-administration records, free-text fields with abbreviations the standard medical NLP pipelines may misread. The maintainers publish version-to-version errata for downstream fixes. 7-point deduction reflects the irreducible noise of real clinical record-keeping (which is itself a real-world signal, not a data flaw).

BIDMC EHR (Epic) → BIDMC clinical data warehouse → MIT LCP secondary-use pipeline → de-identification → PhysioNet distribution. Each hop is documented. The pipeline is reproducible from the published code. Schema mappings between the source EHR and the released schema are documented. Compared to web-crawl-derived datasets where the provenance chain is "Common Crawl → filter → publish" with millions of unique upstream sources, MIMIC-IV's single-institution chain is far more auditable. 8-point deduction reserved for the BIDMC EHR upstream layer itself, which is closed by definition (Epic is proprietary) — not actionable, but a fact procurement audits should note.

HIPAA Safe Harbor methodology under §164.514(b)(2): all 18 identifiers removed or surrogate-shifted, dates shifted within patient, ages capped at 90+ to avoid small-cell-population re-identification risk. For structured fields, this is the gold-standard methodology. The deduction is concentrated in two places: (a) free-text clinical notes, where automated PHI removal (the MIT LCP pipeline uses a regex + rule-based scrubber for notes) cannot perfectly eliminate every indirect identifier — rare disease descriptions, distinctive event sequences, contextual geographic clues; (b) the de-identification standard is HIPAA-grade, which is the right standard for US deployment but not equivalent to GDPR Article 4(5) pseudonymization for EU deployment. Procurement teams shipping models into EU jurisdictions should layer additional pseudonymization or use a GDPR-grade derivative.

Cohort-definition code is public (mit-lcp/mimic-code repo). Standard analysis scripts (mortality prediction, length-of-stay, sepsis cohort) are published with the dataset. The de-identification pipeline is described in detail though not fully open-source — the rule-base for note scrubbing is partially proprietary to MIT LCP. Anyone with credentialed access can fully reproduce published downstream analyses; the de-identification step itself has a closed component. 12-point deduction reflects this.

The PhysioNet Credentialed Health Data License 1.5.0 is unambiguous in its terms but procurement-unusual: not an open license, not a commercial license, but a credentialed-access framework specific to healthcare research data. Researchers must complete CITI Data or Specimens Only Research training, attest to a specific use case, sign the data-use agreement, and undergo credentialed-user verification on PhysioNet. Commercial use requires a separate agreement directly with MIT LCP. For procurement teams accustomed to scanning for Apache 2.0 / MIT / CC-BY / proprietary, this falls outside the standard taxonomy. 30-point deduction is not a quality complaint — it reflects that downstream procurement automation has to handle this case specially. The license terms themselves are appropriate for the data category.

Single-site Boston urban academic medical center. Patient population skews Northeast US (BIDMC catchment area), predominantly insured under Massachusetts payer plans (Mass General Brigham, BCBS-MA, Medicare, Medicaid), with the specific demographic mix of an Alewife-to-Mattapan catchment that does not represent the US population, much less the world. Time window 2008–2019. For models intended for nationwide US deployment, this is the largest dimension gap. The data is not "biased" in a moral sense — it accurately reflects the population BIDMC serves. The procurement question is whether your downstream deployment population matches.

BIDMC's clinical practice patterns are not portable. Drug formulary (which brand of an antibiotic is on formulary at BIDMC vs. a community hospital), ordering patterns (which tests are routinely ordered for sepsis workup at a teaching hospital vs. a rural ED), billing-code application (which ICD-10 codes are routinely applied for the same clinical scenario at BIDMC vs. elsewhere), and EHR free-text conventions (BIDMC's documentation templates and shorthand) all create a site-specific signature. Models that perform well on MIMIC-IV held-out test set often degrade on data from other institutions — this is a well-documented phenomenon in the clinical-AI literature. For FDA SaMD pre-cert and 510(k) submissions, multi-site external validation is essentially required regardless of MIMIC-IV training performance.