跳到正文
DEV Community· Abdurrahman hakan acar·· 1 天前AI 評分38

數秒內生成 1,000,000 份合成醫療紀錄

Generating 1,000,000 Synthetic Medical Records in Seconds

AI 導讀

作者打造兩階段生成引擎,以 Class-Conditional Gaussian Copulas 和 Decision Trees 生成合成醫療紀錄,在標準 CPU 上約 6.9 秒生成 1,000,000 份。測試顯示臨牀規則符合率為 100.0%,與真實數據集沒有重複紀錄;以合成數據訓練的分類器在真實測試數據上的 Δ ROC-AUC ≤ 0.0013。

正文

Abdurrahman hakan acar

Generating realistic tabular medical data usually comes with a frustrating trade-off: deep learning models (CTGAN, Diffusion) are slow, GPU-heavy, and prone to violating clinical rules.

To solve this, we built a transparent, ultra-fast Two-Stage Generative Engine combining Class-Conditional Gaussian Copulas and Decision Trees.

⚡ Key Benchmarks:

  • Speed: 10,000 patients in 0.17s | 1,000,000 patients in ~6.9s (standard CPU)
  • Privacy: 0 duplicate records against real datasets (Gower DCR: 0.0789)
  • Clinical Logic: 100.0% rule compliance (0 boundary violations, non-asthmatics strictly 'N/A')
  • ML Utility (TSTR): Classifiers trained on synthetic data achieve $\Delta \text{ROC-AUC} \le 0.0013$ on real test data

🛠️ How It Works:

  1. Prevalence (Stage 0): Bernoulli sampling matching ground-truth diagnosis prevalence (~24.3%).
  2. Class-Conditional Gaussian Copula (Stage 1): Captures non-linear dependencies across 14 features using empirical quantiles and Cholesky decomposition.
  3. Decision Tree (Stage 2): Deterministically assigns Asthma_Control_Level, perfectly decoupling non-asthmatics.

📊 Scientific Validation (4 Pillars):

Pillar Metric Result Status
Statistical Fidelity Mean Diffs / Correlation Diff < 0.6% / 0.0225 PASS
Clinical Logic Boundary Violations / Non-Asthmatic Rule 0 Violations / 100% 'N/A' PASS
Privacy Guarantee Exact Clones vs Real Data 0 Duplicates PASS
ML Utility (TSTR) Random Forest $\Delta \text{ROC-AUC}$ 0.0013 PASS

Provenance note: Fitted on Arif Miah's (miadul) Apache 2.0 dataset on Kaggle. Replicates synthetic data distributions, not human biology.


🚀 Live Interactive App & Resources:


What is your preferred approach for tabular data synthesis: statistical copulas or deep neural nets? Let's discuss below!

來源:DEV Community · dev.to