使用 Scikit-Learn Halving Search 預測嚴重急性呼吸道感染（SARI）死亡率

When dealing with public health data, computational efficiency and rapid experimentation are critical factors. In this tutorial, we will build an end-to-end Machine Learning pipeline using Python and Scikit-Learn to predict the clinical outcome of patients notified with Severe Acute Respiratory Illness (SARI/SRAG) in Brazil.

We will work with a large-scale real-world dataset from DataSUS, clean and preprocess clinical features, handle class imbalance, and use an efficient search strategy called Successive Halving to optimize our Random Forest hyperparameters in a fraction of the time required by exhaustive Grid Search.

1. Context & Objectives

Severe Acute Respiratory Illness (SARI) remains a major global public health concern. Predicting patient outcomes (Recovery vs. Death) based on initial demographics, symptoms, and comorbidities assists healthcare managers in risk stratification and resource allocation.

Our dataset contains over 260,000 patient records from 2025 in Brazil. We will focus on key variables such as:

• Demographics: Biological Sex (CS_SEXO) and Age (NU_IDADE_N).
• Comorbidities: Heart disease, Diabetes, Obesity, Renal disease, Asthma, and Immunosuppression.
• Support: ICU admission (UTI) and Ventilatory support (SUPORT_VEN).

2. Optimized Data Loading with Parquet

Because raw public health datasets contain dozens of unused epidemiological columns, reading uncompressed CSVs consumes excessive memory. We use the Apache Parquet format to enable columnar selection and fast I/O:

import pandas as pd
import numpy as np

file_path = 'INFLUD25-data.parquet'
target_columns = [
    'CS_SEXO', 'NU_IDADE_N', 'EVOLUCAO', 'UTI', 'SUPORT_VEN', 
    'CARDIOPATI', 'DIABETES', 'OBESIDADE', 'RENAL', 'ASMA', 'IMUNODEPRE'
]

df = pd.read_parquet(file_path, columns=target_columns)
print(f"Dataset Dimensions: {df.shape}")

3. Data Cleaning and Binary Normalization

Public health records contain ambiguous missing flags. We filter out unknown outcomes to build our binary target (TARGET), where:

• 0 = Recovery (Cura)
• 1 = Mortality (Óbito)

We then normalize categorical comorbidity features into strict binary (1 = Yes, 0 = No/Unknown) values:

# Filter valid clinical endpoints
df['EVOLUCAO'] = pd.to_numeric(df['EVOLUCAO'], errors='coerce')
df_clean = df[df['EVOLUCAO'].isin([1, 2])].copy()
df_clean['TARGET'] = df_clean['EVOLUCAO'].map({1: 0, 2: 1})
df_clean = df_clean.drop('EVOLUCAO', axis=1)

# Normalize clinical risk factors to binary
comorbidity_cols = [
    'UTI', 'SUPORT_VEN', 'CARDIOPATI', 'DIABETES', 
    'OBESIDADE', 'RENAL', 'ASMA', 'IMUNODEPRE'
]
for col in comorbidity_cols:
    df_clean[col] = pd.to_numeric(df_clean[col], errors='coerce')
    df_clean[col] = df_clean[col].apply(lambda x: 1 if x == 1 else 0)

# Impute and clean demographics
df_clean['NU_IDADE_N'] = pd.to_numeric(df_clean['NU_IDADE_N'], errors='coerce')
df_clean['NU_IDADE_N'] = df_clean['NU_IDADE_N'].fillna(df_clean['NU_IDADE_N'].median())
df_clean['CS_SEXO'] = df_clean['CS_SEXO'].map({'M': 0, 'F': 1, 'm': 0, 'f': 1}).fillna(0)

4. Stratified Train-Test Splitting

Because mortality is an imbalanced class (~8% positive occurrences), standard random partitioning can cause class distribution drift. We apply a stratified split:

from sklearn.model_selection import train_test_split

X = df_clean.drop('TARGET', axis=1)
y = df_clean['TARGET']

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.3, random_state=42, stratify=y
)

5. Efficient Hyperparameter Search: Successive Halving

Traditional GridSearchCV evaluates every parameter combination across the entire training dataset. For large datasets, this approach is computationally prohibitive.

Instead, we use Successive Halving via HalvingRandomSearchCV:

1. It trains an initial set of hyperparameter configurations using a small subsample of data.
2. Only the top-performing candidate models advance to the subsequent iteration.
3. In each subsequent round, the sample size increases by a factor $k$ (factor=3), and surviving models are re-evaluated.
4. The cycle repeats until the highest-performing architecture is trained on full capacity.

from sklearn.experimental import enable_halving_search_cv
from sklearn.model_selection import HalvingRandomSearchCV
from sklearn.ensemble import RandomForestClassifier

param_distributions = {
    'n_estimators': [50, 100, 200, 300],
    'max_depth': [10, 20, 30, None],
    'min_samples_split': [2, 5, 10],
    'min_samples_leaf': [1, 2, 4],
    'criterion': ['gini', 'entropy']
}

rf_base = RandomForestClassifier(random_state=42, n_jobs=-1)

halving_search = HalvingRandomSearchCV(
    estimator=rf_base,
    param_distributions=param_distributions,
    factor=3,
    resource='n_samples',
    scoring='accuracy',
    random_state=42,
    cv=3,
    verbose=1
)

halving_search.fit(X_train, y_train)
best_model = halving_search.best_estimator_
print(f"Optimal Parameters: {halving_search.best_params_}")

6. Real-Time Risk Inference

With our optimized ensemble model persisted, we can evaluate arbitrary, unseen patient records via direct inference:

high_risk_patient = {
    'CS_SEXO': 1,
    'NU_IDADE_N': 68,
    'UTI': 1,
    'SUPORT_VEN': 1,
    'CARDIOPATI': 1,
    'DIABETES': 1,
    'OBESIDADE': 1,
    'RENAL': 0,
    'ASMA': 0,
    'IMUNODEPRE': 0
}

input_df = pd.DataFrame([high_risk_patient])
mortality_probability = best_model.predict_proba(input_df)[0][1]

print(f"Calculated Risk Probability: {mortality_probability * 100:.2f}%")

Conclusion

Using HalvingRandomSearchCV dramatically reduces iteration time on large datasets without sacrificing model performance. When combined with rigorous binary feature normalization and stratified sampling, developers can build robust risk-stratification pipelines ready for healthcare operational workflows.