銀行數據科學：以淺層決策樹尋找明確規則

Data Science in Banking, Part 1: Finding Hard Rules with Shallow Decision Trees

A credit team reviewing an early-arrears model will eventually ask a practical question: which applications does it flag, and why? A short rule can make that discussion concrete. It identifies a segment, shows the conditions for entering it, and lets us count the cases that contradict its prediction.

The difficult part is earning that simplicity. A threshold can look convincing because it describes the training sample, because it uses information from the future, or because someone kept adjusting it after seeing the test results. A readable rule deserves the same scrutiny as a complicated model.

This first article in Data Science in Banking follows one experiment from candidate conditions to a frozen model and its mistakes. The banking question supplies the motivation. The executable experiment uses the public Wine dataset, whose chemical measurements identify cultivars. Wine contains no borrowers, and its results say nothing about lending performance.

Start with a decision that has a time

Consider an illustrative lending task: flag an application for additional review when it is assessed, before approval and disbursement. Define the eventual target as:

early_arrears = 1 if the funded loan reaches at least 30 days past due at any point during the first 90 days after disbursement; otherwise 0, once the full outcome window is observable.

This is a definition for this example, not a universal definition of credit default. Historical evaluation would use funded loans with complete servicing records and sufficient follow-up. A recent loan with no arrears yet is not automatically a negative example. Training on funded loans also means that the measured performance applies to that selected population; rejected applicants have no equivalent repayment outcome.

Potential predictors include income evidence, existing repayment obligations and a bureau snapshot available at the application decision. A later collections event, a balance updated after disbursement, or a column derived from the target cannot enter the feature table. Even an old transaction can leak information if its corrected value only became available later. Both event time and information availability matter.

A shallow tree could search these approved predictors for small segments with different observed arrears rates. Whether a segment should trigger manual review would then depend on its coverage, false alarms, missed arrears and the team's review capacity. We will not invent those banking results. Instead, we will make the mechanics inspectable on a dataset anyone can run.

Give the tree a small budget

A decision tree tries feature thresholds and chooses splits that reduce class impurity. Each path from the root to a leaf becomes a conjunction of conditions. The search is greedy: a good next split need not lead to the best possible rule set. Depth and leaf-size limits help control how much detail it can fit. Scikit-learn's tree guide describes this construction and its limitations.

A hard rule has a definite boundary: a row either satisfies its conditions or does not. That does not make its prediction certain.

The experiment retains the original published search: 24 candidate configurations, plus a constant baseline. It compares ordinary trees, ordered rule lists, and a merge_refit variant that can combine regions with similar class distributions and try splitting them again. In a rule list, the first matching rule wins, so earlier conditions affect what later rules mean.

The base trees have depth limits of 2, 3 or 4 and at most eight leaves. Each tree leaf needs at least 20 training rows, or more if its configured fraction demands it. This is total support, not a minimum number of positive cases. The extensions can produce longer expressions than a base tree's depth suggests; the final rule expressions still need inspection.

We choose the configuration by three-fold cross-validation using macro-F1, which averages the F1 scores of the two classes equally. This avoids selecting on accuracy alone when one class is larger. It is a demonstration criterion, not a cost model for lending.

The simplicity rule is fixed before evaluation: among candidates within 0.005 of the best mean CV score, prefer fewer predicate occurrences, then fewer segments. This tolerance is an engineering choice, not a confidence interval. Counting predicates across complete rules also differs from counting unique splits: one threshold appears in both complementary leaf rules.

Put the external holdout aside first

Wine has 178 rows and 13 numeric measurements. Its original label has three classes; here, original class 0 is cultivar A, and the other two classes form the negative class. The original target is never included among the predictors. Wine dataset description.

The examples use Python 3.13 and these package versions, verified on 28 October 2026:

python -m pip install "splime==0.4.13" "numpy==2.3.5" "pandas==2.2.3" 
"scikit-learn==1.8.0"

Run the Python blocks in order. This first block constructs the public demonstration table, then calls numeric_rule_partition@1 to reserve an external holdout before any feature screening, imputation or rule search:

import pandas as pd
from sklearn.datasets import load_wine
from spl import SPLClient

wine = load_wine(as_frame=True)
TARGET = "is_cultivar_a"
dataset = wine.data.copy()
dataset[TARGET] = (wine.target == 0).astype(int)

client = SPLClient.embedded(
    registry_url="https://splime.io",
    cache_dir=".sph-wine-cache",
    run_receipts=False,
)
BASE = "splime://@splime.project/case5-validation-20261011"

outer_split = client.call(
    f"{BASE}/numeric_rule_partition@1",
    kwargs={"dataset": dataset, "target_col": TARGET},
    trust=True, progress=False,
).value["result"]["default"]
training_pool = dataset.iloc[outer_split["search_input_positions"]].copy()
external_holdout = dataset.iloc[outer_split["external_holdout_positions"]].copy()
assert set(training_pool.index).isdisjoint(external_holdout.index)

Splime Public Hub (SPH) provides the versioned implementation for every search, fitting and evaluation stage below. These public calls require neither registration nor API keys. The first call downloads code from SPH and executes it locally in your Python process; this DataFrame is not sent to cloud training. Optional execution receipts are disabled. trust=True authorizes the downloaded Python code to run with your process's permissions; it is not a sandbox.

The partition stage checks a fixed input contract: real numeric predictors, finite values or missing values, and a complete classification target. Invalid input stops the call; it does not select which columns to keep. It then makes a stratified 75/25 split with seed 42 and returns row positions. This random split is an experimental convenience for Wine, not the proposed validation design for a loan portfolio.

Make the training boundaries visible

The modular route starts with rule_validate@1 and rule_split@1, using the original search settings on this numeric Wine table:

options = {
    "feature_selection_k": 128,
    "feature_selection_min_fraction": 0.5,
    "date_col": None, "group_col": None, "time_gap": 0,
    "max_candidates": 24, "cv_folds": 3,
    "metric": "f1_macro", "random_state": 42,
    "depths": [2, 3, 4], "leaf_fractions": [0.05, 0.1, 0.15],
    "min_leaf_count": 20, "max_leaf_nodes": 8, "max_rules": 3,
}

validated = client.call(
    f"{BASE}/rule_validate@1",
    kwargs={"dataset": training_pool, "target_col": TARGET, "options": options},
    trust=True, progress=False,
).value["result"]["default"]

split = client.call(
    f"{BASE}/rule_split@1", kwargs={"validated": validated},
    trust=True, progress=False,
).value["result"]["default"]

print(len(split["split"]["development"]), len(split["split"]["test"]))
# 106 27

Validation checks the input contract without fitting transformations. Splitting creates an internal test and CV folds within development:

178 Wine rows
├── 133 rows supplied to the rule search
│   ├── 106 development rows: CV selection, then final fitting
│   └──  27 internal test rows: evaluation after selection
└──  45 external holdout rows: never supplied to the search

The 128-feature cap does not remove any of Wine's 13 measurements. Screening still matters: the implementation checks training-only missingness, constants and duplicates, and ranks eligible numeric features. In each CV fold, it repeats screening and learns missing-value medians from that fold's training rows. It then transforms the validation rows using those fitted choices. The final model repeats this process on the 106 development rows.

There is no categorical encoding in this Wine experiment. For a categorical extension, the vocabulary and any encoding statistics would also have to be learned inside each training fold. Encoding the whole table first would break that separation. Scikit-learn's leakage examples explain why preprocessing and feature selection belong inside validation boundaries.

Ask what the extra conditions buy

Now run rule_search@1 as its own published stage:

searched = client.call(
    f"{BASE}/rule_search@1", kwargs={"split": split},
    trust=True, progress=False,
).value["result"]["default"]

comparison = pd.DataFrame(searched["leaderboard"])
print(comparison[[
    "candidate_id", "family", "cv_mean", "mean_predicates", "mean_segments"
]].head().to_string(index=False))
print("Selected candidate:", searched["selected"]["candidate_id"])

The selected candidate is number 13. A useful comparison is:

These complexity counts are averages across CV fits, not the shape of the final fitted tree. Several candidates tied at the best score. The selected one matched the ordinary tree's validation score with fewer conditions. In this run, simplicity did not require sacrificing CV performance.

That is a narrower conclusion than saying that two conditions are always optimal. We tried a bounded set of configurations on a small sample. The three scores for the selected candidate were approximately 0.9042, 0.9689 and 0.9067; the mean hides that variation. Neither the tolerance nor the tie proves that the chosen threshold will be stable in another sample.

Freeze the model, then inspect its exceptions

Fit the chosen configuration on development with rule_fit@1, evaluate it on the internal test with rule_evaluate@1, and create the report with rule_finalize@1. Each stage is independently callable from SPH:

fitted = client.call(
    f"{BASE}/rule_fit@1", kwargs={"searched": searched},
    trust=True, progress=False,
).value["result"]["default"]

evaluated = client.call(
    f"{BASE}/rule_evaluate@1", kwargs={"fitted": fitted},
    trust=True, progress=False,
).value["result"]["default"]

report = client.call(
    f"{BASE}/rule_finalize@1", kwargs={"evaluated": evaluated},
    trust=True, progress=False,
).value["result"]["default"]

model = report["model"]
print(pd.DataFrame(report["rules"])[[
    "rule", "predicted_class", "development_n"
]].to_string(index=False))
print("Internal test accuracy:", report["metrics"]["test"]["accuracy"])

The fitted model contains one split:

proline <= 755  → predict other cultivar
proline >  755  → predict cultivar A

The selected configuration belongs to the merge_refit family, but this fit needed no merge. Its base tree has depth one and two leaves. The development counts make its limitations visible immediately:

Five rows in the upper segment already contradict its prediction. Another positive sits below the threshold. A hard boundary creates a simple decision, not a pure population.

The exact saved expression uses float32(fill_missing('proline', 660)). The median comes from development; it is part of the model, not recomputed for each prediction batch. Wine has no missing measurements, so this run does not establish whether that missing-value policy is useful in practice.

The upper segment's estimated class-1 probability is about 0.858, rather than the raw fraction 34/39. The implementation smooths counts toward the development class proportions. This is not a separately calibrated probability, and certainly not a probability of loan default.

The internal test gives 25 correct predictions out of 27, or 92.6% accuracy. Those 27 rows were not used to select the configuration or fit the final model. They are not subsequently added to training.

Open the external holdout without moving the threshold

Prediction with rule_predict@1 needs only the saved model and measurements. Evaluation with rule_monitor@1 can then add the known answers:

scored = client.call(
    f"{BASE}/rule_predict@1",
    kwargs={"dataset": external_holdout.drop(columns=[TARGET]), "model": model},
    trust=True, progress=False,
).value["result"]["default"]

monitoring = client.call(
    f"{BASE}/rule_monitor@1",
    kwargs={"dataset": external_holdout, "model": model, "target_col": TARGET},
    trust=True, progress=False,
).value["result"]["default"]

predictions = pd.DataFrame(scored["predictions"])
assert predictions["row_position"].tolist() == list(range(len(external_holdout)))
errors = external_holdout.reset_index(drop=True).copy()
errors["prediction"] = predictions["prediction"].to_numpy()
print(errors.loc[errors[TARGET] != errors["prediction"],
                 ["proline", TARGET, "prediction"]].to_string(index=False))
print("External accuracy:", monitoring["quality"]["accuracy"])
print("Confusion matrix:", monitoring["quality"]["confusion_matrix"])

row_position refers to positions in the submitted batch, not the original pandas index. The saved schema retains all 13 measurements, so prediction receives the full feature table even though this fitted tree splits on only one.

The external result is 41 correct predictions out of 45, or 91.1% accuracy. A majority-class prediction would get 30 out of 45 correct. The errors are more informative than that comparison alone:

All 15 cultivar A samples were found, but four other samples also crossed the threshold. There were 19 positive predictions, of which 15 were correct: precision was 78.9%, with recall of 100% on these 15 positives. That small denominator matters.

Each point represents an external holdout row. Vertical jitter separates overlapping points; the vertical categories show the known class. Orange crosses mark the four mistakes.

One mistake has proline = 780, just above 755. Moving the threshold to fix it is tempting. It would also use the final holdout to make a modelling decision. We leave the rule unchanged. A revised rule would need a fresh evaluation sample.

For the banking question, the corresponding discussion would be about borrowers incorrectly referred for review and early-arrears cases missed by the segment. The acceptable balance must come from the intended action and its costs; Wine cannot answer it.

Run the whole experiment in one call

We have now seen what each stage contributes. The published numeric_rule_experiment@1 pipeline repeats the entire procedure: contract validation, outer partitioning, the six training stages, external prediction, evaluation and error reporting.

experiment = client.call(
    f"{BASE}/numeric_rule_experiment@1",
    kwargs={"dataset": dataset, "target_col": TARGET},
    trust=True, progress=False,
).value["result"]["default"]

# The one-call and step-by-step experiments must agree.
assert experiment["outer_split"] == outer_split
for key in ("model", "rules", "metrics", "segment_metrics", "split", "options"):
    assert experiment["report"][key] == report[key], key
assert experiment["external_holdout"]["monitoring"] == monitoring

holdout_result = experiment["external_holdout"]
print(holdout_result["n"] - len(holdout_result["errors"]), "/", holdout_result["n"])
# 41 / 45

This is a published workflow, not a locally assembled export. Its partition helper is the same implementation as the independently callable partition stage; its training, prediction and monitoring calls pin the public versions shown above. Both routes produced the same model, rules, selections and metrics in the verification run; candidate execution times naturally differ.

The external holdout is inside the orchestration, but outside model fitting and selection. It reaches the fitted model only for prediction and evaluation. The pipeline returns errors for inspection and makes no attempt to fix them. It downloads its implementation and dependencies as needed and runs them locally, with receipts disabled.

What must change before using this on loans

The modular experiment checks statistical separation for these inputs. It does not certify a banking dataset as free of leakage.

The original find_rules@4 release reproduced these same Wine results, but has a concrete implementation limitation: before its internal split, it checks numeric usability across the whole supplied table. In a verification probe, replacing a proline value only in an internal-test row with text caused the whole feature to be dropped and changed the fitted model. The clean Wine run is unaffected, but this behaviour prevents a general claim that held-out inputs cannot influence preprocessing. The modular validator route preserved training and raised an error when that malformed value reached evaluation. The complete experiment for this article uses a fixed numeric input contract: invalid types and ranges stop execution instead of changing the feature set. Statistical screening then runs only within the training partitions.

There is a second limit in the historical release: find_rules@4 recognizes optional user_id and application_date_utc columns, but invalid metadata can be excluded and the splitting strategy can fall back. Its one-block time gap is a gap between distinct timestamps, not a 90-day outcome-maturity rule. Inspecting split.strategy is necessary, but insufficient for banking validation. The new experiment deliberately declares an IID random-split contract; it does not claim to solve temporal or borrower-aware validation.

For the lending task defined here, the experiment design needs the following explicit controls:

• Later, mature evaluation cohorts. Reserve the external holdout chronologically. At every internal validation boundary, allow a training loan only if its full 90-day outcome window and reporting delay have elapsed before that boundary. The evaluation cohort must itself have complete follow-up when scored retrospectively. Sorting application dates alone does not establish either condition.
• A policy for repeat borrowers. For an evaluation intended to assess new borrowers, remove borrowers appearing in validation or test from the corresponding training partition, including each CV fold. Record the rows lost to that purge. A repeat-borrower use case needs its own chronological design and past-only histories; group-disjoint evaluation answers a different question.
• An approved feature snapshot. Exclude the target, target derivatives, future servicing fields and post-decision updates before modelling. IDs and dates used for splitting are metadata, not predictors. A column name cannot prove when its value was available.
• Training-only transformations and choices. Fit imputation, encoding, screening and models inside each training fold. Use development validation to select rules and any operating threshold. Freeze those decisions before opening the final holdout. Do not encode all categories or choose a convenient arrears cut-off after seeing holdout performance.

A bank would also need to examine segment event counts, missingness, changes in customer mix, stability over later periods and the consequences of the resulting action. A short rule is easy to inspect, but may still be unstable, unfair in its effects or operationally unhelpful. Neither predictive association nor interpretability establishes causality.

The useful result of this experiment is a compact, testable decision: one threshold, two segments and four visible external mistakes. Extra conditions did not improve the best cross-validation score here. That gives us a reason to keep the simpler model for this demonstration. In banking, a hard rule earns its place only when the same scrutiny survives the timing of the decision, the formation of the target and evaluation on later loans. If short rules are too weak under those constraints, that is a useful finding too.