PFI

class ordinal_xai.interpretation.pfi.PFI(model, X: DataFrame, y: ndarray = None, n_repeats=5, random_state=42, show_intervals: bool = False)[source]

Bases: BaseInterpretation

Permutation Feature Importance (PFI) interpretation method for feature importance.

This class implements the PFI method for analyzing feature importance in ordinal regression models. It measures the impact of each feature by evaluating how model performance changes when the feature values are randomly permuted.

Parameters:

  • model (object) – The trained ordinal regression model. Must implement fit and predict methods.

  • X (pd.DataFrame) – Dataset used for interpretation. Should contain the same features used during model training.

  • y (np.ndarray, optional) – Target labels. Required for evaluation but optional for initialization.

  • n_repeats (int, default=5) – Number of times to permute each feature. Higher values provide more robust importance estimates but increase computation time.

  • random_state (int, default=42) – Random seed for reproducibility of permutations.

available_metrics

Dictionary of available evaluation metrics and their corresponding functions.

Type:

dict

n_repeats

Number of permutation repeats.

Type:

int

random_state

Random state for reproducibility.

Type:

int

original_predictions

Predictions from the original model.

Type:

array-like

original_results

Evaluation results from the original model.

Type:

dict

Raises:

ValueError – If X is not a pandas DataFrame or if model predictions fail.

__init__(model, X: DataFrame, y: ndarray = None, n_repeats=5, random_state=42, show_intervals: bool = False)[source]

Initialize the Permutation Feature Importance interpretation method.

Parameters:

  • model (object) – The trained ordinal regression model

  • X (pd.DataFrame) – Dataset used for interpretation

  • y (np.ndarray, optional) – Target labels

  • n_repeats (int, default=5) – Number of times to permute each feature

  • random_state (int, default=42) – Random seed for reproducibility

Raises:

ValueError – If X is not a pandas DataFrame or if model predictions fail

_create_scoring_func(metric_name, metric_func)[source]

Create a scoring function for a specific metric.

This method creates a scoring function that can be used with sklearn’s permutation_importance. The function handles both class predictions and probability predictions, and ensures proper sign for different metric types.

Parameters:

  • metric_name (str) – Name of the metric to create scoring function for

  • metric_func (callable) – The metric function to use for scoring

Returns:

A scoring function that takes (estimator, X, y) as arguments and returns a score

Return type:

callable

Raises:

ValueError – If score calculation fails for the metric

explain(observation_idx=None, feature_subset=None, plot=False, metrics=None, title=True)[source]

Generate Permutation Feature Importance explanations.

This method computes feature importance by measuring how model performance changes when feature values are randomly permuted. The importance score represents the average change in model performance across multiple permutations.

Two modes are supported: 1. Global mode (default): Evaluates feature importance across the entire dataset. 2. Local mode (when observation_idx is provided): Evaluates feature importance for a specific instance by permuting feature values for that instance only.

Parameters:

  • observation_idx (int, optional) – Index of specific instance to analyze (local explanation). If None, computes global PFI.

  • feature_subset (list, optional) – List of feature names or indices to consider (for permutation and output only)

  • plot (bool, default=False) – Whether to create visualizations

  • metrics (list, optional) – List of metrics to use for feature importance calculation

  • title (bool, default=True) – Whether to add a suptitle to the visualization

Returns:

Dictionary containing feature importance scores for each metric. In local mode, returns per-feature importance for the specified instance.

Return type:

dict

_plot_feature_importance(results, metrics=None, title=True)[source]

Plot feature importance scores for each metric.

This method creates bar plots showing feature importance scores for each metric. The plots are arranged in a grid, with one subplot per metric. Features are sorted by importance score, and error bars show the standard deviation across permutations.

Parameters:

  • results (dict) – Dictionary containing all results from the PFI analysis

  • metrics (list, optional) – List of metrics to plot (defaults to all in results)

  • title (- Other metrics are shown in green with "Drop" in the) – Whether to add a suptitle to the visualization

  • title

  • title

  • readability (- Feature names are rotated 90 degrees for better)

  • bar (- Score values are displayed in the middle of each)

_abc_impl = <_abc._abc_data object>