elbow_helper package

Submodules

Module contents

elbow-helper — noise-robust knee/elbow detection.

A conservative wrapper around a from-scratch, NumPy-only difference-curve locator. Point-estimate knee detectors propose where plausible knees are; this package decides whether any candidate is strong, unique, persistent, reproducible and unlikely under a no-knee model and otherwise abstains explicitly.

Public API

robust_knee(x, y=None, curve=None, direction=None, config=None) -> ClearKnee | NoClearKnee. y may be omitted (x is then the y-values alone, against an implicit 0, 1, ..., n-1); curve and direction are inferred from the data when omitted. robust_elbow(x, y=None, config=None) is the convex-decreasing convenience. RobustKneeConfig holds every threshold. KneeLocator is the from-scratch locator, usable standalone.

robust_knees(x, y=None, config=None) (plural) -> Knees | InvalidKnees: how many knees, if any, a curve genuinely has, via dynamic-program search and a Bonferroni-gated modified-BIC criterion (see ELBOW-en.tex and research/multiknee/RESULTS.md). RobustKneesConfig holds its settings.

Author

Warith Harchaoui, <warith.harchaoui@deraison.ai>

class elbow_helper.ClearKnee(reason, diagnostics=<factory>, knee_x=0.0, knee_x_norm=0.0, knee_index=0, ci90=(0.0, 0.0), detection_rate=0.0, smoothing_window=1, sensitivity=1.0, prominence=0.0, slope_contrast=0.0, bic_improvement=0.0, null_p_value=1.0)[source]

Bases: KneeResult

A knee accepted by every stage of the pipeline, with uncertainty.

Parameters:
knee_x

The detected knee, in the caller’s original data units.

Type:

float

knee_x_norm

The detected knee, normalized to [0, 1].

Type:

float

knee_index

Index of the knee on the cleaned, sorted curve.

Type:

int

ci90

90% bootstrap interval for the knee location, data units.

Type:

tuple of float

detection_rate

Fraction of bootstrap replicates in which the knee was redetected.

Type:

float

smoothing_window, sensitivity

The scale-space setting that produced the winning candidate.

Type:

int, float

prominence

Topographic prominence of the winning candidate’s peak.

Type:

float

slope_contrast

Normalized Theil-Sen slope contrast across the knee.

Type:

float

bic_improvement

BIC improvement of the broken line over a single line.

Type:

float

null_p_value

Monte-Carlo p-value from the no-knee null test.

Type:

float

bic_improvement: float = 0.0
ci90: Tuple[float, float] = (0.0, 0.0)
detection_rate: float = 0.0
knee_index: int = 0
knee_x: float = 0.0
knee_x_norm: float = 0.0
null_p_value: float = 1.0
prominence: float = 0.0
sensitivity: float = 1.0
slope_contrast: float = 0.0
smoothing_window: int = 1
class elbow_helper.InvalidKnees(reason, diagnostics=<factory>)[source]

Bases: MultiKneeResult

Preprocessing failed: the input could not be searched at all.

Parameters:
class elbow_helper.KneeEstimate(x, x_norm, index, slope_left, slope_right, fwer_p_value=None)[source]

Bases: object

One accepted breakpoint from elbow_helper.robust_knees().

Segments are independent (discontinuous) OLS lines, not the continuous broken-line model ClearKnee uses; slope_left/slope_right are each segment’s own fitted slope, in data units and need not agree at the breakpoint.

Parameters:
x

The breakpoint, in the caller’s original data units.

Type:

float

x_norm

The breakpoint, normalized to [0, 1].

Type:

float

index

Index of the breakpoint on the cleaned, sorted curve.

Type:

int

slope_left, slope_right

Each neighbouring segment’s own fitted slope, data units.

Type:

float

fwer_p_value

The sequential FWER test’s p-value for this breakpoint, None when the FWER gate was not run (require_fwer_confirmation=False).

Type:

float, optional

fwer_p_value: float | None = None
index: int
slope_left: float
slope_right: float
x: float
x_norm: float
class elbow_helper.KneeLocator(x, y, S=1.0, curve='concave', direction='increasing', interp_method='interp1d', online=False, polynomial_degree=7)[source]

Bases: object

Locate the point of maximum curvature (knee/elbow) of a curve.

A NumPy-only implementation of the difference-curve locator, exposing the public surface used by this package: knee, norm_knee, all_knees, all_norm_knees, x_difference / y_difference and the extrema indices.

Parameters:

  • x (array-like) – Input coordinates, equal length. x must be strictly increasing.

  • y (array-like) – Input coordinates, equal length. x must be strictly increasing.

  • S (float, optional) – Sensitivity; larger values are more conservative. Default 1.0.

  • curve (str, optional) – "concave" to detect knees, "convex" to detect elbows.

  • direction (str, optional) – "increasing" or "decreasing".

  • interp_method (str, optional) – "interp1d" (identity fit) or "polynomial" (numpy.polyfit).

  • online (bool, optional) – If True, keep correcting the knee while traversing; if False, return the first knee found. Default False.

  • polynomial_degree (int, optional) – Degree used when interp_method="polynomial". Default 7.

property all_elbows

Alias for all_knees, for callers who think in “elbows”.

property all_norm_elbows

Alias for all_norm_knees, for callers who think in “elbows”.

property elbow

Alias for knee, for callers who think in “elbows”.

find_knee()[source]

Traverse the difference curve and return (knee, norm_knee).

Returns:

(knee, norm_knee) in data-space and normalized-space coordinates respectively, or (None, None) if no candidate clears its sensitivity threshold.

Return type:

tuple of (float or None), (float or None)

property norm_elbow

Alias for norm_knee, for callers who think in “elbows”.

static transform_y(y, direction, curve)[source]

Orient y to a concave, increasing frame (elbows become knees).

Parameters:

  • y (numpy.ndarray) – Normalized y values.

  • direction (str) – "increasing" or "decreasing".

  • curve (str) – "concave" or "convex".

Returns:

y, flipped and/or mirrored so the concave-increasing bump logic in find_knee() applies unchanged.

Return type:

numpy.ndarray

class elbow_helper.KneeResult(reason, diagnostics=<factory>)[source]

Bases: object

Base class for the tagged union returned by robust_knee().

Parameters:
diagnostics: Dict
property is_clear: bool

True for ClearKnee, False for NoClearKnee.

reason: str
class elbow_helper.Knees(reason, diagnostics=<factory>, knees=<factory>)[source]

Bases: MultiKneeResult

A valid multi-knee result: zero or more accepted breakpoints.

Unlike NoClearKnee, an empty knees list here is not an abstention: it is the pipeline’s confident conclusion that the data has no real breakpoint, having survived the same search and false-positive gates a nonempty result would have to survive.

Parameters:
property k: int

Number of accepted breakpoints.

knees: List[KneeEstimate]
class elbow_helper.MultiKneeResult(reason, diagnostics=<factory>)[source]

Bases: object

Base class for the tagged union returned by elbow_helper.robust_knees().

Parameters:
diagnostics: Dict
property is_valid: bool

True for Knees, False for InvalidKnees.

reason: str
class elbow_helper.NoClearKnee(reason, diagnostics=<factory>)[source]

Bases: KneeResult

An explicit abstention: no knee is strong enough to report.

Parameters:
class elbow_helper.Reason[source]

Bases: object

Stable, machine-readable abstention (and status) reason codes.

ALL_CANDIDATES_WEAK = 'ALL_CANDIDATES_WEAK'
BOOTSTRAP_MULTIMODAL = 'BOOTSTRAP_MULTIMODAL'
BOOTSTRAP_UNSTABLE = 'BOOTSTRAP_UNSTABLE'
BOUNDARY_KNEE = 'BOUNDARY_KNEE'
CLEAR_KNEE = 'CLEAR_KNEE'
INCOMPATIBLE_GLOBAL_SHAPE = 'INCOMPATIBLE_GLOBAL_SHAPE'
INSUFFICIENT_DATA = 'INSUFFICIENT_DATA'
INTERNAL_NUMERICAL_FAILURE = 'INTERNAL_NUMERICAL_FAILURE'
INVALID_INPUT = 'INVALID_INPUT'
KNEES_FOUND = 'KNEES_FOUND'
MULTIPLE_PLAUSIBLE_KNEES = 'MULTIPLE_PLAUSIBLE_KNEES'
NO_KNEE_CANDIDATES = 'NO_KNEE_CANDIDATES'
NO_PERSISTENT_CLUSTER = 'NO_PERSISTENT_CLUSTER'
NULL_NOT_REJECTED = 'NULL_NOT_REJECTED'
SEGMENTED_MODEL_NOT_BETTER = 'SEGMENTED_MODEL_NOT_BETTER'
WEAK_SLOPE_CHANGE = 'WEAK_SLOPE_CHANGE'
ZERO_RANGE = 'ZERO_RANGE'
class elbow_helper.RobustKneeConfig(min_samples=20, smoothing_fractions=(0.0, 0.02, 0.03, 0.05, 0.08, 0.12, 0.18, 0.25), sensitivity_fractions=(0.0, 0.01, 0.02, 0.05), min_spearman_abs=0.6, max_direction_violation_rate=0.25, boundary_margin=0.1, min_side_points=5, min_prominence=0.05, min_noise_prominence_ratio=4.0, cluster_tolerance=0.06, min_consecutive_scales=3, min_sensitivity_support=0.7, max_cluster_mad=0.03, max_neighbor_shift=0.07, secondary_support_frac=0.3, min_dominance_ratio=2.0, slope_left_window=(0.15, 0.03), slope_right_window=(0.03, 0.15), min_slope_contrast=0.3, min_cv_improvement=0.1, min_bic_improvement=10.0, cv_folds=5, bootstrap_replicates=100, min_bootstrap_detection_rate=0.9, max_ci90_width=0.1, min_primary_cluster_rate=0.8, max_secondary_cluster_rate=0.15, max_bootstrap_median_shift=0.03, null_replicates=200, max_null_p_value=0.01, random_seed=None)[source]

Bases: object

Immutable bundle of thresholds and search settings.

Use with_() to derive a tweaked copy (the dataclass is frozen).

Parameters:

  • min_samples (int)

  • smoothing_fractions (Tuple[float, ...])

  • sensitivity_fractions (Tuple[float, ...])

  • min_spearman_abs (float)

  • max_direction_violation_rate (float)

  • boundary_margin (float)

  • min_side_points (int)

  • min_prominence (float)

  • min_noise_prominence_ratio (float)

  • cluster_tolerance (float)

  • min_consecutive_scales (int)

  • min_sensitivity_support (float)

  • max_cluster_mad (float)

  • max_neighbor_shift (float)

  • secondary_support_frac (float)

  • min_dominance_ratio (float)

  • slope_left_window (Tuple[float, float])

  • slope_right_window (Tuple[float, float])

  • min_slope_contrast (float)

  • min_cv_improvement (float)

  • min_bic_improvement (float)

  • cv_folds (int)

  • bootstrap_replicates (int)

  • min_bootstrap_detection_rate (float)

  • max_ci90_width (float)

  • min_primary_cluster_rate (float)

  • max_secondary_cluster_rate (float)

  • max_bootstrap_median_shift (float)

  • null_replicates (int)

  • max_null_p_value (float)

  • random_seed (int | None)

bootstrap_replicates: int = 100
boundary_margin: float = 0.1
cluster_tolerance: float = 0.06
cv_folds: int = 5
max_bootstrap_median_shift: float = 0.03
max_ci90_width: float = 0.1
max_cluster_mad: float = 0.03
max_direction_violation_rate: float = 0.25
max_neighbor_shift: float = 0.07
max_null_p_value: float = 0.01
max_secondary_cluster_rate: float = 0.15
min_bic_improvement: float = 10.0
min_bootstrap_detection_rate: float = 0.9
min_consecutive_scales: int = 3
min_cv_improvement: float = 0.1
min_dominance_ratio: float = 2.0
min_noise_prominence_ratio: float = 4.0
min_primary_cluster_rate: float = 0.8
min_prominence: float = 0.05
min_samples: int = 20
min_sensitivity_support: float = 0.7
min_side_points: int = 5
min_slope_contrast: float = 0.3
min_spearman_abs: float = 0.6
null_replicates: int = 200
random_seed: int | None = None
secondary_support_frac: float = 0.3
sensitivity_fractions: Tuple[float, ...] = (0.0, 0.01, 0.02, 0.05)
slope_left_window: Tuple[float, float] = (0.15, 0.03)
slope_right_window: Tuple[float, float] = (0.03, 0.15)
smoothing_fractions: Tuple[float, ...] = (0.0, 0.02, 0.03, 0.05, 0.08, 0.12, 0.18, 0.25)
with_(**changes)[source]

Return a copy of this config with changes applied.

Parameters:

**changes – Field overrides, e.g. config.with_(bootstrap_replicates=500).

Returns:

A new, independent configuration.

Return type:

RobustKneeConfig

class elbow_helper.RobustKneesConfig(min_samples=20, k_max=4, min_seg_fraction=0.08, fwer_alpha=0.05, fwer_permutations=200, require_fwer_confirmation=True, random_seed=None)[source]

Bases: object

Immutable settings for elbow_helper.robust_knees() (plural).

The multi-knee search ships the combination validated in research/multiknee/RESULTS.md: dynamic-program search, the subtractive-sign modified BIC as the selection criterion, and a Bonferroni-gated sequential permutation test layered on top by default, matching this package’s design priority of minimising false-positive knees. Use with_() to derive a tweaked copy.

Parameters:

  • min_samples (int)

  • k_max (int)

  • min_seg_fraction (float)

  • fwer_alpha (float)

  • fwer_permutations (int)

  • require_fwer_confirmation (bool)

  • random_seed (int | None)

fwer_alpha: float = 0.05
fwer_permutations: int = 200
k_max: int = 4
min_samples: int = 20
min_seg_fraction: float = 0.08
random_seed: int | None = None
require_fwer_confirmation: bool = True
with_(**changes)[source]

Return a copy of this config with changes applied.

Return type:

RobustKneesConfig

elbow_helper.robust_elbow(x, y=None, config=None)[source]

Convenience wrapper for the classic convex-decreasing elbow.

Equivalent to robust_knee() with curve="convex" and direction="decreasing", the k-means inertia / scree-plot case. y may be omitted, as in robust_knee().

Parameters:

config (RobustKneeConfig | None)

Return type:

KneeResult

elbow_helper.robust_knee(x, y=None, curve=None, direction=None, config=None)[source]

Detect a knee conservatively or abstain with a reason.

Parameters:

  • x (array-like) – The curve: x[i] maps to y[i]. x need not be sorted or unique; preprocessing handles cleaning, sorting, deduplication and normalization. y may be omitted, in which case x is taken to be the sequence of y-values alone and the implicit x-axis 0, 1, ..., n-1 is used.

  • y (array-like) – The curve: x[i] maps to y[i]. x need not be sorted or unique; preprocessing handles cleaning, sorting, deduplication and normalization. y may be omitted, in which case x is taken to be the sequence of y-values alone and the implicit x-axis 0, 1, ..., n-1 is used.

  • curve (str, optional) – "concave" (knees) or "convex" (elbows). If omitted, inferred from the data: a curve lying above the chord connecting its endpoints is concave, below is convex.

  • direction (str, optional) – "increasing" or "decreasing". If omitted, inferred from the sign of the trend between x and y.

  • config (RobustKneeConfig, optional) – Thresholds and replicate counts. Defaults to RobustKneeConfig.

Returns:

A ClearKnee (with location, 90% interval and diagnostics) or a NoClearKnee (with a reason code and diagnostics).

Return type:

KneeResult

elbow_helper.robust_knees(x, y=None, config=None)[source]

Detect zero or more knees, with the same abstain-rather-than-guess discipline.

Unlike robust_knee(), an empty result is not an abstention: it is the pipeline’s confident conclusion that the data has no real breakpoint, having survived the same search and false-positive gates a nonempty result would have to survive. Only preprocessing failures (bad input, too little data, zero range) return InvalidKnees.

Parameters:

  • x (array-like) – The curve: x[i] maps to y[i]. y may be omitted, in which case x is taken to be the y-values alone against an implicit 0, 1, ..., n-1, as in robust_knee(). No curve or direction is needed: segments may alternate slope sign freely.

  • y (array-like) – The curve: x[i] maps to y[i]. y may be omitted, in which case x is taken to be the y-values alone against an implicit 0, 1, ..., n-1, as in robust_knee(). No curve or direction is needed: segments may alternate slope sign freely.

  • config (RobustKneesConfig, optional) – Search size, false-positive-control settings. Defaults to RobustKneesConfig.

Returns:

A Knees (with zero or more KneeEstimate, and diagnostics from every stage) or an InvalidKnees (with a reason code, for unusable input only).

Return type:

MultiKneeResult