RadiusNeighbors / Classifier Layer

Radius-based Nearest Neighbors Classifier - Adaptive neighborhood classification. Classifier implementing a vote among neighbors within a given radius.

Mathematical form: $\overset{y}{^} (x) = ar g y max i : d (x, x_{i}) \leq r \sum w_{i} I (y_{i} = y)$ where:

$r$ is the fixed radius
$d (x, x_{i})$ is the distance between points
$w_{i}$ is the weight of the i-th neighbor
$I ()$ is the indicator function

Comparison with k-NN:

Neighborhood size:
- k-NN: Fixed number of neighbors
- Radius: Variable number based on density
Density handling:
- k-NN: May stretch in sparse regions
- Radius: Adapts to local density
Empty neighborhoods:
- k-NN: Never empty (always k points)
- Radius: Possible in sparse regions
Computational aspects:
- k-NN: Predictable computation time
- Radius: Variable depending on density

Key characteristics:

Variable neighborhood size
Density-adaptive classification
Handles non-uniform sampling
Distance-based decisions
Local density awareness

Common applications:

Variable density datasets
Spatial classification
Anomaly detection
Density-sensitive problems
Point cloud classification
Spatially varying sampling

Advantages:

Better for non-uniform sampling
Natural density adaptation
More interpretable distance threshold
Automatic local scaling

Limitations:

Risk of empty neighborhoods
Variable computation time
Requires careful radius selection
Sensitive to scale

Outputs:

Predicted Table: Input data with predictions
Validation Results: Cross-validation metrics
Test Metric: Test set performance
ROC Curve Data: ROC analysis information
Confusion Matrix: Classification breakdown
Feature Importances: Distance-based importance

Note: Particularly effective when data sampling density varies significantly across feature space

Table

Predicted Table

Validation Results

Test Metric

ROC Curve Data

Confusion Matrix

Feature Importances

SelectFeatures

[column, ...]

Feature columns for Radius Neighbors:

Requirements:

Numerical features
No missing values
Finite values
Distance-compatible

Preprocessing guidelines:

Scaling (critical):
- StandardScaler recommended
- Affects radius interpretation
- Consistent distance scaling
Density consideration:
- Check local densities
- Monitor sparse regions
- Consider density variations
Feature engineering:
- Distance-relevant features
- Density-aware transformations
- Meaningful spatial relations

If empty, uses all numeric columns except target.

SelectTarget

column

Target column for radius-based classification:

Requirements:

Categorical labels
No missing values
At least two classes
Properly encoded

Density considerations:

Class-wise density variations
Local class distributions
Empty neighborhood handling
Spatial class patterns

Quality checks:

Density-based validation
Class distribution analysis
Spatial distribution checks
Coverage verification

Params

oneof

DefaultParams

Optimized default configuration for Radius Neighbors:

Default settings:

radius: 1.0 (unit hypersphere)
weights: uniform (equal importance)
algorithm: auto (adaptive selection)
leaf_size: 30 (balanced tree)
power: 2 (Euclidean distance)

Best suited for:

Variable density data
Scaled features
Initial exploration
When density matters

Note: Radius choice critical for performance

CustomParams

Fine-grained control over Radius Neighbors parameters:

Parameter groups:

Neighborhood definition (radius)
Distance computation (weights, power)
Algorithm optimization (algorithm, leaf_size)

Trade-offs:

Coverage vs. specificity
Computation speed vs. accuracy
Memory usage vs. performance

Note: Parameter interactions particularly important for density variations

Radius

f64

Radius for neighborhood definition:

Effect: $N (x) = {x_{i} : d (x, x_{i}) \leq r}$

Selection guide:

Small radius: High precision, risk of empty neighborhoods
Large radius: Better coverage, higher computation cost
Consider local density variations

Typical ranges:

Dense regions: 0.1 - 1.0
Sparse regions: 1.0 - 5.0
Scale with feature normalization

Weights

enum

uniform

Neighbor weighting scheme within the fixed radius:

Mathematical form: For a point x and neighbors within radius r: $f (x) = i : d (x, x_{i}) \leq r \sum w_{i} y_{i}$

Comparison with k-NN weights:

k-NN: Weights applied to fixed k points
Radius: Weights applied to variable number of points

Impact on density:

Affects contribution of points at different distances
Particularly important for varying density regions
Helps manage boundary effects near radius limit

uniform ~

Equal weights for all neighbors within radius:

Formula: $w_{i} = 1$ for all points where $d (x, x_{i}) \leq r$

Properties:

Simple majority voting
Equal influence within radius
Sharp boundary at radius edge
More sensitive to radius choice

Best for:

Well-separated classes
When distance within radius not informative
Computational efficiency
Clear decision boundaries

distance ~

Inverse distance weighting within radius:

Formula: $w_{i} = \frac{1}{d ( x , x _{i} )}$ for points where $d (x, x_{i}) \leq r$

Properties:

Smoother distance decay
Reduces radius boundary effect
More weight to closer points
Natural density weighting

Best for:

Continuous feature spaces
Gradual class transitions
Variable density regions
When closer points more reliable

Note: More robust to radius choice than uniform weights

Algorithm

enum

auto

Algorithm for computing nearest neighbors:

Selection criteria:

Data dimensionality
Sample size
Distance metric
Memory constraints

auto ~

Automatic algorithm selection:

Chooses based on:

Dataset size
Feature dimensionality
Metric type
Available memory

ball_tree ~

Ball Tree algorithm:

Best for:

High dimensions (n > 3)
Complex distance metrics
Variable density data
Memory efficiency needed

kd_tree ~

KD Tree algorithm:

Best for:

Low dimensions (n ≤ 3)
Euclidean distance
Uniform density data
Fast queries needed

brute ~

Brute force search:

Best for:

Small datasets
Very high dimensions
Custom metrics
When exact NN needed

LeafSize

u32

Tree algorithm leaf size parameter:

Impact on performance:

Construction time: $O (D N lo g N / leaf_size)$
Query time: Varies with leaf_size
Memory: Increases with smaller leaf_size

Trade-offs:

Small values: Faster queries, more memory
Large values: Slower queries, less memory
Optimal range: 10-50 for most cases

Note: Only affects tree-based algorithms

Power

f64

Minkowski distance power parameter:

Distance formula: $d (x, y) = (i \sum ∣ x_{i} - y_{i} ∣^{p})^{1/ p}$

Common values:

p=1: Manhattan distance
p=2: Euclidean distance (default)
p→∞: Chebyshev distance

Selection criteria:

Feature space geometry
Data distribution
Domain knowledge
Computational needs

Note: Affects radius interpretation

GridSearch

Hyperparameter optimization for Radius Neighbors:

Search strategy:

Radius optimization (primary focus)
Weight and distance metric selection
Algorithm and performance tuning

Key considerations:

Data density variations
Empty neighborhood handling
Computational resources
Coverage requirements

Radius

[f64, ...]

Radius values to evaluate:

Common grids:

Conservative: [0.5, 1.0, 1.5]
Wide range: [0.1, 1.0, 10.0]
Log-scale: [0.1, 0.3, 1.0, 3.0]

Selection strategy:

Start with log-spaced values
Monitor empty neighborhoods
Consider feature scaling
Check density distribution

Note: Most critical parameter for optimization

Weights

[enum, ...]

uniform

Neighbor weighting scheme within the fixed radius:

Mathematical form: For a point x and neighbors within radius r: $f (x) = i : d (x, x_{i}) \leq r \sum w_{i} y_{i}$

Comparison with k-NN weights:

k-NN: Weights applied to fixed k points
Radius: Weights applied to variable number of points

Impact on density:

Affects contribution of points at different distances
Particularly important for varying density regions
Helps manage boundary effects near radius limit

uniform ~

Equal weights for all neighbors within radius:

Formula: $w_{i} = 1$ for all points where $d (x, x_{i}) \leq r$

Properties:

Simple majority voting
Equal influence within radius
Sharp boundary at radius edge
More sensitive to radius choice

Best for:

Well-separated classes
When distance within radius not informative
Computational efficiency
Clear decision boundaries

distance ~

Inverse distance weighting within radius:

Formula: $w_{i} = \frac{1}{d ( x , x _{i} )}$ for points where $d (x, x_{i}) \leq r$

Properties:

Smoother distance decay
Reduces radius boundary effect
More weight to closer points
Natural density weighting

Best for:

Continuous feature spaces
Gradual class transitions
Variable density regions
When closer points more reliable

Note: More robust to radius choice than uniform weights

Algorithm

[enum, ...]

auto

Algorithm for computing nearest neighbors:

Selection criteria:

Data dimensionality
Sample size
Distance metric
Memory constraints

auto ~

Automatic algorithm selection:

Chooses based on:

Dataset size
Feature dimensionality
Metric type
Available memory

ball_tree ~

Ball Tree algorithm:

Best for:

High dimensions (n > 3)
Complex distance metrics
Variable density data
Memory efficiency needed

kd_tree ~

KD Tree algorithm:

Best for:

Low dimensions (n ≤ 3)
Euclidean distance
Uniform density data
Fast queries needed

brute ~

Brute force search:

Best for:

Small datasets
Very high dimensions
Custom metrics
When exact NN needed

LeafSize

[u32, ...]

Leaf size values for tree algorithms:

Common ranges:

Small data: [10, 20, 30]
Medium data: [30, 50, 70]
Large data: [50, 100, 150]

Trade-offs:

Build time vs query time
Memory usage vs speed
Tree balance vs efficiency

Note: Only relevant for tree-based algorithms

Power

[f64, ...]

Minkowski distance powers to evaluate:

Common combinations:

Standard: [2.0] (Euclidean)
Extended: [1.0, 2.0, 3.0]
Complete: [1.0, 1.5, 2.0, 3.0]

Impact:

Affects radius interpretation
Changes neighborhood shape
Influences distance weighting
Computational complexity

RefitScore

enum

Accuracy

Performance metric for model evaluation:

Selection criteria:

Default: Model's built-in scoring
Accuracy: Overall correctness
BalancedAccuracy: For imbalanced data
LogLoss: Probability quality
RocAuc: Threshold-independent

Choose based on:

Class distribution
Problem requirements
Prediction type needed

Default ~

Uses estimator's built-in scoring method:

For Bernoulli NB:

Returns accuracy score
Equal weight to all samples
Fast computation

Best for:

Quick evaluation
Balanced datasets
Initial testing

Accuracy ~

Standard classification accuracy score:

Formula: $Accuracy = \frac{TP + TN}{TP + TN + FP + FN}$

Properties:

Range: [0, 1]
Perfect score: 1.0
Baseline: max(class proportions)

Best for:

Balanced classes
Equal error costs
Simple evaluation

BalancedAccuracy ~

Class-weighted accuracy score:

Formula: $Balanced Accuracy = \frac{1}{n _{c l a sses}} i = 1 \sum n_{c l a sses} \frac{T P _{i}}{T P _{i} + F N _{i}}$

Properties:

Adjusts for class imbalance
Range: [0, 1]
Random baseline: 0.5

Best for:

Imbalanced datasets
When minority classes matter
Uneven class distributions

LogLoss ~

Logarithmic loss (cross-entropy):

Formula: $- \frac{1}{N} i = 1 \sum N j = 1 \sum M y_{ij} lo g (p_{ij})$

Properties:

Penalizes confident mistakes
Range: [0, ∞)
Perfect score: 0.0

Best for:

Probability calibration
When confidence matters
Probabilistic predictions

RocAuc ~

Area Under Receiver Operating Characteristic Curve:

Properties:

Threshold-independent
Range: [0, 1]
Random baseline: 0.5
Perfect score: 1.0

Best for:

Binary classification
Threshold tuning
Ranking evaluation
Imbalanced datasets

Note: For multiclass, computes average ROC AUC

Split

oneof

DefaultSplit

Standard train-test split configuration optimized for general classification tasks.

Configuration:

Test size: 20% (0.2)
Random seed: 98
Shuffling: Enabled
Stratification: Based on target distribution

Advantages:

Preserves class distribution
Provides reliable validation
Suitable for most datasets

Best for:

Medium to large datasets
Independent observations
Initial model evaluation

Splitting uses the ShuffleSplit strategy or StratifiedShuffleSplit strategy depending on the field stratified. Note: If shuffle is false then stratified must be false.

CustomSplit

Configurable train-test split parameters for specialized requirements. Allows fine-tuning of data division strategy for specific use cases or constraints.

Use cases:

Time series data
Grouped observations
Specific train/test ratios
Custom validation schemes

RandomState

u64

Random seed for reproducible splits. Ensures:

Consistent train/test sets
Reproducible experiments
Comparable model evaluations

Same seed guarantees identical splits across runs.

Shuffle

bool

true

Data shuffling before splitting. Effects:

true: Randomizes order, better for i.i.d. data
false: Maintains order, important for time series

When to disable:

Time dependent data
Sequential patterns
Grouped observations

TrainSize

f64

0.8

Proportion of data for training. Considerations:

Larger (e.g., 0.8-0.9): Better model learning
Smaller (e.g., 0.5-0.7): Better validation

Common splits:

0.8: Standard (80/20 split)
0.7: More validation emphasis
0.9: More training emphasis

Stratified

bool

false

Maintain class distribution in splits. Important when:

Classes are imbalanced
Small classes present
Representative splits needed

Requirements:

Classification tasks only
Cannot use with shuffle=false
Sufficient samples per class

Cv

oneof

DefaultCv

Standard cross-validation configuration using stratified 3-fold splitting.

Configuration:

Folds: 3
Method: StratifiedKFold
Stratification: Preserves class proportions

Advantages:

Balanced evaluation
Reasonable computation time
Good for medium-sized datasets

Limitations:

May be insufficient for small datasets
Higher variance than larger fold counts
May miss some data patterns

CustomCv

Configurable stratified k-fold cross-validation for specific validation requirements.

Features:

Adjustable fold count with NFolds determining the number of splits.
Stratified sampling
Preserved class distributions

Use cases:

Small datasets (more folds)
Large datasets (fewer folds)
Detailed model evaluation
Robust performance estimation

NFolds

u32

Number of cross-validation folds. Guidelines:

3-5: Large datasets, faster training
5-10: Standard choice, good balance
10+: Small datasets, thorough evaluation

Trade-offs:

More folds: Better evaluation, slower training
Fewer folds: Faster training, higher variance

Must be at least 2.

KfoldCv

K-fold cross-validation without stratification. Divides data into k consecutive folds for iterative validation.

Process:

Splits data into k equal parts
Each fold serves as validation once
Remaining k-1 folds form training set

Use cases:

Regression problems
Large, balanced datasets
When stratification unnecessary
Continuous target variables

Limitations:

May not preserve class distributions
Less suitable for imbalanced data
Can create biased splits with ordered data

NSplits

u32

Number of folds for cross-validation. Selection guide: Recommended values:

5: Standard choice (default)
3: Large datasets/quick evaluation
10: Thorough evaluation/smaller datasets

Trade-offs:

Higher values: More thorough, computationally expensive
Lower values: Faster, potentially higher variance

Must be at least 2 for valid cross-validation.

RandomState

u64

Random seed for fold generation when shuffling. Important for:

Reproducible results
Consistent fold assignments
Benchmark comparisons
Debugging and validation

Set specific value for reproducibility across runs.

Shuffle

bool

true

Whether to shuffle data before splitting into folds. Effects:

true: Randomized fold composition (recommended)
false: Sequential splitting

Enable when:

Data may have ordering
Better fold independence needed

Disable for:

Time series data
Ordered observations

StratifiedKfoldCv

Stratified K-fold cross-validation maintaining class proportions across folds.

Key features:

Preserves class distribution in each fold
Handles imbalanced datasets
Ensures representative splits

Best for:

Classification problems
Imbalanced class distributions
When class proportions matter

Requirements:

Classification tasks only
Sufficient samples per class
Categorical target variable

NSplits

u32

Number of stratified folds. Guidelines: Typical values:

5: Standard for most cases
3: Quick evaluation/large datasets
10: Detailed evaluation/smaller datasets

Considerations:

Must allow sufficient samples per class per fold
Balance between stability and computation time
Consider smallest class size when choosing

RandomState

u64

Seed for reproducible stratified splits. Ensures:

Consistent fold assignments
Reproducible results
Comparable experiments
Systematic validation

Fixed seed guarantees identical stratified splits.

Shuffle

bool

false

Data shuffling before stratified splitting. Impact:

true: Randomizes while maintaining stratification
false: Maintains data order within strata

Use cases:

true: Independent observations
false: Grouped or sequential data

Class proportions maintained regardless of setting.

ShuffleSplitCv

Random permutation cross-validator with independent sampling.

Characteristics:

Random sampling for each split
Independent train/test sets
More flexible than K-fold
Can have overlapping test sets

Advantages:

Control over test size
Fresh splits each iteration
Good for large datasets

Limitations:

Some samples might never be tested
Others might be tested multiple times
No guarantee of complete coverage

NSplits

u32

Number of random splits to perform. Consider: Common values:

5: Standard evaluation
10: More thorough assessment
3: Quick estimates

Trade-offs:

More splits: Better estimation, longer runtime
Fewer splits: Faster, less stable estimates

Balance between computation and stability.

RandomState

u64

Random seed for reproducible shuffling. Controls:

Split randomization
Sample selection
Result reproducibility

Important for:

Debugging
Comparative studies
Result verification

TestSize

f64

0.2

Proportion of samples for test set. Guidelines: Common ratios:

0.2: Standard (80/20 split)
0.25: More validation emphasis
0.1: More training data

Considerations:

Dataset size
Model complexity
Validation requirements

It must be between 0.0 and 1.0.

StratifiedShuffleSplitCv

Stratified random permutation cross-validator combining shuffle-split with stratification.

Features:

Maintains class proportions
Random sampling within strata
Independent splits
Flexible test size

Ideal for:

Imbalanced datasets
Large-scale problems
When class distributions matter
Flexible validation schemes

NSplits

u32

Number of stratified random splits. Guidelines: Recommended values:

5: Standard evaluation
10: Detailed analysis
3: Quick assessment

Consider:

Sample size per class
Computational resources
Stability requirements

RandomState

u64

Seed for reproducible stratified sampling. Ensures:

Consistent class proportions
Reproducible splits
Comparable experiments

Critical for:

Benchmarking
Research studies
Quality assurance

TestSize

f64

0.2

Fraction of samples for stratified test set. Best practices: Common splits:

0.2: Balanced evaluation
0.3: More thorough testing
0.15: Preserve training size

Consider:

Minority class size
Overall dataset size
Validation objectives

It must be between 0.0 and 1.0.

Time Series cross-validator. Provides train/test indices to split time series data samples that are observed at fixed time intervals, in train/test sets. It is a variation of k-fold which returns first k folds as train set and the k + 1th fold as test set. Note that unlike standard cross-validation methods, successive training sets are supersets of those that come before them. Also, it adds all surplus data to the first training partition, which is always used to train the model. Key features:

Maintains temporal dependence
Expanding window approach
Forward-chaining splits
No future data leakage

Use cases:

Sequential data
Financial forecasting
Temporal predictions
Time-dependent patterns

Note: Training sets are supersets of previous iterations.

NSplits

u32

Number of temporal splits. Considerations: Typical values:

5: Standard forward chaining
3: Limited historical data
10: Long time series

Impact:

Affects training window growth
Determines validation points
Influences computational load

MaxTrainSize

u64

Maximum size of training set. Should be strictly less than the number of samples. Applications:

0: Use all available past data
>0: Rolling window of fixed size

Use cases:

Limit historical relevance
Control computational cost
Handle concept drift
Memory constraints

TestSize

u64

Number of samples in each test set. When 0:

Auto-calculated as n_samples/(n_splits+1)
Ensures equal-sized test sets

Considerations:

Forecast horizon
Validation requirements
Available future data

Gap

u64

Number of samples to exclude from the end of each train set before the test set.Gap between train and test sets. Uses:

Avoid data leakage
Model forecast lag
Buffer periods

Common scenarios:

0: Continuous prediction
>0: Forward gap for realistic evaluation
Match business forecasting needs