LinearDiscriminantAnalysis / Classifier Layer

Linear Discriminant Analysis Classification: A classifier with a linear decision boundary, generated by fitting class conditional densities to the data and using Bayes' rule. The fitted model can also be used to reduce the dimensionality of the input by projecting it to the most discriminative directions, using the transform method.

Mathematical formulation: $P (y ∣ x) = \frac{P ( x ∣ y ) P ( y )}{P ( x )}$ where:

P(y|x) is the posterior probability
P(x|y) is modeled as Gaussian distribution
P(y) represents class priors

Key assumptions:

Gaussian class distributions
Equal covariance matrices
Linear decision boundaries
Sufficient sample sizes

Advantages:

Dimensionality reduction
Interpretable results
Fast computation
Stable predictions
Probabilistic output

Common applications:

Face recognition
Marketing segmentation
Biomedical classification
Document categorization
Financial analysis

Outputs:

Predicted Table: Results with predictions
Validation Results: Cross-validation metrics
Test Metric: Hold-out performance
ROC Curve Data: Classification analysis
Confusion Matrix: Class predictions
Feature Importances: Discriminant coefficients

Table

Predicted Table

Validation Results

Test Metric

ROC Curve Data

Confusion Matrix

Feature Importances

SelectFeatures

[column, ...]

Feature column selection for Linear Discriminant Analysis:

Data requirements:

Statistical assumptions:
- Gaussian distribution per class
- Equal covariance matrices
- Independent features preferred
- Linear relationships
Preprocessing needs:
- Standardization recommended
- Outlier handling important
- Missing value treatment
- Feature scaling crucial
Feature considerations:
- Avoid perfect collinearity
- Check feature correlations
- Remove redundant variables
- Consider interactions
Dimensionality guidelines:
- n_samples > n_features preferred
- SVD for high dimensions
- Feature selection may help
- Consider dimensionality reduction
Quality checks:
- Verify distributions
- Test covariance equality
- Assess multicollinearity
- Monitor condition numbers

Note: If empty, automatically uses all suitable numeric columns except target

SelectTarget

column

Target column specification for LDA classification:

Requirements:

Data structure:
- Categorical labels
- Unique class values
- No missing labels
- Clear class definitions
Class characteristics:
- Minimum two classes
- Sufficient samples per class
- Well-separated groups
- Meaningful categories
Statistical considerations:
- Class-wise normality
- Similar class variances
- Balanced preferred (but not required)
- Representative sampling
Practical aspects:
- Class encoding verification
- Prior probability estimation
- Class separation assessment
- Error cost evaluation
Validation needs:
- Stratified sampling
- Class distribution preservation
- Performance metric selection
- Model evaluation strategy

Note: Must be a single column containing class labels

Params

oneof

DefaultParams

Default configuration for Linear Discriminant Analysis:

Core settings:

Solver configuration:
- SVD solver (memory efficient)
- No shrinkage
- Tolerance: 1e-4
Computational choices:
- Automatic component selection
- Standard covariance estimation
- No prior probabilities
Memory settings:
- No covariance storage
- Minimal memory footprint
- Efficient computation

Best suited for:

Initial analysis
High-dimensional data
Standard classification tasks
Memory-constrained environments

CustomParams

Customizable parameters for Linear Discriminant Analysis:

Parameter categories:

Algorithmic choices:
- Solver selection
- Numerical precision
- Regularization options
Statistical control:
- Shrinkage parameters
- Covariance estimation
- Numerical stability
Computational aspects:
- Memory usage
- Processing speed
- Numerical accuracy

Trade-offs:

Speed vs accuracy
Memory vs precision
Stability vs flexibility

Solver

enum

Svd

Numerical solvers for LDA computation:

Selection criteria:

Dataset dimensionality
Memory constraints
Numerical stability
Computational speed

Implementation notes:

Different stability-speed trade-offs
Varying memory requirements
Distinct numerical properties
Specific use case advantages

Svd ~

Singular Value Decomposition solver:

Mathematical basis: $X = U Σ V^{T}$

Advantages:

Memory efficient
Numerically stable
No covariance computation
Best for high dimensions

Best for:

Large feature sets
Memory constraints
Standard problems
Default choice

Lsqr ~

Least Squares solver:

Mathematical form: $w min ∣∣ Xw - y ∣ ∣^{2}$

Features:

Direct solution method
Supports shrinkage
Custom covariance
Explicit computation

Best for:

Small-medium datasets
When requiring shrinkage
Custom covariance needs
Explicit solutions

Eigen ~

Eigenvalue Decomposition solver:

Mathematical basis: $S_{w}^{- 1} S_{b} v = λ v$

Characteristics:

Complete decomposition
Supports shrinkage
Custom covariance
Full rank solution

Best for:

Complete analysis
Custom covariance
When stability critical
Research applications

Shrinkage

enum

None

Covariance matrix regularization methods:

Purpose:

Improve stability
Reduce overfitting
Handle collinearity
Optimize estimates

Impact:

Numerical stability
Model generalization
Prediction accuracy
Covariance estimates

None ~

No shrinkage regularization:

Characteristics:

Standard MLE estimation
Full rank assumption
Maximum flexibility
No regularization

Best for:

Well-conditioned data
Sufficient samples
When n >> p
Clean datasets

Auto ~

Ledoit-Wolf automatic shrinkage:

Method: $Σ_{shrunk} = (1 - α) Σ + α diag (Σ)$

Features:

Optimal shrinkage intensity
Data-driven selection
Automatic adaptation
Theoretical guarantees

Best for:

General use
Unknown structure
Automatic tuning
Robust estimation

Custom ~

User-specified shrinkage intensity:

Control: $α \in [0, 1]$ specifies mixing

Usage:

Manual optimization
Expert knowledge
Cross-validation tuning
Specific requirements

Best for:

Expert users
Known structure
Research settings
Parameter tuning

ShrinkageF

f64

Custom shrinkage intensity parameter. Only used when Shrinkage enum is Custom.:

Range: [0.0, 1.0] where:

0.0: No shrinkage
1.0: Maximum shrinkage

Guidelines:

Small (0.1-0.3): Mild regularization
Medium (0.3-0.7): Moderate effect
Large (0.7-0.9): Strong regularization

Use cases:

High-dimensional data
Collinear features
Noise reduction
Stability improvement

Tol

f64

0.0001

Absolute threshold for a singular value of X to be considered significant, used to estimate the rank of X. Dimensions whose singular values are non-significant are discarded.

Purpose:

Controls numerical precision
Filters weak components
Ensures stability
Manages conditioning

Typical values:

Strict: 1e-5 or smaller
Standard: 1e-4 (default)
Relaxed: 1e-3 or larger

Note: Only affects SVD solver

GridSearch

Hyperparameter optimization for Linear Discriminant Analysis:

Search space organization:

Solver selection:
- Numerical method
- Computation approach
- Memory usage
Regularization:
- Shrinkage type
- Shrinkage intensity
- Stability control
Numerical precision:
- Tolerance values
- Convergence control
- Accuracy levels

Best practices:

Start with solver selection
Add shrinkage if needed
Fine-tune tolerance last

Solver

[enum, ...]

Svd

Numerical solvers for LDA computation:

Selection criteria:

Dataset dimensionality
Memory constraints
Numerical stability
Computational speed

Implementation notes:

Different stability-speed trade-offs
Varying memory requirements
Distinct numerical properties
Specific use case advantages

Svd ~

Singular Value Decomposition solver:

Mathematical basis: $X = U Σ V^{T}$

Advantages:

Memory efficient
Numerically stable
No covariance computation
Best for high dimensions

Best for:

Large feature sets
Memory constraints
Standard problems
Default choice

Lsqr ~

Least Squares solver:

Mathematical form: $w min ∣∣ Xw - y ∣ ∣^{2}$

Features:

Direct solution method
Supports shrinkage
Custom covariance
Explicit computation

Best for:

Small-medium datasets
When requiring shrinkage
Custom covariance needs
Explicit solutions

Eigen ~

Eigenvalue Decomposition solver:

Mathematical basis: $S_{w}^{- 1} S_{b} v = λ v$

Characteristics:

Complete decomposition
Supports shrinkage
Custom covariance
Full rank solution

Best for:

Complete analysis
Custom covariance
When stability critical
Research applications

Shrinkage

[enum, ...]

None

Covariance matrix regularization methods:

Purpose:

Improve stability
Reduce overfitting
Handle collinearity
Optimize estimates

Impact:

Numerical stability
Model generalization
Prediction accuracy
Covariance estimates

None ~

No shrinkage regularization:

Characteristics:

Standard MLE estimation
Full rank assumption
Maximum flexibility
No regularization

Best for:

Well-conditioned data
Sufficient samples
When n >> p
Clean datasets

Auto ~

Ledoit-Wolf automatic shrinkage:

Method: $Σ_{shrunk} = (1 - α) Σ + α diag (Σ)$

Features:

Optimal shrinkage intensity
Data-driven selection
Automatic adaptation
Theoretical guarantees

Best for:

General use
Unknown structure
Automatic tuning
Robust estimation

Custom ~

User-specified shrinkage intensity:

Control: $α \in [0, 1]$ specifies mixing

Usage:

Manual optimization
Expert knowledge
Cross-validation tuning
Specific requirements

Best for:

Expert users
Known structure
Research settings
Parameter tuning

ShrinkageF

[f64, ...]

Shrinkage intensity search range:

Search patterns:

Coarse grid:
- [0.0, 0.3, 0.6, 0.9]
Fine grid:
- [0.1, 0.2, ..., 0.9]

Best practices:

Start with coarse grid
Refine around best values
Consider problem size

Tol

[f64, ...]

0.0001

Tolerance threshold search space:

Search ranges:

Standard scale:
- [1e-5, 1e-4, 1e-3]
Fine-tuning:
- [5e-5, 1e-4, 2e-4]

Impact:

Numerical stability
Component selection
Computation time

RefitScore

enum

Accuracy

Performance evaluation metrics for LDA classification:

Purpose:

Model evaluation
Performance comparison
Model selection
Validation assessment

Selection criteria:

Class distribution
Business objectives
Error sensitivity
Application needs

Default ~

Model's native scoring method:

Properties:

Uses prediction accuracy
Fast computation
Standard metric
Equal error weights

Best for:

Initial evaluation
Balanced datasets
Quick assessment
General comparison

Accuracy ~

Standard classification accuracy:

Formula: $Accuracy = \frac{Correct Predictions}{Total Samples}$

Characteristics:

Range: [0, 1]
Intuitive interpretation
Equal class weighting
Fast computation

Best for:

Balanced classes
Equal error costs
General performance
Simple comparison

BalancedAccuracy ~

Class-normalized accuracy score:

Formula: $BalancedAccuracy = \frac{1}{n _{c l a sses}} i = 1 \sum n_{c l a sses} \frac{T P _{i}}{T P _{i} + F N _{i}}$

Properties:

Range: [0, 1]
Class-wise normalization
Imbalance robust
Fair evaluation

Best for:

Imbalanced datasets
Varying class sizes
Fair comparison
When minority classes matter

LogLoss ~

Logarithmic loss metric:

Formula: $LogLoss = - \frac{1}{N} i = 1 \sum N y_{i} lo g (p_{i})$

Characteristics:

Range: [0, ∞)
Probability sensitive
Penalizes confidence errors
Information theoretic

Best for:

Probability estimation
Confidence assessment
Risk modeling
Calibration needs

RocAuc ~

Area Under ROC Curve score:

Measurement: $AUC = P (score (x_{p os}) > score (x_{n e g}))$

Properties:

Range: [0, 1]
Threshold invariant
Ranking quality
Discrimination power

Best for:

Binary classification
Ranking problems
When thresholds vary
Trade-off analysis

Split

oneof

DefaultSplit

Standard train-test split configuration optimized for general classification tasks.

Configuration:

Test size: 20% (0.2)
Random seed: 98
Shuffling: Enabled
Stratification: Based on target distribution

Advantages:

Preserves class distribution
Provides reliable validation
Suitable for most datasets

Best for:

Medium to large datasets
Independent observations
Initial model evaluation

Splitting uses the ShuffleSplit strategy or StratifiedShuffleSplit strategy depending on the field stratified. Note: If shuffle is false then stratified must be false.

CustomSplit

Configurable train-test split parameters for specialized requirements. Allows fine-tuning of data division strategy for specific use cases or constraints.

Use cases:

Time series data
Grouped observations
Specific train/test ratios
Custom validation schemes

RandomState

u64

Random seed for reproducible splits. Ensures:

Consistent train/test sets
Reproducible experiments
Comparable model evaluations

Same seed guarantees identical splits across runs.

Shuffle

bool

true

Data shuffling before splitting. Effects:

true: Randomizes order, better for i.i.d. data
false: Maintains order, important for time series

When to disable:

Time dependent data
Sequential patterns
Grouped observations

TrainSize

f64

0.8

Proportion of data for training. Considerations:

Larger (e.g., 0.8-0.9): Better model learning
Smaller (e.g., 0.5-0.7): Better validation

Common splits:

0.8: Standard (80/20 split)
0.7: More validation emphasis
0.9: More training emphasis

Stratified

bool

false

Maintain class distribution in splits. Important when:

Classes are imbalanced
Small classes present
Representative splits needed

Requirements:

Classification tasks only
Cannot use with shuffle=false
Sufficient samples per class

Cv

oneof

DefaultCv

Standard cross-validation configuration using stratified 3-fold splitting.

Configuration:

Folds: 3
Method: StratifiedKFold
Stratification: Preserves class proportions

Advantages:

Balanced evaluation
Reasonable computation time
Good for medium-sized datasets

Limitations:

May be insufficient for small datasets
Higher variance than larger fold counts
May miss some data patterns

CustomCv

Configurable stratified k-fold cross-validation for specific validation requirements.

Features:

Adjustable fold count with NFolds determining the number of splits.
Stratified sampling
Preserved class distributions

Use cases:

Small datasets (more folds)
Large datasets (fewer folds)
Detailed model evaluation
Robust performance estimation

NFolds

u32

Number of cross-validation folds. Guidelines:

3-5: Large datasets, faster training
5-10: Standard choice, good balance
10+: Small datasets, thorough evaluation

Trade-offs:

More folds: Better evaluation, slower training
Fewer folds: Faster training, higher variance

Must be at least 2.

KfoldCv

K-fold cross-validation without stratification. Divides data into k consecutive folds for iterative validation.

Process:

Splits data into k equal parts
Each fold serves as validation once
Remaining k-1 folds form training set

Use cases:

Regression problems
Large, balanced datasets
When stratification unnecessary
Continuous target variables

Limitations:

May not preserve class distributions
Less suitable for imbalanced data
Can create biased splits with ordered data

NSplits

u32

Number of folds for cross-validation. Selection guide: Recommended values:

5: Standard choice (default)
3: Large datasets/quick evaluation
10: Thorough evaluation/smaller datasets

Trade-offs:

Higher values: More thorough, computationally expensive
Lower values: Faster, potentially higher variance

Must be at least 2 for valid cross-validation.

RandomState

u64

Random seed for fold generation when shuffling. Important for:

Reproducible results
Consistent fold assignments
Benchmark comparisons
Debugging and validation

Set specific value for reproducibility across runs.

Shuffle

bool

true

Whether to shuffle data before splitting into folds. Effects:

true: Randomized fold composition (recommended)
false: Sequential splitting

Enable when:

Data may have ordering
Better fold independence needed

Disable for:

Time series data
Ordered observations

StratifiedKfoldCv

Stratified K-fold cross-validation maintaining class proportions across folds.

Key features:

Preserves class distribution in each fold
Handles imbalanced datasets
Ensures representative splits

Best for:

Classification problems
Imbalanced class distributions
When class proportions matter

Requirements:

Classification tasks only
Sufficient samples per class
Categorical target variable

NSplits

u32

Number of stratified folds. Guidelines: Typical values:

5: Standard for most cases
3: Quick evaluation/large datasets
10: Detailed evaluation/smaller datasets

Considerations:

Must allow sufficient samples per class per fold
Balance between stability and computation time
Consider smallest class size when choosing

RandomState

u64

Seed for reproducible stratified splits. Ensures:

Consistent fold assignments
Reproducible results
Comparable experiments
Systematic validation

Fixed seed guarantees identical stratified splits.

Shuffle

bool

false

Data shuffling before stratified splitting. Impact:

true: Randomizes while maintaining stratification
false: Maintains data order within strata

Use cases:

true: Independent observations
false: Grouped or sequential data

Class proportions maintained regardless of setting.

ShuffleSplitCv

Random permutation cross-validator with independent sampling.

Characteristics:

Random sampling for each split
Independent train/test sets
More flexible than K-fold
Can have overlapping test sets

Advantages:

Control over test size
Fresh splits each iteration
Good for large datasets

Limitations:

Some samples might never be tested
Others might be tested multiple times
No guarantee of complete coverage

NSplits

u32

Number of random splits to perform. Consider: Common values:

5: Standard evaluation
10: More thorough assessment
3: Quick estimates

Trade-offs:

More splits: Better estimation, longer runtime
Fewer splits: Faster, less stable estimates

Balance between computation and stability.

RandomState

u64

Random seed for reproducible shuffling. Controls:

Split randomization
Sample selection
Result reproducibility

Important for:

Debugging
Comparative studies
Result verification

TestSize

f64

0.2

Proportion of samples for test set. Guidelines: Common ratios:

0.2: Standard (80/20 split)
0.25: More validation emphasis
0.1: More training data

Considerations:

Dataset size
Model complexity
Validation requirements

It must be between 0.0 and 1.0.

StratifiedShuffleSplitCv

Stratified random permutation cross-validator combining shuffle-split with stratification.

Features:

Maintains class proportions
Random sampling within strata
Independent splits
Flexible test size

Ideal for:

Imbalanced datasets
Large-scale problems
When class distributions matter
Flexible validation schemes

NSplits

u32

Number of stratified random splits. Guidelines: Recommended values:

5: Standard evaluation
10: Detailed analysis
3: Quick assessment

Consider:

Sample size per class
Computational resources
Stability requirements

RandomState

u64

Seed for reproducible stratified sampling. Ensures:

Consistent class proportions
Reproducible splits
Comparable experiments

Critical for:

Benchmarking
Research studies
Quality assurance

TestSize

f64

0.2

Fraction of samples for stratified test set. Best practices: Common splits:

0.2: Balanced evaluation
0.3: More thorough testing
0.15: Preserve training size

Consider:

Minority class size
Overall dataset size
Validation objectives

It must be between 0.0 and 1.0.

TimeSeriesSplitCv

Time Series cross-validator. Provides train/test indices to split time series data samples that are observed at fixed time intervals, in train/test sets. It is a variation of k-fold which returns first k folds as train set and the k + 1th fold as test set. Note that unlike standard cross-validation methods, successive training sets are supersets of those that come before them. Also, it adds all surplus data to the first training partition, which is always used to train the model. Key features:

Maintains temporal dependence
Expanding window approach
Forward-chaining splits
No future data leakage

Use cases:

Sequential data
Financial forecasting
Temporal predictions
Time-dependent patterns

Note: Training sets are supersets of previous iterations.

NSplits

u32

Number of temporal splits. Considerations: Typical values:

5: Standard forward chaining
3: Limited historical data
10: Long time series

Impact:

Affects training window growth
Determines validation points
Influences computational load

MaxTrainSize

u64

Maximum size of training set. Should be strictly less than the number of samples. Applications:

0: Use all available past data
>0: Rolling window of fixed size

Use cases:

Limit historical relevance
Control computational cost
Handle concept drift
Memory constraints

TestSize

u64

Number of samples in each test set. When 0:

Auto-calculated as n_samples/(n_splits+1)
Ensures equal-sized test sets

Considerations:

Forecast horizon
Validation requirements
Available future data

Gap

u64

Number of samples to exclude from the end of each train set before the test set.Gap between train and test sets. Uses:

Avoid data leakage
Model forecast lag
Buffer periods

Common scenarios:

0: Continuous prediction
>0: Forward gap for realistic evaluation
Match business forecasting needs