LinearSvc / Classifier Layer

Linear Support Vector Classification - Similar to SVC with parameter kernel='linear', but implemented in terms of liblinear rather than libsvm, so it has more flexibility in the choice of penalties and loss functions and should scale better to large numbers of samples. Fast implementation for large-scale applications.

Mathematical form: $f (x) = w^{T} x + b$ Optimization: $w, b min \frac{1}{2} ∣∣ w ∣ ∣^{2} + C i = 1 \sum n ℓ (y_{i} (w^{T} x_{i} + b))$ where:

$w$ is the weight vector
$b$ is the bias term
$ℓ$ is the loss function
$C$ is the regularization parameter

Key characteristics:

Linear decision boundaries
Efficient large-scale learning
Multiple loss functions
L1/L2 regularization
Sparse solution support

Common applications:

Text classification
High-dimensional data
Large datasets
Real-time applications
Feature selection

Advantages over SVC:

Better scaling with samples
More flexible penalties
Lower memory usage
Faster training

Outputs:

Predicted Table: Input data with predictions
Validation Results: Cross-validation metrics
Test Metric: Test set performance
ROC Curve Data: ROC analysis information
Confusion Matrix: Classification breakdown
Feature Importances: Feature coefficients

Note: Optimized for linear classification tasks with many samples

Table

Predicted Table

Validation Results

Test Metric

ROC Curve Data

Confusion Matrix

Feature Importances

SelectFeatures

[column, ...]

Feature columns for Linear SVC:

Data requirements:

Preprocessing:
- Numerical features
- Standardized values
- No missing data
- Finite numbers
Scaling importance:
- StandardScaler preferred
- Uniform feature ranges
- Improved convergence
- Better performance
Feature engineering:
- Polynomial features
- Interaction terms
- Domain-specific transforms
- Sparse representations

Note: If empty, uses all numeric columns except target

SelectTarget

column

Target column for classification:

Requirements:

Data format:
- Categorical labels
- No missing values
- Proper encoding
- Distinct classes
Class properties:
- At least two classes
- Balanced preferred
- Clear separation
- Meaningful categories
Preprocessing:
- Label encoding
- Class weights consideration
- Distribution analysis
- Stratification needs

Params

oneof

DefaultParams

Optimized default configuration for Linear SVC:

Default settings:

Model structure:
- L2 penalty (standard regularization)
- Squared hinge loss (smooth optimization)
- One-vs-rest multi-class
Training parameters:
- C = 1.0 (balanced regularization)
- tol = 0.0001 (convergence tolerance)
- max_iter = 1000 (iteration limit)
Model features:
- fit_intercept = true (bias term)
- intercept_scaling = 1 (standard scaling)
- class_weight = none (equal weights)

Best suited for:

Large-scale applications
Linear classification
High-dimensional data
Production environments

Note: Provides efficient baseline for linear classification tasks

CustomParams

Fine-tuned configuration for Linear SVC:

Parameter categories:

Optimization control:
- Penalty type
- Loss function
- Regularization strength
Model architecture:
- Multi-class strategy
- Intercept fitting
- Class weighting
Training dynamics:
- Convergence criteria
- Iteration limits
- Numerical precision

Note: Parameter interactions affect model performance and training efficiency

Penalty

enum

Regularization norm for weight vector:

Mathematical form:

L1: $∣∣ w ∣ ∣_{1} = \sum_{i} ∣ w_{i} ∣$
L2: $∣∣ w ∣ ∣_{2} = \sum_{i} w_{i}^{2}$

Trade-offs:

Feature selection vs stability
Sparsity vs smoothness
Memory vs computation
Solution uniqueness

L1 ~

L1 norm regularization (Lasso):

Properties:

Produces sparse solutions
Built-in feature selection
Robust to outliers
Non-smooth optimization

Best for:

Feature selection needs
High-dimensional data
Redundant features
Memory constraints

Note: Not compatible with 'hinge' loss

L2 ~

L2 norm regularization (Ridge):

Properties:

Smooth solutions
Stable optimization
All features used
Unique solution

Best for:

General purpose use
Correlated features
Numerical stability
Default choice

Note: Standard SVM regularization

Loss

enum

SquaredHinge

Loss function for optimization:

Role:

Measures prediction error
Defines optimization objective
Influences model behavior
Affects computational efficiency

Selection impact:

Training speed
Solution properties
Model robustness
Optimization stability

Hinge ~

Standard SVM hinge loss: $max (0, 1 - y_{i} (w^{T} x_{i} + b))$

Properties:

Maximum margin classifier
Sparse in dual space
Non-smooth function
Original SVM loss

Best for:

Standard SVM behavior
Margin maximization
L2 regularization

Note: Not compatible with L1 penalty

SquaredHinge ~

Squared hinge loss: $max (0, 1 - y_{i} (w^{T} x_{i} + b))^{2}$

Properties:

Smooth function
Stronger penalties
Differentiable
Faster optimization

Best for:

Faster convergence
Numerical stability
General use
Both L1/L2 penalties

Tol

f64

0.001

Optimization tolerance threshold:

Purpose:

Controls convergence
Affects precision
Influences training time

Typical ranges:

Strict: 1e-4 to 1e-3
Standard: 1e-3
Relaxed: 1e-3 to 1e-2

Trade-off: Precision vs speed

CFactor

f64

Regularization strength (inverse):

Effect: $\frac{1}{C} ∣∣ w ∣ ∣^{2} + \sum ℓ (...)$

Ranges:

Strong reg.: 0.1 - 1.0
Balanced: 1.0 - 10.0
Weak reg.: 10.0 - 100.0

Impact:

Model complexity
Generalization
Overfitting control

MultiClass

enum

Ovr

Determines the multi-class strategy if y contains more than two classes.

Purpose:

Extends binary classification
Handles multiple classes
Defines decision boundaries
Structures model architecture

Trade-offs:

Computational efficiency
Memory requirements
Model complexity
Prediction accuracy

Ovr ~

One-vs-Rest strategy (One-vs-All):

Implementation:

n_classes binary classifiers
Each class vs all others
Independent optimizations
Parallel training possible

Advantages:

Computationally efficient
Linear memory scaling
Easy interpretation
Well-proven approach

Best for:

Large-scale problems
Many classes
Production systems
Default choice

CrammerSinger ~

Crammer-Singer multi-class optimization:

Properties:

Joint optimization
Consistent formulation
Direct multi-class approach
Theoretically motivated

Trade-offs:

Computationally intensive
Higher memory usage
Complex optimization
Rarely better accuracy

Best for:

Research purposes
Small datasets
Theoretical studies
Special cases

FitIntercept

bool

true

Whether or not to fit an intercept. If set to True, the feature vector is extended to include an intercept term: [x_1, ..., x_n, 1], where 1 corresponds to the intercept. If set to False, no intercept will be used in calculations (i.e. data is expected to be already centered).

Purpose:

Adds constant feature
Shifts decision boundary
Improves flexibility

Impact:

Model capacity
Training stability
Solution quality

Default: Enabled for better fit

InterceptScaling

f64

Synthetic feature scaling:

Effect:

Scales intercept term
Controls bias impact
Affects regularization

Usage:

Higher: Reduce reg. on bias
Lower: Stronger reg. on bias
Default: Balanced (1.0)

Note: Only used when fit_intercept=true

ClassWeight

enum

None

Class importance weighting schemes:

Purpose:

Handles class imbalance
Adjusts error penalties
Controls class importance
Influences optimization

Impact:

Training behavior
Decision boundaries
Prediction bias
Model sensitivity

None ~

Equal class weights:

Properties:

All classes weighted equally
Natural class distribution
Standard optimization
No bias adjustment

Best for:

Balanced datasets
Equal error costs
Default scenarios
When prior unknown

Balanced ~

Inverse frequency weighting:

Formula: $w_{j} = \frac{n}{n _{c l a sses} * n _{j}}$

Properties:

Automatic adjustment
Inverse class frequency
Balanced errors
Compensates imbalance

Best for:

Imbalanced classes
Minority class focus
Skewed distributions
Fair classification

RandomState

u64

Controls the pseudo random number generation for shuffling the data for the dual coordinate descent (if dual=True). When dual=False the underlying implementation of LinearSVC is not random and random_state has no effect on the results. 'Dual' is chosen automatically by the model based on the values of n_samples, n_features, loss, multi_class and penalty.

Controls:

Data shuffling
Initialization
Cross-validation

Purpose:

Reproducibility
Debugging
Result validation
Consistent behavior

MaxIter

i64

1000

Maximum iteration limit:

Purpose:

Prevents infinite loops
Controls training time
Resource management

Typical values:

Simple: 500-1000
Standard: 1000-2000
Complex: 2000+

Note: Higher for harder problems

GridSearch

Hyperparameter optimization for Linear SVC:

Search process:

Model structure:
- Regularization types
- Loss functions
- Multi-class strategies
Optimization parameters:
- Regularization strength
- Convergence criteria
- Training limits
Model features:
- Intercept options
- Class weights
- Scaling factors

Computational considerations:

Time complexity: O(n_params * n_samples * features)
Memory usage: O(n_params * features)
Storage: O(n_params * features)

Penalty

[enum, ...]

Regularization norm for weight vector:

Mathematical form:

L1: $∣∣ w ∣ ∣_{1} = \sum_{i} ∣ w_{i} ∣$
L2: $∣∣ w ∣ ∣_{2} = \sum_{i} w_{i}^{2}$

Trade-offs:

Feature selection vs stability
Sparsity vs smoothness
Memory vs computation
Solution uniqueness

L1 ~

L1 norm regularization (Lasso):

Properties:

Produces sparse solutions
Built-in feature selection
Robust to outliers
Non-smooth optimization

Best for:

Feature selection needs
High-dimensional data
Redundant features
Memory constraints

Note: Not compatible with 'hinge' loss

L2 ~

L2 norm regularization (Ridge):

Properties:

Smooth solutions
Stable optimization
All features used
Unique solution

Best for:

General purpose use
Correlated features
Numerical stability
Default choice

Note: Standard SVM regularization

Loss

[enum, ...]

SquaredHinge

Loss function for optimization:

Role:

Measures prediction error
Defines optimization objective
Influences model behavior
Affects computational efficiency

Selection impact:

Training speed
Solution properties
Model robustness
Optimization stability

Hinge ~

Standard SVM hinge loss: $max (0, 1 - y_{i} (w^{T} x_{i} + b))$

Properties:

Maximum margin classifier
Sparse in dual space
Non-smooth function
Original SVM loss

Best for:

Standard SVM behavior
Margin maximization
L2 regularization

Note: Not compatible with L1 penalty

SquaredHinge ~

Squared hinge loss: $max (0, 1 - y_{i} (w^{T} x_{i} + b))^{2}$

Properties:

Smooth function
Stronger penalties
Differentiable
Faster optimization

Best for:

Faster convergence
Numerical stability
General use
Both L1/L2 penalties

Tol

[f64, ...]

0.0001

Tolerance thresholds to evaluate:

Search ranges:

Standard scale:
- [1e-4, 1e-3, 1e-2]
- [0.0001, 0.001, 0.01]
Fine-tuning:
- [0.0005, 0.001, 0.002]
- [0.0001, 0.0005, 0.001]

Trade-off: Precision vs speed

CFactor

[f64, ...]

Regularization strengths to evaluate:

Search spaces:

Log scale (recommended):
- [0.1, 1.0, 10.0, 100.0]
- [0.01, 0.1, 1.0, 10.0]
Fine-grained:
- [0.1, 0.5, 1.0, 2.0, 5.0]
- [1.0, 2.0, 5.0, 10.0]

Note: Critical parameter for performance

MultiClass

[enum, ...]

Ovr

Determines the multi-class strategy if y contains more than two classes.

Purpose:

Extends binary classification
Handles multiple classes
Defines decision boundaries
Structures model architecture

Trade-offs:

Computational efficiency
Memory requirements
Model complexity
Prediction accuracy

Ovr ~

One-vs-Rest strategy (One-vs-All):

Implementation:

n_classes binary classifiers
Each class vs all others
Independent optimizations
Parallel training possible

Advantages:

Computationally efficient
Linear memory scaling
Easy interpretation
Well-proven approach

Best for:

Large-scale problems
Many classes
Production systems
Default choice

CrammerSinger ~

Crammer-Singer multi-class optimization:

Properties:

Joint optimization
Consistent formulation
Direct multi-class approach
Theoretically motivated

Trade-offs:

Computationally intensive
Higher memory usage
Complex optimization
Rarely better accuracy

Best for:

Research purposes
Small datasets
Theoretical studies
Special cases

FitIntercept

[bool, ...]

true

Intercept fitting options:

Search combinations:

Single option:
- [true]: With intercept
- [false]: No intercept
Complete evaluation:
- [true, false]: Compare both

Impact: Model flexibility and bias

InterceptScaling

[f64, ...]

Intercept scaling factors to evaluate:

Search ranges:

Standard range:
- [0.1, 1.0, 10.0]
- [0.5, 1.0, 2.0]
Extended search:
- [0.1, 0.5, 1.0, 2.0, 5.0]
- [1.0, 2.0, 5.0, 10.0]

Note: Only relevant with fit_intercept=true

ClassWeight

[enum, ...]

None

Class importance weighting schemes:

Purpose:

Handles class imbalance
Adjusts error penalties
Controls class importance
Influences optimization

Impact:

Training behavior
Decision boundaries
Prediction bias
Model sensitivity

None ~

Equal class weights:

Properties:

All classes weighted equally
Natural class distribution
Standard optimization
No bias adjustment

Best for:

Balanced datasets
Equal error costs
Default scenarios
When prior unknown

Balanced ~

Inverse frequency weighting:

Formula: $w_{j} = \frac{n}{n _{c l a sses} * n _{j}}$

Properties:

Automatic adjustment
Inverse class frequency
Balanced errors
Compensates imbalance

Best for:

Imbalanced classes
Minority class focus
Skewed distributions
Fair classification

MaxIter

[i64, ...]

1000

Maximum iterations to evaluate:

Search ranges:

Conservative:
- [500, 1000, 2000]
- [1000, 2000, 5000]
Extensive:
- [1000, 2000, 5000, 10000]
- [2000, 5000, 10000, 20000]

Consider: Convergence needs vs time

RandomState

u64

Random seed for reproducibility:

Controls:

Cross-validation splits
Data shuffling
Parameter selection

Important for:

Result reproducibility
Fair comparison
Debugging
Validation

RefitScore

enum

Accuracy

Performance evaluation metrics:

Purpose:

Model selection
Performance evaluation
Optimization criterion
Comparison basis

Selection criteria:

Problem objectives
Class distribution
Error costs
Application needs

Default ~

Model's built-in scoring:

Properties:

Accuracy score
Fast computation
Standard metric
Equal error weights

Best for:

Initial evaluation
Balanced datasets
Quick assessment
Standard problems

Accuracy ~

Classification accuracy score:

Formula: $\frac{correct predictions}{total predictions}$

Properties:

Range: [0, 1]
Intuitive metric
Easy interpretation
Standard benchmark

Best for:

Balanced classes
Equal costs
General evaluation

BalancedAccuracy ~

Class-normalized accuracy:

Formula: $\frac{1}{n _{c l a sses}} \sum_{i = 1}^{n_{c l a sses}} \frac{T P _{i}}{T P _{i} + F N _{i}}$

Properties:

Range: [0, 1]
Class-independent
Balanced evaluation
Robust to imbalance

Best for:

Imbalanced datasets
Varying class sizes
Fair evaluation

Split

oneof

DefaultSplit

Standard train-test split configuration optimized for general classification tasks.

Configuration:

Test size: 20% (0.2)
Random seed: 98
Shuffling: Enabled
Stratification: Based on target distribution

Advantages:

Preserves class distribution
Provides reliable validation
Suitable for most datasets

Best for:

Medium to large datasets
Independent observations
Initial model evaluation

Splitting uses the ShuffleSplit strategy or StratifiedShuffleSplit strategy depending on the field stratified. Note: If shuffle is false then stratified must be false.

CustomSplit

Configurable train-test split parameters for specialized requirements. Allows fine-tuning of data division strategy for specific use cases or constraints.

Use cases:

Time series data
Grouped observations
Specific train/test ratios
Custom validation schemes

RandomState

u64

Random seed for reproducible splits. Ensures:

Consistent train/test sets
Reproducible experiments
Comparable model evaluations

Same seed guarantees identical splits across runs.

Shuffle

bool

true

Data shuffling before splitting. Effects:

true: Randomizes order, better for i.i.d. data
false: Maintains order, important for time series

When to disable:

Time dependent data
Sequential patterns
Grouped observations

TrainSize

f64

0.8

Proportion of data for training. Considerations:

Larger (e.g., 0.8-0.9): Better model learning
Smaller (e.g., 0.5-0.7): Better validation

Common splits:

0.8: Standard (80/20 split)
0.7: More validation emphasis
0.9: More training emphasis

Stratified

bool

false

Maintain class distribution in splits. Important when:

Classes are imbalanced
Small classes present
Representative splits needed

Requirements:

Classification tasks only
Cannot use with shuffle=false
Sufficient samples per class

Cv

oneof

DefaultCv

Standard cross-validation configuration using stratified 3-fold splitting.

Configuration:

Folds: 3
Method: StratifiedKFold
Stratification: Preserves class proportions

Advantages:

Balanced evaluation
Reasonable computation time
Good for medium-sized datasets

Limitations:

May be insufficient for small datasets
Higher variance than larger fold counts
May miss some data patterns

CustomCv

Configurable stratified k-fold cross-validation for specific validation requirements.

Features:

Adjustable fold count with NFolds determining the number of splits.
Stratified sampling
Preserved class distributions

Use cases:

Small datasets (more folds)
Large datasets (fewer folds)
Detailed model evaluation
Robust performance estimation

NFolds

u32

Number of cross-validation folds. Guidelines:

3-5: Large datasets, faster training
5-10: Standard choice, good balance
10+: Small datasets, thorough evaluation

Trade-offs:

More folds: Better evaluation, slower training
Fewer folds: Faster training, higher variance

Must be at least 2.

KfoldCv

K-fold cross-validation without stratification. Divides data into k consecutive folds for iterative validation.

Process:

Splits data into k equal parts
Each fold serves as validation once
Remaining k-1 folds form training set

Use cases:

Regression problems
Large, balanced datasets
When stratification unnecessary
Continuous target variables

Limitations:

May not preserve class distributions
Less suitable for imbalanced data
Can create biased splits with ordered data

NSplits

u32

Number of folds for cross-validation. Selection guide: Recommended values:

5: Standard choice (default)
3: Large datasets/quick evaluation
10: Thorough evaluation/smaller datasets

Trade-offs:

Higher values: More thorough, computationally expensive
Lower values: Faster, potentially higher variance

Must be at least 2 for valid cross-validation.

RandomState

u64

Random seed for fold generation when shuffling. Important for:

Reproducible results
Consistent fold assignments
Benchmark comparisons
Debugging and validation

Set specific value for reproducibility across runs.

Shuffle

bool

true

Whether to shuffle data before splitting into folds. Effects:

true: Randomized fold composition (recommended)
false: Sequential splitting

Enable when:

Data may have ordering
Better fold independence needed

Disable for:

Time series data
Ordered observations

StratifiedKfoldCv

Stratified K-fold cross-validation maintaining class proportions across folds.

Key features:

Preserves class distribution in each fold
Handles imbalanced datasets
Ensures representative splits

Best for:

Classification problems
Imbalanced class distributions
When class proportions matter

Requirements:

Classification tasks only
Sufficient samples per class
Categorical target variable

NSplits

u32

Number of stratified folds. Guidelines: Typical values:

5: Standard for most cases
3: Quick evaluation/large datasets
10: Detailed evaluation/smaller datasets

Considerations:

Must allow sufficient samples per class per fold
Balance between stability and computation time
Consider smallest class size when choosing

RandomState

u64

Seed for reproducible stratified splits. Ensures:

Consistent fold assignments
Reproducible results
Comparable experiments
Systematic validation

Fixed seed guarantees identical stratified splits.

Shuffle

bool

false

Data shuffling before stratified splitting. Impact:

true: Randomizes while maintaining stratification
false: Maintains data order within strata

Use cases:

true: Independent observations
false: Grouped or sequential data

Class proportions maintained regardless of setting.

ShuffleSplitCv

Random permutation cross-validator with independent sampling.

Characteristics:

Random sampling for each split
Independent train/test sets
More flexible than K-fold
Can have overlapping test sets

Advantages:

Control over test size
Fresh splits each iteration
Good for large datasets

Limitations:

Some samples might never be tested
Others might be tested multiple times
No guarantee of complete coverage

NSplits

u32

Number of random splits to perform. Consider: Common values:

5: Standard evaluation
10: More thorough assessment
3: Quick estimates

Trade-offs:

More splits: Better estimation, longer runtime
Fewer splits: Faster, less stable estimates

Balance between computation and stability.

RandomState

u64

Random seed for reproducible shuffling. Controls:

Split randomization
Sample selection
Result reproducibility

Important for:

Debugging
Comparative studies
Result verification

TestSize

f64

0.2

Proportion of samples for test set. Guidelines: Common ratios:

0.2: Standard (80/20 split)
0.25: More validation emphasis
0.1: More training data

Considerations:

Dataset size
Model complexity
Validation requirements

It must be between 0.0 and 1.0.

StratifiedShuffleSplitCv

Stratified random permutation cross-validator combining shuffle-split with stratification.

Features:

Maintains class proportions
Random sampling within strata
Independent splits
Flexible test size

Ideal for:

Imbalanced datasets
Large-scale problems
When class distributions matter
Flexible validation schemes

NSplits

u32

Number of stratified random splits. Guidelines: Recommended values:

5: Standard evaluation
10: Detailed analysis
3: Quick assessment

Consider:

Sample size per class
Computational resources
Stability requirements

RandomState

u64

Seed for reproducible stratified sampling. Ensures:

Consistent class proportions
Reproducible splits
Comparable experiments

Critical for:

Benchmarking
Research studies
Quality assurance

TestSize

f64

0.2

Fraction of samples for stratified test set. Best practices: Common splits:

0.2: Balanced evaluation
0.3: More thorough testing
0.15: Preserve training size

Consider:

Minority class size
Overall dataset size
Validation objectives

It must be between 0.0 and 1.0.

TimeSeriesSplitCv

Time Series cross-validator. Provides train/test indices to split time series data samples that are observed at fixed time intervals, in train/test sets. It is a variation of k-fold which returns first k folds as train set and the k + 1th fold as test set. Note that unlike standard cross-validation methods, successive training sets are supersets of those that come before them. Also, it adds all surplus data to the first training partition, which is always used to train the model. Key features:

Maintains temporal dependence
Expanding window approach
Forward-chaining splits
No future data leakage

Use cases:

Sequential data
Financial forecasting
Temporal predictions
Time-dependent patterns

Note: Training sets are supersets of previous iterations.

NSplits

u32

Number of temporal splits. Considerations: Typical values:

5: Standard forward chaining
3: Limited historical data
10: Long time series

Impact:

Affects training window growth
Determines validation points
Influences computational load

MaxTrainSize

u64

Maximum size of training set. Should be strictly less than the number of samples. Applications:

0: Use all available past data
>0: Rolling window of fixed size

Use cases:

Limit historical relevance
Control computational cost
Handle concept drift
Memory constraints

TestSize

u64

Number of samples in each test set. When 0:

Auto-calculated as n_samples/(n_splits+1)
Ensures equal-sized test sets

Considerations:

Forecast horizon
Validation requirements
Available future data

Gap

u64

Number of samples to exclude from the end of each train set before the test set.Gap between train and test sets. Uses:

Avoid data leakage
Model forecast lag
Buffer periods

Common scenarios:

0: Continuous prediction
>0: Forward gap for realistic evaluation
Match business forecasting needs