DecisionTree / Classifier Layer

Split quality measurement criteria:

Purpose:

• Evaluates potential splits
• Guides tree construction
• Measures node purity
• Optimizes classification

Selection impact:

• Tree structure
• Learning bias
• Computational cost
• Model performance

The strategy used to choose the split at each node.

Impact:

• Tree construction speed
• Split optimality
• Model randomization
• Computation time

Trade-offs:

• Speed vs accuracy
• Deterministic vs random
• Memory usage
• Search complexity

The maximum depth of the tree. If 0, then nodes are expanded until all leaves are pure or until all leaves contain less than min_samples_split samples.

Values:

• 0: Unlimited depth
• >0: Maximum levels from root

Effects:

• Controls model complexity
• Prevents overfitting
• Impacts memory usage
• Affects prediction speed

Guidelines:

• Small (1-3): Simple rules
• Medium (4-10): Balanced models
• Large (>10): Complex patterns

The minimum number of samples required to split an internal node:

Constraint: $N\ge N_{min}$ where N is node samples

Purpose:

• Prevents excessive splitting
• Controls overfitting
• Ensures statistical significance
• Manages leaf size

Typical values:

• 2: Maximum splitting (default)
• 5-10: Moderate regularization
• >20: Strong regularization

The minimum number of samples required to be at a leaf node. A split point at any depth will only be considered if it leaves at least min_samples_leaf training samples in each of the left and right branches.

Constraint: $L\ge L_{min}$ where L is leaf samples

Effects:

• Ensures leaf node significance
• Prevents class isolation
• Smooths predictions
• Reduces variance

Guidelines:

• 1: Maximum detail (default)
• 5-10: Balanced smoothing
• >20: Strong smoothing

The minimum weighted fraction of the sum total of weights (of all the input samples) required to be at a leaf node:

Constraint: $L\ge L_{min}$ where L is leaf samples

Properties:

• Range: [0.0, 0.5]
• Weighted sample control
• Alternative to min_samples_leaf
• Handles imbalanced weights

Use cases:

• Weighted samples
• Cost-sensitive learning
• Imbalanced problems

Feature subset selection strategy for splits:

Purpose:

• Controls feature randomization
• Manages computational complexity
• Reduces overfitting
• Influences tree diversity

Impact:

• Training speed
• Model variance
• Feature exploration
• Memory usage

Note: Critical parameter for high-dimensional datasets

Custom feature subset size:

Requirements:

• Value >= 1
• <= total features

Usage:

• Active when max_features=Custom
• Direct control over feature sampling
• Problem-specific optimization
• Manual tuning capability

Maximum number of leaf nodes:

Values:

• 0: Unlimited leaves
• >0: Maximum leaf count

Effects:

• Controls tree size
• Alternative to max_depth
• Affects memory usage
• Impacts model complexity

Trade-offs:

• Model size vs accuracy
• Memory vs precision
• Speed vs detail

A node will be split if this split induces a decrease of the impurity greater than or equal to this value:

Constraint: $I_{parent}-{\sum }_c\frac{n_c}{n}{I}_c\ge I_{min}$

where:

• $I_{parent}$ is parent node impurity
• $I_c$ is child node impurity
• $I_{min}$ is minimum decrease threshold
• $n_c/n$ is child/parent sample ratio

Purpose:

• Prevents weak splits
• Controls tree growth
• Ensures meaningful splits
• Pre-pruning mechanism

Typical values:

• 0.0: All splits allowed
• 0.0001-0.001: Weak pruning
• >0.01: Strong pruning

Class importance weighting scheme:

Purpose:

• Handles class imbalance
• Adjusts misclassification costs
• Controls class importance
• Influences tree splits

Impact:

• Training bias
• Error penalties
• Decision boundaries
• Model sensitivity

Note: Critical for imbalanced classification tasks

Random number generator seed:

Controls randomization in:

• Feature selection
• Split point selection
• Sample ordering

Importance:

• Reproducibility
• Debugging
• Result validation
• Experimental control

Cost-Complexity Pruning alpha:

Formula: $R_{\alpha }(T)=R(T)+\alpha |T|$ where:

• R(T): Tree error rate
• |T|: Number of leaves
• α: Complexity parameter

Effects:

• Post-pruning control
• Complexity reduction
• Overfitting prevention
• Size optimization

Values:

• 0.0: No pruning
• >0.0: Increases pruning strength

Split quality measurement criteria:

Purpose:

• Evaluates potential splits
• Guides tree construction
• Measures node purity
• Optimizes classification

Selection impact:

• Tree structure
• Learning bias
• Computational cost
• Model performance

The strategy used to choose the split at each node.

Impact:

• Tree construction speed
• Split optimality
• Model randomization
• Computation time

Trade-offs:

• Speed vs accuracy
• Deterministic vs random
• Memory usage
• Search complexity

Tree depth ranges:

Common patterns:

1. Basic: [3, 5, 7, 10]
2. Extensive: [3, 5, 7, 10, 15, 20]
3. Unlimited: Include [0]

Impact:

• Model complexity
• Memory requirements
• Training time
• Prediction speed

Node splitting thresholds:

Search ranges:

1. Fine: [2, 5, 10]
2. Medium: [2, 5, 10, 20, 50]
3. Coarse: [10, 30, 50, 100]

Considerations:

• Sample size
• Noise level
• Overfitting risk
• Statistical significance

Leaf size constraints:

Common ranges:

1. Detailed: [1, 2, 4]
2. Balanced: [1, 5, 10, 20]
3. Smooth: [10, 25, 50]

Effects:

• Prediction stability
• Model variance
• Generalization
• Leaf purity

Weighted leaf constraints:

Typical ranges:

1. Light: [0.0, 0.01, 0.05]
2. Moderate: [0.0, 0.05, 0.1, 0.2]
3. Heavy: [0.1, 0.2, 0.3, 0.4]

Applications:

• Weighted samples
• Imbalanced data
• Cost-sensitive learning
• Rare event detection

Feature subset selection strategy for splits:

Purpose:

• Controls feature randomization
• Manages computational complexity
• Reduces overfitting
• Influences tree diversity

Impact:

• Training speed
• Model variance
• Feature exploration
• Memory usage

Note: Critical parameter for high-dimensional datasets

Custom feature counts:

Search patterns:

1. Small: [1, 2, 3]
2. Medium: [2, 4, 6, 8]
3. Large: [5, 10, 15, 20]

Note:

• Only used with max_features=Custom
• Must be <= total features
• Affects computation speed

Leaf node constraints:

Search spaces:

1. Limited: [10, 20, 50]
2. Extended: [20, 50, 100, 200]
3. Include unlimited: [0, 50, 100]

Impact:

• Tree size
• Memory usage
• Model complexity
• Training speed

Impurity threshold ranges:

Common values:

1. Fine: [0.0, 0.0001, 0.001]
2. Medium: [0.0, 0.001, 0.01, 0.1]
3. Coarse: [0.01, 0.05, 0.1]

Effects:

• Split quality
• Tree growth
• Pruning level
• Model sparsity

Class importance weighting scheme:

Purpose:

• Handles class imbalance
• Adjusts misclassification costs
• Controls class importance
• Influences tree splits

Impact:

• Training bias
• Error penalties
• Decision boundaries
• Model sensitivity

Note: Critical for imbalanced classification tasks

Random seed control:

Ensures:

• Reproducible results
• Consistent splits
• Deterministic search
• Comparable outcomes

Important for:

• Research
• Debugging
• Validation
• Benchmarking

Cost-complexity pruning alphas:

Search ranges:

1. Fine: [0.0, 0.001, 0.01]
2. Medium: [0.0, 0.01, 0.05, 0.1]
3. Aggressive: [0.1, 0.2, 0.3]

Effects:

• Tree size reduction
• Complexity control
• Overfitting prevention
• Model simplification

Performance evaluation metrics for Decision Tree classification:

Purpose:

• Model evaluation
• Cross-validation scoring
• Model selection
• Performance monitoring

Selection criteria:

• Problem objectives
• Class distribution
• Error costs
• Application requirements

Random seed for reproducible splits. Ensures:

• Consistent train/test sets
• Reproducible experiments
• Comparable model evaluations

Same seed guarantees identical splits across runs.

Data shuffling before splitting. Effects:

• true: Randomizes order, better for i.i.d. data
• false: Maintains order, important for time series

When to disable:

• Time dependent data
• Sequential patterns
• Grouped observations

Proportion of data for training. Considerations:

• Larger (e.g., 0.8-0.9): Better model learning
• Smaller (e.g., 0.5-0.7): Better validation

Common splits:

• 0.8: Standard (80/20 split)
• 0.7: More validation emphasis
• 0.9: More training emphasis

Maintain class distribution in splits. Important when:

• Classes are imbalanced
• Small classes present
• Representative splits needed

Requirements:

• Classification tasks only
• Cannot use with shuffle=false
• Sufficient samples per class

Number of cross-validation folds. Guidelines:

• 3-5: Large datasets, faster training
• 5-10: Standard choice, good balance
• 10+: Small datasets, thorough evaluation

Trade-offs:

• More folds: Better evaluation, slower training
• Fewer folds: Faster training, higher variance

Must be at least 2.

Number of folds for cross-validation. Selection guide: Recommended values:

• 5: Standard choice (default)
• 3: Large datasets/quick evaluation
• 10: Thorough evaluation/smaller datasets

Trade-offs:

• Higher values: More thorough, computationally expensive
• Lower values: Faster, potentially higher variance

Must be at least 2 for valid cross-validation.

Random seed for fold generation when shuffling. Important for:

• Reproducible results
• Consistent fold assignments
• Benchmark comparisons
• Debugging and validation

Set specific value for reproducibility across runs.

Whether to shuffle data before splitting into folds. Effects:

• true: Randomized fold composition (recommended)
• false: Sequential splitting

Enable when:

• Data may have ordering
• Better fold independence needed

Disable for:

• Time series data
• Ordered observations

Number of stratified folds. Guidelines: Typical values:

• 5: Standard for most cases
• 3: Quick evaluation/large datasets
• 10: Detailed evaluation/smaller datasets

Considerations:

• Must allow sufficient samples per class per fold
• Balance between stability and computation time
• Consider smallest class size when choosing

Seed for reproducible stratified splits. Ensures:

• Consistent fold assignments
• Reproducible results
• Comparable experiments
• Systematic validation

Fixed seed guarantees identical stratified splits.

Data shuffling before stratified splitting. Impact:

• true: Randomizes while maintaining stratification
• false: Maintains data order within strata

Use cases:

• true: Independent observations
• false: Grouped or sequential data

Class proportions maintained regardless of setting.

Number of random splits to perform. Consider: Common values:

• 5: Standard evaluation
• 10: More thorough assessment
• 3: Quick estimates

Trade-offs:

• More splits: Better estimation, longer runtime
• Fewer splits: Faster, less stable estimates

Balance between computation and stability.

Random seed for reproducible shuffling. Controls:

• Split randomization
• Sample selection
• Result reproducibility

Important for:

• Debugging
• Comparative studies
• Result verification

Proportion of samples for test set. Guidelines: Common ratios:

• 0.2: Standard (80/20 split)
• 0.25: More validation emphasis
• 0.1: More training data

Considerations:

• Dataset size
• Model complexity
• Validation requirements

It must be between 0.0 and 1.0.

Number of stratified random splits. Guidelines: Recommended values:

• 5: Standard evaluation
• 10: Detailed analysis
• 3: Quick assessment

Consider:

• Sample size per class
• Computational resources
• Stability requirements

Seed for reproducible stratified sampling. Ensures:

• Consistent class proportions
• Reproducible splits
• Comparable experiments

Critical for:

• Benchmarking
• Research studies
• Quality assurance

Fraction of samples for stratified test set. Best practices: Common splits:

• 0.2: Balanced evaluation
• 0.3: More thorough testing
• 0.15: Preserve training size

Consider:

• Minority class size
• Overall dataset size
• Validation objectives

It must be between 0.0 and 1.0.

Number of temporal splits. Considerations: Typical values:

• 5: Standard forward chaining
• 3: Limited historical data
• 10: Long time series

Impact:

• Affects training window growth
• Determines validation points
• Influences computational load

Maximum size of training set. Should be strictly less than the number of samples. Applications:

• 0: Use all available past data
• >0: Rolling window of fixed size

Use cases:

• Limit historical relevance
• Control computational cost
• Handle concept drift
• Memory constraints

Number of samples in each test set. When 0:

• Auto-calculated as n_samples/(n_splits+1)
• Ensures equal-sized test sets

Considerations:

• Forecast horizon
• Validation requirements
• Available future data

Number of samples to exclude from the end of each train set before the test set.Gap between train and test sets. Uses:

• Avoid data leakage
• Model forecast lag
• Buffer periods

Common scenarios:

• 0: Continuous prediction
• >0: Forward gap for realistic evaluation
• Match business forecasting needs