The most important features which one can tune in decision trees are:
- Splitting criteria
- Min_leaves
- Min_samples
- Max_depth
In machine learning interviews, when asked about hyper-parameters of decision trees, you can mention several key hyper-parameters that are commonly used to tune and optimize the performance of decision trees. Some of these include:
- Maximum Depth (
max_depth): This hyper-parameter controls the maximum depth of the decision tree. A deeper tree may capture more complex patterns in the training data, but it also increases the risk of overfitting. - Minimum Samples Split (
min_samples_split): It represents the minimum number of samples required to split an internal node. This parameter helps control the growth of the tree and prevent overfitting. - Minimum Samples Leaf (
min_samples_leaf): This hyper-parameter sets the minimum number of samples required to be in a leaf node. It works similarly tomin_samples_splitbut applies to the leaves. It can also help prevent overfitting. - Maximum Features (
max_features): It determines the maximum number of features considered for splitting a node. Setting it to a lower value can help reduce overfitting. - Criterion: The function used to measure the quality of a split. Common values include “gini” for the Gini impurity and “entropy” for information gain.
- Splitter: The strategy used to choose the split at each node. It can be “best” to choose the best split or “random” to choose the best random split.
- Class Weight: This parameter is used to assign weights to classes, which is useful in imbalanced datasets. It helps the model give more importance to minority classes.
When discussing these hyper-parameters, you may also want to mention their impact on the model, how they help in controlling overfitting, and the trade-offs involved in tuning them.