Monotonic and Interaction Constraints in Boosted Ensembles

A tree ensemble fitted to observational data will happily encode a relationship that is known to be false. If higher income happens to correlate with default in a sample, the fitted function will use income as evidence for risk, and no amount of regularization removes the direction of an effect it has already learned. The usual answer is to constrain the shape of the fitted function: monotonicity in chosen features, and restrictions on which features may interact.

Both are enforced inside the tree-building loop rather than applied afterwards, which is what makes them guarantees instead of hopes. Neither is free, and knowing where the cost lands is most of the practice.

Bounds as the representation of monotonicity

A tree is piecewise constant, so a monotone tree is one whose leaf values are ordered along the constrained feature. The standard construction assigns every node an interval $[\ell, u]$ that constrains the value of any leaf beneath it.

At the root the interval is unbounded. When a node splits on a constrained feature with an increasing constraint, the left child, which receives the smaller values, gets $[\ell, m]$ and the right child gets $[m, u]$, where $m = (\ell + u)/2$. When the split is on an unconstrained feature, both children inherit the parent’s interval unchanged. When the constraint is decreasing, the halves are swapped.

Leaf values are then clipped: after the objective computes the unconstrained optimum $w^$, using the formula given in the regularized split gain, the stored value is $\text{clip}(w^, \ell, u)$. The gain attributed to the tree is computed from the clipped values, so the split search sees the true objective of the constrained model rather than the value it would have had unconstrained.

def fit_node(rows, lo, hi, idx, increasing):
    if stop(rows) or hi - lo < 1e-9:
        return Leaf(clip(opt_value(rows), lo, hi))
    j, t = best_split(rows)                 # ignores constraints during the scan
    mid = 0.5 * (lo + hi)
    if j == idx and increasing:
        left, right = fit_node(rows_left, lo, mid, idx, increasing), \
                      fit_node(rows_right, mid, hi, idx, increasing)
    elif j == idx:
        left, right = fit_node(rows_left, mid, hi, idx, increasing), \
                      fit_node(rows_right, lo, mid, idx, increasing)
    else:
        left = fit_node(rows_left, lo, hi, idx, increasing)
        right = fit_node(rows_right, lo, hi, idx, increasing)
    return Split(j, t, left, right)

The interval shrinks by half at each constrained split, so after $d$ consecutive splits on that feature the interval has width $(\text{range})/2^d$. This is a resolution limit, not a capacity limit: a deep enough tree can express any monotone step function, but each step’s value is confined to a band fixed by the number of constrained splits above it.

Two details decide whether the guarantee survives contact with the data. Missing values need a consistent side: if a node routes missing rows to the left child while its sibling routes them right, a row with a missing value can end up on the wrong side of the order and break monotonicity. Standard implementations attach the default direction per node and treat it as part of the split, so the ordering constraint must account for it. And the clipping must happen at the leaf, not by post-processing the tree; clipping after the fact can invert the order of two adjacent leaves.

Why per-tree constraints compose

Boosted predictions are sums of the kind built one stage at a time in gradient boosting. If every tree is non-decreasing in feature $j$, the sum is non-decreasing, because the sum of two non-decreasing functions is non-decreasing and a positive learning rate preserves the direction. This is the entire reason the constraint can be enforced one tree at a time without looking at the ensemble: the inductive step is trivial and requires no global bookkeeping.

The property does not extend to interactions between constrained features. A model can be monotone in $j$ and monotone in $k$ while the effect of $j$ at fixed $k$ varies enormously across the range of $k$. Monotone in each coordinate is a much weaker statement than a monotone response surface, and it is the strongest statement a tree model can make cheaply.

Nor does the guarantee survive a monotone transformation of the output. Applying a link function with a negative slope, taking a reciprocal, or ranking scores in the opposite direction all invert the direction. The constraint belongs to the scale in which the trees are fitted, which for logistic loss is the log-odds scale, not the probability scale. Monotonicity in log-odds implies monotonicity in probability because the logistic function is increasing, but a custom objective’s link may not preserve the ordering.

Restricting interaction

The second constraint family limits which features may appear on the same root-to-leaf path. The mechanism is a set carried by each node. At the root the allowed set is every feature. When a node splits on feature $f$, each child’s allowed set becomes the parent’s allowed set intersected with the features that are permitted to interact with $f$; the split search at the child then considers only that set.

Features are organized into groups, and interaction is permitted within a group. The practical effect is that a path can only mix features that share a group, so the tree’s interaction order is bounded by the group structure rather than by depth. If features are placed in singleton groups, the first split determines the only feature the rest of that path may use, and the tree degenerates into a set of axis-aligned steps in one dimension per branch. The ensemble can still represent interaction through the sum across trees, since different trees can use different features, but no single tree can.

The rule as stated is coarser than it first appears because the restriction is inherited by every descendant. A split near the root on a feature from group $A$ removes all features outside $A$ from the entire subtree below, including the sibling branch. Group design therefore matters more than group count: putting two features in different groups asserts that they never interact, and that assertion is enforced globally for the tree in which the split occurs.

What the constraints cost

Monotone constraints reduce the feasible set, so the training loss at a fixed number of rounds can only worsen. The size of the effect depends on how far the unconstrained fit was from monotone. A feature whose true effect is monotone loses little; a feature whose effect reverses in a subpopulation loses exactly the capacity to represent the reversal, and the model will express it through other features if they are available, which can be worse than the original violation.

Interaction constraints cost more when the truth has cross-group interactions. Bounding the interaction order is a strong assumption, and a model trained under it will underfit in a way that no number of rounds repairs, because the representable function class does not contain the target.

There is also a sampling cost. Constraints interact with the split search: a constrained node may have no admissible split with positive gain, in which case it becomes a leaf early, and the tree is shallower than the depth parameter suggests. Ensembles compensating for this need more rounds, and the shrinkage and round count then have to be retuned rather than carried over from an unconstrained run.

What the constraints cannot promise

Monotonicity is a statement about the fitted function, not about causality. Making a model monotone in a feature does not make that feature’s coefficient interpretable, and it does not remove confounding: the model can satisfy the constraint while attributing the effect to the wrong variable, as long as the ordering is right. A monotone constraint is a hard prior on shape, and priors can be wrong.

The constraints also do not survive every post-processing step that people apply to ensembles. Averaging several independently fitted constrained models with non-negative weights preserves monotonicity; stacking them behind a second model does not. Quantizing leaf values for compact storage can break ordering if the quantization is not monotone. Rounding scores to a fixed number of decimals preserves order, but clipping the output range to a deployment range does not, if the clip is applied per tree rather than to the sum.

Finally, the two constraint families have different failure signatures, which is useful when diagnosing a model that behaves strangely in production. A monotone model that is directionally wrong is a prior error and shows up as a consistent bias along the constrained feature. An interaction-constrained model that misses structure shows up as a residual pattern that depends on two features jointly, and the diagnostic is a two-way residual plot rather than a one-dimensional partial dependence. Neither signature appears in aggregate error, which is why both are usually discovered by someone who knows what the relationship should look like.