[CUDA] [fix] enable max_depth in cuda tree learner - fix build

[lorenzolucchese]

Small changes to fix the cuda build for the PR #6994. I also added a test checking the max_depth parameter is indeed enforced to the python package tests.

I tested this locally as follows:

1. Clone, modify and install lightgbm locally

git clone --recursive https://github.com/microsoft/LightGBM.git
cd LightGBM
git checkout fix-6969
# specific settings for my system
export CMAKE_ARGS="-DCMAKE_CUDA_ARCHITECTURES=80 -DCMAKE_CUDA_COMPILER=/usr/local/cuda-12.6/bin/nvcc -DCMAKE_CXX_FLAGS=-w"
sh ./build-python.sh install --cuda

My system:

• GPU: NVIDIA A100-SXM4-80GB
• Driver Version: 565.57.01
• CUDA Version: 12.6
• gcc Version: 13.4.0

2. Test that max_depth is indeed enforced (cf. Issue #6969)

Reproducible example

import lightgbm as lgb
from sklearn.datasets import make_regression

# Generate synthetic regression data
X, y = make_regression(n_samples=1000, n_features=20, noise=0.1, random_state=42)

# Create and fit the LGBMRegressor with GPU support
model = lgb.LGBMRegressor(
    objective="regression",
    device="cuda",  # Use CUDA
    max_depth=10,
)
model.fit(X, y)

print("lightgbm version:", lgb.__version__)
print(model.booster_.trees_to_dataframe().groupby("tree_index")["node_depth"].max().value_counts().sort_index())

lightgbm version: 4.6.0.99
node_depth
8      3
9     11
10    86
Name: count, dtype: int64

[lorenzolucchese]

@microsoft-github-policy-service agree company="GSA Capital"

[jameslamb]

Thanks for this. I'll let @shiyu1994 comment on the C++ changes.

Please see my suggestions on the testing changes. I do agree that #6994 needs at least one test like this, but I'd change the way it's written.

If you don't have the time or interest in addressing that feedback, please let me know and I'll just push the changes directly in a separate PR.

[lorenzolucchese]

@shiyu1994 do you think you could take a quick look? I think this should be good to merge. Thank you!