Importance of sample size, data type and prediction method for remote sensing-based estimations of aboveground forest biomass
Abstract
Estimates of forest biomass are needed for various technical and scientific applications, ranging from carbon and bioenergy policies to sustainable forest management. As local measurements are costly, there is a great interest in obtaining reliable estimates over large areas from remote sensing data. Currently, such estimates are obtained with a variety of data sources, statistical methods and prediction standards, and there is no agreement on what are best practices for this task. To improve our understanding of how these different methods affect prediction quality, we first conducted a systematic review of the available literature to identify the most common sensor types and prediction methods. Based on the review, we identified sample size of the reference points on the ground, prediction method (stepwise linear regression, support vector machines, random forest, Gaussian processes and k-nearest neighbor), and sensor type as the main differences that could potentially affect predictive quality. We then compared those factors in two case study areas in Germany and Chile, for which airborne discrete return Light Detection And Ranging (LiDAR) and airborne hyperspectral as well as airborne discrete return LiDAR and spaceborne hyperspectral data were available. For each factor combination, we calculated Pearson's coefficient of correlation between observations and predictions (r²) and root mean squared error (RMSE) for bootstrapped estimates using k-fold cross-validation with a varying number of folds. Finally, Analysis of Variance (ANOVA) was used to quantify the influence of the factors on the predictive error of the biomass models. Our results confirm previous findings that predictor data (sensor) type is the most important factor for the accuracy of biomass estimates, with LiDAR being preferable to hyperspectral data. In contrast to some previous studies, complementing LiDAR with hyperspectral data did not improve predictive accuracy. Also the prediction method had a substantial effect on accuracy and was generally more important than the sample size. In most cases, random forest performed best and stepwise linear models worst, judging from r² and RMSE under cross-validation. Additional results suggested that r² may deliver unrealistically large values when the hold-out sample during the cross-validation is too small. In conclusion, our literature review revealed that different methods for biomass estimation are currently used, with no general agreement on best practices. In our case studies, we found substantial accuracy differences between those methods, with LiDAR data, in combination with a random forest algorithm and a large number of reference sample units on the ground yielding the lowest error for biomass predictions. The comparatively high importance of the statistical prediction method seems particularly relevant, as they suggest that choosing the appropriate statistical method may be more effective than obtaining additional field data for obtaining good biomass estimates. Considering the costs of improving accuracy of global and regional biomass estimates by ground measurements, it seems sensible to invest in further comparative studies, preferably with a wider range of sites and including also RADAR sensors, to establish robust best-practice recommendations for obtaining regional and global biomass estimates from remote-sensing data.
What the paper shows and why it matters
(AI-generated)
Estimates of forest biomass are needed for carbon accounting, bioenergy planning and forest management. Ground measurements are costly, so maps over large areas are built by training statistical models on a limited number of field plots, using remote sensing data such as LiDAR (laser scanning, which captures canopy height) and hyperspectral imagery (light recorded in many narrow wavelength bands). The paper’s own systematic review of 113 studies found a wide mix of sensors, prediction methods, sample sizes and validation practices; results are therefore hard to compare, and there is still no agreement on best practice. More field plots are one way to improve estimates, but they are expensive. The paper asks how much the number of plots, the sensor type and the prediction method each contribute to accuracy.
The study finds that, in two case studies on forest biomass estimated from remote sensing data, the choice of statistical prediction method affects accuracy more than the number of field plots and second only to the type of sensor. The case studies are in Germany and Chile; the study resampled the field data, fitted five prediction methods and used analysis of variance (ANOVA, which splits variation in accuracy among factors) to rank the factors. LiDAR performed better than hyperspectral data, and adding hyperspectral data to LiDAR did not improve accuracy. Random forest, which averages many decision trees, usually performed best, and stepwise linear regression was among the worst methods. A second result concerns how accuracy is judged: when cross-validation hold-out sets were very small (three to four samples, at the Chilean site Monte Oscuro), r², the correlation between observed and predicted values, became unrealistically high, whereas the root mean squared error (RMSE) was not notably affected.
The paper reads these results as a sign that choosing the statistical method well can be more effective than collecting additional field data, although a minimum number of plots is still needed. Within the limits of the data and methods tested, it advises LiDAR with many reference plots and random forest, and reporting at least RMSE and one correlation measure rather than r² alone. It names its limits as two sites, two kinds of sensor and five methods, and calls for repeats on more sites and sensors, including RADAR.