In interannual to decadal predictions, forecast quality may arise from the initial state of the system, from long-term changes due to external forcing such as the increase in greenhouse gases concentrations, and from internally generated variability in a model. In this study, we use a new framework to investigate the achievable skill of decadal predictions by comparing perfect-model prediction experiments with predictions of the real world in order to identify margins for possible improvements to prediction systems. In addition, we assess the added value from capturing the initial state in the climate system over changes due to climate forcing in decadal predictions focusing on annual average near-surface temperature. We find that ideal initialization may substantially improve the predictions during the first two lead years particularly in parts of the Southern Ocean, Indian Ocean, the tropical Pacific and North Atlantic, and some surrounding land areas (the lead time is the elapsed time since the beginning of a prediction). On longer time scales, the predictions rely more on model performance in simulating low-frequency variability and long-term changes due to external forcing.. This framework identifies the limits of predictability using the NCAR Community Climate System Model Version 4, and clarifies the margins of achievable improvements from enhancing different components of the prediction system such as initialization, response to external forcing and internal variability. We encourage similar experiments to be performed using other climate models, to better understand the dependence of predictability on the model used.