Recommender systems add value to vast content resources by matching users with items of interest. In recent years, immense progress has been made in recommendation tech- niques. The evaluation of these has however not been matched and is threatening to impede the further development of rec-ommender systems. In this paper we propose an approach that addresses this impasse by formulating a novel evalua-tion concept adopting aspects from recommender systems research and industry. Our model can express the quality of a recommender algorithm from three perspectives, the end consumer (user), the service provider and the vendor (business and technique for both). We review current bench- marking activities and point out their shortcomings, which are addressed by our model. We also explain how our 3D benchmarking framework would apply to a specific use case.