Context: Organizational performance measurements in software product development have received a lot of attention in the literature. Still, there is a general discontent regarding the way performance is evaluated in practice, with few studies really focusing on why this is the case. In this paper research focusing on the context of developing software-intensive products in large established multi-national organizations is reported on. Objective: The purpose of this research is to investigate performance measurement practices related to software product development activities. More specifically, focus is on exploring how managers engaged in software product development activities perceive and evaluate performance in large organizations from a managerial perspective. Method: The research approach pursued in this research consist of exploratory multiple case studies. Data is collected mainly through 54 interviews in five case studies in large international organizations developing software-intensive products in Sweden. Focused group interviews with senior managers from eight companies have also been used in the data collection. Results: The results of this research indicate that managers within software product development in general are dissatisfied with their current way of evaluating performance. Performance measurements and the perception of performance are today focused on cost, time, and quality, i.e. what is easily measurable and not necessarily what is important. The dimensions of value creation and learning are missing. Moreover, measurements tend to be result oriented, rather than process oriented, making it difficult to integrate these measurements in the management practices. Conclusion: Managers that are dissatisfied with their performance measurement system and want to improve the current situation should not start by focusing on the current measurements directly; instead they should focus on how the organization perceives performance and how important performance criteria are being developed. By developing relevant performance criteria the first step in developing an effective performance measurement system is made. Moreover, it is concluded that manager's perception of performance is affected by the currently used measurements, hence limiting the scope of the performance criteria. Thus, a change in the way managers perceive performance is necessary before there can be any changes in the way performance is evaluated.