Performance regression testing is a difficult task with several intricacies and complexities. In the absence of analysis tools, manual analysis must be conducted which is undoubtedly infeasible. Thereby, in this thesis, an automated performance analysis framework is proposed, aiming to mitigate the faced issues. To make this possible, the adequacy of the data needed to be established. Additionally, a fault detection algorithm had to be developed. From investigating the current state-of-the-art of performance anomaly detection, evidently, statistical models have been utilised far more than classical machine learning, and deep learning. Consequently, based on this knowledge and based on the types of anomalies present in the data, a cumulative sum based statistical method is proposed. The findings demonstrate that the data is adequate for detecting faults, and verifying their validity, as they are consistently observable in several test configurations. However, tests are not performed frequently enough which consequently leads to challenges in identifying the exact locations of faults. The algorithm was evaluated on artificial data with injected faults and could detect over 90 % of anomalies if they were prominent enough. Longer sequences before fault deviations occur, improved the ability of detecting the faults. Thus, further motivating the need to collect data more frequently. On a final note, the automated performance analysis framework successfully improved the efficiency of fault detection, and greater contextual data awareness were achieved through the visualization features. Manual analysis can however detect faults with greater accuracy. On that ground, these results should be interpreted with caution.