Integrating power-to-heat (P2H) assets in combined heat and power plants (CHPs) is an attractive option, which can improve the flexibility in CHPs. This paper compares the potential benefits of integrating an electrical boiler (EB) and a heat pump (HP) in a CHP from providing flexibility services in both the day-ahead market and the frequency regulation market. An optimization model is developed for the operation of P2H assets and the CHP to maximize the profit. A case study is carried out using the data of a real CHP and electricity prices of Nord Pool. It is found that when an EB or a HP is integrated, the annual profit of the studied CHP from providing frequency regulation can be increased by 3.1 % (EB) or 27.7 % (HP) respectively compared to the CHP without P2H. Despite the high capital cost, a HP can increase the net present value up to 21.8 %, and achieve a payback period of 3 year, which are better than an EB (0.8 % and 5 year). Sensitivity analysis shows that prices of fuel and electricity have significant impacts on the net present value and payback period for the integration of P2H assets. Even though the increase of the fuel price decreases the NPV, it can lead to a decline in the payback period. Meanwhile, the increase of the electricity price results in a large growth in the profit and NPV, but a big reduction in payback period.