There are two ways a Conservative Party leadership contest can be triggered. The first is if the current leader resigns.
The most recent example of this was back in May 2019, when Theresa May announced her resignation as Leader of the Conservative Party and Prime Minister once her successor, Boris Johnson, was elected in July 2019.
The UK has de facto caretaker governments during general elections but not during leadership challenges. So, the prime minister in place during a leadership contest would still have all their normal prime ministerial powers.