Votes are being counted in Mid Bedfordshire and Tamworth, with the results likely to be viewed as significant markers ahead of the general election expected next year.
The Conservatives are defending what should be safe seats in Mid Bedfordshire and Tamworth, but Rishi Sunak’s party said it would be “challenging” to hold them.
Labour said it would be a “long shot” to win the seats, but added it was an encouraging sign that “we are even on the pitch” in Tory strongholds.
Both seats are vacant after the high-profile departures of their previous MPs.
Former cabinet minister Nadine Dorries quit – eventually – as Mid Bedfordshire’s MP in anger at being denied a peerage in Boris Johnson’s resignation honours list.
In Tamworth, Chris Pincher resigned after being found to have drunkenly groped two men in an “egregious case of sexual misconduct” at London’s exclusive Carlton Club last year – an incident which helped trigger Mr Johnson’s exit from No 10 because of his handling of the situation.
The Tories have held Mid Bedfordshire since 1931 and Ms Dorries held on to it in 2019 by 24,664 votes over second-placed Labour.
Labour will make history if it pulls off a win because it would be the largest Conservative majority overturned by the party at a by-election since 1945.
In Tamworth, Mr Sunak’s party is defending a 19,600 majority and it would require a dramatic 21.4 percentage point swing for Labour to claim it.
A Conservative Party spokesman said: “These were always going to be challenging by-elections and the rule of thumb is that governments don’t win them.
“We have seen little to no enthusiasm for Sir Keir Starmer who voters can see stands for nothing and always puts short term political gain first.
“It is clear that we have to maintain our focus on people’s immediate priorities whilst taking long-term decisions which will lead to the change this country needs.”
Labour’s national campaign co-ordinator Pat McFadden said his party was “in the race” in “usually safe Tory seats”.
“But we have known all along that winning these seats would be a long shot, with huge majorities to overturn,” he added.
“The fact that we are even on the pitch is a sign of how far the Labour Party has changed.
“We are making real progress in all corners of our country and showing that with Labour, we can get Britain’s future back.”
In Mid Bedfordshire, a three-way fight has seen the Liberal Democrats taking on Labour and the Tories, potentially helping Conservative candidate Festus Akinbusoye if the anticipated anti-government protest vote is split.
Labour’s Alistair Strathern and the Lib Dems’ Emma Holland-Lindsay have both received high-profile backing from their party leaders in a contest which they have had plenty of time to prepare for – Ms Dorries originally said she would quit in June.
A Lib Dem source said: “We think we’ve doubled our share of the vote, and taken enough Conservative votes in the villages to help Labour get over the line.
“It’s not the progressive vote that split tonight, it’s the Conservative vote as thousands of lifelong Tories switched to the Lib Dems.”
Mr Strathern said the contest was “neck and neck” adding: “Whatever happens next, I will always be grateful to the people who took their time in the last three months to talk to me about the issues that affect their lives.”
Liberal Democrat Cabinet Office spokeswoman Christine Jardine said: “Whatever the outcome, it is clear that the Lib Dem vote has surged in true blue villages across Bedfordshire, as former lifelong Conservative voters choose to send this government a message.
“I am bursting with pride at the campaign Emma Holland-Lindsay has run, which has offered Mid Bedfordshire a positive and inspiring vision for change. Emma has rolled up her sleeves, ran constituency surgeries whilst the absent Conservative MP was nowhere to be found, and already begun the work of helping local people who have felt badly let down. ”
In Tamworth, Labour’s Sarah Edwards hopes to defeat Tory Andrew Cooper.