The Mexican had three quick laps deleted for failing to stay within the lines and will start 15th in Sunday's Austrian Formula One Grand Prix at Spielberg's Red Bull Ring.
"Everybody's fully behind (Perez)," Horner told Sky Sports in response to speculation about possible driver changes.
"Any talk of replacing Checo is wide of the mark."
Without the lap deletions, Perez would have sailed through to the final top 10 shootout and Horner said that was even more frustrating.
"He’s got the pace today. He’s got a car that was easily capable of being on the first or second row. He was matching Max’s times - stay in the white lines, you know," said the Briton.
"It was 'strike one, strike two, Checo just stay in the white lines, strike three', and that was it.
"Just hugely frustrating because he could have been there, he could have done it. So that's the frustration. It’s fantastic to have got the pole, but it feels not complete."
Horner said the radio instructions to Perez, recovering from sickness on Thursday, had been 'crystal clear'.
"It's just a great shame because I think this would have really kick-started things for him. And of course the frustration for us as a team is now we have two Ferraris that are a little closer versus one Red Bull."
Ferrari's Charles Leclerc (25) and Carlos Sainz (28) qualified second and third.
Verstappen, on pole for the fourth race in a row, also had some tense moments early on when he called the track limits "a joke".
"At least he responded," said Horner. "We said 'Stay in the white lines' and he did that, so he built a conservatism into his laps to make sure that he had a wheel inside the line. He was driving with a bit of restraint".
Perez, who has finished off the podium in the last three races while Verstappen has won four in a row, said he was blocked at the end but stewards saw it differently.
"There are so many things I can control but unfortunately this one - you are closing a good lap and all of a sudden you are blocked and you have a penalty. I think the system is wrong," he said.