J.J. Hoover had allowed only a pair of homers all season. No. 3 came at a very bad time.

Matt Carpenter hit a tying homer in the eighth off Hoover, and Randal Grichuk connected in the 13th inning, rallying the St. Louis Cardinals to a 4-3 victory over the Cincinnati Reds on Wednesday night.

Grichuk was moved up to second in the Cardinals' struggling batting order. He doubled home a run in the sixth inning and hit his 15th homer in the 13th off Dylan Axelrod (0-1), Cincinnati's eighth pitcher.

Seth Maness (4-1) gave up a pair of walks in two innings.

"As everybody anticipated it would, it came down to one swing of the bat," Reds manager Bryan Price said. "We were able to string things together, but we couldn't come up with that knockout blow. We had a 3-0 lead. We just couldn't put it away."

Manager Mike Matheny changed the batting order — Grichuk went from batting eighth on Tuesday to second — to try to spark his struggling offense, but the Cardinals didn't get much going until late in the game.

Left-hander David Holmberg allowed two runs, including Grichuk's RBI double off the top of the wall in center that cut it to 3-2 in the sixth.

Carpenter tied it with a homer in the eighth off Hoover, only the third that the reliever has allowed this season. It was Carpenter's fifth homer in his last six games.

"I was trying to throw a fastball up and away, but I pulled it across the plate middle-in," Hoover said. "That's not what I wanted to do."

The Reds strung together walks and infield hits while scoring three times off Carlos Martinez.

They loaded the bases in the third with a walk, an infield single and a fielding error by shortstop Jhonny Peralta. Brayan Pena's opposite-field double landed just inside the left-field line and made it 2-0.

The Reds loaded the bases again in the fourth with a pair of walks — one to Holmberg — and Brandon Phillips' single just out of the reach of second baseman Kolten Wong. Todd Frazier's soft groundout got in another run.