Rivera homered and drove in four runs, Erasmo Ramirez allowed one hit over five innings and the Rays beat the Yankees 6-1 on Thursday night.
After losing the series opener 11-5, the Rays battled back to win the final three games.
"We played really good baseball the last three nights," Tampa Bay manager Kevin Cash said.
Yankees starter Chase Whitley (1-2) left with two on and two outs in the second due to a right elbow injury. He was replaced by Esmil Rogers, who gave up a three-run homer on his first pitch to the light-hitting Rivera that put Tampa Bay up 4-0.
"I was looking for something in the strike zone so I could put a good swing on it," Rivera said.
Whitley will undergo imaging tests and further examination Friday.
"Obviously a concern about this," Yankees manager Joe Girardi said. "It's in that area you don't want to talk about."
Ramirez (1-1) didn't allow a hit after Jacoby Ellsbury opened the first with a single. The right-hander struck out four and walked two.
Alex Rodriguez homered in the ninth off Matt Andriese, who went four innings and completed the five-hitter for his second save.
A-Rod got his 1,000th RBI with the Yankees on his 663rd career homer.
The Rays have given up two runs or fewer 19 times this season, tops in the majors.
Rivera, who entered hitting .126 with one homer and five RBIs, made it 5-0 on an RBI single off Rogers in the fourth. Rivera's career high for RBIs is five set on April 28, 2014, for San Diego against San Francisco.
"We needed that, Rene needed that," Cash said.
James Loney opened the scoring with a first-inning RBI single off Whitley, charged with three runs, two hits and two walks.
Girardi said left-hander Chris Capuano, who is coming back from a quadriceps injury and pitched six innings for Triple-A Scranton/Wilkes-Barre on Tuesday night, will take Whitley's spot in the rotation.
Whitley has been filling the spot that opened when Masahiro Tanaka was injured.
"It's frustrating but I've been dealing with (tightness) for a while," Whitley said. "Of course there's a concern but I'll know more about it tomorrow."
The announced crowd was 11,977. The four-game set drew just 44,937.
Yankees: Girardi said INF Brendan Ryan (calf) left an extended spring training game early due to heat exhaustion. ... Tanaka (right wrist and forearm) will have his second bullpen session Friday.
Rays: RHP Alex Cobb had elbow ligament-replacement surgery. ... LHP Matt Moore (elbow surgery) will have his first simulated game Monday.
Andriese was optioned to Triple-A Durham after the game to make room for closer Jake McGee, who is coming back from offseason elbow surgery and was reinstated from the 15-day disabled list.
Rodriguez has 1,990 RBIs, which is two away from tying Babe Ruth for fourth place all-time since RBIs became an official stat in 1920. ... Four of his nine homers this season have come off Tampa Bay pitching. ... He has 51 homers in 212 games against the Rays. ... His solo homer stopped a stretch of 26 consecutive innings without an extra-base hit for the Yankees.
Yankees: RHP Michael Pineda (5-0) will face Kansas City RHP Chris Young (2-0) on Friday night. Pineda struck out 16 Sunday against Baltimore, matching the second-highest total in team history.
Rays: RHP Jake Odorizzi (3-2) and Minnesota RHP Phil Hughes (2-4) are Friday night's scheduled starters.