Robin van Persie hit a dramatic hat-trick as Manchester United fought back to win 3-2 at Southampton in Sir Alex Ferguson's 1,000th league game in charge, while Arsenal piled on the misery for struggling Liverpool with a 2-0 victory at Anfield on Sunday.
Van Persie looked like ruining Ferguson's landmark celebrations when he missed a penalty with United trailing 2-1 midway through the second half at St Mary's.
But the Dutch forward showed why Ferguson splashed out £24 million to sign him from Arsenal last month with two clinical finishes in the final three minutes to snatch the points from Southampton's grasp.
Rickie Lambert had headed Southampton into the lead from Jason Puncheon's 16th minute cross, but van Persie equalised seven minutes later with a fine strike after Nathaniel Clyne slipped while trying to reach Antonio Valencia's cross.
Southampton, who are bottom of the table after losing their first three matches back in the top-flight after a seven-year absence, regained the lead 10 minutes after half-time when midfielder Morgan Schneiderlin directed a diving header past United goalkeeper Anders Lindegaard.
Van Persie won a penalty when Jos Hooiveld fouled him in the 69th minute, but Kelvin Davis saved the Dutchman's chipped spot-kick.
Determined to make amends for that blunder, van Persie bagged the 87th minute equaliser with a close-range finish before heading the winner from Nani's corner in stoppage-time.
Earlier on Sunday, Arsenal had ended their goal drought in emphatic fashion as Lukas Podolski and Santi Cazorla inspired an impressive 2-0 win at Liverpool.
Arsene Wenger's side had drawn a blank in their first two Premier League matches as they struggled to adapt to life without van Persie.
But the Gunners, held to goalless draws by Sunderland and Stoke, were back on target at Anfield thanks to two of Wenger's pre-season signings.
Germany striker Podolski netted his first goal for the club in the first half to end Arsenal's barren run of 211 scoreless minutes.
Then Spanish midfielder Cazorla bagged his first Gunners goal in the second half with a low shot that Liverpool goalkeeper Jose Reina should have saved.
That was more than enough to clinch Arsenal's first win of the season and leave struggling Liverpool with just one point from their first three matches.
After Liverpool missed out on long-term transfer target Clint Dempsey and failed to make any other signings on deadline day, this lacklustre display was a painful end to a dispiriting week for Reds manager Brendan Rodgers.
"The sharpness gets better from game to game. Overall we looked solid," Wenger said. "We missed opportunities on the counter-attack at the start of the game but we had a convincing team performance."
Rodgers added: "I am disappointed with the scoreline and the goals we conceded.
"We have things to work on, needing to keep the ball better and Arsenal did that well in the final period.
"Giving the ball away has nothing to do with the style of play. There is a long way to go."
In the day's other game Aston Villa earned their first point under new boss Paul Lambert with a 1-1 draw against Newcastle.
Villa had lost the first two matches of Lambert's reign, but they took the lead at St James's Park through Ciaran Clark's 22nd minute header.
French midfielder Hatem Ben Arfa salvaged a draw for Newcastle with a brilliant strike in the 59th minute.
