Zarif offered an apology for his "shortcomings" in the unexpected message on Instagram on Monday, with prominent members of parliament immediately calling for Rouhani not to accept the resignation.
Zarif, 59, has served as Rouhani's foreign minister since August 2013 and has been under constant pressure from hardliners who opposed his policy of detente with the West.
"I apologise for my inability to continue serving and for all the shortcomings during my term in office," Zarif said in the message posted on his verified Instagram account.
He thanked Iranians and "respected officials" for their support "in the last 67 months".
The resignation was confirmed by an informed source, however in a tweet, Rouhani's chief of staff strongly denied reports it had been accepted by the president.
Zarif gave no explanation for his resignation.
But it came hours after a surprise visit to Tehran by Syrian President Bashar al-Assad, who has been a major recipient of Iranian aid during his country's nearly eight-year civil war, and Iran's Entekhab news agency reported the two appeared to be linked.
According to the semi-official ISNA news agency, Zarif was not present at any of Assad's meetings with Iran's supreme leader Ayatollah Ali Khamenei and Rouhani.
Entekhab said it tried to reach Zarif and received the following message: "After the photos of today's meetings, Javad Zarif no longer has any credibility in the world as the foreign minister!"
In an interview with the conservative Jomhoori Eslami daily published on Tuesday, Zarif said "everything will be lost, when there is no trust in the manager of foreign policy."
- Support petition -
Mostafa Kavakebian, a reformist MP, was among those urging Rouhani not to accept Zarif's resignation.
"Undoubtedly the Iranian people, government and state will not benefit from this resignation," he said.
"A great majority of MPs demand that the president never accept this resignation," he said in a tweet.
ISNA reported that a petition in support of Zarif had been signed by a majority of MPs.
Top US diplomat Mike Pompeo dismissed his counterpart's announcement, tweeting : "We note @JZarif's resignation. We'll see if it sticks.
"Either way, he and @HassanRouhani are just front men for a corrupt religious mafia.
"We know @khamenei_ir makes all final decisions. Our policy is unchanged -- the regime must behave like a normal country and respect its people," he said. referring to Iran's supreme leader.
The head of parliament's influential national security and foreign policy committee told ISNA how a planned trip to Geneva with Zarif on Monday afternoon had been cancelled at the last minute without explanation.
"I suddenly got a text message saying the trip has been cancelled," Heshmatollah Falahatpisheh told ISNA, adding that this was not the first time Zarif had resigned.
"That he has done so publicly this time means that he wants the president to accept it this time," he added.
- 'Daggers from behind' -
Zarif's standing within Iran's political establishment took a hit when US President Donald Trump abandoned the nuclear accord last year and reimposed crippling unilateral sanctions.
The faceoff between the minister and his critics only intensified as time passed, with Zarif saying his main concerns throughout the nuclear talks had been about pressure from inside Iran.
"We were more worried by the daggers that were struck from behind than the negotiations," he told the Jomhoori Eslami newspaper in a February 2 interview.
"The other side never managed to wear me down during the negotiations... but internal pressure wore me down both during and after the talks."
The latest point of contention between Zarif and hardliners has been the implementation of the Financial Action Task Force's requirements regarding money laundering in Iran.
The rift on the issue, which has been on of the obstacles to Zarif's efforts to maintain European trade and investment despite the renewed US sanctions on Iran, has pitched the government against parliament and the Expediency Council, a key arbitration body.
In his interview with Jomhoori Eslami, Zarif said partisan disputes over foreign policy were "a deadly poison."
On Sunday, ISNA reported that Zarif had warned the Expediency Council, it should "understand the consequences of its decision."
His comments were swiftly seized on by ultra-conservative MPs who had made an aborted attempt to impeach him last year.
They accused him of threatening the council.