Kavanaugh has denied the claim and others that later surfaced.
While President Trump railed against Democrats and apologized to Kavanaugh "on behalf of our nation" during that ceremony, the new justice seemed to be trying to bring the temperature down after his narrow confirmation.
Trump slammed Kavanaugh's opponents for a "campaign of personal destruction".
Retired Justice Anthony Kennedy, whom Kavanaugh once clerked for and whose place he has taken on the court, administered the oath of office.
"On behalf of our nation, I want to apologize to Brett and the entire Kavanaugh family for the awful pain you have been forced to endure", Trump said today during a ceremonial swearing-in at the White House.
The president said Kavanaugh overcame intense scrutiny by Democrats who considered him guilty and didn't want to give him due process.
Kavanaugh, who was accused of sexual assault while a teenager, was confirmed by the Senate on Saturday and sat with the court for the first time on Tuesday.
A 'Family Favorite' Emerges As Possible Replacement For Nikki Haley, Report Says
Trump called Haley a "very special" person, adding that she told him six months ago that she might want to take some time off. She did not say what she will do next, except that it will be in the private sector, according to her letter of resignation .
"However much we might sympathize or agree with EPA's policy objectives, EPA may act only within the boundaries of its statutory authority", Kavanaugh wrote in the ruling.
Kavanaugh's nomination had appeared safe until Christine Blasey Ford, a university professor in California, last month went public with allegations that Kavanaugh had sexually assaulted her in 1982 when they were high school students in Maryland. Only Samuel Alito and Stephen Breyer participated in a White House event after they had been sworn-in and begun work as a justice, according to the court's records on oath-taking by the current crop of justices. Either way, he added, for Trump to declare that Kavanaugh was proven innocent is a "wild overstatement to say the least, and very likely an inaccurate read of what happened".
During Kavanaugh's confirmation process, dozens of women had dressed up in similar costumes to represent the TV show's depiction of women living under a dystopian fundamentalist regime. He had already hired four law clerks, all women.
By the way, given the volume and shrillness of anti-Kavanaugh protests - from his original hearings all the way through to his delayed confirmation - this is an interesting detail, isn't it? "We wish you a long and happy career in our common calling".
Some analysts said the court's reputation could suffer as it becomes perceived as a political, rather than a legal, institution.
Senator McConnell called the current partisan divide a "low point", but he blamed Democrats.
Republicans hold a 51-49 majority in the Senate, with several seats up for grabs in November.
He thanked the Republicans who stood with Kavanaugh and said "it was a great honor to be involved in this situation".