The comment came as allegations that Mr Allen had sexually abused his daughter, returned to haunt the 80-year-old veteran US director.
Mr Allen, whose film Cafe Society, premiered on Wednesday, was seated in the front row at the opening ceremony when Mr Lafitte referred to rape charges in a gag that drew gasps from the audience.
"It is a pleasure to see you in France, because you have shot many films in Europe in recent years, and yet in the United States you haven't even been convicted for rape," Mr Lafitte said.
It was not clear if the comment was directed at Mr Allen or fellow director Roman Polanski who has been living in Europe to escape charges he raped a 13-year-old girl in the United States.
Mr Lafitte tweeted:
I wanted to be funny, impertinent and quirky, I acknowledge that what I said was inappropriate or could be misinterpreted. (1/2)— Laurent Lafitte (@LafitteOfficiel) May 13, 2016
I'm deeply sorry for all those that I offended. (2/2)— Laurent Lafitte (@LafitteOfficiel) May 13, 2016
Joke about 'American puritanism'
Allegations had resurfaced on Wednesday that Mr Allen sexually abused his daughter Dylan Farrow, now 30, in the early 1990s.
His estranged son Ronan Farrow published a column in trade magazine The Hollywood Reporter lashing out at the media for failing to ask hard questions about the allegations.
Ronan said that his mother Mia Farrow decided not to press charges at the time due to Dylan's "fragility."
Mr Lafitte had earlier told The Hollywood Reporter that the punchline had been misconstrued.
"When I wrote this joke, it was more a joke about Europe and why one of the greatest American directors spent years in Europe, while Mr Allen didn't have to because he wasn't accused of rape in his own country, compared to Roman Polanski," he said.
"It was as a joke about American puritanism and the fact that it is surprising that an American director wants to do so many movies in Europe. I didn't know about the other stuff," he added.
Mr Lafitte said that festival director Thierry Fremaux had approached Mr Allen at the after party to ask if he was offended by the joke "but he said he was not."
"He said, 'I thought that guy was funny.' It's a storm in a tea cup, and it's a shame, because I tried as much as I could to write something funny every 20 seconds," Mr Lafitte said.
"In a show lasting 45 minutes, this is the only thing that will remain," he said.
Mr Allen told Variety magazine on Thursday: "It would take a lot to offend me."
"I'm a comic myself and I feel they should be free to make whatever jokes they want," said Mr Allen.