All may not be going well for Bernard Arnault, who owns luxury brands Louis Vuitton and Dior, after a whopping $54 billion wipeout. LVMH’s Arnault was once considered to be the world’s richest man, but has now fallen to 5 th place. With an aim to stay in his role longer, the founder of the Paris-based behemoth pushed to raise the retirement age from 75 to 80 in 2022.

“I have five members of the family working in the group, he told Bloomberg in an interview this year. “Let’s see if one of them has the capacity to take over.” The 75-year-old boss had an estimated net worth of $231 billion in March, falling to $200 billion by June, before eventually plummeting to $177 billion on Monday.

While he still remains wealthier than former Microsoft ( MSFT ) CEOs Bill Gates ($163 billion) and Steve Ballmer ($150 billion), he now ranks lower than Elon Musk ($265 billion), Jeff Bezos ($216 billion), Mark Zuckerberg ($200 billion) and Larry Ellison ($178 billion), according to the Bloomberg Billionaires Index . The drop in Arnault’s rank comes after the company’s stock price dipped 20%, wiping out $54 billion from his net worth. LVMH reported a fall in revenues, with its wine and spirits section being hit the hardest.

According to its earnings report, Champagne sales declined 15% in the first half of the year compared to the same period a year ago. But LVMH is not alone . Analysts at Bank of America Securities ( BAC ) recently downgraded European luxury stocks, citing sector.