Fresh allegations are again surrounding Prince Andrew, with a new book set for release next week containing further claims about the Duke of York’s private and public life. Entitled: The Rise and Fall of the House of York by Andrew Lownie revisits Andrew’s ties to convicted sex offender Jeffrey Epstein and even alleges he was involved in a physical altercation with Prince Harry after making comments about Meghan Markle. A spokesperson for Prince Harry told PEOPLE the claims were false, insisting there had never been a fight nor such remarks.

The book also delves into Andrew’s controversial relationship with alleged “Chinese spy” Yang Tengbo and the long-running “siege of Royal Lodge,” during which King Charles withdrew his brother’s £1 million-a-year personal allowance.

According to The Daily Telegraph’s royal correspondent Hannah Furness, palace officials are quietly discussing ways to further distance the Royal Family from Andrew. One rarely-used mechanism, she notes, would allow Parliament to remove not only a dukedom but even the title of “Prince,” though it would require legislation and Royal Assent.

Royal historian Christopher Wilson suggested that such action is more likely under Prince William, should he become king, describing him as “ruthless” about preserving the monarchy’s image in the social media era. By contrast, Wilson said King Charles has largely followed the late Queen Elizabeth II’s policy of avoiding overt family ruptures.