Prince Harry is speaking out after losing a legal appeal over his security arrangements in the U.K., revealing the toll it’s taken on his relationship with his family.

In an interview with BBC News on May 2, the Duke of Sussex reflected on the judge’s dismissal of his challenge to the reduction of his security detail, which changed after he stepped back from royal duties in 2020.

“I would love reconciliation with my family. There’s no point in continuing to fight anymore,” Harry said. “Life is precious. I don’t know how much longer my father has.” Referring to King Charles’ ongoing cancer treatment, he added, “He won’t speak to me because of this security stuff, but it would be nice to reconcile.”

Harry believes the security issue could be resolved through his father, saying, “There is a lot of control and ability in my father’s hands… This whole thing could be resolved through him. Not necessarily by intervening, but by stepping aside, allowing the experts to do what is necessary.”

However, the palace maintains that King Charles does not have the authority to reinstate Harry’s security.

Tensions between Harry and the royal family became public in 2020 when he and Meghan Markle stepped away from their roles. While several factors contributed to the rift, security remains a central concern. A source told PEOPLE last year that Harry’s calls and letters about his father’s health have gone unanswered. “He gets ‘unavailable right now,’” the friend said.

Harry returned to London in April for a two-day hearing seeking the reinstatement of state-funded security during U.K. visits. Leaving court on April 9, he told PEOPLE he felt “exhausted and overwhelmed,” adding, “My worst fears have been confirmed by the whole legal disclosure in this case — and that’s really sad. This one has always mattered the most.”

On May 2, Judge Sir Geoffrey Vos ruled against Harry, stating that RAVEC’s (the Royal and VIP Executive Committee) decision to downgrade his protection was “understandable and perhaps predictable.” The judge added that further risk assessments would likely not have changed the outcome.

In response, Harry announced he would write to the Home Secretary urging a review of the RAVEC process, calling the legal action “a last resort” that had “uncovered shocking truths,” including the revelation that the Royal Household played a key role in decisions about his safety.

In the BBC interview, Harry said he “can’t see a world” where he would bring Meghan or their children—Prince Archie and Princess Lilibet—back to the U.K. under the current circumstances. “The things that they’re going to miss is… well, everything,” he said. “I love my country. I always have… I miss the U.K., of course I do.”

Reflecting on the broader situation, he added, “A decision was made to keep us under the roof… but once they realized we were happier outside of the institution, then just look at the facts, look at the risk, look at the impact. If anything were to happen to me, my wife, or my children, look where the responsibility lies.”