A new report from The Telegraph states that King Charles fears that he may be in “legal jeopardy” if he tries to reestablish a relationship with his son, Prince Harry.

Published on Saturday, Nov. 2, The Telegraph‘s report included insight from a constitutional expert who said the Duke of Sus𝑠e𝑥’s long-running legal case against the U.K. government to reinstate his security when he visits the country may be hindering his relationship with his father.

“Here you have the infelicitous situation where the King’s son is suing the King’s ministers in the King’s courts. That is pulling the King in three directions,” a senior constitutional expert and adviser to the royal family told royal biographer Robert Hardman, per the outlet.

“You also have the situation where the King’s son publishes accounts of private conversations, some of which have been, shall we say, wrong,” Hardman added, referring to reports that Harry misremembered some of the private conversations between family members mentioned in his 2023 memoir Spare.

“So imagine the situation if the prince were to talk to his father about his court case and then later to describe that conversation – or, worse, a conversation which was not entirely accurate. There would be serious legal jeopardy. Harry would only have to say, ‘My father said this’ and a court case could collapse,” added Hardman, author of Charles III: New King. New Court. The Inside Story.

Back in January 2020, Prince Harry pleaded his case at the Sandringham Summit, which decided his and his wife Meghan Markle’s fate as non-working royals. He argued that their security should remain intact and that they should receive police protection when in the U.K.

RAVEC (the government organization that includes the Home Office, or the U.K. version of Homeland Security) eventually downgraded their security — despite the revelation from court documents showing that Queen Elizabeth, who died on Sept. 8, 2022, deemed it “imperative” for the Duke and Duchess of Sus𝑠e𝑥 to have “effective security.”

A key factor in Prince Harry’s decision to stay in California with Meghan, 43, and their two children was the ability to have armed private security in the U.S. The prince previously pointed to daily social media threats, potential home intrusions and alarming incidents such as a chaotic paparazzi chase in New York City in May 2023.

Harry is currently in the process of appealing the court’s decision to withdraw their security, and his spokesperson told The Telegraph: “The Duke of Sus𝑠e𝑥 hopes he will obtain justice from the court of appeal. The Duke is not asking for preferential treatment, but for a fair and lawful application of Ravec’s own rules, ensuring that he receives the same consideration as others in accordance with Ravec’s own written policy.”

The palace declined to comment when reached by PEOPLE. A representative for the Duke of Sus𝑠e𝑥 did not immediately respond to a request for comment.

Amid the estrangement, Prince Harry remains deeply concerned for the safety of his wife and children, Prince Archie, 5, and Princess Lilibet, 3, and has, according to sources, repeatedly asked his father for help.

“Harry is frightened and feels the only person who can do anything about it is his father,” a royal insider previously told PEOPLE.

Another source close to the situation said: “Harry is determined to protect his own family at all costs.”