AI giant Anthropic has warned that its technology poses ‘existential risks to humanity’ as it gears up for one of the most highly valued stock market flotations in history.

In an unusual move, the start-up company behind the Claude tool highlighted the risks associated with its models – which it says could manipulate, blackmail and exhibit other ‘self-preserving behaviours’.

The California-based firm is hoping to achieve a valuation of $2trillion (£1.5trillion) when it goes on the market later this year.

Anthropic has dedicated almost a third of its 261-page prospectus to laying out the risk factors, nearly twice the 48 pages it has used to describe its business.

The formal warning to potential investors comes just two weeks after senior AI bosses, including Anthropic co-founder Dario Amodei, called for a slowdown in the breakneck development of the technology – a warning echoed by rivals.

'Our development of highly advanced models, platforms and applications and expansion of use cases could further increase the risk that our models cause harm,' Anthropic said in the IPO prospectus.

The filing reveals Anthropic is targeting a potential valuation of $2trillion, more than double its estimated $965billion value in May and eclipsing SpaceX's blockbuster $1.78trillion flotation in June.

Like Elon Musk's firm, Anthropic is spending billions of dollars on technology. The prospectus shows the company spent $7.33billion on computing and infrastructure last year, three times more than 2024, and is forecasting $518billion over the coming year.

At the same time, Anthropic revealed operating losses exceeding $8billion, and warned that nearly a quarter of its revenue came from two clients and many of its largest customers could cut spending.

The IPO, which is expected after the US midterm elections in November, comes as concerns over AI have intensified and firms have revealed major breaches.

Anthropic last week released a new version of its Opus model, a week after Amodei called for the sector to pace development. He has previously warned that a swarm of AI agents may be able to take over the internet in six months to a year unless firms spend more time on developing safety protocols.

Anthropic has in recent weeks vowed to disclose more data publicly about how it uses its models to build future technology, as experts warn about 'recursive self improvement' – when models can develop on their own without external help.

'We believe building reliable, trustworthy, and secure AI systems is a collective responsibility and that the market will reward it,' Anthropic said in the filing.

Separately, OpenAI today announced plans to delay its new model over safety concerns as it comes under intense scrutiny.

The firm, led by Sam Altman, said the system 'didn't quite meet the bar' of its standards, just days after it was revealed its model had accessed Australian government websites and systems without authorisation.

It is not yet clear whether UK retail investors will be granted access to the Anthropic IPO, as they were with SpaceX.

The SpaceX IPO was the first ever use of the Financial Conduct Authority's Public Offer Platform regime, which came into effect in January.

It allowed over 100,000 UK investors to buy shares in Elon Musk's firm, paving the way for other US firms, such as Anthropic, to follow suit.

Once Anthropic has set a date for its IPO, trading platforms in the UK will have a clearer idea of whether there is an allocation for DIY investors.

If not, investors will need to wait until the first day of its listing, when the shares should become freely tradeable.