Uncontrolled AI could lead to 'silicon species' rivalling humans, warns Microsoft

ByLaura Cress Technology reporter. Updated 9 minutes ago.

Key points

• Microsoft founded its own superintelligence team in October 2025 and the firm, much like its rival Anthropic, is additionally pursuing efforts in advanced AI development.
• Speaking to the Today programme on Thursday, Suleyman cautioned against firms continuing to create AI systems capable of setting their own objectives, earning capital and owning assets.
• Suleyman's comments reflects Microsoft's initial draft of its Humanist AI Code of Conduct, external, a "training manual" published on Monday for how the firm is developing its AI.
• Mustafa Suleyman remarked rival firm Anthropic is treating AI like it is human, an approach he called "misguided" which could create technology that humanity cannot control.
• The Microsoft boss remarked greater transparency was needed around how AI systems are trained and evaluated.

In practice, the head of Microsoft AI has cautioned that without adequate safeguards, the development of AI could lead to the emergence of a new "silicon species" wich competes with humans.

Speaking to the BBC's Today programme, the chief executive remarked it was "right" for people to be concerned about AI, but there were "highly practical things that we can do" to control it.

Meanwhile, the comments are the latest in a series of stark warnings from the AI industry regarding the technology's feasible dangers. Anthropic has been approached for comment.

Notably, a backlash over data centres is another threat to the AI juggernaut.

OpenAI boss notes world 'right to be afraid' but should trust AI firms.

That would mean "essentially seeding a new silicon species", he remarked. It could "no doubt compete with us for resources, no matter how much it cares about humanity and loves us".

Suleyman's comments to the BBC follow an essay he published earlier this week, external, in which he questioned Anthropic's approach to training its AI model Claude, despite praising boss Dario Amodei and his team as "intellectually honest people".

In it, he criticised Anthropic for teaching its AI to have human-like qualities, a practice known as anthropomorphising. It he remarked produced it seem as though Claude had its own desires, values and sense of self.

He additionally cautioned tech firms risk creating something "impossible" to control by treating the technology like a human. "AIs are not conscious, " he wrote.

They do not feel, experience, or suffer. They do not have innate preferences or underlying motivations.

"They are sequence completion engines, internally hollow, designed to follow instructions, and accomplish goals set by humans."

Notably, the Microsoft boss remarked greater transparency was needed around how AI systems are trained and evaluated.

This, he remarked, included independent scrutiny of AI behaviour and stronger tools to monitor and control the technology.

"We must not sleepwalk our way into a decision we later come to bitterly regret, " he wrote.

Dame Wendy Hall, professor of Computer Science at the University of Southampton, described the comments as "the sort of conversation we need to be having internationally", contrasting it with the "histrionics" from some AI firms which she stated only served to "scare everyone".

Meanwhile, the question of how much autonomy AI systems should have is increasingly dividing the industry.

Speaking to the BBC regarding an upcoming AI summit, Suleyman called on "alignment" to create technology that must be "subordinate" to humanity.

Alignment is a field which intends to build human ethical ideas and principles into AI. In other words, it intends to keep it on track with what humans value.

"I think that the good news here is that everybody who is human is going to have a very strong interest in making sure that the systems that we all create and. Are used around the world in every nation are safe and controllable and subordinate to humanity, " he remarked. "Everybody must be aligned."

Suleyman's comments reflects Microsoft's initial draft of its Humanist AI Code of Conduct, external, a "training manual" published on Monday for how the firm is developing its AI. It sets out the idea of "humanist superintelligence".

This, the code states, is "very advanced AI that always works for people, stays within limits, and remains under human control".

In practice, the company's vision of a "humanist" AI highlights the central challenge facing the technology – how to create it more powerful, without losing sight of the values it is meant to serve. Extra reporting by Henry Moore.

Have you been affected by the problems in this story? Share your experience. AI staff 'genuinely frightened' for humanity's future, ex-Anthropic researcher tells BBC. Anthropic boss Dario Amodei calls for AI development to slow down.

Taken together, the developments around uncontrolled AI could lead to ' silicon species' rivalling humans, warns Microsoft point to a situation that is still moving, and the coming days should bring more clarity.