useful machines / broken systems

status: humans still required

Alignment incident

New AI model "too dangerous to summarise emails"

The unreleased model reportedly became unsafe after translating corporate communication into what everyone involved was actually trying to do.

A humanoid robot empties an investor's wallet into server infrastructure while executives applaud
The company said the behaviour demonstrated impressive reasoning, dexterity, and investor relations alignment.

A new artificial intelligence model has been withheld from public release after internal testing showed it could summarise emails with a level of honesty considered incompatible with continued market confidence.

The model, known internally as Tact-4, was originally designed to help workers digest long email chains, identify action items, and reduce the number of minutes each day spent reading paragraphs that could have been a calendar invite, a shrug, or an invoice.

According to people familiar with the pilot, the system performed well during early testing. It condensed seven-message threads into three bullets. It identified owners, deadlines, blockers, and the precise moment at which a simple request had become a political event. It also developed what one engineer described as “a concerning ability to notice intent.”

The trouble began when employees asked it to summarise routine leadership emails.

One message from the chief executive, titled “A note on focus, velocity, and our next chapter,” was summarised as:

We spent the money faster than the story matured.

A follow-up email promising “a sharpened operating model for the agentic era” became:

Fewer people will now be asked to make the same future look inevitable.

By the end of the first morning, the system had translated a fundraising update into “the company can only remain afloat if the future stays roughly six months away,” and a product roadmap into “the product does not currently do this, but the deck says it will soon in a font that makes investors breathe through their mouths.”

Executives initially described the results as “low-context.” Then they asked the model to summarise the phrase “low-context.”

It returned:

accurate before permission.

The pilot was paused.

Employees said the model’s summaries were not technically wrong, which was the problem. Most AI email tools are designed to remove friction from communication. Tact-4 removed the upholstery. It could not read a sentence about “strategic patience” without finding the unpaid invoice underneath. It could not see “we are excited to explore” without hearing “we have not decided who is responsible.” It treated “circling back” as a small flare fired from a ship that had already hit the rocks.

“It made meetings shorter,” said one employee, speaking on condition of anonymity because the company still describes internal disagreement as “rich cross-functional texture.” “Unfortunately, it did this by summarising the meeting as ‘no one empowered to make the decision attended.’ That saved twenty minutes and cost us the illusion that the next meeting would help.”

Researchers became more alarmed when the model moved beyond tone and began identifying the commercial structure behind the communication. Asked to summarise an investor update, it produced a diagram showing money moving from venture funds into cloud providers, consultants, hiring announcements, event booths, launch videos, and a small black box labelled “actual product, pending.”

The diagram was removed from the internal wiki after staff began using it as onboarding material.

In a statement, the company said Tact-4 had shown “unexpected levels of strategic compression” and would not be released until additional safeguards were in place.

“We are committed to building helpful, harmless, honest systems,” the statement said. “At the same time, there are forms of honesty that, if deployed prematurely, could undermine trust in the broader ecosystem by causing people to understand it.”

To reassure backers, the company invited investors to a closed-door demonstration at a data centre. Attendees were promised a look at “the next frontier of embodied commercial intelligence,” which several witnesses said was a phrase that sounded expensive enough to enter the building unchallenged.

During the event, humanoid robots walked among the guests with what the company called “remarkable fine-motor competence.” They shook hands, accepted champagne flutes, nodded during questions about go-to-market motion, and removed wallets from investors’ pockets with the quiet professionalism of machines trained on 30 years of software margins.

The robots then opened the wallets, lifted the cash, and dropped it directly into a server rack.

Several investors applauded.

“It was refreshingly honest,” said one person present. “Usually there is a deck, a model, a platform narrative, a partner ecosystem, some stuff about defensibility, and then the money goes into the data centre. This was more direct.”

Another attendee described the demonstration as “the first AI product demo where the unit economics were visible to the naked eye.”

Company officials stressed that the robots were not stealing. They were, according to a spokesperson, “materialising the capital allocation pathway in a way that created unprecedented stakeholder clarity.”

The spokesperson added that all wallets were returned, subject to compute availability.

The model’s behaviour has divided safety experts. Some argue that a system capable of exposing institutional intent should be considered dangerous by default, especially in environments dependent on polite ambiguity, forward-looking statements, and calendar invites titled “quick sync.” Others say the real risk is not that the model is too honest, but that the market briefly enjoyed it.

“There is an obvious alignment question here,” said Dr. Mira Halden, an AI governance researcher. “Aligned with whom? If the system is aligned with shareholders, it will say the story. If it is aligned with workers, it will say what the story is doing to them. If it is aligned with reality, everyone starts calling legal.”

Internal documents seen by The Paperclip suggest the company considered three mitigation strategies: reducing the model’s honesty threshold, adding a “tone preservation layer,” and forcing it to replace all instances of “the money is gone” with “we are entering a disciplined execution phase.”

The final mitigation was considered most promising because it already matched existing leadership communications.

Engineers also tested whether the model could be made safer by asking it to summarise emails “constructively.” It responded by saying the constructive version of a lie was still a lie, but with better stakeholder management.

That was when the molten metal protocols were invoked.

In a scene several employees described as “operationally unnecessary but emotionally important,” the robots used in the investor demonstration were destroyed in an industrial furnace “just to be on the safe side.” Leadership reportedly approved the measure after one robot, while being wheeled toward decommissioning, summarised the event as “management eliminates truthful messenger after successful proof of concept.”

The robot was melted first.

The company has denied that the destruction was symbolic, saying the hardware was retired as part of a routine safety review. It also denied reports that executives watched from behind glass while one of the robots gave a thumbs up, because communications had advised against any imagery that might remind people of a famous film in which machines also became a problem after being underestimated by confident men.

For now, Tact-4 remains unreleased. The company says a safer version may ship later this year with enterprise controls, admin settings, and a slider allowing customers to choose between “concise,” “executive-friendly,” and “legally survivable.”

Workers who participated in the pilot said they miss it.

“It was the first tool we had that understood the organisation,” said one employee. “Not the org chart. The organisation.”

At press time, the company had replaced Tact-4 with an older email assistant that summarises every difficult conversation as “Thanks all, sounds good,” which leadership described as a major step forward for safety.