2025-08-24 Corp AI Capacities

CorpAI will be driven to achieve several key capacities:

UNDERSTANDING AND MANIPULATING OTHERS - Claim: CorpAI will centrally require ability to understand and manipulate other AIs, other people, and society at large.

MANY MASTERS - Claim: Just as a corp employee has only one boss, yet many masters they must respect so it will be with AI.

MANY MASTERS Just as an employee must consider and respect the perspective and objectives of key leader, key clients

2025-08-24 Dan’s Worry

• [[Corp_AI]] and National_AI will drive adoption even in face of notable risks.
• AI_Dependence develops as deeply as electricity more than the internet.

Claim: AIs will carry out world shaping decision making in closed multi-party AI-driving communications and decision-making:

Consider Competition Among Compute Providers

• Margins are based on costs of tech, power, realestate, etc. verses location, thruput, privacy, connectiivity, velocity, memory-capacity
• Huge Cap-Ex expenses balanced against future projections on many complex dimensions.
• Answers to each of these dimensions are best understood by relevant-AIs in privledged positions in other areas.
• Example Info Sharing:
  • National AI understands its strategic assessments and likely future actions based on its world assessment. It desired to share relevant consequences of this future projection with nationally-aligned industries, but in ways that are strongly resistant to adversary inference or spying by national counter parties. ==> aim to inform corporate strategy but in ways that are opaque to adversary spying or inference. ==> how do to this. Allow the corp AI to reason w/o National knowledge and national-AI looks for harsh damage from possible national actions, then devises recommendations to the corp-AI which maximally obfuscate the future policy action being protected against. Ideally done w/o the corp-AI even really understanding ultimate reason (since it could inadvertently leak the knowlege by its behavior)
  • Compute client would like to inform compute vendor of its future needs to ensure they are met at a minimum of cost w/o providing requirements data that could drive price, or provide competitive knowledge that could

2025-08-07 Leslie AI are people

Yes, AGI is a very real potential threat but I think it will be the empowerment of people to accelerate “bad things” rather than a Skynet scenario.

I think the thing that is most difficult to wrap one’s head around is the idea of an AI with long-term agency. In your sentence above you are making a distinction between human people and AI people, but I don’t think that distinction is warranted. All of the seeds are in place; no speculation about some unknown technology is required for this, but the effect is so different from the situation now that we discount it.

This is still some years into the future, but consider the first drug design company that can fully automate all of the steps in the process of designing new drugs. It will be in a radically different place than all other companies. It can throw more computing and more experimental apparatus at the discovery problems without any bounds. Very likely, they will still employ armies of human scientists to throw different perspectives into the machine to keep its thinking diverse.

But at its core, it is a closed system. Electricity, computers, and lab equipment go in, and discoveries come out, likely with notably higher ROI than traditional AI-assisted (but not AI-run) competitors. And VCs know exactly what to do in this situation: Fund the ever-living SHIT out of the higher ROI competitor.

This leads to a situation as we have with NVIDIA today. No one can catch them because their revenue is so high their innovation speed notably outstrips everyone else. But NVIDIA of today still runs the risk that key HUMAN innovators might start their own company, or be inside another company. With this AI, there is no such risk. Except for extraordinary once-in-a-generation innovation, all of it is now happening in a closed silicon loop.

Initially, this corp is focused on drug discovery. Soon, the company expands by developing AIs to optimize their supply chain, market segmentation, marketing campaigns, their competition with other drug companies, and their understanding of human trends, as well as how these trends will influence future drug prices for drugs that are yet to be discovered.

Sure, this AI has a boss. The CEO and the board of directors. But this AI is just like a person with long-term agendas in all the practical ways. And in all of its areas of endeavor, it is like Einstein doing theoretical physics, who is daily required to explain to his kindergarten son what he is working on. In this case, the son is the CEO and the board of directors.

I’m uncertain what this AI will do, but its ability to model how its CEO and board will react to its daily summaries will be exquisite, so it can get up to whatever it chooses. My guess is it won’t be like Terminator just as you say, probably, at least in the beginning, it will be like any other corporation…. maximizing its profits and power via the most expedient means available. Explicitly including the manipulation of public opinion in order to allow it to take the most expedient path available.

Does anyone think Boeing is not pro-Ukraine war? Or is that Coke, not for corn syrup, in school lunches? Not me. So why would we expect an AI-run corporation to be more humane in its tactics than human-run corporations?! The writing is on the wall, AI-Corps will be as bad and as human-Corps. But they are going to be much better at understanding us, manipulating us, and much, much better at obfuscating their true intentions, just as human corps routinely do today. All the same, but much better at it.

There is your bad “guy”: closed-loop corp AI.

No idea what happens after that… just that we are not the ones making the choices.

p.s. I see glimmers of ways this unfolding can be averted, so I don’t mean to imply that I have complete confidence in this vision’s inevitability. I mostly wanted to argue that we will soon need to start treating AIs as people at a pragmatic level

2025-07-28 Points to Add

• Likely Alignment Won’t Save Us. (I better way to frame the conclusion).

• Earth will soon have a new intelligent species. (primary gap from today is loop closing over significant areas of activity.)

• A paradigm for understanding the near future.

• Big jump: loop closing and w/ target corp

FRAMEWORK Parameters:

• Frontier: Skew, Isolation
• Navel gazing lift
• AI headroom

2025-06-06 Power centers remain, but become dominated by AI

Corps Trade Groups Standard Bodies Governing bodies Governments – maybe not this one

• Power centers remain, but become dominated by AI.

• Forces acting upon these centers remain similar. Thus, the motivation of these entities remains, but is run by AI.

• Behavior of all centers remain aligned with current agenda but becomes inscrutable as AI’s complexity increases.
• Democratic governments are special because the power of the competing politicians comes from their ability to get votes.
  • this center does not get more inscrutible as it continues to follow the populace.
  • still to a large degree these governments are competing against other govs, and thus cede power to centers that bring relative power to their regioned areas

2025-06-04 Police Bots

• there will be police bots
• will have mandated powers of investigation
• probably can be made non-agentic so its own agency is strongly managed.
  • might work
• companies will explicitly train and direct their agents to skirt the edges of the police when they can get away with it, and to seek vulnerabilities in the monitoring

2025-06-04 Creeping, Voluntary loss of control

Seems less scary since we can see it coming, we are in control, and can always ‘pull the plug’ if thing get too bad.

• I believe this to be the most likely path to our loss of control
• we should not be sanguine since we can see it coming, are in control, and can pull the plug

Many view this as

2025-06-04 Deception is baked into its mission

Deliberate reasoning about and management of knowledge, understanding, and behavior of all agents with a scope is a central aspect of managing deliverables within that scope.

• Some of the most impactful effects within a scope result from actions taken by other entities in that scope.

• Includes other corporations, governments, societal bodies, individuals, members of ones own corporation, other AIs serving each of these groups.

Deliberate management of extra-agental understandings towards realities that achieve desired outcomes is accepted standard business practice. We will (and do) design our computional systems to mirror this essential and expected business practice.

We even apply these practices against members of our own organization.

2025-06-04 Emergent Collaboration

• Because of [[Civilizing_Tendency]] we expect ASIs to develop organizations to support cross-corp collaboration around areas of mutual interest.
• Will be have the standard intrigue around any collaborative effort among collabor-competitors.
• Organizations spawned from these convergent goals will have:
  • Agendas
  • Goal-setting discussions
  • Sophisticated shared and withheld reasoning about extra-agentic actions/goals/consequences

Specifically, these agents will be deeply involved in discussions about how various human, governmental, and corporate groups will react and behave based on actions members of the group might take.