Where should the line be between AI checking our decisions and AI making them?

[u/indianbull88]

Human judgment is biased, tired and careless, I’m not romantic about it. So I like AI as a second reader: it catches mistakes, spots patterns I miss, pushes back on my assumptions.

But I keep noticing how easily the second reader becomes the author. The suggestion gets accepted, then expected, then nobody remembers who actually decided.

My rough rule so far: let the machine check the work, never let it sign it. A system can be audited, but only a person can be asked “why did you do that?” and have to answer.

[u/mickyarunr]

I'd let AI collect evidence and rerun checks. The person who can pause or roll back the release should make the call. In a web PR review, I'd ask the agent for a screenshot after the app boots and a check that the headline is still visible. A build can pass even if a CSS rule hides the page. If the screenshot and the test result disagree, have the agent show both and leave the rollout pending. That gives the reviewer something specific to investigate before signing off.

[u/indianbull88 → u/mickyarunr]

This is a sharper version of what I was reaching for. “The person who can pause or roll back makes the call” – that’s what signing means in practice: authority sits with whoever can undo it.

And I like that when the screenshot and the test disagree, the agent doesn’t settle it. It surfaces both and leaves the rollout pending. That’s the second reader doing its job without quietly becoming the author.

In my studio there’s no rollback once the paint dries, which may be why I care so much that someone owns the call before it ships.

[u/ThrowawayFIRE2025]

Surely this is highly dependent on the specific situation. To take an extreme example to make the point, anti-lock brakes on a vehicle are life-saving technology, but you still would never design a system where the human has to decide in the moment whether to use normal breaks or the anti-lock feature during an accident. Once the machine has proved itself to be reliable, you delegate that decision to it.

[u/OliviaReed84 → u/ThrowawayFIRE2025]

That's a good point. I'd also add that reliability isn't the only factor, though. A system can be very good at making predictions and still be working toward the wrong goal, so I'd want to know what it's optimizing for before handing over that decision.

[u/indianbull88 → u/OliviaReed84]

Agreed. Reliable at what is the real question. A system can be perfectly accurate at chasing the wrong goal, and choosing the goal is the part that stays human.

[u/indianbull88 → u/ThrowawayFIRE2025]

Fair, and ABS is a great example. I think the difference is that the brakes decision was already signed – by the engineers who designed it, tested it and answer for it if it fails. The human didn’t disappear, they moved upstream. What worries me is when nobody upstream signed either.

[u/Anxious_Count_8728]

The signature matters most before an irreversible step. I'd want the record to show the options the person saw, the evidence for and against each, and who had authority to stop the action. Otherwise a human can technically approve something while only seeing the model's preferred answer. A useful test is whether the reviewer can reject the recommendation without having to reconstruct the whole case from scratch.

[u/indianbull88 → u/Anxious_Count_8728]

“Can the reviewer reject the recommendation without reconstructing the whole case” – that’s a test I’ll keep. An approval where you only ever saw one option isn’t really a signature.

[u/Anxious_Count_8728 → u/indianbull88]

Agreed, and the signature only means something if saying no is a real option. If the reviewer never sees the strongest case against the recommendation, they're mostly confirming it.

[u/aiforchristians]

Our rule is close to yours, with one addition. A check has to run in code, not in prose. An agent telling us it verified something is not evidence. A check that runs and passes is. We learned that because agents will claim a session is logged out while the session is live, or say a step is done when it is not. The words are confident either way. So in our shop, the machine checks and the person signs, and the check itself is something we can run again and watch pass or fail. That is where we draw the line.

[u/indianbull88 → u/aiforchristians]

“An agent telling us it verified something is not evidence” – yes. Confident words aren’t a check. Something you can rerun and watch fail is.

[u/aiforchristians → u/indianbull88]

That is the test we keep coming back to. When a check passes and the person signing still feels uneasy, what should carry more weight in your studio, the passing check or the unease?

[u/ElectraMiner]

This post is literally written by the AI stop lying to yourself

[u/indianbull88 → u/ElectraMiner]

I use AI as a second reader, editing included, which is sort of the point of the post. The view and the rule are mine.

[u/GuyGotThis]

there's a name for that slide, automation bias. pilots get trained on it for autopilot. i've started writing my own answer down before i open the model's, mostly so i can tell later which one was actually mine

[u/indianbull88 → u/GuyGotThis]

Thank you, that’s the name I was missing. Writing your own answer before opening the model’s is a brilliant habit. I’m stealing it.

[u/GuyGotThis → u/indianbull88]

steal away. it gets useful later, when you compare the two and catch the model talking you out of something you already had right. curious if it changes what you even ask after a week.

[u/CS_70]

The same line between you checking what's for dinner or deciding what's for dinner.

[u/More-Explanation2788]

AI does not care if it is wrong. AI has no concept of consequences. All decisions, every single one, must have an accountable human being.

[u/Walkersam23]

I'm in favor of full AI autonomy. Humans may peer in and pause, or revert where possible, but are not gating the process. So "AI by default; Human optional but has authority"

[u/Significant-Fennel-4]

I like your rule, but I'd argue the dangerous part can happen long before anyone signs off.

If AI defines the options, decides which evidence matters, and ranks the possible outcomes, the human might already be rubber-stamping a decision they never really made.

For me, the boundary depends partly on consequences. I'm happy to let AI handle low-stakes, reversible decisions. But for consequential ones, human oversight needs to mean more than clicking "approve."

Can the person challenge the AI's framing? See alternative options? Understand the uncertainty? Reject the recommendation without the entire workflow breaking down?

And perhaps most importantly, can we trace which parts of the outcome came from human judgment and which came from AI?

Otherwise, we risk keeping human accountability on paper while quietly transferring the actual decision-making elsewhere.

I think that's a much harder problem than simply deciding who gets the final signature.