Evaluate Before You Automate: Using the IDEA Framework

When a task is tedious or repetitive, it’s very tempting to jump straight to: “How can I automate this?” Especially now that generative AI makes it possible to build things that would have required much more technical skill a few years ago.

But the more useful first question is: should I automate this at all?

Sometimes automation saves a huge amount of time. Sometimes you spend three hours building and maintaining a system that saves you five minutes. Sometimes the process works perfectly until one small thing changes, and suddenly the whole thing breaks. So before building anything, it helps to slow down and figure out what problem you’re actually trying to solve.

That’s the idea behind the IDEA framework, which is outlined in Systems Thinking for AI & Automation, part of the Applied AI: Data Analysis, Workflows, and Decisions series.

What is the IDEA Framework?

IDEA stands for Identify, Document, Experiment, and Adjust. It’s not meant to be a rigid recipe. It’s a way to think through whether automation is actually useful before you invest a bunch of time building it.

• Identify — What is the actual goal, what triggers the task, and what does a successful outcome look like?
• Document — What are the pieces of the workflow? What stays the same, and what changes each time?
• Experiment — What is the smallest version you can test before building the whole thing?
• Adjust — What happens when the real world inevitably does something your system wasn’t expecting?

Here’s what that can look like in practice.

1. Identify the Real Goal

The first thing is to separate the goal from the solution you already have in mind.

“I want to automate my team’s attendance email” sounds like a goal, but it’s actually a proposed solution. The goal might be: “I want my team to receive an accurate attendance summary every Monday morning without someone having to assemble it manually.”

That distinction matters because once you know what you’re actually trying to accomplish, you may realize there’s a much simpler way to do it. Automation is one possible solution. It shouldn’t automatically be the starting point.

2. Document the Workflow

Next, break the task into its basic pieces. Where does the information come from? What has to happen to it? Where does it need to end up?

For an attendance email, some things might stay fixed: who gets the email, the general format, or the kinds of information it includes. Other things change every week: the dates, attendance numbers, or notes.

Once you can see which parts are fixed and which parts vary, it becomes much easier to figure out what can be templated, what might benefit from AI, and what still needs human judgment.

3. Experiment Small

This is the part where it’s best to avoid the temptation to build the giant automated system first.

Try the smallest version that could possibly be useful. Maybe you give the raw attendance information to an AI tool and ask it to format a draft, but you still review and send the email yourself. If that works reliably, you can think about automating another piece.

There’s a big difference between proving that one step works and assuming an entire workflow will work because the demo looked good. Keeping a human in the loop while you experiment also makes it much easier to catch problems before they become automated problems.

4. Adjust for Reality

If the experiment works, then you have to ask what happens outside of the ideal case.

What if the attendance sheet changes format? What happens when someone new joins the team? How will you know if the system makes a mistake? Who is going to fix it when something breaks six months from now?

This is the part that gets skipped surprisingly often. An automation isn’t finished just because you got it to work once. If it’s going to become part of how you actually work, it also has to be maintainable.

Why Human Judgment Still Matters

Good AI use isn’t about automating as much as possible. It’s about being able to recognize when automation is genuinely useful and when it just adds another layer of complexity.

That also means being willing to decide that something is not worth automating. The time it takes to build a system, the time it takes to maintain it, the consequences when it fails, and the amount of human oversight it still needs are all part of the cost.

The IDEA framework is really just a way to make those trade-offs visible before you commit to a solution. Identify the problem, understand the workflow, test the smallest useful version, and then adjust based on what actually happens.

That kind of thinking will stay useful even as the specific AI tools keep changing—which, at the rate things are moving, they definitely will.

Take the Next Step

Applied AI: Data Analysis, Workflows, and Decisions is a three-course Michigan Online series about using AI for data analysis, automation, and decision-making. The focus throughout the series is on the principles underneath the tools: how to define the problem, work with information, evaluate AI outputs, and decide when AI is actually useful.