How Judgment Outperforms Speed in AI-Driven Data Analysis

In this conversation, ex-Google data scientist Sundas Khalid reveals that AI-powered data analysis isn’t replacing human analysts--it’s redefining their role in a way that rewards judgment, systems thinking, and validation over raw coding speed. The non-obvious implication? The real competitive advantage no longer lies in who can run the fastest query, but in who can ask the most incisive questions, spot flawed outputs, and guide AI through messy real-world data. This is essential reading for data professionals, marketers, and product leaders who want to avoid career-damaging errors from blind AI trust--and instead position themselves as the indispensable translators between machine output and business reality. Those who master this workflow won’t just survive the AI wave--they’ll lead it.

Why the "Set and Forget" AI Mentality Fails in Data Analysis

Most teams treat AI tools like Codex or Gemini as magic wands: upload data, type a prompt, and expect truth. Sundas Khalid’s workflow exposes a dangerous flaw in that logic. AI doesn’t understand data--it manipulates patterns. It can generate a cohort analysis in nine minutes, yes. But it can also confidently misstate causality, misalign timeframes, or hallucinate trends--all while sounding definitive. The real work begins after the AI finishes.

Khalid’s approach is a masterclass in consequence-mapping. She doesn’t hand off the entire analysis. She delegates the mechanics--data parsing, chart generation, SQL drafting--but retains ownership of the meaning. This creates a feedback loop: AI accelerates the tedious, humans validate the critical. The immediate benefit is speed. The downstream effect? A sharper analyst, not a lazy one. Because when you’re forced to check every conclusion--“Does the math even make sense?”--you develop deeper intuition. You start seeing where the model could fail, not just where it succeeded.

And failures are inevitable. Khalid notes that Codex, like most agentic tools today, assumes the data it’s given is clean and complete. It won’t pause and say, “Hey, 30% of your mobile session data is missing--should we impute or exclude?” It just plows forward. That’s a silent killer. A flawed input creates a flawed output, but the output looks polished--charts, slides, executive summaries. The leadership deck is ready by 2 PM. The mistake isn’t caught until Q3 revenue tanks.

"It’s possible that AI is going to give you something wrong and if you don't have the analytical thinking and the domain knowledge you're going to get wrong answers and that's going to hurt your career more than it's going to help."

-- Sundas Khalid

This is where conventional wisdom fails. The belief that “AI will replace analysts” collapses under systems thinking. Yes, AI can write Python. But it can’t care if the conclusion is right. It can’t feel the weight of presenting flawed data to a room of VPs. It can’t anticipate the follow-up question: “What did we do differently the week before?” That’s human work. And it’s becoming more valuable, not less.

The Hidden Cost of Pretty Decks and Fast Answers

Here’s the kicker: the faster the output, the higher the risk. When AI delivers a full cohort analysis and leadership deck in under ten minutes, it creates a false sense of completion. The system rewards speed. Stakeholders see slides and say, “Great, we’re done.” But the real analysis hasn’t even started.

Khalid points out that Codex’s decks “don’t look the prettiest,” but that’s almost a feature, not a bug. If the output were flawless, teams might skip validation entirely. The roughness forces engagement. You have to read the numbers. You have to check the logic. You have to ask, “Why did retention spike the week before the drop?”

This is where delayed payoff creates separation. Teams that rush to present lose. Teams that slow down to validate--despite the pressure of a Friday 2 PM deadline--build credibility. They catch the mismatch between revenue drop (12%) and retention drop (30%) that doesn’t add up. They notice the email campaign exposure collapse to “1 5”--a clear parsing error. They fix it. And in doing so, they become the person leadership trusts, not just the one who delivers first.

The system responds. When your peers are chasing speed, your willingness to inject friction--by double-checking, questioning, refining--becomes your moat. It’s uncomfortable in the moment. It feels like you’re moving slower. But over six months, you’re the only one whose insights consistently hold up.

And here’s what most miss: the AI isn’t just analyzing data. It’s revealing where human judgment matters most. Khalid’s example of finding data tables at Google using AI? That’s not about coding. It’s about framing the problem correctly. The AI didn’t decide which tables were relevant--it responded to a precise, experience-shaped prompt. The skill isn’t in using AI. It’s in knowing what to ask it, and when to disbelieve it.

What Happens When Your Data Is Dirty (And AI Doesn’t Know)

AI tools assume clean data. Real-world data is never clean.

Khalid flags this as a silent failure point: “if your data has missing values or if it's incorrect then your answers that you're going to get is incorrect.” The AI doesn’t flag gaps. It doesn’t hesitate. It generates. And because the output is structured and confident, it’s assumed correct.

But in systems terms, this creates a compounding error. Garbage in, gospel out. The first analysis is wrong. The leadership acts on it. The next quarter’s strategy is misaligned. The feedback loop breaks. No one realizes the root cause was never the mobile app launch--it was a data pipeline failure two weeks prior that AI never questioned.

This is where the experienced analyst separates from the novice. The novice sees the AI’s output and thinks, “It must know.” The expert thinks, “What’s it missing?” They run the validation layer: pivot tables, manual spot-checks, cross-referencing with known benchmarks. They ask, “Does this feel right?”--a judgment built on years of seeing what real retention drops look like.

And here’s the real shift: the value isn’t in doing the analysis. It’s in designing the analysis. Khalid’s workflow assumes the AI will make mistakes. So she builds in human checkpoints. She doesn’t trust the cohort chart until she’s seen the raw weekly retention trend. She doesn’t accept the root cause until she’s ruled out alternatives.

"The way I treat it is like let's say if I have an intern who is working for me... I'll just give the crunching part and the coding part to my intern and yes the intern will come back with something but I need to sit down with the intern and figure out what exactly makes sense."

-- Sundas Khalid

That’s the new model. AI is the intern. Fast, eager, error-prone. The human is the manager. Slower, skeptical, accountable.

The 18-Month Payoff Nobody Wants to Wait For

Most teams adopt AI to go faster. The ones who win use it to go smarter.

The immediate payoff of AI in data analysis is obvious: reduce a 10-hour task to 10 minutes. But the lasting advantage comes from reinvesting that time. Instead of celebrating early completion, the best analysts use the saved hours to ask better questions. They don’t stop at “Why did retention drop?” They go to “What drove the spike the week before?” or “Can we isolate the impact of the email campaign from the mobile crash?”

This is where others won’t go. It’s easier to deliver a deck than to interrogate it. But over 12--18 months, the pattern becomes clear: the teams that consistently refine their questions, validate their tools, and challenge their assumptions develop a deeper organizational memory. They stop repeating mistakes. They anticipate problems. They become the go-to source for insight, not just reporting.

And because they’ve trained themselves to spot AI’s blind spots--missing data, flawed assumptions, misaligned metrics--they become more valuable, not less. Their role evolves from data cruncher to data sense-maker. That’s not a job at risk. It’s a job in demand.

"Data analytics and data science is a lot more than just like coding or doing analysis in excel. It's a lot more about stakeholder management, applying critical thinking and applying your analytical thinking."

-- Sundas Khalid

The system rewards depth. AI handles breadth. The future belongs to those who can bridge both.

Key Action Items

• Validate every AI output with manual checks -- Spend 10--15 minutes verifying core numbers (e.g., retention drop %, active users) using pivot tables or quick calculations. This pays off immediately in credibility and avoids career-damaging errors.

• Start every project by auditing data quality -- Before prompting AI, ask: Are there missing values? Is the schema consistent? Is this the right dataset? Do this now to prevent downstream garbage-in-garbage-out results.

• Use AI as an intern, not a replacement -- Delegate coding, charting, and SQL drafting, but retain ownership of interpretation, validation, and presentation. This creates a sustainable workflow where speed and accuracy coexist.

• Build leadership decks that anticipate follow-ups -- Include slides that answer likely objections (e.g., “Why was retention high the week before?”). Over the next quarter, this habit will position you as proactive, not reactive.

• Invest saved time in better questions, not early delivery -- When AI cuts your analysis time from 8 hours to 30 minutes, use the remaining 7.5 hours to explore second-order insights. This pays off in 12--18 months as you develop deeper business intuition.

• Only use enterprise-approved AI tools for company data -- Never upload sensitive data to public models. Use internal, secure platforms (e.g., HubSpot’s AI, Google’s Gemini Enterprise). This protects you and the company--non-negotiable from day one.

• Practice with dummy data to build judgment -- Ask AI to generate fake datasets and walk you through analysis. This builds pattern recognition and validation skills without risk. The payoff is faster, more accurate real-world analysis over time.