Research is Short on Judgment, Not Tools: The Case for a Federation of Insights

This blog is adapted from the CEO of Knit, Aneesh Dhawan’s talk at the Terry College of Business advisory board meeting at the University of Georgia.

‍

Every company says it wants the customer at the center of the business. Most don’t get there, and the usual excuse is that research takes too much time and costs too much money.

That excuse used to be understandable: research with a traditional agency can take weeks to months and cost hundreds of thousands of dollars (one company told us they spent six to eight weeks and $100,000 on a single packaging decision). Organizations make most decisions without customer feedback if that’s its price — 80% of them, by our estimates. 

AI, however, is shaking up the industry. AI-moderated interviews, automated thematic coding and analysis, and faster questionnaire and deliverable generation make it possible to run research at a scale and speed that wasn’t realistic before. AI removes old time and cost constraints, enabling non-researchers to run studies themselves. This results in an unprecedented level of access. While this may sound positive on the surface, it creates a new problem: what happens when everyone can run research?

‍

Access Without Expertise
‍

We call this problem vibe research: studies run without the proper methodology and rigor only trained researchers can bring. Vibe research leads to research slop, including leading questions that sway results, misread data, and generic answers that sound right but are wrong.

This slop can be dangerous. According to GWI, 80% of US marketers say their organization has made a business decision based on incorrect AI “insights.”

Knit’s original AI Trust Index corroborates this finding:
‍

• Researchers are 2.1× more likely to worry about non-researchers using AI without methodology or oversight (45%) than about AI replacing their own role (21%).
• 92% say at least one non-researcher AI risk scenario is already happening at their organization.
• 47% say AI-generated insights are already reaching senior leaders with no researcher review.

As such, the industry is dealing with a shadow analytics issue. Company leaders and other stakeholders make decisions on AI-generated insights no expert ever touched (90% of AI outputs require editing or major rework before they’re usable anyway). While it may sound good on the surface, this kind of research democratization doesn’t integrate customer input the way organizations hope. 

‍

‍

A Federation of Insights
‍

The answer isn’t to totally lock down or grant full research access. We support a different solution: a federation of insights. The dictionary definition of federation involves a group in which smaller divisions keep some degree of autonomy. So, while democratization means everyone has unsupervised access to AI-powered research, a federation means everyone has access, but an expert owns the standards.

It works in three layers:
‍

1. Expert researchers set the foundation: the standards, the best practices, and a deep understanding of the company’s customers.
2. Marketers and product leaders run research on that foundation, with AI agents doing the heavy lifting and vendors like Knit working as an extension of their team.
3. The right people get access, and experts stay accountable. The standards hold because someone owns them.

We see this structure working for one of our customers, T-Mobile. The broader team can interact with voice-of-the-consumer data, but that data is only trustworthy because the insights team governs the foundation itself. 

‍

Democratization means everyone has unsupervised access to AI-powered research; a federation means everyone has access, but an expert owns the standards.

‍

From Line Cook to Head Chef
‍

We like to think of research as a kitchen. Historically, researchers were line cooks, executing tasks like fielding, cleaning, coding, and first-pass decks. Now that AI takes over the line prep,  researchers are elevated to head chefs and are responsible for writing the recipe, checking the ingredients, and tasting everything before it’s served. 

The head chef is accountable for three things:
‍

• Governance: The standards, design, and integrity of how research is conducted.
• Contextual judgment: Outcomes that reflect compounding knowledge of how the business and its customers think and act.
• Transparency: Every insight traces back to the customers and data behind it.

Researchers are already asking for exactly this. In the AI Trust Index, the top wish in respondents’ open-ended responses (19%) was for AI that understands their specific business context, followed by traceability and transparency. 94% won’t share raw AI outputs with executives. None of the top trust drivers are about speed or features, but about credibility: context, transparency, traceability, and rigor.

‍

The Payoff
‍

When research gets dramatically faster and more affordable, companies don’t do the same amount for less; they conduct much more (a classic case of Jevons’ Paradox). On average, research with Knit is 10x more cost-effective and 10x faster than with traditional agencies, and customers tell us about half of the work they do with us is net-new research that wouldn’t have happened under the old model.

This new model opens up new possibilities, including:
‍

• Current research, faster. Returning to our T-Mobile example, when leadership asked a critical question on a Friday, our team was in the field that night and delivered a full, senior-leadership-grade report by Monday. The speed comes from accumulated context and a durable understanding of how T-Mobile wants findings presented.
  
  
• Research that wasn’t possible before. After T-Mobile assembled the largest data collection in its history (one enterprise-wide segmentation, nine customer clusters), the challenge was activating it across hundreds of teams. Synthetic personas let teams query those segments in natural language, and they stay grounded because ongoing primary research feeds back into them.
  ‍

Both are built on the same federation principles. We build research best practices into our agents (so research stays governed), customers’ Research Context Libraries carry context from project to project, and our agents cite their sources to ensure everything stays transparent. 

A major question in the industry is whether or not AI is replacing researchers. Our answer is a resounding no. It’s replacing phases of projects, such as compressing discovery, but the savings get reinvested in more rigorous execution downstream.

‍

What This Means for the Next Generation
‍

If the line cook role in research is moot, the industry needs to adjust how it trains newcomers. 

The good news is that the job is getting better, not smaller. 96% of professionals in the AI Trust Index see AI as a tool, not a threat, when they direct it. 94% say it makes them a better researcher. The hours saved on transcription and crosstabs go to the parts of research that are most interesting and hardest to automate: methodology, interpretation, recommendations, and storytelling.

Students entering the field need four skills: 
‍

• The why behind the methods. When anyone can generate a questionnaire in 30 seconds, knowing what a good one looks like is rarer and more critical.
• Soft skills. Working cross-functionally, collaborating with stakeholders, and getting to business objectives.
• Business context. A head chef’s compounding knowledge of the business is essential. Students need to learn industries deeply, not just tools. 
• Orchestration. The ability to direct AI agents, not just use them, including writing the recipes and managing the line prep. 

A federation runs on experts who stay accountable, which means it needs a steady supply of them. The industry’s problem won’t be a shortage of research tools; it will be a shortage of people qualified to hold them accountable.

‍

The Industry’s Future
‍

Today, fewer than 20% of decisions at the world’s biggest brands are informed by customers, but the tools necessary to change that already exist. What the industry lacks is judgment. The companies that win will build a federation of insights, where experts set the standards and deploy both humans and AI agents to deliver the highest-quality, decision-ready insights.

That’s how we get to a future where customers inform every business decision with answers leaders can trust. 

‍

{{cta-1}}