Reflections on agentic analytics 6-months-in

At the beginning of 2025, we decided to pivot TrueState away from a generic AI platform and double-down on our roots in data science and advanced analytics with an agentic analytics product that blends a capable analytics AI agent with high-performance analytics infrastructure.

A few months in, I had seen enough progress that I wrote a blog post about it, proudly declaring how I thought the next phase of analytics would play out.

Six months is a long time in AI and with that time, we've seen immense progress at TrueState and in the technology we rely on. We've developed the world's first advanced analytics platform designed for AI to operate, as well as "Delegate" our an agentic AI framework that wrangles the best LLMs into reliable, testable agents and assistants.

TrueState has helped customers around the world use AI to automate their most time consuming data analytics work (think weekly reporting) while giving them the tools and expert assistance to dive deep on advanced analytics use-cases (like churn forecasting).

But it's not done so completely in the way I expected. Below are some of my reflections on the (current) state of agentic analytics 6 months since I last wrote about it.

1 - Tools and conventions not roles

In my original post I outlined a vision for agentic analytics where there were agents paired to traditional human roles in analytics. Analyst, data scientist, data engineer. I thought this was going to be the case because the LLMs we used at the time didn't seem to mix roles well and responded well to role-playing.

That might sound ridiculous but such is the nature of "programming" with LLMs. You look for tricks to get LLMs to behave in certain ways, and roleplaying around traditional human roles was, at the time, the way to get the maximum performance for analytics related tasks.

As models have evolved they've been able to handle more complex system instructions and a greater number of tools. For analytics agents, this has meant they've collapsed into a single analytics agent, with access to the full suite of analytics tools and superior performance.

2 - AI-first infrastructure is weird

The tools available to your analytics agents really matter, in a way I had not anticipated before. When designing AI-first infrastructure, an understanding of transformer models is critical for maximising performance.

Understanding that the percentage chance of error increases with the number of tokens generated, you shift to designing APIs to be incredibly high-leverage, but also expressive. Your aim becomes allowing the agent to have maximum control over its environment (in our case the advanced analytics stack) with the minimum number of tokens.

Understanding the importance of the attention mechanism, guiding the agents to inject relevant context ahead of their system instructions (e.g. rationale, conventions followed) leads to consistently better outcomes.

After you've applied principles of tool leverage and context injection, you end up with an effective but bizarre interface to your core infrastructure, as judged by conventional software standards.

Different as it may be, having truly AI-first infrastructure is incredibly important in agentic analytics.

3 - Conventions matter in analytics

Each customer we've partnered with has had some unusual business logic that was needed for some of their analytics. As anyone who's worked in analytics before, this is hardly surprising.

Business specific terminology, calculations and other analytics convetions are critical for reliable analytics at scale.

In retrospect it's shocking that I didn't consider this a key feature. How times change.

We've now integrated conventions as a key capability in TrueState. Our agent both detects potential conventions and respects user defined analytics, resulting in consistent calculation of business critical metrics at scale.

4 - Human analysts are being elevated, not automated

The last six months have shown me that, in analytics at least, AI is a tool for elevation, not elimination of human work. Analysts using TrueState go through a pretty interesting and consistent journey.

They first get their data integrated, then they automate their regular reporting tasks. Before they've had time to have a long winded conversation about the future of their job with HR, they're asking questions about advanced analytics, vibe coding their first machine learning pipelines and looking to make a real difference with their newly elevated analytics skills.

While an awesome testament to TrueState as a product, I think it's an even better testament to analytics professionals. They're both driven and curious people (sometimes to a fault) who got into analytics answer hard questions for their team. They didn't get into analytics to make weekly marketing reports or stale dashboards.

5 - We're just getting started

With the foundations now laid, we're busy working with customers to further refine the TrueState experience. From improved explainability of machine learning models, to the UX of common user flows to pushing the capabilities of our analytics agent further, we're sure the next six months will be even more transformative.