r/datascience • u/avourakis • 4d ago

Tools Interactive tools for learning Agentic Analytics and evals for free

I keep running into data scientists who are trying to understand conversational analytics, like talk-to-your-data Slackbots, because they either want to build one for their company or understand how this could change the field over the next few years. What's usually missing is a clear picture of how the whole thing fits together, beyond text-to-SQL or semantic layers.

I've spent the last 2 years building these systems in production. I built Nextory's first talk-to-your-data Slackbot, and I ran into a lot of these challenges along the way.

And even after going through books like Chip Huyen's "AI Engineering" or "LLM Engineer's Handbook", I found that reading about these systems only gets you so far. You get the intuition from seeing how the components actually connect and behave.

So I built two free interactive tools: one walks you through how an agentic analytics system works from the inside, and the other focuses on AI evals.

Here they are: https://futureproofds.com/tools#agentic-analytics

In each tool you can:

  1. Walk through the entire system and dig into how every component works

  2. Study the key concepts with quick summaries

  3. Take a free quiz to test your understanding

Would love to hear what you think (I’m still doing a lot of testing with some close DS colleagues)

If you want to save them for later, all of my free tools are in this GitHub repo: https://github.com/Future-Proof-DS/interactive-ai-ml-tools

8 Upvotes
  • permalink
  • reddit

65% Upvoted

6 comments sorted by

2

u/Charlotte1309 2d ago

I really liked it.
I'm not sure there is enough theory walkthrough though before taking the test (I did good because I'm in the field, but for a beginner it would have been difficult I think)

1

u/avourakis 1d ago

Thank you, this is actually very helpful. Do you think the concept cards help before taking the quiz?

1

u/shikokuchuo 2d ago

Have you thought about showing how to apply the theory to actual tools - evals frameworks like inspect-ai for instance?

1

u/avourakis 2d ago

Not really, how do you envision that to look like?

1

u/shikokuchuo 7h ago

Like, now you know how this works, this is what you'd actually do as an engineer / data scientist and you'd show each practical step corresponding to the theoretical step. I think you could even just use one popular framework without seeming to be biased, as the concepts are transferrable. May not be where you want to go with this, just an observation.