Instant Search & Visualizations For All Your LLM Logs

For teams building AI products, the ability to quickly understand what's happening in your systems is critical to success. That's why we've made major updates to Freeplay's observability dashboard to more quickly find relevant data and visualize trends in data, and then take immediate action to improve — all in one place, without the need to bounce between tools.

Below is a quick demo video of what the new changes look like. Read on for more detail!

Why Fast Log Search Matters for AI Teams

Production AI systems generate enormous amounts of data — both in terms of the volume of logs, and the size of the payloads from large prompts/context windows and large LLM outputs. The insights needed to improve your AI products can be hidden within those logs, and speeding up your product development cycle depends on finding issues and being able to quickly fix them.

AI engineers, product managers, and data experts need to constantly filter and search log data for a variety of reasons, for example:

• Identifying sessions with negative customer feedback to understand friction points

• Locating production examples where automated evaluations failed for a specific prompt, or even specific versions of a prompt

• Finding completions containing specific phrases, either in the inputs or LLM outputs

• Detecting high-cost or high-latency completions that impact performance

• Filtering by contextual metadata like customer ID, model used, or environment

Many teams resort to manually writing complex queries or stitching together multiple tools to complete these tasks, creating bottlenecks that delay improvements. Freeplay's observability dashboard solves these challenges by putting powerful visualization and filtering capabilities at everyone's fingertips—regardless of their technical background.

Lightning-Fast Filtering for Precision Data Analysis

Behind Freeplay's visualization capabilities is a high-performance search infrastructure that delivers exceptional speed at scale. This enables:

• Complex multi-condition filters: Combine filters across prompts, evaluations, metadata, cost, and content to find exactly what you need

• Near real-time indexing: New completions become searchable within seconds

• Enterprise-grade performance: Get sub-second query response times even when filtering through millions of completions

Creating complex queries is straightforward through the Freeplay interface. For example, to review all completions where your assistant mentions a specific phrase and that have been evaluated with a certain score:

Once created, these filters can also be saved and shared with your team, creating persistent monitoring views that help everyone stay on the same page:

Powerful Visualization Tools for Better Insights

After you search and filter to find the data you want, you still need to make sense of it and rarely have time to review it all. Our updated visualization tools help make patterns immediately apparent.

• Rich, interactive visualizations: A range of chart types make it simple to spot trends, anomalies, and patterns across prompts and evaluations

• Flexible time views: Toggle between daily and weekly views to balance granular debugging with identifying broader trends

• Shareable, screenshot-ready graphs: Generate clean visualizations you can instantly share with stakeholders or include in reports

We've also made it easier to make use of these visualizations with:

• Searchable metric library: Quickly find relevant charts by searching for prompt names, eval types, or specific metric names — even when you have hundreds in a given project

• Bookmark your favorites: Pin frequently-viewed graphs to the top of your list for easy access

• Interactive elements: Deselect graph elements from legends to focus on specific data subsets, hover for detailed metrics, and zoom into date ranges of interest

Closing the Loop: From Insights to Action

Seeing your data is rarely enough. Generally the goal is to learn and improve. Freeplay makes it easy to integrate your log data with your broader AI development workflow and close the feedback loop, getting you to a high-quality product faster:

• Identify issues using graphs and advanced filters

• Save custom filters as persistent monitoring dashboards

• Select relevant completions in bulk

• Add production examples to Review Queues for human evaluation and labeling

• Curate relevant examples into Datasets for testing improvements or fine-tuning

• Run experiments to improve your product quality

Getting Started

All these features are all available now in Freeplay. To get started:

1. Navigate to your project dashboard

2. Explore the refreshed graphs

3. Click the filter icon to create your first advanced filter

4. Try combining filters and saving your most useful views

Under the hood, Freeplay's search and visualization capabilities are built on well-established open-source technologies, designed for reliability, performance, and familiar maintenance. For customers with stringent data privacy requirements, it's straightforward self-host in your own VPC and comply with existing enterprise security models.

New to Freeplay? Get in touch to see a demo and learn more how we can help.