As generative AI becomes increasingly integral to building and running software products, managing the associated costs is essential. Everyone thinks about inference costs to simply serve the product, but what about the additional costs to experiment, test, and evaluate quality?
It needs to be easier to know and control what you’re spending across the full lifecycle of operating AI products, including across models and providers.
That’s why Freeplay is excited to introduce our new Usage Dashboard and Spend Management features.
See The True Cost of AI Products
To manage costs effectively, product development teams and leadership need a shared understanding of costs and a comprehensive view of spend across their AI investments.
Freeplay's new features address this complexity head-on, offering a level of granularity and control that goes beyond what any LLM providers offer. Specifically, new Usage dashboards make plain what you’re spending on each prompt, feature, or project.
You can also easily see where the spend is coming from including:
Your core application inference from the completion logs you send Freeplay via API or SDK
Auto-evals that run on those logs (aka “Live evals”)
Batch tests, including any auto-evals that run to score those tests
Playground experimentation
These costs are now visible in two places in Freeplay:
An Organization-level Usage Dashboard, which looks at costs across Projects, so leaders can easily see total spend and where it’s coming from. (Under Settings > Usage)
A Project-level Usage Dashboard, which shows costs within a given Project, so team members responsible for a project can easily keep track of the spend they’re responsible for. (Under Project > Usage)
In all cases, spend is aggregated across models and providers (e.g. Google Vertex, OpenAI, Anthropic, etc.) so you can quickly see the total regardless of which models you use. It’s also rolled up across multiple environments (dev, staging, prod, etc.) so even if you use different API keys for core app inference or testing in each environment, you can see the total spend in one place.
A picture of the new Project Usage dashboard is above. Below is the new org-level dashboard showing a rollup across Projects. You can find this view under Settings > Usage.
Spend Limits
Knowing what you’re spending is great, but you also need ways to control it. This problem has been complicated until now because spend happens across API keys, projects/features, and models & providers.
The new Spend Limit settings make it easy to control any LLM spend happening via Freeplay, giving you fine-grained control over your LLMOps infrastructure costs. This applies to:
Any auto-evals run by Freeplay
Any Tests that are executed via the Freeplay UI
Any use of the Freeplay Playground or auto-eval alignment features
Importantly: Spend limits in Freeplay will never apply to your use of your LLM API keys in your code. Any completions that run in your core app code and any tests you initiate using the Freeplay API or SDKs will NOT be impacted.
You can set spending caps at both the organization and individual project levels. Once these are set, Freeplay will pause your ability to continue using LLMs via the Freeplay application until your billing period resets or the spend limit increases.
Note that only users with “Admin” permissions can make these changes. Find them under Settings > Spend Limits.
The Power of Informed Decision-Making
By providing this level of detail and control, Freeplay empowers product teams to make data-driven decisions about their AI investments. With these tools, you can:
Identify cost-intensive areas of your AI operations and optimize accordingly
Better explain AI expenditures to leadership and finance teams with concrete data
Allocate resources more effectively across different projects and initiatives
Forecast future AI costs with greater accuracy, facilitating better budgeting and planning
Do these insights and controls sound helpful?
Freeplay customers can start using them today. If you’re not using Freeplay, get access today or reach out to learn more about how we can help your team build better generative AI products.
