Customer Story
How Upstart Accelerates Innovation with GrowthBook
Upstart (Nasdaq: UPST) is the leading AI-lending marketplace, connecting millions of consumers to more than 100 banks and credit unions that leverage Upstart’s AI models and cloud applications to deliver superior credit products. Their engineering team is committed to expanding access to credit through continuous experimentation and innovation.
Upstart
Faster Experimentation
Launch in hours, not days
Tool Consolidation
From 3 platforms to 1
Resource Optimization
Significant cost savings
Ready to get started?
The Challenge: Multiple Tools and High Costs Slowing Innovation
Upstart’s experimentation and feature-flagging process had become complex and somewhat manual. The company was using multiple tools, including a third-party solution for feature flagging and two separate in-house solutions for experimentation, which created bottlenecks and slowed decision-making.
"We had to be familiar with these different tools and rely too heavily on our analytics team for
data analysis rather than having software that could handle some of that for us,” said Diego Accame, who leads the engineering teams supporting Upstart's Growth vertical. “It was difficult not having experiment results available at all times, and it was slowing us down when rapid iteration was critical."
The delays had an additional impact as well: the potential for suboptimal results from the experiments that were in progress. Diego’s team decided it was time to look for a solution.
The Search for a Unified Solution
Recognizing the need for a flexible, efficient platform, Upstart explored several alternatives:
- Investing in and expanding their in-house solutions
- Evaluating three paid third-party tools for feature flagging + experimentation to fully cover their needs
- Evaluating two free open-source solutions, which would require additional investment to meet their needs
These alternatives did not fully address Upstart’s key requirements:
- Support for their diverse tech stacks: Flexibility was crucial given that different teams used different programming languages.
- Strong data privacy and security: As a financial services company, Upstart needed to securely host sensitive data in-house.
- Cost-effectiveness: The solution needed to be affordable given Upstart’s current size and needs, and as it scaled its operations
- Ease of use and autonomy: Engineers needed a tool that would allow them to design, launch, and analyze experiments according to their own specifications.
The GrowthBook ROI
Upstart had considered creating its own platform, but that would have cost about the same as a team of four engineers in the first year and at least two engineers every year thereafter. Also, it’s not a core expertise of the company. That’s why GrowthBook emerged as the clear solution because it eliminated the need to build an experimentation framework while dramatically reducing delays in getting results.
GrowthBook’s open-source, cost-effective model provided a ready-to-use solution that saved Upstart significant resources from day one. The platform empowered Upstart’s engineers to self-serve and, with real-time access to data, teams could make faster decisions, enhancing their ability to innovate at speed.
Furthermore, GrowthBook offered the ability to host all data in-house, ensuring full control over sensitive information, which was critical for Upstart’s compliance and security requirements.
GrowthBook’s open-source, cost-effective model provided a ready-to-use solution that saved Upstart significant resources from day one. The platform empowered Upstart’s engineers to self-serve and, with real-time access to data, teams could make faster decisions, enhancing their ability to innovate at speed.
Furthermore, GrowthBook offered the ability to host all data in-house, ensuring full control over sensitive information, which was critical for Upstart’s compliance and security requirements.
"With the kinds of experiments we run and the sensitive data we handle, data security is paramount. The fact that GrowthBook offered us the ability to keep that data in-house was a key reason why we chose to work with them," Accame said.
Upstart’s decision to adopt GrowthBook was further reinforced by a successful proof of concept during a hack week, where the platform’s UI and capabilities were put to the test by a small team.
“Our strength is as an AI-powered lending marketplace, not an experimentation framework company. GrowthBook lets us focus our resources where they matter most
— on growing our core business.”
— on growing our core business.”
DIEGO ACCAME, Director of Engineering, Growth, Upstart
The Migration Process: Updating Complex Systems
Though GrowthBook is the right solution, migrating isn’t without its challenges. Upstart has to consolidate three existing flagging and experimentation platforms into one new solution. This means cleaning up a large number of feature flags and experiments that use their old systems, educating and onboarding GrowthBook to all teams with their codebases, and finally deprecating and removing the old tooling after the cleanup was complete. Using one tool instead of three is also easier for maintenance while reducing the cognitive load for all engineering teams writing experiments. The full migration is expected to be complete early in 2025.
Accame explained, "We had to rally our teams for this project and get our old tools cleaned up, but the process is significantly improving the quality of our systems and reinforcing best practices around experiments and feature-flagging hygiene."
Results and Benefits of Adopting GrowthBook
Since adopting GrowthBook, Upstart has seen multiple key improvements:
- Consistency in Metrics: GrowthBook provided preloaded, standardized metrics that resolved the inconsistencies sometimes caused by human error.
- Faster Decision-Making: Engineers are able to assess experiment performance in real time, allowing for immediate, data-driven actions. This newfound autonomy has led to quicker identification of underperforming experiments and faster iterations. "We've been able to see underperforming experiments quickly and react immediately, empowering our engineers to make decisions independently,” Accame said.
- Improved Collaboration: Upstart’s experiment dashboards are easily accessible and shareable, promoting transparency and encouraging cross-team discussions.
- Cost Efficiency: GrowthBook’s open-source model provided a budget-friendly alternative to other solutions, helping Upstart stay within their financial targets while improving their experimentation processes.
- Reduced Technical Debt: As mentioned above, the migration to GrowthBook led to the cleanup of old feature flags and experiments, improving overall system hygiene and maintainability.
Looking Ahead: Future Growth with GrowthBook
Upstart is excited about several upcoming features in GrowthBook’s roadmap, including:
- Sticky Bucketing Support: This feature will be critical for Upstart’s experimentation processes and is already in development.
- Improved Support for Ephemeral Environments: Upstart’s workflow relies heavily on ephemeral environments, and they are eager to see better integration with GrowthBook.
- Feature Flag and Experiment Hygiene Tools: GrowthBook’s ongoing focus on improving experiment management aligns perfectly with Upstart’s need to maintain clean and scalable systems.
Conclusion
Upstart's journey with GrowthBook demonstrates that the right tool doesn’t just solve technical problems—it transforms how teams work and innovate. By switching from a fragmented system that included third-party and in-house tools to GrowthBook, Upstart saved time and resources while improving their ability to make data-driven decisions. Upstart is now able to launch experiments in hours when it had taken days in the past.
GrowthBook has empowered Upstart's engineering teams to innovate faster, reduce dependency on manual analytics processes, and ensure data privacy with its on-premises solution.
GrowthBook has empowered Upstart's engineering teams to innovate faster, reduce dependency on manual analytics processes, and ensure data privacy with its on-premises solution.
"GrowthBook has changed the way we think about experiments,” Accame said. “It allowed us to uplevel our code, speed up decision-making, and focus on what we do best—building a world-class AI lending marketplace."
Enjoy unlimited experiments for unlimited traffic. All for free.
No credit card required