ICONIQ Releases: 2025 AI Builder's Handbook - A Comprehensive Guide to Building, Scaling & Optimizing AI Products

catalogs

1. Introduction

We believe that building and operating AI products is the new frontier of competition, and the voices of the architects, engineers, and product leaders driving this effort deserve more attention. While last year's AI report focused on the buying journey and enterprise adoption dynamics, the 2025 report shifts the focus to the "how": how to conceptualize, deliver, and scale AI-powered products from start to finish.

This year's report delves into the core dimensions of the builder's manual:

1. Product Roadmap and Architecture: Emerging best practices for balancing experimentation, go-live speed, and performance at each stage of model evolution.
2. Market Entry Strategy: How teams can adapt pricing models and market entry strategies to reflect AI's unique value drivers.
3. People and Talent: Assemble the right team to leverage AI expertise, foster cross-functional collaboration, and sustain innovation over time.
4. Cost Management and ROI: Spending strategies and benchmarks related to building and launching AI products.
5. Internal Productivity and Operations: How companies are embedding AI into their daily workflows and the biggest drivers of productivity.

2. Building generative AI products

2.1 AI Product Types

• Proxy workflow and application layeris the most common type of product built by AI-native and AI-enabled companies. Notably, about 80% of AI-native companies are currently building agent workflows.

2.2 Model utilization

• Most companies building AI applications rely on third-party AI APIs; however, high-growth companies are also fine-tuning existing base models and developing proprietary models from scratch.

2.3 Key Considerations in Selecting a Base Model

• product development: When selecting a base model for customer use cases, companies prioritize model accuracy over all other factors.

In last year's AI report, cost was ranked as the lowest key purchase consideration compared to other factors such as performance, security, customizability and control. Notably, cost was much higher in this year's data, which may reflect the commoditization of model layers with the rise of more cost-effective models like DeepSeek.

2.4 Top Model Providers

• OpenAI's GPT modelremains the most widely used model; however, many companies are increasingly adopting a multi-model approach to developing AI products for different use cases.

marginal notes:: (1) Companies that build end-user applications and core AI models/technologies.

Companies are increasingly adopting a multi-model approach to AI product development, leveraging different providers and models based on use cases, performance, cost and customer requirements. This flexibility allows them to optimize for diverse applications such as cybersecurity, sales automation, and customer service, while ensuring cross-regional compliance and a superior user experience. Architectures are being built to support rapid model exchange, with some companies favoring open source models for cost and speed of reasoning advantages.

Typically, most respondents are using OpenAI models and 1-2 other models from other providers.

We use different proprietary and third-party models because our customers have different needs. Proprietary models allow us to better customize the experience for our clients and their use cases - sales automation, customer service agents and internal tools. In addition, we can offer our clients more flexible price points and options, and can constantly experiment with new models and business opportunities.

--VP of Product at a full-stack AI company with over $1 billion in annual revenue

3. Market entry strategies and compliance

3.1 AI Product Roadmap

• For AI-enabled companies, about 20-35% of the product roadmap is focused on AI-driven features, while high-growth companies devote 30-45% of their roadmap to AI-driven features.

3.2 Main pricing models

• Many companies use hybrid pricing models, including subscription/plan-based pricing and usage-based or outcome-based pricing.

3.3 AI feature pricing model

• Currently, most AI-enabled companies either offer AI features as part of a premium product or offer AI features for free.

ICONIQ Cross-Functional Insights: In our State of GTM 2025 report, we asked GTM leaders the same question, and their responses were highly aligned with R&D leaders - further reinforcing the continued consistency of this trend in the marketplace.

3.4 Pricing changes

• 40% companies have no plans to change pricing, but 37% respondents are exploring new pricing models based on consumption, ROI, and usage tiers.

Consider the return on investment::

"We wanted to integrate willingness to pay and a clear link to ROI outcomes into our pricing model."

--VP of Product at a full-stack AI company with $100 million to $150 million in annual revenue

"We're watching to see if AI capabilities bring additional value to our customers. Once we have key adoption rates and proof of value-add, we may segment the current tiers of our platform (i.e., create an advanced tier with full AI/agents, limit the basic tier, and create an enterprise tier)."

--VP of Product at a full-stack AI company with $100 million to $150 million in annual revenue

Consumption and outcome-based pricing::

"We will be supplementing the advanced tier model pricing with a pricing model around consumption. I expect we will also experiment with outcome-based pricing, but it's not clear how we will structure pricing so that clients can accurately budget for those costs."

--VP of Product at a full-stack AI company with $100 million to $150 million in annual revenue

"The subscription model doesn't work for us. Heavy users tend to use a lot, which leads to negative margins given the LLM API costs, and those who don't use risk churn. Given the high variable costs, we plan to move to a usage-based model but bundle usage into a subscription, for example, a 100 million token per year package."

--VP of Product at a full-stack AI company with $100 million to $150 million in annual revenue

3.5 AI Interpretability and Transparency

• As AI offerings expand, it becomes even more critical to provide detailed model transparency reports or basic insights into how AI affects outcomes.

3.6 AI Compliance and Governance

• Most companies have guardrails around AI ethics and governance policies, and most respondents use human oversight to ensure AI is fair and safe.

4. Organizational structure

4.1 Specialized AI/ML leadership

• Many companies have dedicated AI leaders by the time they reach $100 million in revenue, likely due to increased operational complexity and the need to have a centralized owner responsible for AI strategy.

4.2 AI-specific roles

• Most companies now have dedicated AI/ML engineers, data scientists and AI product managers, with AI/ML engineers taking the longest to hire on average.

4.3 Speed of recruitment

• Attitudes toward the speed of hiring were relatively balanced among respondents, with those who felt that hiring was not fast enough primarily citing a lack of qualified candidates as a major limiting factor.

4.4 Proportion of AI engineering team

• On average, companies plan to focus 20-30% of their engineering teams on AI, with high-growth companies having a higher percentage of their engineering teams focused on AI.

5. AI costs

5.1 AI development expenditure

• On average, companies allocate 10-20% of their R&D budgets to AI development, with most planning to increase AI spending in 2025.

5.2 Budget allocation

• Talent costs as a total percentage of spending tend to decline as AI products expand; conversely, infrastructure and computing costs tend to increase as products begin to gain market traction.

5.3 Infrastructure costs

• Of the various infrastructure costs, respondents pointed to API usage fees as the most difficult to control, indicating that companies face the greatest unpredictability in terms of variable costs associated with external API consumption.

5.4 Cost optimization

• To reduce AI infrastructure costs, organizations are exploring open source models and ways to optimize inference efficiency.

5.5 Model Training

• Most respondents trained or fine-tuned their models at least monthly, with estimated monthly model training costs ranging from $160,000 to $1.5 million, depending on product maturity.

5.6 Deployment costs: reasoning

• Reasoning costs spiked after launch, with high-growth AI companies spending 2x more than their peers in the GA and scale-up phases.

5.7 Deployment costs: data storage and processing

• Data storage and processing costs also rise sharply from the GA stage, with high-growth AI builders spending more on data storage and processing than their peers.

6. Internal productivity

6.1 Annual internal productivity budget

• In 2025, internal AI productivity budgets will nearly double across all revenue tiers, with companies spending between 1-81 TP3T of total revenue.

6.2 Internal productivity budget sources

• R&D budgets continue to be the most common source of internal productivity budgets for AI in enterprise companies; however, we're also starting to see headcount budgets being used for internal productivity spending.

6.3 AI access and utilization

• While about 70% of employees have access to a variety of AI tools for internal productivity, only about 50% use AI tools on a consistent basis, and adoption is much more difficult in larger organizations (over $1 billion in annual revenue).

Don Vu (Chief Data & Analytics Officer, New York Life): Just deploying tools is a recipe for disappointment, especially for large organizations. To truly empower employees, you need to pair usability with a support structure that includes training, highlighting champions and, most importantly, relentless senior-level support.

6.4 Key Considerations for AI Tool Adoption

• Cost is the most important consideration when choosing a base model to use for internal use cases, followed by accuracy and privacy.

Whereas accuracy is ranked as the most important factor when deploying external AI products, cost is the most important consideration when choosing an internal AI use case model. Privacy also becomes a more important consideration in internal use cases compared to external.

6.5 Model deployment challenges: internal use cases

• The biggest challenges organizations face when deploying AI for internal use cases tend to be more strategic (i.e., finding the right use cases and proving ROI) than technical.

6.6 Number of use cases

• Companies typically explore multiple GenAI use cases across functions, with companies with high adoption rates using GenAI in seven or more use cases.

6.7 AI adoption

• AI Tools Active Usage is lower than 20% Employees: Average number of use cases = 3.1
• Employees with AI tool active usage of 20-50%: Average number of use cases = 4.9
• AI Tools Active Usage Over 50% Employees: Average number of use cases = 7.1

6.8 Top use cases: by popularity

• R&D and Sales & MarketingUse cases lead in popularity, while General & Administrative use cases still lag behind.

6.9 Top use cases: by impact

• In terms of actual impact on productivity, the top use cases echoed the usage trends, with coding assistance far outstripping the other use cases and having the most significant impact in terms of productivity.

6.10 Attitudes towards internal AI adoption

• High-growth companies tend to experiment and adopt new AI tools more aggressively, suggesting that leading companies see AI as a strategic lever and are integrating it into internal workflows more quickly.

6.11 Tracking return on investment

• Most companies are measuring the productivity and cost savings of internal AI use.

7. AI builder technology stack

7.1 The most commonly used tools: model training and fine-tuning

key point

Frameworks and hosting platforms

Core deep learning frameworks remain popular, with PyTorch and TensorFlow accounting for more than half of all respondents' usage. But they're almost matched by fully hosted or API-driven offerings - the popularity of AWS SageMaker and OpenAI's fine-tuning service suggests that teams are split between a "build your own" and "let other people run it" approach. Let others run it" approach.

Ecosystem players get attention

• The Hugging Face ecosystem and Databricks' Mosaic AI Training are carving out meaningful niches that provide a higher level of abstraction for primitive frameworks.
• Meanwhile, more specialized or emerging tools (AnyScale, Fast.ai, Modal, JAX, Lamini) appear in single-digit percentages, suggesting that experimentation is underway but widespread adoption is still in its infancy.

Enterprise Requirements

• Later-stage companies typically have larger data teams, more complex pipelines, and more stringent requirements for security, governance, and compliance.
• Databricks' "Lakehouse" architecture (which blends data engineering, analytics, and ML) and AnyScale's hosted Ray clusters (which simplify distributed training and hyperparameter tuning) directly address these enterprise needs, with more respondents in the $500M+ annual revenue range using these Solutions.

Most widely used tools

7.2 Most commonly used tools: LLM and AI application development

key point

Orchestration frameworks dominate

• Top frameworks used include LangChain and Hugging Face's toolset, which shows the team's clear emphasis on simplifying cue chains, batch processing, and high-level libraries that interface with public or self-hosted models.
• Approximately 701 TP3T of respondents also noted that they use private or custom LLM APIs.

Security and higher level SDKs gain attention

• About 3/10 respondents use Guardrails to perform security checks, and nearly 1/4 utilize Vercel's AI SDK (23%) for rapid deployment, indicating a growing awareness of the need for guardrails and simplified integration layers for production LLM applications.

the long tail experiment

• The weak use of emerging players such as CrewAI, Modal Labs, Instructor, DSPy, and DotTXT suggests that while experiments are widespread, extensive standardization has not yet been established beyond the major players.

Most widely used tools

7.3 Most commonly used tools: monitoring and observability

key point

Existing infrastructure remains dominant

• Nearly half of the teams rely on their existing APM/logging stack (Datadog, Honeycomb, New Relic, etc.) rather than adopting ML-specific tools - highlighting the fact that ease of integration and organizational standardization often outweigh the benefits of custom AI monitoring.

ML Native Platform Gains Early Attention

• Both LangSmith and Weights & Biases have made breakthroughs, reaching an adoption rate of ~17%, demonstrating that there is a real demand for out-of-the-box solutions that can detect cue chains, track embeddings, and find drift without the need for add-ons to legacy systems.

The long tail of fragmentation and the knowledge gap

• Beyond the first two ML-native names, usage quickly fragmented among players such as Arize, Fiddler, Helicone, Arthur, etc., and 10% of respondents did not know what tools they used; this indicates both the nascent state of the ecosystem and the "observational" nature of generative AI that people are confused by the " concept is confusing.

Most widely used tools

7.4 The most commonly used tool: inference optimization

key point

NVIDIA's control in production

• Together, TensorRT and Triton Inference Server accounted for more than 60% of market share, highlighting the continued dominance of NVIDIA's stack in compressing latency and throughput in GPU deployments.

Cross-platform alternatives gain market share

• ONNX Runtime (18%) is the top-tier non-NVIDIA solution, reflecting the team's need for hardware-independent acceleration across CPUs, GPUs and gas pedals.
• TorchServe (15%) similarly shows that there is still a place for pure PyTorch services, especially for CPU-only workloads or simpler containerized setups.

Knowledge gaps and untapped potential

• The fact that 171 TP3T respondents did not know which optimizations they use and 141 TP3T respondents reported "none" suggests that there is significant confusion or inexperience with inferential tuning, which suggests that there are educational (and tooling) opportunities for quantification, pruning, and efficient runtime - especially for large-scale operational teams. -especially for large-scale operational teams.

Most widely used tools

7.5 Most commonly used tool: model hosting

key point

Direct access from the provider is king

• Most teams have direct access to model hosts through OpenAI, Anthropic, etc., which highlights the fact that resistance resistance is still through calls to the vendor's own inference APIs, rather than building or integrating through a middle tier.

The mega-sizers are close behind

• AWS Bedrock and Google Vertex AI have captured a significant share, reflecting market demand for unified, enterprise-class ML platforms that bundle hosting with governance, security and billing in one pane.
• In particular, more late-stage companies ($500M+ in annual revenue) report using hyperscaler solutions.

Fragmentation alternatives and new players

• Beyond the big three, usage quickly fragmented among players like Fireworks, Modal, Together.ai, AnyScale, Baseten, Replicate, Deep Infra, and others. This long tail indicates that teams are still exploring specialty hosts, often driven by unique pricing, performance SLAs, or feature sets (e.g., custom runtimes, in-field options).

Most widely used tools

7.6 Most commonly used tool: model evaluation

key point

No clear independent leader

• Nearly a quarter of teams primarily use the built-in evaluation functionality of platforms such as Vertex, Weights & Biases, or Galileo, while 20% of respondents simply "don't know" what tools they are using, suggesting that many organizations are still relying on evaluation functionality built into existing ML stacks rather than adopting specialized frameworks. functionality in existing ML stacks, rather than adopting specialized frameworks.

Emerging professional frameworks

• LangSmith and Langfuse lead the ranks of purpose-built evaluation tools, with HumanLoop and Braintrust also showing traction; these platforms have gained market share by offering richer tip-level metrics, customizable test suites, and out-of-the-box drift detection.

Knowledge gaps and DIY

• Almost a quarter of respondents did not know which assessment tool they use, or did not have an assessment tool at all, indicating both confusion about the concept of "assessment" in generative AI and the risk of unmonitored model regression.
• At the same time, some respondents are building their own assessment pipelines, suggesting that off-the-shelf tools do not yet cover all use cases.

Most widely used tools

7.7 Most commonly used tools: data processing and feature engineering

key point

Classic big data tools still dominate

• Apache Spark (441 TP3T of respondents) and Kafka (421 TP3T of respondents) lead the way, highlighting that at scale, teams default to using battle-tested distributed batch processing and streaming frameworks for ETL and real-time data ingestion.

A Strong Foundation for Python

• Despite the heavy big data footprint, 41% respondents continue to rely on Pandas - suggesting that the simplicity and flexibility of in-memory Python tools are still indispensable for smaller datasets, prototyping, or edge cases.

Features stored on the horizon

• Only 171 TP3T people are using specialized feature stores, indicating that while the "build once, use everywhere" feature concept is gaining popularity, most organizations have yet to implement it at scale.
• As maturity grows, we may see feature stores and lightweight coordinators (Dask, Airflow, etc.) rise in the rankings - but for now, the Apache ecosystem dominates.

Most widely used tools

7.8 Most commonly used tools: vector databases

key point

Search Engines Evolve into Vector Storage

• Elastic and Pinecone led the adoption, reflecting teams either adapting existing full-text search platforms for embedding or adopting specialized hosted vector engines.

Redis and the Long Tail

• Redis shows the appeal of leveraging the in-memory data store you already run, while other solutions such as Clickhouse, AlloyDB, Milvus, PGVector, and others show that many organizations are experimenting with different backends to balance cost, latency, and functionality needs.

The rise of open source solutions

• Specialized open-source tools, such as vector plug-ins from Chroma, Weaviate, Faiss, Qdrant, and Supabase, are cannibalizing the early leaders, suggesting a competitive battleground for ease of use, scalability, and cloud-native integration.

Most widely used tools

7.9 Most commonly used tools: synthetic data and data enhancement

key point

In-house production dominates

• More than half of the teams (52%) build their own tools, suggesting that off-the-shelf providers still struggle to cover every use case or integrate with existing pipelines.

Scale AI is the clear vendor leader

• With an adoption rate of 21%, Scale AI is the preferred third-party synthetic data platform - but even it is only adopted by 1/5 of organizations.

Early attraction of the program framework

• Snorkel AI and Mostly AI show that program markup and generation tools are gaining market share, but still lag far behind custom solutions.

Most widely used tools

7.10 Most commonly used tools: coding aids

key point

Pioneer dominance

• GitHub Copilot Used by nearly three-quarters of development teams, thanks to its tight integration with VS Code, multi-language support, and the backing of GitHub's large user base, Copilot's network effect and product-market adaptability make it difficult to displace, but the strong performance of Cursor (used by 50% respondents) suggests that there is a demand for a diverse range of IDE integrations.

Long-tail products lagging behind

• After the top two, adoption rates dropped dramatically and long-tail solutions fragmented, suggesting that while most teams have tried at least one helper, few have standardized on alternatives. Low-code or no-code solutions such as Retool, Lovable, Bolt and Replit also received honorable mentions, demonstrating the market's growing interest in idea-to-application solutions.

Most widely used tools

7.11 Most used tools: DevOps and MLOps

key point

MLflow leads - but doesn't have a monopoly

• MLflow was used by 36% of respondents and was the clear frontrunner for experiment tracking, model registries, and basic pipeline orchestration - this was only slightly more than one-third of teams, suggesting that there is still plenty of room for alternatives.
• Weights & Biases also has a strong market share, used by 20% of respondents, reflecting its appeal as a hosted SaaS for tracking, visualization and collaboration.
• Beyond the top two, usage is quickly fragmenting - 16% "doesn't know" which tools support their MLOps and other tools mentioned include Resolve.ai, Cleric, PlayerZero, Braintrust, etc. This shows both confusion about responsibilities (DevOps vs. MLOps) and that the market is still sorting itself out.

Gap between tracking and full-scale operations

• The dominance of trace-first platforms like MLflow and W&B suggests that many teams have yet to adopt an end-to-end suite of MLOps - continuous delivery, drift monitoring, or automated rollback are still works in progress for most.

Most widely used tools

7.12 Most used tools: product and design

key point

Figma's near-universal influence

• With an adoption rate of 87%, Figma has actually become the established standard for UI/UX and product design - teams are overwhelmingly sticking to their real-time collaboration, component libraries, and plugin ecosystems rather than looking to AI-specific design tools.

Miro for higher level collaboration

• With an adoption rate of 37%, Miro remains the first choice for wireframes, user journey mapping, and cross-functional brainstorming; its whiteboard-style interface complements Figma's pixel-level canvas, especially in the early ideation phase.

The rise of AI-enabled product line block diagrams

• Design teams are not yet feeling the urgent need for AI-native product/design platforms, however, many are using low/no-code solutions for rapid prototyping such as Bolt, Lovable, and Vercel V0.

Most widely used tools

8. Internal productivity use cases (part I)

8.1 Use cases

Key trends

For more information on specific tools in each category, contact ICONIQ Insights.

Sales productivity

• Many teams get their AI-driven sales capabilities directly from Salesforce - suggesting that an easy path is to rely on your existing CRM's built-in recommendations, forecasting, and opportunity scoring, rather than attaching a separate service.
• Other respondents are also using sales engagement platforms like Apollo, Salesloft, and Gong, while others are turning to AI-powered lead generation tools like Clay and People.ai.
• As embedded functionality matures, we may see clearer differentiation around a few platforms or from point solution newcomers.

Marketing Automation and Content Generation

• Marketers have turned almost exclusively to Canva's generative capabilities for brand visualization and rapid content iteration, making it by far the most common "AI" touchpoint in the marketing stack.
• Many respondents are also using n8n or on-premise solutions, suggesting that marketing use cases sometimes require a high degree of in-house customization.
• Many respondents are also using specialized AI writing tools, such as Writer and Jasper, with adoption rates higher for later-stage companies (over $1 billion in annual revenue).

Customer Engagement

• Teams rely almost exclusively on Zendesk or Salesforce's built-in AI capabilities for customer interactions, demonstrating that it's still better to rely on the platforms you're already using rather than adopting a standalone conversational AI platform.
• When they need deeper bot customization, self-service wizards, or tightly built-in support widgets, a large minority relies on specialized tools such as Pylon, Forethought, Grano.la, or Intercom - showing that when off-the-shelf AI isn't good enough, boutique tools still have a place.

Documentation and knowledge retrieval

• Most teams either build on existing wiki and note-taking tools or standardize on Notion; this suggests that organizations often default to using knowledge capture tools that already exist before attempting AI-driven overlays.
• However, a large proportion of respondents are also turning to specialized AI tools such as Glean and Writer for indexing and semantic search.

9. Internal productivity use cases (part II)

9.1 Use cases

Key trends

For more information on specific tools in each category, contact ICONIQ Insights.

IT and Security

• ServiceNow (used by 33% respondents) and Snyk (used by 30% respondents) lead the way, suggesting that large organizations are still defaulting to their existing ITSM and security scanning platforms rather than building entirely new AI tools.
• Zapier and Workato were also frequently mentioned, highlighting the team's strong focus on low-code orchestration for combining alerts, ticket creation, and remediation scripts across different tools.

legislation

• The legal sector is mainly through ChatGPT and temporary scripts dabble in AI, but specialized legal assistant platforms are starting to gain traction.
• As regulations and safety concerns increase, we may see a divergence: mainstream LLM for informal research and compliance-centric suites for mission-critical contract workflows.

Human resources and recruitment

• Nearly half of teams rely on LinkedIn's built-in AI features-profile suggestions, candidate matching, and promotion sorting-indicating that recruiters are relying on a platform they already use every day, rather than integrating standalone solutions. However, niche platforms like HireVue (for AI-driven video interviewing) and Mercor (for candidate engagement) are also starting to see moderate adoption.

FP&A Automation

• Many teams use Ramp for FP&A automation, potentially leveraging its spend management and data synchronization capabilities in an all-in-one platform.
• Specialized suites such as Pigment, Basis, and Tabs are also starting to gain traction, indicating a growing interest in driver-based planning and multi-scenario modeling platforms.
• About one-third of respondents are also using on-premise solutions, reflecting an investment in custom scripts, Excel macros, and custom pipelines to bring ERP, billing, and BI systems together.

10. Conclusion

Technology is important, strategy is important, but the most important thing is people.

 

Original: https://www.iconiqcapital.com/growth/reports/2025-state-of-ai

Report Download: https://cdn.prod.website-files.com/65d0d38fc4ec8ce8a8921654/685ac42fd2ed80e09b44e889_ICONIQAnalytics_Insights_The_AI_ Builders_Playbook_2025.pdf