Investor POV

Reimagination of everything: How intelligence-first design and the Next Stack will unlock human+AI collaborative reasoning

Ganesh Bell, Sophie Beshar, Ash Tutika, Michael Spiro | November 08, 2024| 13 min. read

We stand at the intersection of possibility and uncertainty. AI has been hailed as the ‘new electricity’ and the ‘biggest architectural shift’ — yet in the same breath, it’s often reduced to ‘easy to incorporate and just an API call.’ This tension between bold promises and growing skepticism has created a moment ripe for reimagination.

As observers and students of this rapidly shifting landscape, we see something bigger emerging — a new stack and a design paradigm that moves beyond the generative AI hype. We call this intelligence-first design, a shift that positions AI within applications as a true collaborator, capable of amplifying human reasoning.

The real transformation won’t come from layering incremental product updates; it will come from fundamentally redesigning applications and systems with intelligence and AI as a first-class citizen.

A multi-trillion-dollar opportunity to reimagine software, services, and work

GenAI is not only transforming traditional enterprise software but also expanding the boundaries of the addressable market for software. In the early 1990s, business software was still a cottage industry. It has since grown into a $650B sector in 2024, driven by various architectural transitions — from mainframes to client/server systems, and then the cloud and mobile era.

During the cloud and mobile era of the 2010s, new applications like Airbnb and Uber redefined their respective industries of lodging and transportation, setting new precedents for how software could reshape the world. Despite these shifts, the core designs and “legacy applications,” which were built for an era of manual data entry, persist today.

GenAI is driving a fundamental shift in knowledge management, software development, and productivity, creating hybrid teams where AI assistants, copilots, and AI workers collaborate with humans. We believe this shift will extend even beyond traditional software to include services and redefine the very nature of work itself.

Although the last decade has accelerated the digital transformation of industries, the rewiring of business through software and data is still in its early stages. Many industries remain at varying levels of digitization, with significant sectors still underserved or untouched by modern software solutions. Now, with the rise of AI, driven by advancements in both hardware and software architectures, we are witnessing the birth of new systems that are helping automate and unlock more human reasoning.

It feels like the perfect time to reimagine everything: not just revisiting what we’ve built before, but also envisioning entirely new possibilities for what we can create moving forward. Service-based businesses can now be built with technology or intelligence-first approaches, and work can be enhanced through automation, reasoning, and co-intelligence. It is a pivotal moment in the brief history of (business) software. The pie has just grown dramatically for software/AI entrepreneurs and digitally skilled incumbents alike.

addressable market software ai

The Next Stack: Foundational infrastructure for reimagination is here

We are witnessing the fast sedimentation of the Next Stack, which we discussed back in March 2023 in our original post on the impact of genAI. At that time, we outlined how AI was laying the foundation for a new technological stack, but today, its evolution is even more evident. Foundational models have since gone multi-modal, becoming larger, smaller, open, composable, and increasingly agentic, with deliberate thinking, planning, and reasoning abilities.

In past architectural eras, “data stacks” and “application development stacks” often lived separate lives. However, we’re now seeing the modern data stack evolving to support AI with a focus on unstructured data orchestration and multimodal capabilities.

Databricks is at the forefront of this with its Mosaic AI platform, which enhances its data lakehouse to handle AI workloads and unstructured data more effectively. Vector databases like Pinecone and Weaviate enable storage and retrieval of ‘vectorized’ data, while pre-existing databases such as SingleStore and MongoDB, already holding vast amounts of company data, have added strong vector search capabilities in recent months. Companies like LlamaIndex, Unstructured, and Superlinked are working to ensure relevant data and context can be retrieved by LLMs, agents, and other AI systems at inference. Platforms like Atlan, Astronomer, Patronus, and Braintrust, on the other hand, are working to embed lineage, data governance, orchestration, and model evaluation into AI development workflows.

AI development frameworks have exploded, from multi-modal programming abstractions to advancements in chaining, RAG, reinforcement learning, and the latest breakthroughs in agents. Our colleagues have offered a detailed take on the disruption of agents, available here. Additionally, we’ve seen the evolution of ML Ops to LLM Ops, with growing maturity in tools that can ensure safe deployment, observability, and alignment. Companies like CrewAI, Gradient, Contextual, LangChain, Fireworks.ai, and Unstructured are seeing strong developer adoption and are delivering significant ROI to customers.

As we reflect on the pace of these changes and the rich ecosystem, one thing is certain: While the Next Stack is seeing hyper-innovation and fast sedimentation, we would argue the infrastructure and tools needed to reimagine and build for the future have already started to arrive.

Emerging design patterns of intelligence-first applications

Among the significant advances this year are OpenAI’s o1 ability to execute long internal chains of thought for complex reasoning, the rise of agentic architectures, and Google’s ‘mini ChatGPT moment’ with NotebookLM.

Despite fears around ‘no moat in AI’ and criticisms that ‘all apps are just thin wrappers on models,’ many intelligence-first application builders have been quietly busy, creating. It turns out that building useful, meaningful, and complex systems that humans can interact with is difficult — and requires real craft.

These intelligence-first apps seem to share a set of core design patterns.

Intelligence-first design patterns

New knowledge representation and memory systems

Storing data in third normal form may have been optimal for simple database apps, but next-gen applications will need dynamic, richer data architecture systems that can update and retrieve knowledge contextually across structured, semi-structured, unstructured, time-series, feature stores, graphs, and real-time data.

Data can now be stored in superstructures, maintaining relationships across data types such as a knowledge graph and without rigid structures through a schema-on-read approach. This also allows for flexibility in how the data is consumed and transformed based on AI models’ needs vs human retrieval needs. Additionally, these systems must enable continuous learning, where AI co-workers improve over time by adapting to new information, refining their memory, and building on past experiences.

RAG as new query

Why just query a customer record when the intent or purpose for the query can make it as important to retrieve all related knowledge? Traditional queries are not sufficient in systems with new knowledge representation.

RAG enhances the query process by breaking down questions, retrieving real-time data from diverse sources like relational, graph, and vector databases, and refining the results using methods like RAG-fusion and re-ranking. This enables AI systems to deliver richer, context-aware, augmented responses from a variety of data formats.

Companies like Glean and Harvey use RAG workflows to optimize queries, making this approach essential for apps that rely on dynamic, real-time insights.

Networked reasoning and cognition engines

For decades, applications have relied on rules engines, but now, we can build them with native cognition and reasoning capabilities. In intelligence-first applications, reasoning will be distributed across interconnected systems, mimicking the way human brains process information across multiple nodes.

This distribution enhances robustness and scalability, particularly for large-scale applications where localized or specialized reasoning engines lack the processing power or contextual understanding needed for general intelligence. Runway and Replit are examples of companies pushing the boundaries of distributed AI, where cognitive processes are distributed across cloud-based systems, enabling them to handle larger, more complex tasks like video editing and software development with real-time collaboration.

Multi-agentic workflow orchestration

Agentic technology is set to become a fundamental building block for intelligence-first applications. AI agents focused on individual tasks can now be coordinated to plan and collaborate in multi-agent systems, enabling more complex tasks to be completed autonomously.

In a prior blog, AI Agents are Disrupting Automation, we explored the emerging Mixture of Experts architecture, where AI Agents segment workflows into task planning, functional tasks, and reflection. Companies like CrewAI are streamlining the deployment of these multi-agent systems. We also expect many advances in agentic infrastructure and frameworks to mimic human thinking and enable guardrails for observability, alignment, and compliance.

Agents represent a significant area of advancement and research in how we move from understanding and generation to reasoning and taking actions.

Adaptive multi-modal user experiences

When user interaction journeys are no longer pre-determined, and humans are not the only or primary users of systems, how can they be re-imagined? Next-gen apps will move beyond code-determined UX to on-the-fly generated user experiences.

Instead of relying on pre-built forms, narrow apps, or static dashboards, these applications will provide situation or surface-aware interactions, meeting users wherever they work — whether via voice, text, or visual systems with dynamic user journeys. While chat is the current AI UX trend, it mirrors the skeuomorphism of older graphical UIs — effective for some tasks but inadequate for the fluid, intuitive interactions needed in complex workflows.

For apps to evolve from co-pilots to true AI co-workers, they need to become as natural to interact with as another human across modalities and canvases. While this topic has been an area of research dating back to DARPA CwC (Communicating with Computers), recent breakthroughs like GPT 4o (where interactions can be interrupted) point to collaborative interaction.

Early examples of this new UX direction include NotebookLM and MakoAI, which explore more fluid, proactive collaboration between humans and AI.

Applications of the past and future

Many traditional software applications (including most built for enterprises) are at risk of falling behind in this transformation. Most packaged applications were built for an era where humans were the primary drivers — entering data and interacting with screens, even when APIs were available. These systems rely on deeply configured, rigid workflows, siloed data storage, and countless surrounding apps, integration services, and custom solutions to accomplish complex end-to-end processes. In this model, humans essentially stitched apps and workflows together and were often the primary source of intelligent decision-making.

In contrast, intelligence-first applications promise to be fundamentally different. Built using the design patterns described above, these applications have the potential to reshape how work is done. We see a broad spectrum of application archetypes emerging, poised to evolve not only with breakthroughs in technology but also alongside changes in buyer and user behavior.

How different can they be? Consider a warehouse management system sold by a leading enterprise vendor that hasn’t changed in core design in three decades, still built around humans as the primary workers. Now, contrast that with a system designed for a warehouse where robots, machines, and AI co-workers outnumber humans. The difference is stark.

For the latter to be possible, intelligence must be a native, first-class citizen. Simply put, the value will shift to applications that are only possible because of advancements in AI — leveraging both deterministic and generative models.

Key archetypes of “intelligence-first” applications

Deep AI apps

Deep AI apps are advanced applications that solve highly complex, domain-specific problems and are often characterized by full-stack optimization from the models layer up. AI-driven breakthroughs in design, research, and simulation are spurring a new generation of applications. These are companies whose innovations would not be possible without advanced AI model capabilities.

Companies like BeyondMath are using advanced models to analyze the aerodynamic efficiency of vehicles without spending millions on research and development. Profluent has built OpenCrispr, an open-source AI gene editor powered by its large language model trained on millions of proteins and RNA. Ambience is building an AI-powered operating system for healthcare, inclusive of a suite of AI-native products.

Co-pilots

Co-pilots can provide interactive support, helping humans make decisions in real time while leaving the user in control. They leverage natural language interfaces, APIs, and unstructured data to enhance day-to-day workflows across multiple tools. Co-pilots can augment existing applications or serve as primary interfaces to orchestrate tasks across various systems. Co-pilot as a pattern for tools or assisting tools across applications is here to stay as we navigate the vast landscape of existing software.

Writer’s co-pilots can help automate key workflows in human resources, marketing, sales, and even finance by leveraging their own domain-specific models and knowledge graphs. NormAI is leveraging AI models that understand government regulations and compliance policies, along with a set of AI agents that automate compliance analysis on top of such regulations and corporate policies.

Autopilots

Autopilots work on their own to automate entire workflows and tasks — not just repetitive tasks but also those requiring intelligent decisions. These agentic systems go beyond co-pilots to enable autonomous intelligence at scale. Autopilots integrate with a variety of third-party tools and systems aiming to ensure they can complete the task(s) at hand. Able to work 24/7, autopilots drive productivity beyond what a human alone is capable of.

Cognigy’s AI agent can automate and summarize omnichannel conversations via phone or chat, freeing hundreds of hours for human customer experience agents. Bland’s AI voice agent enables organizations to “call” customers, users, patients, and more without needing to pick up a phone — saving organizations time and resources and delivering strong outcomes. We’re also seeing autopilots that fail gracefully, handing off tasks to a human+co-pilot when necessary, and using the feedback provided to continuously improve via Reinforcement Learning from Human Feedback (RLHF).

AI Coworkers

AI Coworkers go beyond task and workflow automation, working side-by-side with humans, ushering in the age of true collaborative intelligence. With reasoning and cognition capabilities, AI Coworkers can handle a variety of tasks and learn from their work and the workers around them, much like human coworkers.

Design approaches are emerging — from targeting specialized roles to building general-purpose coworkers with foundational skills for later specialization. Unlike humans, some AI Coworkers can manage vast complexity without needing structured applications. Instead of building rigid accounting software, why not create a virtual accountant capable of autonomously managing and organizing all data?

Although widespread adoption may take time due to inherent complexities, companies like Cognition (Devin) and Relevance AI are showing early signs of what is possible.

AI + human work fabric

As AI Coworkers and Autopilots become more integrated into work environments, we believe the future of software will need to evolve to accommodate this shift. This raises key questions:

Do we need a new fabric of computing to organize, integrate, and manage human-AI collaboration and workflows?
Will we shift value from old systems of record to fundamentally new ones?

The answer is likely yes, which will enable the future of work that envisions human and AI workforces. This is likely to happen faster in traditional services industries, where the transformation opens greenfield opportunities for new players and incumbents alike. Services businesses can now be reimagined as human+AI fabrics and delivered in a “Services-as-Software” model where human and AI work is highly integrated to deliver outcomes. This could also shift business models from per-seat or consumption-based pricing to outcome-based, where results delivered by human-AI collaboration can drive value.

Solutions like CrewAI, Ema, and Agentforce (by Salesforce) are already orchestrating real-time collaboration, where AI can autonomously interact and collaborate with human teams. CrewAI is building a developer platform to manage complex multi-agent AI systems, automating numerous tasks. Ema is creating a ‘Universal AI employee’ that currently handles a variety of roles. Similarly, Sema4 is building AI Coworkers that automate tasks like invoice reconciliation, regulatory compliance, and receivables matching.

An emerging wave of “intelligence-first applications”

While still early, we are seeing a wave of new startups leveraging these design patterns and programming models across Deep AI apps, Copilots, Autopilots, AI Coworkers, and AI+Human Operating Systems. Some of these companies include:

Relevance AI

Relevance AI is pioneering the concept of an “AI Workforce” with its platform that allows businesses to create multi-agent systems tailored to specific tasks. These AI agents integrate with tools like LLMs to automate workflows, enhance personalization, and tackle complex tasks such as marketing automation and customer data analysis. Their focus on integrating unstructured data from multiple sources and creating scalable AI SDRs and lifecycle marketers makes them a great example of an AI Co-Worker.

Ema

Ema enables enterprises to deploy “AI Employees” across a range of functions like customer support, HR, and compliance. Ema’s EmaFusion model blends various LLMs to maximize accuracy while reducing operational costs. This AI assistant integrates with over 200 enterprise apps and is optimized to handle both structured and unstructured data. Ema works with human employees, existing tools and apps, making it a key example of the AI & Human Work Fabric.

Torq

Torq’s Socrates AI agent is an integral part of their security hyper-automation platform. Socrates autonomously handles 90% of Tier-1 security tasks, such as alert triage, incident investigation, and threat response. By integrating with various security tools and leveraging threat intelligence, Socrates enriches and prioritizes security events. Its ability to consolidate data from disparate sources makes it an essential part of the Data Fusion Stack for security operations, empowering SOC teams to manage high-volume alerts more effectively. Torq is a strong example of an AI coworker for SOC teams.

CrewAI

CrewAI is an AI framework designed to simplify the management and deployment of multi-agent AI systems. It enables agents to coordinate, interact, and automate tasks efficiently. CrewAI uses a role-based architecture, allowing agents to be assigned specific functions, such as data extraction or task execution, making it adaptable for various workflows. The platform integrates tools for agents to perform complex actions like searching the web or analyzing unstructured data. Its modular design ensures scalability, allowing organizations to add or modify agents without disrupting existing workflows. This framework plays a key role in enabling multi-agent collaboration and orchestration, empowering businesses to scale their AI-driven processes across industries.

Sixfold

Sixfold is building an AI Copilot for insurance underwriters. Sixfold ingests data from a variety of platforms underwriters interact with (emails, PDFs, website data, third-party data, etc). They then feed this data into Sixfold’s AI models (powered by a variety of open-source and closed-source AI models), which understand a carrier’s risk appetite for a given insurance line and can make optimal recommendations. Sixfold boosts efficiency and outcomes for insurance underwriters and carriers, bringing AI and automation to a historically manual workflow.

Can incumbents redesign to become intelligence-first?

Each major architectural shift in technology has historically brought new winners in platforms, infrastructure, and applications. In some cases, new upstarts have become the new winners, leveraging their agility and pace of deployment to win. In other scenarios, incumbents have rebuilt their platforms or acquired newer companies to drive their innovation forward. For example, Oracle, long a dominant database vendor, underestimated the complexities of building businesses in layers above them and eventually acquired companies like PeopleSoft, Siebel, Netsuite, and BEA Systems to fill that gap.

Unpacking the dynamics of disruption is an intriguing exercise, as there are many attributes to evaluate in each product category — the pace of industry change, quality of incumbents’ tech talent, data gravity of existing solutions, business model, distribution, brand trust, and user experience, to name a few.

HubSpot’s evolution toward a next-gen CRM with their Breeze AI agents and Salesforce’s release of AgentForce exemplify how incumbents are attempting to reinvent themselves in the AI era by layering intelligence into traditional applications. Yet, the question remains: Can they truly inject intelligence into decades-old architectures, or will the emergence of intelligence-first applications give rise to a new CRM that looks nothing like the CRM of the past?

For incumbents like Salesforce and HubSpot, retrofitting AI into legacy systems might seem like the easiest path. The benefit of this approach is the power of existing distribution — many companies already use Salesforce, so encouraging adoption of their agentic features is a simpler task than replacing a company’s entire CRM. However, the ultimate value-add from these AI-enhanced features may be limited by their own legacy design and architectures.

Today, many argue that incumbents will have the edge, given their distribution advantages and perceived agility. But is it as simple as plugging into ‘electricity’? Analogies are often imperfect, but initial attempts by incumbents feel like adding electric lighting to steam-powered factories rather than reimagining and redesigning manufacturing for an electric era. Becoming AI-native or intelligence-first demands the same kind of reimagination and redesign.

Given the distribution benefits and inertia of incumbents, this new wave of intelligence-first applications needs to be, materially, 10x better than existing applications with AI enhancements to win. Switching from Salesforce + Agentforce to a new, AI-first CRM won’t make sense if the AI-first CRM is only marginally better. Each of the design patterns we discussed above presents an approach to create new value and could even become future control points to shift value away from legacy solutions, even without a full replacement.

We believe, however, that a truly reimagined intelligent CRM has the potential to drive 10x, even 100x, user experiences and desired outcomes as it will be created for greater autonomy, reasoning, scalability, adaptability, and seamless integration of AI across all workflows.

Looking ahead

While the last few paragraphs have focused on the CRM, we believe a similar paradigm shift will be felt across many categories of software — and surrounding industries being reimagined by software and AI. Many of the companies that emerge as leaders in the next generation of AI-driven software are likely to be the ones that start fresh with intelligence-first architectures at their core. For builders, the real opportunity lies not in who can glue AI onto old platforms, but in who can build new, intelligence-first applications from day one. Those are the companies that will likely redefine entire industries and labor and services markets, and they are the founders we are most excited to support.

The AI world is changing rapidly. We believe that the applications being built today and tomorrow will be fundamentally different from the ones of yesterday.

The AI world is changing rapidly. We believe that the applications being built today and tomorrow will be fundamentally different from the ones of yesterday. The future of work will entwine AI workers and human workers together, and we are excited to learn more about the systems designed to optimize for that future.

If you are a founder/re-imagineer building in the space, we would love to meet with you. We are eager to collect more feedback in this ever-changing world, so if you have any additional viewpoints to share, feel free to reach out. We welcome the discussion.

Editor’s note: BeyondMath, Profluent, Writer, Cognigy, CrewAI, SingleStore, Acceldata, Atlan, Astronomer, Relevance AI, and Torq are Insight portfolio companies. Insight has also invested in Databricks.

References, in no particular order: