Focus on AI applications, not just workflows: Andrew Ng’s keynote at ScaleUp:AI ‘24

AI luminary and Founder of DeepLearning.AI, Andrew Ng, discussed the exciting trends in AI Agents and applications. In recent years, the spotlight has been placed on large language models (LLMs) in AI. Building on them, a new set of design patterns is emerging with the rise of AI Agents and agentic reasoning, in which systems can reflect, plan, use tools (function calls), and collaborate in multi-agent designs. These design patterns are proving to be both cost-effective and powerful for quickly building numerous new applications.

Key takeaways

• AI’s real value lies in applications: Ng emphasized that AI is a general-purpose technology, but the greatest opportunity is in building applications that drive revenue and sustain the ecosystem.
• Agentic AI is transforming workflows: AI Agents can iterate, research, and refine outputs, leading to significantly better results in industries like law, healthcare, and government.
• The AI stack is evolving: The emergence of an agentic orchestration layer, with tools like LangGraph, is making it easier to build AI applications.
• AI trends are reshaping industries: Faster text generation, generative AI-driven rapid prototyping, and the growing role of data engineering are accelerating innovation.
• Companies need a structured AI strategy: Ng shared a five-step process for corporate AI adoption, starting with executive training and systematic opportunity identification.
• AI governance should focus on applications: Rather than regulating AI technology itself, Ng argues that governance should address specific use cases to ensure responsible AI development.

These insights came from our ScaleUp:AI event in November 2024, an industry-leading global conference that features topics across technologies and industries. Watch the full session below:

Agentic AI: The next big leap in AI capabilities

Ng identified agentic AI as the most significant AI technology trend. He contrasted the traditional non-agentic workflow with agentic workflows, explaining that the latter allows AI to engage in iterative processes similar to human thinking, research, and revision, leading to a “much better work product.” He provided data showing a substantial improvement in coding benchmark scores (HumanEval) when using agentic workflows with models like GB 3.5 and GPT-4.

Ng also noted that agentic workflows are proving highly effective in applications such as processing legal documents, assisting with medical diagnoses, and handling complex government compliance paperwork.

The rise of the agentic orchestration layer

This shift has led to the emergence of an agentic orchestration layer in the AI stack, with tools like LangChain (evolving to LangGraph) facilitate the building of agentic applications. However, Ng reaffirmed that “most of the value had better be at the application layer.”

Beyond agentic AI, Ng outlined several other important AI trends:

• The benefit of cheaper and faster text generation is crucial for the text-intensive nature of agentic workloads.
• Generative AI is significantly speeding up prototyping, reducing development time from months to potentially just ten days, which accelerates corporate innovation and allows for rapid experimentation with a “move fast and be responsible” approach.
• The gravity of data is decreasing due to the compute-intensive nature of generative AI, making it more feasible to transmit data across different clouds for processing.
• Data engineering for unstructured data (text and images) is becoming increasingly important as generative AI can now extract significant value from these data types.

A five-step approach to corporate AI innovation

Regarding corporate innovation in AI, Ng shared a structured process that includes executive training on AI technology to ensure leaders understand its capabilities.

1. Equip leaders with AI literacy: A strong AI strategy starts at the top. Ng stressed the importance of executive education, citing courses like Generative AI for Everyone as essential for helping leaders understand AI’s capabilities and identify strategic opportunities.

2. Brainstorm AI applications with task-based analysis: Rather than looking at automating entire jobs, Ng advised companies to break roles down into individual tasks and identify where AI can have the most impact. This approach often uncovers unexpected, high-value opportunities.

3. Evaluate feasibility with due diligence: Not every AI idea is worth pursuing. Companies must rigorously assess both technical feasibility and business value before committing resources to a project.

4. Decide: build, buy, or invest?: With generative AI still in its early stages, many solutions don’t exist in the market yet. Ng encouraged companies to carefully weigh their options — whether to build in-house, acquire, or even spin out a startup to develop AI solutions more efficiently.

5. Develop long-term AI strategy & workforce training: Sustained AI success requires ongoing investment in strategic planning and workforce training. Ng highlighted that organizations need to continuously refine their AI approach and upskill employees to maximize AI’s potential.

Why regulation should focus on applications, not technology

Finally, Ng addressed AI governance, strongly advocating for governing applications rather than the technology itself. He used the analogy of an electric motor to illustrate that the risks are tied to the specific application of a technology, not the technology in isolation. He expressed concerns about lobbying efforts aimed at stifling open-source AI model development and reiterated that “governance safety applied at the application layer makes more sense than [the] technology layer.”

Ng believes that AI, driven by agentic workflows and other advancements, is rapidly expanding possibilities and creating significant opportunities for both startups and large companies.

Watch more sessions from ScaleUp:AI, and see scaleup.events for updates on ScaleUp:AI 2025.