Introducing Orca: A Groundbreaking Human-in-the-Loop AI System from UC San Diego

Researchers at UC San Diego have unveiled a pioneering open-source system named Orca, which exemplifies how large language models (LLMs) can enhance user experiences on the web. Contrary to notions of AI taking control away from users, Orca serves as a supportive guide—or a decision-making ‘co-pilot’—to facilitate web interactions.

Enhancing Web Interaction with Orca

The research team’s findings, recently published in a peer-reviewed white paper, highlight significant improvements in task speed and accuracy when employing Orca. This early evidence underlines the potential of integrating human-in-the-loop AI systems in everyday workflows, showing that technology can enhance human capabilities rather than replace them.

Key Features of Orca

Orca is specifically designed to extract meaningful insights from online content. Its capabilities range from summarizing lengthy web pages to extracting structured data from otherwise unstructured sources. Additionally, it can track changes over multiple browsing sessions and compare claims across various sources. This versatility empowers users to delegate repetitive or complex tasks to Orca, all while maintaining ultimate control over the browsing process.

Real-World Applications: A User-Centric Approach

A lab study involving eight participants revealed that Orca notably accelerated web exploration. Users reported feeling more confident in their information foraging, as they could visually organize various web pages and selectively delegate tasks to the AI. This design resonates with many; one participant effectively utilized Orca to compare Yelp options side-by-side, while another harnessed its capabilities to filter Reddit posts for product research.

The spatial organization and batch interactions provided by Orca were especially praised for minimizing context-switching costs, making complex workflows more manageable and intuitive. This focus on simplicity and usability is crucial for integrating advanced technology into everyday tasks.

Shared Control as a Core Design Principle

A standout feature of Orca’s design is its emphasis on shared control. Users initiate actions and maintain command throughout their interactions, fostering greater trust in the AI’s assistance. This principle of shared control cultivates transparency, a key factor the researchers identified as essential for building user confidence and ensuring agency in AI-assisted workflows. Orca stands out in offering a collaborative partnership rather than a domineering presence.

Technical Architecture and Functionality

Orca operates as an Electron application featuring a React-based frontend. Each web page loads within its own isolated web view, while the ‘Web Canvas’—an interface used for organizing and interacting with multiple pages—leverages the open-source tldraw library for seamless navigation and task management.

The application’s language capabilities—summarization, extraction, and automation—are powered by the Claude 3.7 Sonnet model. To enhance efficiency, Orca employs a custom HTML distillation and agentic pipeline architecture. This system transforms unrefined web content into structured representations that are immediately useful to the LLM, allowing user interventions as necessary during execution.

Research Prototype with Future Potential

While Orca’s capabilities are impressive, it is currently positioned as a research prototype rather than a fully-fledged production tool. The team aimed to offer developers a foundation for exploring future collaborative agent workflows. However, the researchers caution that performance may degrade under heavy workloads. For instance, an M4 Max MacBook Pro with 36GB of unified memory can handle approximately 80 webpages before experiencing performance issues.

Exploring the Future of Collaborative AI

Orca exemplifies the promising benefits of human-in-the-loop systems, offering a glimpse into what the future may hold for collaborative user-agent interactions. The intention is clear: AI is here to assist, not replace, empowering users through high-context, decision-heavy workflows.

Orca in the Broader Landscape of AI Tools

As Orca emerges in this innovative space, it is not operating in isolation. It shares a philosophy with other developing tools, such as OpenAI’s Operator and the redesigned Opera Neon browser, that seek to facilitate collaborative workflows. Together, these systems could shape the future of how we interact with multiple information sources and technologies in real time.