Query Processing on Unstructured Multimodal Data with Accuracy Guarantees
ReDD (pronounced "ready") is a novel research project developing the first query processing framework for unstructured multimodal data with formal accuracy guarantees. We adopt a fundamentally different approach from traditional RAG systems by extracting query-specific schemas and translating natural language queries to SQL for analytical query support.
Our vision: Execute natural language queries against arbitrary document repositories with error bounds and formal guarantees.
About ReDD
We develop a query engine for unstructured documents. Our vision is to be able to receive natural language queries and execute them against arbitrary document repositories. Unlike Retrieval Augmented Generation (RAG), in ReDD we adopt a fundamentally different approach aiming to serve a very different purpose.
Given a natural language query, we extract a schema from the data tailored to the query, and then we populate the schema with data that will answer the query correctly. Thus we can support analytical queries fully. We translate the question to SQL and execute the SQL query against the collection of extracted tables.
Think about DeepResearch or DeepSearch, but for answering relational queries over documents with error guarantees. We are working to address many challenges in this space.
Research Challenges
Our research addresses several fundamental challenges in query processing over unstructured data:
Natural Language to SQL Generation
Developing robust translation models that convert natural language queries into syntactically and semantically correct SQL, accounting for the variability in human language and query complexity.
Ad-hoc Schema Extraction
Creating dynamic, query-specific schemas from unstructured data that may include multiple tables with constraints, ensuring the extracted structure accurately represents the information needed to answer the query.
Query Optimization Framework
Optimizing the trade-off between data extraction and code generation in this novel framework to maximize efficiency while maintaining accuracy guarantees.
Abstention Mechanisms
Developing reliable methods for the system to recognize queries it cannot answer with sufficient confidence and appropriately abstain or ask for clarification.
Conversation Frameworks
Creating interactive systems that can inject relevant knowledge during query execution and steer the process toward correctness through conversational guidance.
Research Directions
We are developing the first query processing framework on unstructured multimodal data with accuracy guarantees. We are exploring the following research directions:
Query Processing with Accuracy Guarantees
Exploring schema extraction, query processing and optimization, enhanced with various aspects of filtering for guaranteed performance.
Query execution over unstructured data: NL → SQL → Document Results
This research direction formalizes the pipeline from natural language queries to SQL translation to result extraction from documents with verifiable accuracy bounds.
Multimodal Query Processing
Documents have images and tables that interact in various ways. We are developing extensions of popular query processing frameworks for queries involving multimodal data with guarantees.
Multimodal query touching tables, images and text
Our framework handles heterogeneous data types and their interactions, providing a unified query interface for multimodal document repositories.
Reasoning Models Integration
Reasoning models such as Gemini-3-Pro have superior accuracy for data that fit in their context window. We explore query processing when one can obtain superior accuracy on finite windows of data.
Query + Unstructured Data + Reasoning Model
This research investigates optimization strategies and accuracy implications when integrating large language models with bounded context windows into the query processing pipeline.