AI PoC: Testing a sovereign AI agent for document research.

Quai des Notaires is a fast-growing French LegalTech company specializing in the digital transformation of the notarial sector. With a team of around 50 employees, the company has established itself as a major player in the sector, serving nearly 24% of notarial offices in France, representing more than 8,500 daily users.

Their objective was to integrate AI broadly across their operations. Given the sensitivity of the data processed, the client required a sovereign AI solution, either French or self-hosted, while prioritizing local hosting and national technology partners.

Selected technical solution: a sovereign architecture combining the Qwen model for OCR and the Mistral Small 24B model for metadata extraction. The entire solution is orchestrated by Apache Airflow, while hosting is provided by a French sovereign cloud, OVHcloud.

This technology choice was driven by an absolute requirement for digital sovereignty and confidentiality of notarial data, requiring all processing to take place exclusively within France. The use of "Open Weights" models such as Mistral provides an auditable and secure solution, unlike foreign proprietary solutions.

From an operational standpoint, Apache Airflow was selected for its ability to manage complex data flows with a high level of reliability, providing automatic retry mechanisms in the event of an error. Finally, we recommended OVH Public Cloud given the need to adjust performance after validating the solution under real-world conditions, while also meeting sovereign hosting requirements.

Phase 1: Business requirements qualification

This initial stage focused on precisely defining the scope of the engagement for Quai des Notaires. The work involved identifying all the documents to be processed and the exact nature of the information to be extracted in order to address the business requirements. At the same time, we identified the key elements of the client's expectations while incorporating the strict legal constraints related to data sovereignty. This phase established the regulatory and operational framework required before any technical research.

Phase 2: State-of-the-art review and analysis of existing solutions

The study identified and compared several OCR technologies and AI models against performance and confidentiality criteria. Several options were explored, including Mistral OCR, LlamaOCR, Qwen, Tesseract and Docling.

Phase 3: Experimentation and PoC results

The practical implementation of the PoC served as a test bench for the previously shortlisted solutions. This experimentation phase was crucial in assessing the tools against real-world conditions and identifying specific technical challenges, particularly in processing complex and handwritten documents, where the success rate was capped at 60%.

Following these comparative tests, Qwen was selected for the OCR component because it was the most pragmatic solution for immediate local deployment without relying on an external connection. For semantic analysis, Mistral Small 24B was selected for its effectiveness and full compliance with sovereignty requirements.

Phase 4: Hosting and sovereignty assessment

Based on the performance observed during the PoC, a dedicated study was conducted to define the production infrastructure. OVH Public Cloud was selected to ensure that all data processing takes place within France, thereby meeting the confidentiality requirements of the notarial sector. This stage made it possible to accurately size the GPU server requirements needed to run the Mistral and Qwen models, while ensuring strict separation between test and production environments.

Phase 5: Financial assessment and industrialization roadmap

The study concluded with a detailed cost analysis and a roadmap for scaling up. This assessment highlighted an initial investment to build the platform and significant monthly operating costs linked to server computing power. Optimization options were proposed, such as using shared models as a service, to reduce hosting costs. This final section provides full visibility into the development effort and budget required to sustainably integrate this solution within Quai des Notaires.

The experiment demonstrated excellent processing efficiency: a heterogeneous batch of 52 documents was fully processed in 4 minutes and 33 seconds. This time covers the entire pipeline, from retrieval from the SFTP server through to final result writing, validating the architecture's ability to handle real production workloads without saturation.

However, extraction reliability varies significantly depending on the document type:

• Simple certificates: The system achieves a 100% success rate, correctly identifying the document type and its metadata without errors.
• General data extraction: For standard information (addresses, cadastral references), accuracy reaches 83%, providing a solid basis for data-entry assistance.
• Complex cases: For multi-certificate files or blank documents, the success rate falls to 60%. The errors observed mainly involve "hallucinations" (inventing a non-existent certificate) or omitting information embedded in dense layouts.

Regarding the limitations encountered, three structural factors explain the inaccuracies observed in complex cases:

• Annotation timing: OCR extracts text in raw form without being able to distinguish the original printed text from handwritten notes added later (often crossed-out text or pen corrections). The AI receives an inconsistent mixture that it struggles to reconcile.
• Non-linearity: Documents containing complex tables or information written vertically in the margins led to OCR reading errors, which subsequently disrupted the agent's semantic analysis.
• Semantic density: On some very dense urban planning certificates, the AI struggled to extract the exact legal "substance", sometimes limiting itself to an overly literal interpretation.