Master's Thesis

Enhancing Semantic Search with NLP for Secure and Scalable Document Retrieval: A Metadata-Enriched, ACL-Aware Approach

Francisco Carlos Qiu Mexia de Azeredo — 2025

Key information

Authors:

Francisco Carlos Qiu Mexia de Azeredo (Francisco Carlos Qiu Mexia de Azeredo)

Supervisors:

Filipe Correia; Sérgio Luís Proença Duarte Guerreiro (Sérgio Luís Proença Duarte Guerreiro)

Published in

December 5, 2025

Abstract

This thesis designs and implements a retrieval-augmented, agentic semantic search system for corporate document management, developed with Edoclink. Corporate repositories contain rich metadata and organizational structure that traditional keyword search cannot exploit. We explore a newsroom scenario where journalists query documents to fact-check articles; the system retrieves relevant content, follows cross-references for programmable multi-hop retrieval, and synthesizes cited answers for efficient verification. We leverage advances in NLP, centering on RAG and AAI. The pipeline (i) ingests and normalizes heterogeneous documents, (ii) indexes content in a VD with metadata-aware schemas supporting explicit cross-references, and (iii) uses LLMs orchestrated via PE, to ground answers in retrieved evidence. We study retrieval in isolation—structure-aware chunking, metadata filtering, and hybrid search—then implement Agentic RAG, where agents perform query analysis, collection selection, iterative retrieval, and cited synthesis. A key contribution of this work is the development of an MCP server, implemented in Go and TypeScript, that exposes Weaviate’s schema-aware querying and cross-reference traversal as typed tools—enabling agents to navigate organizational structures programmatically. The system’s capabilities are demonstrated through a newsroom scenario featuring multi-hop queries. Retrieval performance is validated on synthetic enterprise information (1,000 files, 300 QA pairs) as well as on temporal knowledge bases. Using Youden-optimized thresholds, the agentic system achieved a 60.4% retrieval accuracy (compared to 13.8% naive RAG baseline), and correctly classified 61% of answers (versus 19% naive RAG baseline), representing a 27.6× cost increase—demonstrating a favorable trade-off for precision-critical applications.

Publication details

Authors in the community:

Supervisors of this institution:

Fields of Science and Technology (FOS)

electrical-engineering-electronic-engineering-information-engineering - Electrical engineering, electronic engineering, information engineering

Publication language (ISO code)

eng - English

Rights type:

Embargo lifted

Date available:

November 16, 2026

Institution name

Instituto Superior Técnico