Loading project...
Back to Work
RizalEngine
RizalEngine is a hybrid information retrieval system for exploring recurring themes — education, colonialism, justice, sacrifice — across José Rizal's Noli Me Tangere and El Filibusterismo. Built as an undergraduate thesis with two classmates, it solves a problem plain keyword search can't: a search for "edukasyon" (education) misses passages that express the same theme through different words.
My role was engineering the team's retrieval and validation design into the production system shown here — a FastAPI backend, Next.js frontend, and PostgreSQL/pgvector database — including a staged pipeline that tries exact keyword matches first and falls back to semantic search only when needed, plus a validation layer that stops the system from guessing on queries outside its literary domain.
103 chaptersAcross both novels, ~15,000–20,000 word primary corpus
3.00–3.24 / 4.00Expert validation rating ("Valid")
+21 ptsSummary-mode retrieval accuracy vs. full-text mode
Features
Hybrid Search
Search tries exact keyword matching first, falling back to XLM-RoBERTa semantic search only when direct matches run thin. A Lexical-Priority Reservation keeps exact matches on top even when semantic fallback activates, so literal word matches are never buried under passages that are merely similar in meaning. Every result shows its lexical and semantic scores side by side — the basis for a match is never a black box.
Domain-Adaptive Validation Guard
Because the underlying model is trained on modern multilingual web text, a query outside 19th-century Philippine literature could otherwise return a false "closest match." Every query is checked against a blocklist of anachronistic terms and separately scored against the literary corpus and its themes — rejected only if it fails both. This means the system correctly returns "no relevant results" for out-of-scope queries instead of guessing.
Scores reflect retrieval confidence, not accuracy — Level 5's zero score is the domain-validation guard correctly rejecting an out-of-domain query, not a retrieval failure.
Dataset Normalization
The raw 19th-century Tagalog source text — pulled from public-domain digitized copies — needed to be modernized before a search index or language model could work with it reliably. I built a normalization pipeline that standardizes archaic spelling and punctuation into modern Filipino orthography while preserving proper nouns and sentence structure — producing the dataset the rest of the system runs on.
View repo
System Architecture
Passages are stored in a sentences table (chapter, sentence text, source type, and a pre-computed 768-dimensional embedding vector) alongside a themes table used for thematic classification and query validation. A user submits a search query through the Next.js frontend, which sends an HTTP request to the FastAPI backend. The backend's RizalEngine converts the query into a 768-dimensional vector using an XLM-RoBERTa embedding model, then uses that vector to query the PostgreSQL/pgvector database for the most relevant matches, returning ranked candidate passages and their scores back to the frontend as JSON.
Technologies
Next.js 16TypeScriptReact 19Tailwind CSS 4ZustandTanStack QueryFramer MotionPython 3.11FastAPISQLAlchemysentence-transformerspgvectorPostgreSQL 15Docker ComposeXLM-RoBERTa
Engineering Decisions
01
Lexical-Priority Reservation
Exact keyword matches are pinned above semantically-similar results, even when semantic fallback activates — preventing a direct textual match from being buried under a passage that's merely topically related. This held retrieval scores at 79–80% for direct-match queries across evaluation.
02
Domain-Adaptive Validation Guard
Query validation runs in two layers: a hard blocklist rejects anachronistic terms outright, while a separate scoring layer checks each query's alignment against the literary corpus and its themes — catching out-of-domain queries that dodge the blocklist but have no real connection to the source material.
03
Cross-Lingual Support
A dynamic English-to-Tagalog mapping layer lets the system handle queries typed in English, routing them through the same retrieval pipeline used for native-language queries, with fallback handling for terms outside the initial mapping.
RizalEngine was developed collaboratively with my thesis teammates, Ian Placencia and Dominic Vilog, under the guidance of our adviser, Dr. Melvin Ballera. What's shown in this case study reflects my own contribution: engineering the team's retrieval and validation design into the production system — the backend pipeline, the frontend, and the tooling described above. Other parts of RizalEngine were built by my teammates and aren't represented here.