RAG inside a Laravel app, without a Python service

Every RAG tutorial assumes you are starting a new Python project. Most of my clients are not. They have a Laravel or Symfony application with five years of data in MySQL, a team that knows PHP, and no appetite for a second service to deploy and monitor.

Keep retrieval next to the data

The documents you want to search already live in your application database, with permissions attached. Moving them into a separate service means duplicating both the content and the access rules — and access rules are exactly what you cannot afford to get wrong when an answer is generated from private records.

What the pipeline actually looks like

Ingestion runs as a queued job: extract text, chunk with overlap, embed, store the vector alongside the record. Retrieval is a similarity query filtered by the same authorisation scope the rest of the app uses, then a rerank pass, then the model call with sources pinned into the prompt. Nothing exotic. The work is in the details.

Chunking is the whole game

Bad answers are almost never a model problem. They are a chunk problem: a paragraph split mid-sentence, a table flattened into noise, a contract clause separated from the definition it depends on. I spend more time on extraction and chunk boundaries than on prompts.

Log everything from day one

Store the question, the retrieved chunk ids, the prompt, the answer and the latency. Without that you cannot tell whether last week's prompt change helped, and you cannot explain to a client why one answer was wrong.

Let the client own the provider

With an abstraction layer — Prism in Laravel, or your own interface — swapping Anthropic for a local Ollama model is a config change. For clients in healthcare or legal work, that switch is the difference between a signed contract and a polite no.