JOURNAL

XERJ’s Open-Source AI Search Engine: What the Numbers Actually Say

A fast-growing open-source search engine claims 40x fewer tokens than grep for AI coding agents. Its own benchmark says 2.7x. Here's what the repo, security docs, and case study actually show.

Conceptual illustration of agent tools, memory and quality checks.

Read with AI

Choose content to copy and paste into your AI assistant. Nothing is sent automatically. CMS content is converted to Markdown; original Markdown is used when available.

60-second summary

XERJ (github.com/xerj-org/xerj) is a free, Apache-2.0, single-binary local search server that indexes any folder — code, docs, logs, PDFs — so a coding agent can query a passage by meaning instead of grepping whole files into its context window. It is young (created 2026-06-30, already at 6,976 stars and 355 forks) but under active development, last pushed the day before this was written. The GitHub repo’s own one-line description claims “40x fewer tokens than grep.” The project’s own published benchmark, run against a real coding agent on 8 tasks across 4 languages, actually measured 2.7x fewer output tokens than a grep-driven agent at the same 16/16 solve rate — still a real, reproducible win, just not the number on the tin. If your team runs AI coding agents against private or recently-written code, this is worth a 20-minute trial before you trust either number.

The problem it’s aimed at

An AI coding agent working from memory alone retries endlessly on any API or internal convention it hasn’t memorized — there’s nothing to retrieve, so it guesses. Grep fixes the “nothing to retrieve” part but not the cost: it finds candidate files, then the agent has to read them whole into context to figure out which lines actually matter, which burns tokens fast on anything beyond a small file. For a team running agents against a large private monorepo, or against code written after a model’s training cutoff, that token cost compounds across every task, every day. XERJ’s pitch is to shrink the “find the right lines” step from reading whole files to returning the exact passage, indexed once and queried many times, via an interface that’s compatible with existing Elasticsearch clients.

A concrete scenario

Picture a 12-person engineering team maintaining a five-year-old internal billing service in Rust, with conventions and helper libraries no public model has ever seen. Every time someone asks Claude Code or Cursor to add a feature, the agent either invents a plausible-looking function that doesn’t match the codebase’s real patterns, or burns tens of thousands of tokens reading adjacent files to find the real one. The team doesn’t need a new chat product — it needs its existing agent to stop reading the whole barn to find one bolt. That’s the specific, narrow case XERJ’s own benchmark was built to measure, and it’s also the case where the savings are least likely to be marketing noise: code the model has no chance of already knowing from training.

A test before you commit, not a definitive result

Before wiring this into daily use, the cheap test is to run XERJ’s own published comparison shape on your own codebase rather than trusting either vendor’s headline number: take 8-10 real past tickets your team already solved, run an agent against them three ways (memory only, grep, XERJ-indexed) with token counts and pass/fail logged for each, and compare against how your team actually solved them. This test has not been run for this article — XERJ’s cited numbers come from its own published case study, not from independent verification here. The number that should decide adoption isn’t the repo tagline; it’s whether the token savings hold on your codebase’s actual pass rate, not just a demo corpus.

Getting it running

  • Single static binary, no JVM, installed with a one-line script: curl -fsSL https://xerj.org/get | sh (github.com/xerj-org/xerj).
  • Start a local node, index a project, and query it: xerj --insecure --data-dir ./data &, then xerj autoindex ~/my-project, then xerj search "how do we rotate the WAL segment" for ranked file:line passages (same source).
  • Two extra CLI commands worth knowing: xerj def "symbol_name" for go-to-definition (file:line plus signature), and xerj gain, which the project describes as reporting what the tool actually did for you, “counted, not estimated” (same source).
  • Same primitive, four other listed uses: codebase Q&A/RAG with file:line citations instead of a directory listing; security audits querying for sink patterns and secrets across file types; log/incident analysis via typed indices with Elasticsearch-style aggregations; and agent long-term memory stored per namespace under /_memory, recalled by meaning in later sessions (same source).

How it actually fits together

Diagram showing a coding task triggering the --insecure trust boundary, then autoindex, then an agent querying by meaning instead of grep, writing code, and sending a field report back
Start the node inside its one-user trust boundary, index the project once, then let the agent query by meaning instead of grepping whole files into context — the measured result was 2.7x fewer output tokens than a grep-driven agent at the same solve rate.

XERJ’s own case study measured the full loop end to end: 8 tasks, 4 languages, 16 runs per arm, real claude -p token counts. Working from memory only cost 260,916 output tokens and $11.18, solving 11 of 16. A grep-driven agent cost 26,477 tokens and $3.27, solving 16 of 16. An XERJ-backed agent cost 9,982 tokens and $1.58, also solving 16 of 16 — 2.7x fewer tokens than grep and 26x fewer than memory alone at the same solve rate, with up to 278x fewer on a single Java task in the same run (xerj.org/case-studies/reference-coding). Separately, on Elasticsearch-API compatibility and raw performance, the project publishes an 88-cell comparison scorecard against Elasticsearch itself: 55 wins, 26 ties, 4 losses, 3 not applicable, with 1.72x higher bulk-ingest throughput and 1.61x smaller on-disk size (176.2 MB vs. 283.0 MB) on identical corpus and hardware (xerj.org/benchmarks) — and it explicitly publishes the losses alongside the wins rather than only the favorable cells.

What this costs and where it breaks

Read the gap between the tagline and the body text before you quote either one internally. The GitHub repository’s own one-line description advertises “40x fewer tokens than grep.” The README’s own measured result, the only controlled benchmark the project points to, is 2.7x fewer output tokens than a grep-driven agent at the same solve rate (26x fewer than memory alone) — the 40x figure doesn’t appear anywhere in the measured results themselves. That’s not evidence of bad faith; “field reports” (user-submitted, not controlled) put everyday savings around 5x. But it’s a reminder to read past the badge row before repeating a vendor’s own headline number as fact, the same discipline this blog has applied to every other vendor-claimed benchmark.

The project is explicit about where the win disappears: the value is “gated by memorisation” — it helps on private, internal, niche, or post-training-cutoff code, and is “neutral to harmful” on popular public libraries a model already knows well from training. If your team’s codebase is mostly glue around well-known public frameworks, don’t expect the same multiple.

Security is the other line item to budget for before production use. Running XERJ requires an explicit --insecure flag to start without TLS or API-key authentication; in that mode, every caller that can reach the port is a superuser who can read or delete every index, including agent-memory namespaces. A fresh node binds to 127.0.0.1 only by default, and the project documents this plainly as a “one user on a dev machine” trust boundary, not a production security posture (docs/SECURITY_MODEL.md). The same document is unusually specific for an open-source project this young: every authorization claim is tied to a named source file and line number, including a note that path segments are percent-decoded before classification specifically so an encoded path can’t slip past a check written against the decoded form — a level of documented rigor worth noting, though it describes the code’s intent, not an independent security audit.

Finally, the project’s only ask in exchange for the free binary is a short “field report” pull request — what you indexed, what you used it for, your honest verdict, explicitly including negative verdicts — and these are CLA-exempt, merging without a signature requirement; both humans and agents are invited to submit one (github.com/xerj-org/xerj). That’s a low-friction way to contribute, but it also means the project’s own growth signal (star count, report volume) is self-reported by design, not a neutral third-party metric.

Sources

Where to go next

If your team is deciding whether to wire an indexed-search tool into your AI coding agents, the question worth five minutes isn’t the repo’s tagline — it’s whether the token savings hold on your own codebase’s pass rate, and whether your deployment can tolerate the current single-user security boundary before you open it to a shared team server. If you want help scoping a benchmark like the one above against your own ticket history before adopting a tool like this, Thuận Lương’s consulting practice helps engineering teams run exactly that kind of pilot; reach out through luonghongthuan.com/en/contact.

Discussion

Comments are reviewed before publication. Your email is kept private.

← Back to allĐọc tiếng Việt