What our agent refuses to collect, and what that costs

An agent that runs inside somebody else's production process is trusted
with everything that process can see. The interesting part of building one is
not what it collects. It is the list of things it could collect, that would be
useful, and that it does not.

Here is ours, with what each one costs.

Query plans on MySQL and MongoDB

A plan change is the most actionable finding this product can produce. "This got
slower" invites a shrug; "this stopped using index_orders_on_user_id" names the
fix. We capture plans on Postgres and refuse to on MySQL and MongoDB.

The difference is one Postgres feature. EXPLAIN (GENERIC_PLAN) plans a
parameterised statement without binding a parameter — there is no step at
which a value could enter the plan. MySQL has no equivalent: it can only explain
a statement that still has its values in it. Mongo can only explain a filter
that still has its values in it.

So capturing a plan there would mean holding onto somebody's query values in
order to compose a command out of them. The signal is worth a great deal. It is
not worth that.

Postgres plans are refused too when the statement arrived carrying literals —
what an application with prepared statements disabled produces — because our
EXPLAIN runs on the flush rather than inline, and queueing such a statement
would mean keeping the values until then.

Bind parameters

The PDO instrumentation reads $statement->queryString and never
$params. Doctrine's takes the SQL from prepare(), so the values bound
afterwards are never in scope. This costs us the ability to show you the query
that was actually run, which is the single most requested feature of every tool
in this category.

The arguments in a backtrace

Call sites are captured once per statement, ever — a call site is a property of
the query, not of the call — with DEBUG_BACKTRACE_IGNORE_ARGS. The frames
give file, line and function name. What was passed is the customer's data and is
never read.

Values inside a document filter

SQL can be lexed: literals come out, identifiers stay. A Mongo filter has no
text to lex — the values sit beside the keys, arbitrarily deep. So identity is
not stripped out of it, it is built from it, keys only:

find orders {filter{user_id},limit,sort{created_at}}
aggregate orders [$match{status}][$lookup{from}][$unwind]

The distinction matters more than it looks. A redaction pass can miss a case. A
construction that only ever emits keys has no branch where a value could appear.

What we do send, and why it is enough

Rows returned. Queries issued. Round trips per cursor. Time in your own code
versus below it. The normalized shape of the statement, which is schema rather
than data.

That set is enough to catch a query that went from 30 rows to 30,000, a method
that went from one query to twenty, and a read that started fetching in three
hundred batches. Those are the changes that take production down, and none of
them requires knowing a single value.

The one we could not close

Exception messages. We record the class and the message, and a driver will
happily quote a value back at you: Duplicate entry '[email protected]' for key 'users_email_unique'.

So messages are normalized before they leave the process, with the same rules in
every agent: addresses, URLs, UUIDs, tokens, long hex runs and digit runs are
replaced, and a quoted string survives only if it looks like a program
identifier. That message arrives as Duplicate entry '<value>' for key 'users_email_unique' — the useful half intact, the value gone.

It is a heuristic, and it is deliberately biased towards redacting, because a
lost debugging hint is recoverable and a leaked address is not.