The People Who Run Your Systems Don’t Trust Your Vendors

The people who actually run production systems, the ones who carry a pager and answer the 2 AM page, do not trust the vendors they buy software from. The trust gap is not new, but it has gotten wider in…

A single brass handshake with one side held back on dark wood, dim warm amber side light, deep navy shadows, no people, no logos.

The people who actually run production systems (the ones who carry a pager, the ones who answer the 2 AM page) do not trust the vendors they buy software from. The trust gap is not new, but it has gotten wider in the last three years, and it is now wide enough that the average enterprise security team is treating vendor announcements as marketing copy to be verified rather than facts to be acted on. This is a healthy instinct. It is also expensive, and it is not what the vendors want.

The reason the trust gap exists is structural. The vendor incentive is to ship a release on schedule, with the marketing claim that the release closes a particular vulnerability, supports a particular standard, or implements a particular feature. The operator incentive is to know whether the claim is true, on the version of the software they are actually running, with the configuration they are actually using, in the environment they are actually deploying to. The vendor is incentivised to be optimistic. The buyer is incentivised to be sceptical. The incentives on the two sides are not aligned, and the result is a permanent gap in the trust between the people who build the software and the people who run it.

What the operators do not trust

The specific things the operators do not trust, ranked by how often we hear them in the field.

Security advisories that say “patch immediately” without specifying the exploit chain. A CVE number, a CVSS score, and a paragraph of marketing prose does not tell the operator whether the vulnerability is reachable in the operator’s environment. The buyer wants to know: what sits as the actual exploit, what are the prerequisites, and what stands as the detection signature? The advisory that does not say amounts to the advisory that gets deprioritised, because the operator has a hundred of them and the deprioritisation sits as the only way to get through the queue.

Compliance certifications that say “SOC 2 Type II” without specifying the scope, the period, or the auditor. A SOC 2 report that covers only the corporate IT environment and the corporate IT period is not a SOC 2 report for the production environment the operator is running. The buyer wants to know: what was in scope, what was tested, what was the period, who was the auditor, and what were the exceptions? The certification that does not say runs as the certification that does not move the needle on the vendor review.

SLA claims that say “99.9% uptime” without specifying the definition of uptime, the credits, and the carve outs. An SLA that credits the customer with 10% of the monthly fee for a downtime event the vendor gets to define is not the SLA the buyer thinks they are buying. The buyer wants to know: how is uptime measured, what amounts to the credit, and what is excluded? The SLA that does not say serves as the SLA that does not survive the contract review.

Pricing models that say “per user per month” without specifying the user definition, the minimum commitment, and the overage. A per user per month price that bills on the active user in the last 30 days and counts the API integration as 50 users is not the per user per month price the buyer thinks they are buying. The buyer wants to know: what counts as a user, what amounts to the minimum, and what runs as the overage? The pricing that does not say amounts to the pricing that gets renegotiated after the first quarterly bill.

Reference customers who turn out to be the marketing team, not the engineering team. A reference call that the vendor schedules is a reference call with the vendor’s chosen customer, talking to the vendor’s chosen stakeholder, about the vendor’s chosen topic. The buyer wants to talk to the on call engineer at a real customer, about what breaks at 2 AM, about what the support response looks like when the customer is not on the reference list. The vendor that does not provide that call counts as the vendor that does not survive the reference check.

What the operators do instead

Operators have learned, over years of being burned, to verify the vendor claims in their own environment, on their own time, before they trust the claim. The verification looks like this.

The proof of concept, on the production stack. Not on the vendor’s reference architecture, not in the vendor’s free tier, not in the demo environment. On the production stack, with the production data, with the production configuration, for at least a month. The PoC that is not on production becomes the PoC that does not tell the operator anything useful.

The incident test, before the incident. A controlled failure injected into the vendor’s software, at a time the on call engineer chooses, to see how the vendor’s stack handles the failure. The incident test that the vendor does not let the operator run amounts to the incident test that tells the operator everything they need to know.

The exit interview, before signing. A conversation with a customer who has stopped using the vendor. The exit interview amounts to the conversation the vendor does not want to facilitate, and the exit interview counts as the conversation that tells the operator whether the vendor’s lock in is real and whether the support is what it says it is.

What the vendors should do about it

The vendors that are closing the trust gap are the ones that have stopped treating the operators as the audience for the marketing claim and started treating the operators as the audience for the engineering documentation. The vendors that are widening the trust gap are the ones that are still putting the press release in the first paragraph of the security advisory.

The operators have voted with their feet. The vendors that earn the trust get the renewals, the case studies, and the word of mouth. The vendors that do not earn the trust get churned out at the end of the contract, with a post mortem that the vendor sales team will not want to read.

The trust gap is not going to close on its own. The trust gap is going to close because the operators, who are the ones who actually know whether the software works, are the ones who actually decide whether the contract gets signed.

The People Who Run Your Systems Don’t Trust - inline
Key points from The People Who Run Your Systems Don’t Trust

The bottom line

The patterns the post covers have been showing up in production for long enough that the patterns have names, the failures, the mitigations, the gaps. The work the security team and the engineering team and the operations team are quietly doing today sits as the work that decides whether the practice the post names sits as a tool the team uses or a liability the team is paying for.

Sources & Further Reading

All claims in this article are sourced from primary documentation, vendor advisories, and reputable security researchers.

Spotted an error? Email the editor. Corrections are issued with a visible correction note.

Editorial standards. Every article on humanrequired.org is reviewed by a human editor before publication. AI may assist with drafting or research; final editorial control is human. Read the full standards.

Continue reading