Methodology

How we rank, and what we will not do for money.

Every ranking on this site is built the same way, from evidence a reader can check themselves. This page is the standard those rankings are held to. We rank two different things, the agencies you hire and the tools you buy, and they are judged on different criteria, because what makes an agency worth hiring is not what makes a tool worth paying for.

Every agency ranking scores the same six things

The weights change between categories. The criteria do not. That is what makes two agency rankings on this site comparable rather than two separate opinions.

Independent recognition. Awards, evaluations and accreditations from a body that is not us and not the firm. Verified on the awarding body’s own page or the firm’s own announcement, never on another ranking.

Client evidence. Who the firm has actually worked for, named on its own site, and how relevant that work is to the list you are reading.

Specialisation and fit. Whether this is the firm’s core business or one service line among many, and whether how it works suits the buyer this list is written for.

Commercial transparency. Whether the firm publishes what it costs. Not whether it is cheap.

Tenure. How long it has been trading.

Documented outcomes. Results the firm has published and attached to named or specifically described work.

Every tool ranking scores the same five things

You are buying a product, not hiring a team, so the questions are different ones. The weights change between categories; the criteria do not.

Core capability. Whether it does the category’s core job as well as or better than the standard, including whether it was built for that job or had it added to something else. A purpose-built tool and a suite with the feature bolted on are not the same product, and the difference usually shows.

Value and real cost. Whether the price is published at all, and what it actually costs at realistic usage: add-ons, base plans a feature quietly requires, capability held back for a higher tier. Not whether it is cheap. An advertised entry price nobody actually pays is worse than no price.

Ease of use. Setup, daily use, and how long it takes before the output is worth anything. This is the one criterion a vendor’s own website cannot answer, so we score it only where we have used the product, and we say plainly when we have not. Where nobody on the team has run the tools in a ranking, it is left unscored and the other four criteria carry the weight proportionally, rather than a trial length being substituted for an answer.

Integrations and extensibility. What it connects to natively, whether there is an API, and whether you can get your own data back out of it.

Support and documentation. Whether the documentation answers real questions, and which support channels your tier actually buys.

We do not score tools on awards or on how long the vendor has existed. Whole software categories are younger than their first award programme, and scoring on age would rank the calendar rather than the product. A new tool is not marked down for being new. It is marked down only if it does less.

We do not score tools on their customer logos. A recognisable name on a homepage tells you that company signed a contract once, not that the product is good, and logo walls go stale for years. Named customers still appear in a review as context. They do not earn points.

Why the two sets are not the same

An agency is a group of people, and the risk you are managing is whether they will do good work for you specifically. So the evidence that matters is what they have already delivered, for whom, and for how long.

A tool is a product, and the risk you are managing is whether it does the job and what it will really cost. Its track record matters far less. You can cancel a subscription in a month, and a two-year-old tool can be better than a ten-year-old one.

Forcing both through one scoring sheet would mean either judging tools on their age or judging agencies on their feature list. So we publish two sets, and every ranking says which one it used.

Why the weights differ, and where to find them

A criterion only earns weight where it can actually separate one option from another. Where the evidence for a criterion barely exists across a category, weighting it heavily would rank that category’s disclosure habits instead of its firms or its products.

So the weights are set per ranking, and every ranking states its own weights at the top of the page, before the list starts. If a ranking has adjusted the weights for its category, it says what it changed and why it changed it. You should never have to come to this page to find out how a ranking you are reading was scored.

What does not move is the criteria themselves, and the rules below.

How a claim becomes a fact on this site

Awards and accreditations are verified at the source. On the awarding body’s own page, or the firm’s own announcement of it. Never on another ranking or directory. That rule exists because we have repeatedly found errors circulating between competing lists, including agencies placed in the wrong country by pages that rank them.

Facts about a firm come from that firm’s own site. A competitor’s list is somewhere to discover a name, never somewhere to learn a fact.

Prices are only printed when the firm published them. A figure the firm states on its own site is used as given. A figure it has self-declared on a third-party profile is used and attributed to that profile. Our own estimate is never printed as a price and never earns transparency credit. If a firm publishes nothing, its entry says so, because "ask them" is the honest answer and a guess would not be.

Absence of an award is not evidence of weakness. Most award and evaluation programmes require entry, and often payment. A firm that does not enter cannot win. Each ranking says so on its own page.

A vendor’s own comparison page is not evidence. Every tool in a contested category publishes a page explaining why it beats the others. Those pages tell you what a vendor thinks its differentiator is. They do not tell you whether it is true.

Software prices carry the date we checked them. Pricing changes faster in software than in anything else we cover, and an undated price is a claim about today that was true in March. Each entry also says when it was last verified.

A withdrawn product is removed, not repriced. Where a product has been folded into a parent plan or discontinued, the entry says so. We have already found every page-one competitor in one category quoting a standalone price for a product that no longer exists.

Where a category needs a different rule

The default standard for client evidence is at least three verifiable examples of relevant work. One category cannot meet it for a legitimate reason, and rather than quietly scoring those firms down, the rule changes and the change is stated on the page.

Cybersecurity. Naming clients is against policy for most security firms, and for good reason: it advertises who runs which defences. So for security companies the criterion accepts anonymised but specifically described engagements, published threat research, public detection engineering and disclosed vulnerabilities. A firm is not penalised for a confidentiality practice its buyers actively want.

Any future deviation follows the same rule. It is stated on the ranking that uses it, not applied silently.

What these rankings cannot tell you

Worth being blunt about, because no ranking of this kind can measure it. Nothing here observes whether a project ran to time, whether the budget held, whether the team that pitched is the team that delivered, or how the handover went. For security firms, nothing here observes whether the analyst escalated the alert that mattered at three in the morning.

For tools, nothing here observes how one behaves six months in. Whether it stayed up, whether support answered on your tier, whether it still does what it did when you signed, whether your data volume broke it, or whether the feature you bought it for is still on the roadmap.

Where we have used a tool ourselves over a real period, its review says what we used it for, how long, and when. That is the only part of this site that speaks to living with a tool rather than evaluating one, and it appears only where it is true.

What a ranking answers is which firms or products are strongest against evidence you can check. Deciding which is right for you is a different question, which is why every list ends with a section routing by budget, timeline and the actual constraint you are working under.

Every entry carries a stated limitation. An entry with no downside tells you nothing, and it is the clearest single sign that a placement was bought.

What money can and cannot buy

Position is not for sale, at any price. There is no rate for a higher placement, no package that includes one, and no arrangement under which a payment moves a firm up a list.

Paid placements are labelled on the entry itself and carry the link attributes search engines expect. A reader can always see which entries involve money and which do not, without reading the small print.

Being listed is free and is not requested. Firms appear because our research put them there. Nobody has to know we exist to be included.

These rules apply to every ranking on the site, whether or not any money is involved in it.

Corrections, and what happens if you disagree with your placement

We fix factual errors. Wrong founding year, wrong location, a service we misdescribed, an award we missed or recorded incorrectly. Send it with a source on your own site and it gets corrected.

We add evidence as it appears. A newly won award, a newly published price, a new accreditation. Once verified, it counts, and it may change where a firm sits.

We do not move a placement on request. Not for a client, not for a payment, not for a persistent email thread. The way to move up is to hold evidence that scores higher against criteria that were published before anyone asked.

We will tell you why you placed where you did. Every entry has an internal note recording what decided its position. Ask and you get a straight answer rather than a brush-off. That note is not published, because an argument about ordering is not what readers came for.

Exclusion on integrity grounds is rare and it is not a ranking judgement. Where we cannot substantiate what a firm claims about its own work, it is left off rather than ranked low. That is a decision about evidence, not about quality, and it is reversible the moment the evidence stands up.

Who writes these

Rankings are researched and written by a named author whose byline and last-updated date appear on every page. Dates reflect real revisions, not a refreshed timestamp.

Where demand for a topic is unproven, the ranking says so rather than implying an authority it has not earned. A page built because a category needs covering is a different thing from a page built to answer a question people are demonstrably asking, and the difference is disclosed.

Corrections and new evidence are welcome from any firm we have written about, whether or not it likes where it landed.