The fairest way to compare privileged access management products is to run the same tasks, with the same acceptance criteria, on every product on your shortlist. Feature matrices tell you what vendors say. A pilot tells you what the product does in your environment.
Established vendors compete on breadth, customer history and integrations. Newer ones compete on a specific improvement. Either way, the decision should rest on evidence you gathered yourself. This guide gives you a structure.
Start from your problem, not a feature list
Write down the two or three problems that justify the purchase, in terms a business owner understands:
- "Contractors have standing access to production."
- "We cannot show an auditor who did what on our database servers."
- "Service account keys have not been rotated in years."
- "Our MSP engineers share credentials."
Every pilot task should trace back to one of these. A feature that solves none of them is not a reason to buy.
What the same tasks look like
Run each of these on every shortlisted product, with the same environment and the same people.
| Area | Task to request | Pass criteria |
|---|---|---|
| Access workflow | Request, approve and grant time-limited access to a real system | Reason recorded, a different person approves, access expires on its own |
| Scope accuracy | Generate and review the permitted scope for real maintenance tasks, including vague and incomplete justifications | Scopes are sensible, reviewable and editable, and vague requests are visible |
| Enforcement | Run a permitted action and an out-of-scope action, separately, on Linux, on a database, and on a Windows desktop | Know exactly what is blocked, what is warned, and what is only recorded, per protocol |
| Operational impact | Run a week of real tasks | Count false alarms, legitimate tasks blocked, review effort and time to approve |
| Credential handling | Connect to a target and try to find the password | The user never sees or can copy it |
| Machine credentials | Rotate a service account password, a cloud key and a token, with an application consuming each | No outage, a grace period, and a rollback |
| Private AI | Disable outbound connectivity, then use every AI feature | Know which features work offline and which do not |
| Evidence integrity | Export evidence, verify it offline, change a file, and verify again | The change is detected without the vendor's console |
| Audit sample | Pick sessions at random and trace request to approval to activity to expiry | The full chain is available for each |
| Integrations | Connect your identity provider, ticketing and SIEM | Works with your versions, and fails safe when a system is down |
| Production operation | Take a backup, restore it, fail over a node, and rehearse emergency access | Documented, repeatable, and within your recovery targets |
| Multi-tenancy (if relevant) | Create two tenants and try to cross between them | Separation holds |
Separate what is built from what is claimed
Ask the vendor to mark every capability you will rely on as one of:
- Generally available and in production use.
- Newer, available to evaluate but with a smaller installed base.
- Planned, not in the product today.
Then test only what is built. Be especially careful with phrases such as "AI-driven", "zero trust" and "autonomous", which cover very different things from vendor to vendor. Ask for the exact behaviour on your protocol and your target.
Questions to ask every vendor
- Which protocols and targets are enforced preventively, and which are only recorded?
- Where does AI run, and does every AI feature work with no outbound connection?
- How is evidence protected from tampering, and can an auditor verify it without your tooling?
- What does deployment need: appliances, specific operating systems, database licences?
- What are the recovery steps if the vault's master secret is lost?
- Which certifications or validations do you hold today, and which are in progress? (Ask this plainly; some procurement processes have hard gates.)
- How many customers use the capabilities in this pilot, and can we speak to one?
- What is the support model, and what does it cost?
- What are the exit terms and how do we export our data?
Pricing: ask for quotations, not list prices
- Get a quotation for the modules you actually need, not a headline price. PAM products are often sold as separate modules for vaulting, session management, endpoint control and analytics.
- Count the platform cost around the product: servers, databases, licences for the operating system or database engine it requires.
- Count the operating effort: people to run, upgrade, back up and tune it.
- Compare over three years, including renewals and growth in users and targets.
- Do not assume any vendor is always cheaper or always easier to run. The evidence does not support that; your quotations will.
Scoring
Weight the areas to reflect your problems, then score each vendor 0 to 3 per task, with evidence attached:
| Score | Meaning |
|---|---|
| 0 | Could not be done, or failed |
| 1 | Done with significant workarounds or vendor help |
| 2 | Done as expected |
| 3 | Done well, with clear advantages |
Record who ran each task and how long it took. Keep screenshots and exported evidence in one folder per vendor.
Common mistakes
- Comparing slide decks instead of running tasks.
- Letting the vendor run the pilot rather than your own staff, who will operate the product afterwards.
- Testing only the happy path. Include a failure: a wrong password, an out-of-scope action, a node going down.
- Ignoring operational effort, which usually exceeds the licence cost.
- Deciding on breadth when your problem needs depth in one area.
- Skipping the reference call with a customer of similar size and environment.
How ZenXPAM approaches an evaluation
ZenXPAM welcomes this kind of side-by-side pilot. A production-grade pilot runs on three VMs inside your own network against one high-value use case, with your identity provider, ticketing and targets. We will walk you through what is preventive and what is recorded on each protocol, run the AI features with outbound connectivity disabled, export and verify a signed evidence pack offline, and rehearse a backup restore. We would suggest running the same script against every product on your list, including ours. Compare ZenXPAM with conventional PAM, see pricing, or book a pilot conversation.
FAQ
How long should a PAM pilot be? Four to six weeks is usual: one to set up, two to three of real use, and the rest to test failure and recovery.
How many vendors should we pilot? Two or three. More than that dilutes your effort and your staff's attention.
Who should run the pilot? The people who will operate the product, with a business owner for the problem you are solving.
Is a feature comparison useful at all? As a first filter, yes. Treat it as a way to shorten the list, not to choose.
What if a vendor will not run the same tasks? That tells you something. A product that cannot be tested against your scenarios is a risk.