How well does the vendor meet the security requirements we set?
Does not meet / Partly meets / Meets / Exceeds / Not assessed
Scored on its own line, so a strong score elsewhere cannot hide a weak security answer.
Third-party vendor evaluation: score criteria one at a time
A third-party vendor evaluation records each evaluator's score for each criterion, the must-have requirements the vendor did or did not meet, and any relationship the evaluator has with the vendor. Write the criteria, the gates and the scale before anyone sees a demo, so the scores judge the vendor against your needs rather than its presentation. Keep the scores separate rather than averaging them into one total, because a strong demo should not hide a weak security answer.
Question bank
Scores for each criterion, not one overall rating
How well does the product cover the workflows we need?
Does not cover / Covers with workarounds / Covers / Covers and extends
Tests fit against your real tasks, not against the feature list in the vendor's presentation.
How clear and complete are the vendor's contract and data terms?
Unclear or missing / Partly clear / Clear / Clear and reviewed by counsel
Separates a commercial question from a legal one. Counsel confirms the reading; the score records your view of it.
How realistic is the implementation plan for our team?
Not realistic / Some risks / Realistic / Realistic and confirmed by references
Judges the plan against the people and time you actually have, which a demo rarely tests.
How well does the vendor support customers after the sale?
Not confident / Somewhat confident / Confident / Very confident
Support quality is hard to see in a demo. Ask the references, and score what they said.
Field blueprint
Eleven fields for one evaluator, one vendor
| Field | Type | Required | Why you ask it |
|---|---|---|---|
Your name | Short text | Required | Each score belongs to a named evaluator, so the panel can see who judged what. |
Vendor name | Short text | Required | Use the legal name from the contract, so the scores match the vendor record. |
Your areaProcurementSecurityProductEngineeringFinanceOperationsOther | Dropdown | Required | Each area scores what it can judge. Security should not be scored by the product team alone. |
Do you have a personal or financial relationship with this vendor?NoYes, I will explain below | Single choice | Required | Declares a possible conflict before the scores are compared. A yes does not disqualify you, but it has to be visible. |
Explain the relationship | Long text | Optional | Only needed if you answered yes above. Keep it factual. |
Does the vendor meet every must-have requirement?YesNoNot yet known | Single choice | Required | A no stops the evaluation whatever the scores say. Write the must-haves in the evaluation plan before anyone scores. |
Which must-haves are not met yet? | Long text | Optional | Names each gap, so the vendor can be asked about it and the decision can be traced later. |
Link to the vendor's security documentationLink to a document in a store your team controls, not a copy of it. https:// | Short text | Optional | The evidence behind the security score. Link the document rather than copying its contents into the form. |
Notes for the panelDo not paste the vendor's confidential material here. | Long text | Optional | A sentence or two that explains a score, so the panel can read it before the discussion. |
RecommendationProceed to contractProceed to a pilotDo not proceedNeed more information | Single choice | Required | Your recommendation, kept separate from the scores, so the panel can see when the two disagree. |
Date of this evaluation | Date | Required | Scores go stale when a vendor changes its product or terms. A dated record shows which version you judged. |
At a glance
How long one evaluator's form takes
11Fields
7Required fields
5Estimated time (minutes)
Estimate based on question types; actual completion time varies.
Read the results
Compare evaluators before the discussion
Compare criterion by criterion
Average each criterion across evaluators, then look at the spread. A wide gap on one criterion can mean people read the vendor's answer differently, and that disagreement is the discussion to have.
Read the gates before the totals
A vendor that fails a must-have is out, whatever its scores. Check the gate answers first, then compare only the vendors that passed on the criteria.
Read recommendations next to scores
When a vendor scores well and an evaluator recommends against it, the notes should explain why. Read them before you choose between the two signals.
Keep the method fixed for every vendor in a round
Use the same criteria, scale and weights for each vendor in the same round. Changing the scale halfway through makes the comparison meaningless.
Mistakes to avoid
Mistakes that make a vendor choice hard to defend
Using one overall score
A single total hides a failed security answer inside a good demo. Score each criterion on its own, and keep the gates separate from any average.
Writing the criteria after the demo
Criteria written after a persuasive demo tend to reward what the vendor showed. Write the criteria, the gates and the weights first, then score.
Expecting the form to add up the scores
This form does not calculate totals, weights or averages. Export the responses to CSV, and calculate them in a spreadsheet with the same weights for every vendor.
Copying the vendor's confidential material into responses
Price sheets, security reports and roadmap slides belong in a store your team controls. Link to them, and keep their contents out of the responses.
Skipping the conflict question
A relationship nobody declared tends to surface after the contract is signed, when it is much harder to explain. Ask the question on every evaluation.
Frequently asked questions
What should a third-party vendor evaluation include?
Include the vendor name, the evaluator's area, a declared relationship, the must-have gate, a score for each criterion, links to the evidence, and a recommendation. Score security, workflow fit, contract terms, implementation risk and support separately, so each one can be discussed on its own.
Is there a vendor evaluation template I can download?
What this guide offers is the criteria and fields, not a Word or PDF file. Each evaluator submits separately through a shared link, and the CSV export feeds the comparison.
Can the form calculate weighted scores?
No. The form has no calculated fields, so weights and totals are your job. Export the responses to CSV and calculate in a spreadsheet, using the same weights for every vendor.
Should evaluators score in a meeting or on their own?
On their own first. Each evaluator submits a form before the group discusses the vendor, so the first person to speak does not set everyone else's view. Then the group reads the spread of scores and talks through the criteria where they differ.
Browse by category
More surveys in Product & Engineering
Surveys
Product Feedback
Surveys
Sprint Retrospective
Applications
User Research Screener
Requests
API Access Request
Requests
Architecture Review Request
Applications
Beta Tester Signup
How to build it
Choose a category and a form type in the wizard. Answer three Atty Suggestions questions, then give the form a title. Atty builds the fields in the background. Edit the fields and ask Atty to refine them, then publish and share the /f/ link.
