How to build ATS requirements and evaluation criteria

Here’s what happens to most requirements documents: they start as focused lists of real problems, then grow into 47-page monsters that include every feature anyone’s ever mentioned. By the time you’re done, you’re not evaluating ATS systems. You’re looking for unicorns that don’t exist at any reasonable price point.

Keep your requirements ruthlessly practical. For each item, ask: “If this feature didn’t exist, would our hiring process break?” If the answer is no, it’s not a must-have. This doesn’t mean nice features aren’t valuable, but be honest about what’s essential versus what’s aspirational.

Weight your requirements based on impact, not excitement. Advanced AI features might be fascinating, but if your biggest problem is recruiters spending 2 hours a day on admin tasks, prioritize workflow automation over machine learning algorithms. Solve your actual problems first.

Consider the 80/20 rule: 80% of your hiring benefit will come from 20% of available features. Identify that crucial 20% and make sure any system you consider handles it brilliantly. Everything else is secondary.

Most ATS evaluations follow the same pattern: watch demos, compare feature lists, negotiate prices. This approach misses crucial factors that determine real-world success. You need to evaluate not just what the system can do, but how well it’ll work in your specific environment.

Start with elimination criteria. Create a shortlist of 3-5 vendors maximum. Any more and you’ll suffer from decision paralysis. Any fewer and you might miss better options. Use your must-have requirements to eliminate vendors quickly. Don’t waste time on detailed evaluations for systems that can’t meet basic needs.

Once you’ve shortlisted your vendors, you can then focus on demos and hands-on testing.  If you’re running a formal RFP process, you might want to see initial demos and then issue your RFP to better understand available solutions, then conduct final demos after receiving proposals. Both approaches work — choose based on your organization’s procurement requirements and timeline constraints.

Go beyond standard demos. Most vendor presentations show ideal scenarios with perfect data and simple workflows. Ask to see how the system handles your specific edge cases: What happens when a candidate applies to multiple roles? How do you manage interview scheduling across time zones? Can you bulk-update candidate statuses when requirements change?

Establish clear decision-making authority before you start evaluations. Who makes the final call? What happens if stakeholders disagree? How do you handle situations where no option is perfect?

Consider using a structured decision-making approach like a weighted scoring matrix, but don’t let it become the only input. Numbers can’t capture everything that matters. Cultural fit, vendor relationships, and gut feelings about long-term partnerships all matter too.

Create a scoring system that reflects reality. If candidate experience is crucial for your employer brand, make it worth more points than advanced reporting. If your hiring managers are drowning in volume, prioritize automation over customization.

Set clear evaluation criteria upfront: what does success look like? How will you measure it? Most teams focus on features during selection but judge success on outcomes later. Bridge that gap early.

Build consensus where possible, but don’t let perfect be the enemy of good. If you’re 80% aligned on a solution that meets your core needs, move forward. Waiting for 100% agreement often means missing implementation windows or losing momentum.

Document your decision rationale. Future you will thank you when someone asks why you chose system A over system B. More importantly, clear reasoning helps you evaluate whether your decision was sound if problems arise later.

Remember: the perfect ATS doesn’t exist. You’re looking for the best fit for your specific situation, not the highest-scoring system on a generic checklist.