Evaluation Criteria for a System Selection (or any selection!)

Whether you are selecting a new software package or a new car, or a quart of ice cream, it is important to have a set of Evaluation Criteria against which to make your choice.  You need to identify which attributes of whatever you are buying, before you try to make a decision on y0ur purchase. (And although this advice is useful for the car or ice cream, I’ll just focus on the software package from here on.)

And there are typically two levels of criteria. The first level is the “GO-NO GO” criteria. If a prospective vendor doesn’t fit even one of these criteria, the vendor is out. For example if your budget is under $20,000 for the software and implementation, you know that will rule out many vendors, so don’t even bother sending them an RFP. Or if your IT standards require a MAC solution or an Adabas Database, limit your RFP to only vendors which are compatible with those criteria. Note for example that if you had those two mutually exclusive criteria (i.e., Adabas doesn’t run on a MAC) you’d have very few (None?) bidding on your requirements.

A visit to the vendors web site or your Selection consultant can be helpful here. Several of these “Pre-Criteria” might include:

  • Value/Cost Range (in addition to the budget example above, if you have a sizeable budget, you will not be considering an “App” from the iPhone or Google app store.)
  • Vendor financial viability & track record in the market (for example, this may rule out vendors which have only come on the market in the last 6-12 month, or a vendor you know that has recently filed for bankruptcy.)
  • Support for new functions that your users require (i.e., you need more than “contact management software for event management or capital campaign reporting .)
  • The software enhances productivity
  • Reliability of the software and hardware

With the criteria above driving which vendors you may consider in your bid/selection process, once you have sent out an RFP or Requirements document, you will need a more detailed set of criteria to evaluate 1st the vendor proposals, and 2nd their scripted demonstrations to your users. These second level criteria may include:

  1. Specific Functionality (e.g., are you looking for just basic “CRM” capabilities, OR are you looking for CRM, plus major gifts fundraising, direct marketing fundraising, any kind of membership support, events management, volunteer management, etc.)
  2. Querying and Reporting – some of my clients have bundled this with Specific Functionality above. I suggest having it as a separate criterion. And “reporting” should include all outputs – printed reports, dashboards, exports to Excel or Word Processing, .PDF  files, etc.
  3. Ease of Use – Is using the software intuitively obvious or does it require significant training? Does the system have good documentation including user manuals and context sensitive help?  Is data logically organized? Is the menu/tab system friendly? In general, is there a low level of demand for added technical resources/
  4. Productivity – the system increases the productivity of the staff in measurable ways. This goes beyond ease of use above.
  5. Flexibility – the system offers options within the functions allowing for addition of new applications in the future. For example, the system has a robust database with no (practical) limits to attributes, codes or fields for data capture, including user-definable fields. It has a rich comments capability.
  6. Vendor Support and responsiveness – this may be the vendor alone, or a wide base of consultants using the system, and a good user community. Training quality/quantity is sometimes included here and sometimes as a separate criterion. Vendor history of providing regular upgrades and enhancements.  There is a vendor hot-line group which is responsive.
  7. Integration of Packages – The objective is to experience “seamless integration” of user packages. Any user should be able to access all data bases from a single access point. This would also include the availability of industry-standard, end-user analysis and reporting tools.
  8. Future-oriented technology
  9. Technological Compatibility
  10. Speed/Timeliness of Implementation

Note that some of these criteria may be evaluated as part of the proposal review, others may be observed in the software demonstration, and still others should be confirmed in contacting references or participating in site-visits. Much more detail is incorporated into an example for a client; I wanted to introduce the concept here on my blog.

Over the years I have gotten a sense of which criteria should be (and typically are) weighted higher or lower and a reasonable range of values. The weightings are determined by the client and are often a reflection of the pain points from their prior software. I try to keep them each in a reasonable range with the total hitting 100%.

So which criteria are more important in your car purchase – the potency of the new car smell or the gas mileage and service record? By establishing the criteria up front the final decision is more objective. And if the selection criteria are agreed upon by stake holders, the actual choice of a package is more straight-forward. It is then driven by the users and not by the vendor’s dog and pony show which typically focuses on known strengths and ignoring weaknesses and omissions. And for ice cream, does it have chocolate chips or not?  That’s a Go-NoGo.

I’d welcome any comments/questions via e-mail at CharliesBlog@riarlington.com  And please sign up for my periodic blog posts in the space in the column on the right!

SDG, Charlie Hunsaker


This entry was posted in Blog, Fundraising Software, System Selection and tagged . Bookmark the permalink.