How to Evaluate Vendors

Business intelligence solutions are famously complex. Those assessing BI systems need the means to distinguish necessary functionality from the superfluous. A BI system’s true value is twofold: first, the insight it provides on business operations; and, second, the opportunities it creates for better decision-making.

Understanding the business requirements for a BI project is crucial to a selection strategy that aligns with key business objectives and addresses end-user needs. Failure to integrate business requirements into the selection process can result in serious costs and project failure.

Business intelligence solutions are costly both from a licensing perspective and in terms of the resources needed to implement them. Failing to choose the right one is a cost that IT managers cannot afford. The ultimate determinant of success of a BI solution is end-user adoption and key stakeholder satisfaction. Remember, a system that fails to address business needs will not be adopted by users. Business requirements also have a valuable role in short-listing vendors.

Studies show that organizations that place greater emphasis on the vendor evaluation process tend to have more successful projects. These organizations also place more importance on functionality. The bottom line is that organizations need to select a product that has the requisite functionality to meet business requirements.²²

Business intelligence software selection and implementation are major undertakings. Once the basics have been considered concerning the potential supplier—number of years in business; how long the BI product has been on the market; functionality for the type of business considering the purchase; stability and reputation; ability to service and support the software; and, of course, a road map for the future of the product—an exhaustive requirements analysis should be undertaken. This 10-step process contains the essential steps to purchasing a BI solution:²³

1. Do a comprehensive assessment of your existing business processes and the scale of operations to identify the need for a BI system.
2. Define the objective of your selection process. Considerable background research is necessary before exploring the market for BI software.
3. Form a selection or evaluation committee for carrying out the evaluation process. The evaluation committee should typically include functional experts from different departments, top management (CIO, COO, or IT director), consultants, and key supply chain partners.
4. Define your product requirements in terms of both core functionality and advanced modules and features.
5. Develop pre-evaluation selection criteria to permit the evaluation of the available packages on the same parameters; this will help identify the top potential BI providers.
6. Create a shortlist of four to five BI solutions. Besides using pre-evaluation selection criteria, BI software vendors can also be shortlisted with the help of external consultants by analyzing your organization and identifying solutions that are being used successfully by peer companies.
7. Conduct extensive research on the shortlisted solutions. The selection committee should gather more information about shortlisted packages through independent industry experts, user reviews, and research agencies.
8. Formulate the final selection criteria. Even before marketing executives of BI vendors make presentations for their tools, be ready with a comprehensive list of questions for them addressing all your concerns.
9. Evaluate vendors based on selection criteria and product demonstrations. The vendors’ responses to your questions will help to eliminate a vendor or strengthen its case. You can also prepare minutes of the meeting and make the vendor sign them. This will prevent the vendor from making false claims and hold it accountable in case of failure to deliver promised results.
10. Check for successful BI implementations by the vendor. Before making the final call, visit and consult companies where the selected vendor’s BI software has already been implemented.