Software Evaluation Project: Selecting a DS Data Management System

A Case Study from the University of Nevada, Reno

Summary: Have you ever asked colleagues, “What data management system do you use?  What’s good?” See how a collaborative group at University of Nevada, Reno used data and a system to guide their decision-making as they explored software options for the disability resource office data management system. This case illustrates a starting point for looking into data management software.  Even with a Vendor Rubric in place, this can be a challenging process.

Background

I work in Enrollment Services, and along with other responsibilities related to policies and procedures on campus, I coordinate our Electronic Information Technology committee, and work closely with the DRC (Disability Resource Center). About a year ago, the DRC indicated they needed to schedule exams and have a better calendar system.  We were using five disparate systems to manage data and sharing Outlook calendars among twenty people. Clearly this was not working well.

Because of my background in IT, I was familiar with system analysis and software evaluation. Together, we sat down and decided to look at what options exist. First we had to answer the question, “What is it we are trying to fix? What is the problem we are trying to solve?”

Identifying Needs and Getting Data from the Vendors

I wasn’t familiar with any of the products that were available on the market to manage the day-to-day operations of a DRC office, so I researched the products that UNR was considering. I used some online resources (see Mary’s list of resources below) to help me draft a spreadsheet with general categories of things I thought should be included and I asked my colleagues in the DRC office, “Are these the software features you’re looking for?”

We gathered some research and I spoke with some people who have varying job functions in the DRC office for some specific information. “What tasks do you need to do in your job specifically? What three things cause the most problems for you? Do you have suggestions for how to fix those problems? And if we haven’t fixed them, why do you think that is?”

We put all this data into a spreadsheet.  We knew we needed to buy software that would address as many of our problems as it could and that it had to integrate well with our existing campus student information system and our IT infrastructure. It also needed to be a thoughtful purchase. The software didn’t cost so much that we needed to extend an RFP (Request For Proposal), but never the less, we strive to be good stewards of our resources, so we chose an organized approach.

We formatted the spreadsheet to work like a rubric and sent it to the vendors of the products we were considering, asking them to fill it out and send us their replies.  Fortunately, the vendors were eager to cooperate with this request.

Sorting Through the Vendor Feedback

Once we heard back from each of the vendors, we analyzed the results. We looked at which features were critical to us and which were optional. We knew from our earlier conversations that calendars and scheduling exams, auxiliary aids, and alternative testing rooms were the biggest roadblocks right now and must be critical features of the software we would buy. Having had experience working in the IT division, I knew that a purchase this size would be considered fairly small in the larger context of the University and had the potential to be overshadowed by competing demands on the IT department. It was important to purchase a software package that could be outsourced. On the rubric we asked each vendor to indicate if their company hosted the software product or if it had to reside on our campus server.  If the software required a campus-based server (i.e., could not be outsourced), we eliminated that product from our list. Cost was of course another critical feature.  We had to think about budget for the next 3-5 years, not simply the initial purchase.

Now, I love my checklists, spreadsheets, and rubrics, but we needed to remind ourselves that the vendor rubric is largely a wish list. We couldn’t expect to find a product that offered everything on our list. As we looked at the vendor feedback, one product rose to the top fairly quickly. It was a hosted solution, met our needs for data access security, integrated with our SIS, had features to schedule auxiliary aids and track case notes, had an online request form and a robust reporting mechanism. Although we liked these features, this software didn’t have the calendaring functionality we needed so we were faced with compromising our most desired feature. We decided to look around campus and talk with other units that might be using similar calendaring systems. We found that our Continuing Education Division was already using a system that would work well for our calendaring needs so we simply extended our license to include the DRC office. It was a great way to leverage our resources; we had in house training and the system had already been integrated and tested on campus. With the calendaring issue addressed, we were free to select the data management system that best met most of our remaining needs.

How Effective was the Process?

Developing the vendor rubric helped to focus the search and rank features.  Including everyone in the process increased the likelihood that we would get a product that met most of our functional needs and that adoption would be successful.  Collaborating on the rubric was incredibly helpful to the process. The DRC staff brought valuable information from their professional network of colleagues and listserv contacts and experience and observations about the vendor demos. IT staff included questions to ensure we had enough information early in the process to determine if the products under consideration would integrate efficiently into our environment. Everyone’s collective expertise was necessary for the rubric to most useful.

Reflections

So did everything run smoothly and result in a quick purchase?  Absolutely not. This is higher education, after all. We faced some typical challenges along the way. Getting a group of this size together to make a unanimous decision in a short timeframe is nearly impossible. We embarked on this project in March with the intent of signing a contract in June, the end of our fiscal year. We lost our funding that year because we weren’t able to be as thorough as we wanted and do that in three months. We do, however, know exactly what we need and have identified a solution so when funding becomes available again we can act quickly and confidently.

Advice

This experience exposed some thoughts to remember for our next project, which we hope will be implementing a data management system.

Executive sponsorship and funding:  Ideally this will be in place before putting much time and energy into a project. However, sometimes we must design a plan to secure funding and support. We now have that plan.

Timing: This was a process that demanded stamina, consistency and coordination. I know we were not unique in our attempt to act quickly and come to a decision by the end of a fiscal year. Making an informed decision was more important than making a hasty decision. We took that time; about three months to select a product. We should expect to plan six months for the implementation phase.

Leadership: Having a project manager or point person to coordinate vendor demos, meetings, project requirements, etc. was important. Ideally this person will be familiar with policies and procedures in the office (DRC in this case), have knowledge of the procurement process, and be familiar with technology.

Resources That Helped Along the Way

Infotivity User Needs Survey, http://www.infotivity.com/free-system-planning-tool.html[infotivity.com]

Checklist for Evaluating Tech Tools, Apps, Software, and Hardware, http://techpudding.com/2011/04/04/checklist-for-evaluating-technology-software-and-applications/[techpudding.com]

IT Proposal / Software System Selection Tools,
http://www.axia-consulting.co.uk/html/download.html[axia-consulting.co.uk]

Interested in Learning More?

Here’s the rubric that they used at UNR: http://ahead.org/isp/campus-data/data-management/vendor-rubic

 

Thanks to Mary Heid for contributing this case study

Contact Information:

Mary Heid
Coordinator, Assistive Technology
System Administrator, Enrollment Services
University of Nevada, Reno, NV
mheid@unr.edu