Technically, anyone can write a survey. They can sign up for SurveyGizmo or SurveyMonkey (neither sponsoring this blog), or any other similar website. They can write a few questions and read over the answers. But what happens when they realize the questions were not asked in a way to gather the answers they wanted? What happens when more respondents provided data than expected and no one on staff has time, skill, nor the proper software to analyze all that data? And how do they make sense of it to their board of directors, customers, or other stakeholders? The challenge with "big data" is that even small data files can be cumbersome and lose integrity in programs like Excel. What often happens is that the staff, scrambling to reliably turn numbers into talking points for the next board meeting, turn to a consultant who is brought in to save the day, and maybe even the CEO's job.
Contracting with an external researcher for evaluation studies from the start of the process has several advantages. First, an outside evaluator provides a neutral, third-party, and independent perspective to create survey, interview, or focus group questions. Second, outsourcing your survey writing, programming, data collection and management, and data analysis and reporting frees up internal staff time to focus on programmatic tasks and goals, and ultimately your mission. Further, the evaluator can provide methodological and statistical expertise to expand the capacity of a full-time staff. Last but not least, a third-party evaluator might be required by your funding source(s).
Neutrality
Most organizational data can be manipulated in some way to support a predetermined goal or meet the assumptions of a stakeholder. By contracting with a professional researcher, they can approach the process with less bias - from the way they help you word questions or the sampling and data collection process, to conducting analysis using methods that minimize biased results. Ideally, finding someone who is familiar with your organization's content area - but less married to its mission - provides a consultant who can approach the data in context and process the results in a way that is helpful and meaningful - yet neutral - to your staff and stakeholders.
Time is money
This oldie but goodie phrase is applicable to organizations and schools who focus their resources on their programmatic goals, be that fundraising, providing formal or informal educational opportunities, or serving niche at-risk populations. Typically more than money, their resource is time, a scarce commodity among a small staff or staff whose plates are quite full. Delegating the time for evaluation tasks to someone else who can carry that weight efficiently and effectively is a smart strategy.
Data dilemmas and statistical nightmares
It's probably safe to say most people do not love methods and statistics nor have any interest in calculating them. So questions about data use can arise, such as these:
"What do we do with surveys that are half complete?"
"What about those who seemed to answer all "1" to finish and get the gift card?"
"How will that affect our statistical findings?"
Delegating these dilemmas to someone with the specific technical talent to work them out is only part of the advantage. Yes, your team can go home for dinner in the evening instead of being trapped at work staring at a seemingly endless spreadsheet. Most importantly though, an evaluator also can be a thought partner in how to best conduct pre- and post-survey methodology to enhance rather than distract from programmatic goals. They can advise on how to gather a representative focus group, and conduct interviews with stakeholders.
Funding requirements
Government, foundation, and donors' funds typically are awarded with strings attached. Why would anyone pour all this funding into programs with little to no evidence that they are effective? And would they trust the implementation staff to remain unbiased in their evaluations? They don't and they don't. The same way a person usually does not diagnose their own ailments; instead, they go to a third-party professional, a medical doctor who will gather the information and tests they need to make an informed and usually unbiased decision on what might be wrong. In the same way, funders often require that programs partner with a third-party to provide all evaluation services to help identify what's working, and what might need improvements or changes.
Take away
Whether your organization needs neutrality, increased manpower, extended expert capacity, or to meet funding requirements, a research consultant with content knowledge of your industry or mission is key to addressing your evaluation needs and meeting your mission. ||
Ariela Greenberg, Ph.D. is CEO and Founder of The Greenberg Team Research and Evaluation services. Dr. Greenberg is a researcher with a content area focus in a variety of educational topics including but not limited to STEM, early childhood, school leadership, formal and informal instruction and learning. She also specializes in helping Jewish schools and non-profit organizations meet their missions through data-driven decision making. See the Research and Evaluation Services page for more about The Greenberg Team can help you meet your mission.