Making Your Investment Count: Developing Cost-Effective Distance Learning Programs
This abstract was prepared for the Online Educa Berlin 2001 conference.
The successful management of telecommunicated learning programs requires analysis on many levels: needs assessment, market analysis, cost feasibility, assessment of student readiness, evaluation of cognitive and behavioral gains, assessment of satisfaction with programs, technology and services offered. Far too little attention has been given to cost-effectiveness of distance learning programs. This article provides an overview of (1) the roles of various assessments and analyses in the pre-development, mid-development, and full implementation stages of distance learning projects; and (2) cost analysis strategies and their functions.
Assessment and evaluation, defined as a continuous process for the improvement of the teaching-learning process and improvement of the instructional delivery systems, are reviewed in answer to the question, "why evaluate the distance learning program?" Within a multi-faceted evaluation strategy, cost effectiveness evaluation can be thought of as the fifth level of evaluation and a complement to the evaluation of satisfaction, learning, performance and demonstrated results.
Hezel Associates’ Business Model for Higher Education provides the distance education administrator with a means of supporting the planning process to increase the likelihood of satisfactory financial performance of the distance learning initiative. The Model is designed to be an interactive planning tool that allows the user to create multiple scenarios based on different planning assumptions.
Evaluation Elements for Cost-Effectiveness Analysis
The principal measures are organization-centered and customer/learner-centered. We want to know how the organization responded to the demands of distributing education. How well did the organization market and deliver content to those who needed it? To what extent did the management change or adapt to the delivery of distance learning? How well did the organization use funding available for the distance learning enterprise? Those are just a few of the evaluation questions focused on the organization.
The external, or learner oriented, questions apply to the uses and effectiveness of the distance learning effort. How and to what extent did learners make use of the distance learning content and service? How satisfied were the learners with the instruction and delivery? What information, concepts, or analysis did they learn from the content? What behavioral skills have they acquired as a result of the distance learning service?
Financial Viability and Business Planning
Just as the needs assessment is conceived as an up-front analysis of content needs and market demand, the financial feasibility analysis and business plan are management tools to anticipate the cash requirements of the distance learning initiative. The feasibility analysis and business plan should answer the following questions: How will the distance learning program be supported? How strong is the market for the program? What will be the capital/start-up costs? What are the operating costs? What is the source of revenue (e.g., grants, tuition)? And when is the breakeven point, the point at which the revenues begin to cover all the expenses?
Is Distance Learning Worth the Expenditure?
Four methods of analysis to help participants make better investment decisions: these methods were first discussed by Levin (1981) and include cost effectiveness, cost benefit, cost feasibility, and cost utility analyses. Cost effectiveness analysis requires precise evaluation through which to gain a consistent method to measure effectiveness. Cost benefit analysis requires that the benefits of a project can be assessed in terms of monetary value. In cost feasibility analysis the evaluation of a project and its alternatives is based solely on the cost of the program. Finally, under cost utility analysis the evaluation of a project and its alternatives is based on the costs.
Cost benefit and cost feasibility analyses are ordinarily conducted at the early stages of the project, prior to implementation, whereas cost effectiveness and cost utility analyses follow some period of implementation experience. The former, preliminary analyses should predict expenditures--and revenues--at the project's outset. The latter inform managers of actual outcomes measured through evaluation, cost accounting, and return on investment.
Cost effectiveness analysis depends on sound evaluation. After all, the analysis has two principal components: cost and effectiveness. Hezel Associates (1999) notes that the measurement of effectiveness of learning, skills, and satisfaction components of distance learning projects occur through evaluation. The outcome elements to be measured and reported for the analysis may be determined by the evaluator and the manager. Data from those measures are applied against the cost, and the comparison of cost and evaluation-based effectiveness form the basis for the analysis.
The result of a cost effectiveness analysis is a dollar figure: For every dollar (or thousand or million dollars) spent on the project x units of impact have occurred. The analysis assumes that units of effect can be expressed through quantification, such as changes in learning scores, skill performance, or satisfaction levels. In businesses, quantification might be focused on a single determinant: revenue or profit increases and would typically take the form of a Return on Investment calculation. Businesses typically establish a minimum threshold, or hurdle rate, above which an initiative needs to perform to sustain support.
In education, however, arriving at quantifiable measures could be more problematic. What is a 10-point rise in SAT scores or a .3 growth in GPA worth? Educators might blanch at the prospect of attaching dollar figures to some level of educational outcome. Nevertheless, fiscal responsibility requires that a distance education initiative, with its attendant costs and potential revenue opportunities, undergo such analysis. Without that analysis, it would be impossible to determine whether the initiative, as an alternative method to deliver instruction to students, was worth the investment.
The analysis can make a difference in policy and future funding opportunities. Administrators and policy makers need to make decisions about committing the educational institution to an increasingly aggressive distance education initiative. They need to know, among other outcomes, whether the outcome has merited the expenditure. On that basis they might recommend the continuance or discontinuance of the initiative. Return on investment is becoming increasingly interesting to funders and investors. Educators should consider the risks and benefits of minimum impact thresholds on distance education so that investment continues to flow to the highest impact initiatives.
The cost effectiveness analysis provides them with a tool with data and analysis for their investment decisions. Ultimately, this analysis also gives the distance education administrator an implement for results-oriented management. The outcome of the analysis could result in the rapid scalability of the initiative.
References
Hezel Associates. (1999). Cost Effectiveness of Star Schools Funded Projects:
An Evaluation Framework and Case Studies. Report for the United States
Department of Education Office of Educational Research and Improvement.
Levin, H. (1981). Cost-effectiveness: A primer. Newbury Park, CA:
Sage.