Drawing On Eleven Years Experience With A Problem-Based
Learning MBA Program, Curricular Design And Implementation
Issues are Discussed
John E. Stinson and Richard G. Milter
This paper is based on our experience implementing problem-based learning into the Ohio University MBA program. It is not intended to report rigorous research. We have moved from a traditional course-based, discipline-divided MBA to a completely problem-based program during the last eleven years. We started without a road map and made many mistakes along the way. But we also gained some insights that may be of use to others attempting to implement problem-based learning. It is in this spirit that we share our story.
We will first briefly describe the program as it exists at the present time and review the factors that influenced us to make the change to problem-based learning. We will then discuss the process used and the problems confronted when establishing curriculum, designing problems, and extracting learning from problems. Finally, we will discuss implementation issues that are unique to faculty, students, and administrative processes.
The Ohio University MBA
The Ohio University MBA presently is an intense 13-month learning experience, starting in August of year 1 and concluding in September of year 2. There are also part-time programs, "housed" at our regional campus locations, 50 - 90 miles from our main campus. These distance learning programs are lock-step, cohort programs, that require approximately 2 and 1/2 years to complete.
All programs use a problem-based learning format with a theoretical base in cognitive constructivism, a format that places the learner into exactly the type of projects and work situations that he/she will face as a leader of the information age organizations of the 21st century. Students learn basic business concepts, but learn them in the context of their use, maximizing their ability to both recall and apply those concepts as they move back into the work world. Students develop the skills (communication, collaboration, teamwork) and the personal characteristics (initiative, creativity, personal responsibility) that are becoming so necessary for success. Students develop a high level of comfort with information technology as they regularly access information through the resources of the internet, collaborate electronically over time and space, and develop and make professional level computer-driven presentations.
The program centers around eight major projects. The projects tend to be large macro problems that address business holistically. There are, within any project, multiple smaller problems that students must address to manage the total learning problem. Students construct their knowledge of business practices by working their way through the problems. Student learning is aided by the ability to access appropriate content on a just-in-time basis. Students learn content at a time when it will be most useful to them in their management of the learning problems. While some of the problems are designed to challenge individuals separately, most of them are designed to be approached by collaborative learning groups.
While focusing on macro problems and working with larger groups, the problem-based learning process employed is a derivative of Reiterative Problem-Based Learning, which was developed by Howard Barrows (1985), and follows closely the concepts of cognitive constructivism (Savery and Duffy, 1994) and cognitive apprenticeship (Collins and others, 1990). (For a more complete description of our use of the action learning process, see Stinson, 1990, and Milter and Stinson, 1995.)
Developing the curriculum.
Our first movement into a more integrated curriculum using a problem-based learning pedagogy was prompted by criticism of graduate business education. The popular business press published several reports critical of business education during the early eighties. Business schools were chastised for being too theoretical and out of touch with business realities, for producing narrow-minded technicians who lack interpersonal and communication skills, and for concentrating on esoteric research which has little if anything to do with the business world.
While some of the reports were sensationalized and demonstrate a lack of understanding of both business schools and the business world, there was merit to the concerns expressed. Many business schools, including ours, heard from members of their executive advisory boards, that graduates were not well prepared for the business world. They noted that graduates do not have a realistic understanding of the business world, they criticized graduates for ineffective communication skills, they noted the lack of leadership skills, and they commented on the need to train new graduates, teaching them concepts they supposedly learned in school. Similar concerns were expressed in the Business-Higher Education Forum in its May 1985 report to President Reagan, America's Business Schools: Priorities for Change.
In response to those concerns we performed a complete redesign of our curriculum. We used a process of curriculum design that, as it has evolved, has served us well. The process is conducted by an interdisciplinary team who will be the central delivery team for the program. Represented on this team are faculty from each major discipline in the College including: accounting, communications, finance, information systems, management, marketing, and operations.
The process begins by attempting to establish desired student outcomes. To approach that issue, we ask ourselves the question, "what is it that we want our students to know, and know how to do, as they leave our program?" We attempt to answer the question from three different perspectives:
We utilized this process during our last major redesign in 1992 and have performed the same analysis in abbreviated form each year since. (We reinvent the program each year; it is never conducted the exact same way twice). Using this process, we have developed a dozen desired student outcomes. Under each of these outcomes, called "meta outcomes" we developed a more specific set of learning outcomes (some 150) that drive the structure of the program.
The meta outcomes are much broader than those traditionally identified for MBA programs. In addition to knowledge and the ability to apply the knowledge, they incorporate a number of skills and personal characteristics not traditionally addressed. Because of the breadth and the inter relatedness of those outcomes, we concluded some time ago that a typical discipline-based course-structured curriculum would not produce the desired outcomes.
Further, as constructivist research shows (Duffy and Jonassen, 1991) positivist pedagogies encourage the development of personal characteristics counter to those we needed to develop. For example, in the more positivist-based pedagogy, students are encouraged to be passive. Our outcomes, on the other hand, call for them to become active initiators. Traditionally, faculty have taken the responsibility to provide clarity to students. Our outcomes, rather, call for students to clarify their own roles in ambiguous situations. Thus we concluded that a pedagogy based on a constructivist philosophy of learning, problem-based learning, was more appropriate given our desired learning outcomes.
We have now had more than ten years experience implementing problem-based learning. While some may still have concerns about the effectiveness of the process, we do not. Rather, our concerns center around the implementation of problem-based learning. Inappropriately used, problem-based learning will not lead to the potential robust learning. It is our experience that the critical implementation issues, those that may actually limit learning, include incomplete or inappropriate use of the process, faculty capabilities and attitudes, and issues of student transition.
Designing Problems and Extracting Learning
The design and implementation of appropriate problems is central to effective problem-based learning. The learning materials should be tailored to the process. Although problem-situations may be somewhat similar to traditional cases in business, they go well beyond the typical case by emphasizing both the acquisition of information and the structuring of the situation to permit analysis. In the traditional case format, each of these activities is carried out by the case writer and affords no opportunity for student learning of valuable skills.
We agree with Schank and Cleary (1995) that too frequently inadequate attention is given to the design of problems. In our experience, there has been a tendency by less experienced faculty to select a problem because it is convenient or interesting. The tendency is to then "fudge" the development of learning outcomes to fit the selected problem. This, of course, is inappropriate. Effective problem design begins with a set of clearly identified learning outcomes. It is the learning outcomes that should drive problem design, and not the other way around.
What are good problems? In his descriptions of goal-based scenarios, Schank and Cleary note that problems should be both authentic and engaging. Barrows (1985) proposes that problems should be ill-structured. Consistent with these guidelines and drawing upon the constructivist philosophy, we have developed a number of principles that guide our development of problems and projects. These principles include targeting learning outcomes as holistic, and assuring that problems mirror professional practice by being ill-structured and contemporary.
Learning outcomes should be holistic rather than being divided by narrow disciplinary boundaries.
Focusing on content only within narrow discipline boundaries limits potential excursionary learning. Further, discipline boundaries are largely a construct of academic convenience. In practice, there is no such thing as a "marketing problem." Any action taken in the marketing area of the firm impacts on the operational area and the financial area. As firms today are learning the importance of becoming more "boundaryless" and organizing around processes designed to serve customers, learning outcomes need to become more boundaryless. It makes little sense to have to work hard at the end of an educational process to attempt to "integrate" disciplines when the disciplines should never have been differentiated to begin with.
Problems should mirror professional practice.
This meets the criteria of authenticity. Problems should be similar in nature to the problems we find in professional practice or at least call forth the same types of skills and activities. Thus, content will be learned in the context of practice so that, when needed for practice, the content can be more readily recalled and used. More and more over time, as the student confronts and manages authentic situations, the process of learning and doing becomes intertwined and indivisible. The learner manager develops the ability and accepts the responsibility for managing his or her own learning. The manager in practice approaches each situation as a learning experience and has the ability to reflect upon his or her experience and extract "knowledge" that will continually add to her or his capability. The learner manager will evolve into the learning manager.
Problems should be ill-structured.
In practice, managers are seldom confronted with neat, well-structured problems. Rather, they most frequently are confronted with what Ackoff has characterized as "managerial messes" (1979). Students need to develop the ability to confront ambiguous, ill-defined situations and make sense of them. They need to be able to recall concepts and techniques and apply them in this sense-making process. Further, they need to engage in and develop an effective inquiry process. During the process of confronting and managing an ill-structured problem, students are required to gather and interpret information. The information required to manage the problem is not packaged and provided to the student. There is no textbook or written case study.
Once again, this is authentic. The information needed to analyze a situation is not pre-packaged, pre-analyzed, and provided for the manager. There is no "Harvard Case Study" of the day. Managers must be able to determine what information is needed and have the ability to obtain and interpret the information. By repeatedly confronting and managing ill-structured problems, students develop the ability to ask the right questions and to determine what information is needed to frame the situation. Further, they learn where and how to obtain the needed information. They develop the truly requisite business research skills.
Problems should be contemporary.
While authenticity is emphasized in the above principles, engagement is implied. In our experience, authentic problems are engaging. Students see such problems as real and find them stimulating to attack. But these stimulating real problems should also be contemporary. Students are not engaged by a challenge to determine what a company should have done ten years ago. This is a typical problem of "Harvard type" cases. Students do not accept the authenticity of a case set several years in the past and, thus, are not engaged. An additional problem with an historic case is a search for the "right" answer. This gives students an impression that complex business problems are simply puzzles requiring selection of the correct response.
Learning From the Problems
While design of problems is critical, they must also be effectively implemented. In particular, students must learn from their experience and be able to generalize from the specific situation to develop more robust knowledge and understanding.
Albanese and Mitchell (1993), in their review of the problem-based learning in medical education, note that graduates of problem-based medical programs sometimes report a lack of confidence that they have learned as much content as has those who went through a traditional program. They further note research that sought to measure extent of content learning. They conclude that research suggests somewhat less knowledge of problem-based learning graduates. Note, however, that this conclusion reflects a measurement bias of the authors. It is based on measurement of learning via "standardized objective tests." Measures more directly related to professional activity, data from clinical evaluations (evaluations of clinical performance of graduates), were discounted because, "clinical evaluation represents a complex mix of personal and secondary observation of residents (p. 77)." This bias causes us to question the validity of the conclusion. Rather than suggesting a weakness in the concept of problem-based learning as was implied, these reviews may suggest a weakness in the implementation of the learning process.
In our early use of problem-based learning, we experienced similar concerns. Students would learn, but would exit the program not fully comprehending how much they had learned. Further, they could not effectively access their learning in non-associated recall and thus could not demonstrate the extent of their learning on traditional tests.
A review of our use of problem-based learning revealed that we were not effectively helping students make their learning explicit. We were assuming that the students would, as a natural part of the learning process, reflect on their experience and extract abstract knowledge. At most, we would conduct a debrief that focused on how the student "felt" about their experience.
Collins (1990) notes three problems in simply learning by doing. A flexibility problem (students learn to do things in only one way), a learning problem (students do not learn a global framework to organize their learning), and a transfer problem (students do not learn how to apply what they have learned in new situations). He proposes that, to construct robust understanding from situated experience, learners must:
Influenced by Collins as well as Schon (1983) we have now redesigned the curricular structure of our program so that the initial problem enables the students to develop and articulate a global framework - the business concept.
We also implemented a rigorous assessment process that requires students to articulate what they have learned. The assessment also requires them to relate what they learn in any particular problem to their global framework - their understanding of the business concept. Finally, assessments may require them to address a similar situation, but in a different context.
These assessments occur both while the students are addressing the problem - functioning as reflections in action, and after they have completed a problem - reflections after action. The assessments can occur in written form or in oral form, or they may be behavioral demonstrations.
These changes have materially impacted our students' recognition of the breadth and depth of their conceptual understanding and on their ability to engage in non-associated recall and articulate their knowledge.
The Faculty as a Variable
In their review, Albanese and Mitchell (1993) largely ignored faculty attitude, capability, and orientation as variables that have impact on the implementation of problem-based learning. They noted only that the "role of facilitating discussion rather than directing it is foreign to many faculty ..." (pg. 74). They also report some fragmentary research on faculty satisfaction with being a tutor. Our experience suggests that faculty are the central variable in effective implementation of problem-based learning.
The role of the faculty member is quite different in problem-based learning. He or she spends very little time up front, lecturing and transferring information to students. Rather the role becomes a combination of both learning manager and coach.
The faculty is responsible for selecting appropriate learning problem-situations. As noted above, the selection of these situations is critical. They must be involving, relevant, holistic, and at the appropriate level of complexity to be in phase with the students' development. Further, the faculty must insure that appropriate physical resources are present. This requirement ranges from arranging executive panels to review student presentations and provide feedback, to insuring that appropriate data resources are available at the library or via electronic sources.
But the teacher is also a coach (Kraft, 1988). The teacher observes student performance, corrects poor performance and encourages appropriate performance. The coach/teacher "... encourages that the right way of performing be done over and over again until the requisite skill becomes a firm and stable habit of performance" (Kraft, 1988, pg. 1).
Much as happens in apprenticeship (Collins and others, 1990), the teacher also provides a model. At appropriate times as students are involved in a learning situation, or after they have completed it, the teacher provides a thorough and high quality performance so that students can compare their performance to that of an expert. This can be a personal performance demonstrated by the teacher or it can be a real-life or communicated performance by some other expert. Note that this model is normally not provided before students enter into the learning situation. They need some involvement, some understanding of the context, before they can benefit from modeled performance.
Finally, the teacher helps students generalize the learning (Collins and others, 1990). As students express (in oral or written form) what they have learned while confronting the problem-situation, the teacher helps them understand how that same knowledge and skill can be used in other situations. The transition to this richer faculty role can be extremely difficult for some faculty.
Faculty have been trained to become experts in a narrow discipline and share their expertise with students and their professional colleagues. The role models most have been exposed to, primarily by faculty in their Ph.D. programs, have emphasized scholarly research. "Teaching," to the extent it is addressed, emphasizes "covering the content" of the discipline. Lecture/discussion has been the dominant teaching approach. Little questioning is done regarding the relevance of the content to the practice of business or the effectiveness of the pedagogy in the preparation of students to become business people.
Traditional faculty orientations
are strongly imbedded in the culture and in the profession. The traditional
reinforced by the existing structures and reward systems. For
most faculty, the primary reference group comprises the leaders
in their disciplinary field, not practicing business people.
While lip service is given to "teaching," the reward
system, including tenure and promotion, tends to emphasize publication
of research articles in refereed discipline-based journals.
Skills Required of Faculty.
This new faculty role, represents a paradigm shift calling for new skills. The paradigm shift has been expressed as moving from being the "sage on the stage" to serving as a "guide by the side." The basic skills required to be a "guide by the side" (active listening, coaching, mentoring, and facilitation) are not characteristic of a significant number of faculty, and thus they must be developed. Once they successfully navigate this paradigm shift, (usually by starting to participate in problem-based learning and experiencing the AH hA!) most faculty adjust to the situation by learning a new set of skills while developing a new set of work habits.
Making the transition from the traditional faculty role to the richer role required in problem-based learning is not easy. It requires time, experimentation, nurturing, and support. Most central, however, is a willingness to move from a concern for teaching - to a commitment to learning. This attitude shift moves faculty from an input orientation to an outcomes orientation.
Will this transition become widespread? There is some hope. Reports such as the work of Porter and McKibbin (1988) reflect increased concern with preparing graduates to function effectively in business. Likewise, recent changes in American Assembly of Collegiate Schools of Business (AACSB) accreditation standards emphasize the move toward relevancy and outcomes-based education. But the transition, if it is to become more widespread, will have to be championed by visionary educational leaders at the faculty as well as the administrative level. If the transition to establishing more meaningful learning communities is to continue it will need to be reinforced by changing reward structures.
Students often have difficulty initially adapting to the problem-based learning methodology. Students frequently express frustration when they first encounter problem-based learning. Most students have progressed through a typical educational system where knowledge is divided into arbitrary disciplines and taught to them through lectures, discussion sessions, or some combination. The students have learned to memorize information and regurgitate it on multiple choice/true-false or essay examinations. This is a teacher-centered model of education with teacher and/or textbook structuring all dimensions of learning. The teacher takes responsibility for student learning.
Problem-based learning is student-centered. Students are expected to take responsibility for their own learning. The teacher does not tell them the "right answer." The teacher lets them experiment and make mistakes. The teacher makes them go to original sources to get information. The teacher may not even answer their questions directly. They are expected to find their own answers. This creates a very ambiguous situation for students.
"What are we supposed to do?"
"How do we do that?"
"If you would only tell me what you want, I would do it."
These are the types of statements we frequently hear. The situation is often most difficult for students who have been particularly good students in the positivist learning environment. They have functioned well under an external locus of control where their life was structured for them, and perceive being forced to perform using greater internal locus of control as very threatening.
Thus, a great deal of coaching is required as students make the transition into problem-based learning. Students must be helped and encouraged as they start to take on responsibility for their own learning. Rather than just giving an assignment, the teacher must work with the students as they take their first halting steps into an ill-structured problem/situation. Rather than giving them a direct answer to a question, the teacher should talk them through the process of answering their own questions. If coached effectively through the transition, all but the most regimented of students make the transition and eventually thrive in the new learning environment.
We want to emphasize that we do not believe we have found the "right" answers to the problem of how to best develop future business leaders. We have made serious entry into the arena of developing a genuine learning community. We are committed to continuing our search for better ways to assist students with learning. We believe this search will never end. When we begin to feel we are doing it right, it will be time to get out of the business.
Ackoff, R. "The Future of Operations Research is Past," Journal of Operations Research Society, 1979, 30, (2), 93-104.
Albanses, M. A. and Mitchell, S. "Problem-based Learning: a Review of Literature on its Outcomes and Implementation Issues. Academic Medicine, 1993, 68, 52-81.
Barrows, H. S. How to Design a Problem-based Curriculum for the Preclinical Years. New York: Springer Publishing Company, 1985.
Brown, J. S., Collins, A. and Duguid P. "Situated Cognition and the Culture of Learning," Educational Researcher, January-February, 1989, 89-99.
Business-Higher Education Forum. America's Business Schools: Priorities for Change, Report to the President, 1985.
Collins, A., Brown J., and Newman, S. "Cognitive Apprenticeship: Teaching the Craft of Reading, Writing, and Mathematics," in L. B. Resnick (ed.), Cognition and Instruction: Issues and Agendas. Hillsdale N.J.: Lawrence Erlbaum Associates, 1990.
Collins, A. "Generalizing from Situated Knowledge to Robust Understanding," American Educational Research Association Annual Meeting, Boston, April, 1990.
Duffy, T.M., and Jonassen, D.H. "Constructivism: New Implications for Instructional Technology" Educational Technology, 1991, 31 (5), 7-12.
Kraft, R. "Coaching to Learn," The Teaching Professor, January 1988, 2 (1), 1-2.
Milter, R. and Stinson, J. "Educating Leaders for the New Competitive Environment," in W. Gijselaers, Tempelaar, D., Keizer, P., Blommaert, J., Bernard, E., and Kasper, H. (eds.), Educational Innovation in Economics and Business Administration: the Case of Problem-Based Learning. Dordrecht: Kluwer Academic Publishers, 1995.
Porter, L., and McKibbin, L. Management Education and Development: Drift or Thrust into the 21st Century. New York: McGraw-Hill, 1988.
Savery, J. and Duffy, T. "Problem-based Learning: An Instructional Model and its Constructivist Framework," Educational Technology, August, 1994, 1-16.
Schank, R. and Cleary, C. Engines for Education, Technical Report, no. 58, http://www.ils.nwu.edu/~e_for_e/index.html 1995.
Schon, D. The Reflective Practitioner. New York: Basic Books, 1983.
Stinson, J. "Integrated Contextual
Learning: Situated Learning
In The Business Profession," ERIC Clearinghouse on Higher
Education, 1990, no. ED319330, RIE.
John E. Stinson is professor of management in the College of Business
at Ohio University.
Richard G. Milter is associate professor of management in College
of Business at Ohio University.
" Problem-Based Learning in Business Education: Curriculum
Design and Implementation Issues" (with Milter) accepted
for publication, New Directions in Teaching and Learning
in Higher Education, Jossey-Bass, 1996.