The contents of this book derive from many scientific and professional sources. To make Telling Ain’t Training easy to read, we refrained from citing too many references for fear of interrupting the flow of each chapter. Instead, we opted to create a substantial endnotes section keyed to specific topics and/or assertions found within the chapters. What follows is a list of references and interesting readings for those who wish to delve more deeply into a subject of personal interest. Please note that we frequently comment on the references, which are organized by chapter and then by topic. Topics are presented sequentially as they appear in the text. There are no endnotes for chapter 1. We also retained most of the original references from the first edition because, in many cases, they still remain valid and offer excellent information even though some may be dated.

Chapter 2: An Introduction to Some “Familiar Terms”

Training, instruction, education, learning

¹All of the following provide much more depth about what we do as trainers/ instructors/educators and the impact of each on learning:

Driscoll, M.P. (2004). Psychology of Learning for Instruction, 3d edition. Needham Heights, MA: Allyn & Bacon. This textbook covers a wide range of teaching-learning theories and their applications from primary school to corporate learning and development.

Gagné, R., and K.L. Medsker. (1996). The Conditions of Learning: Training Applications. Orlando, FL: Harcourt Brace and Co. Still a foundational document for anyone connected with training.

Sawyer, R.K., ed. (2006). The Cambridge Handbook of the Learning Sciences. New York: Cambridge University Press. This volume provides a modern view of learning, what methods are most effective, and what is wasteful or counterproductive. The chapters provide excellent guidance and examples of how training, instruction, and education combine to produce learning.

Performance-based instruction

²Brethower, D.M., and K.A. Smalley. (1999). Performance-Based Instruction. San Francisco: Jossey-Bass Publishers. See also Pucel, D.J. (2005). Developing and Evaluating Performance-Based Instruction, 3d edition. New Brighton, MN: Performance Training Systems.

Media and learning

³Clark, R.E. (2001). Learning From Media: Arguments, Analysis, and Evidence. Greenwich, CT: Information Age Publishers.

Clark, R.E. (1999). Media Will Never Influence Learning. Educational Technology Research and Development 47(2): 21-29.

Both publications by Clark provide powerful arguments and evidence that media, per se, have little to no impact on learning effectiveness.

⁴A more recent and nuanced summary of the research on media and learning is Fadel, C., and C. Lemke. (2008). Multimodal Learning Through Media: What the Research Says. San Jose, CA: Cisco Systems. One interesting citation is “A recent metaanalysis in which over 650 empirical studies compared media-enabled distance learning to conventional learning found pedagogy (the method of instruction) to be more strongly correlated to achievement than media” (Clark, R.E., and D.F. Feldon. {2005}. Five Common but Questionable Principles of Multimedia Learning. In R.E. Mayer, ed., Cambridge Handbook of Multimedia Learning. Cambridge: Cambridge University Press, 97-116).

Interactive learning

⁵The following provide considerable information on effective means for creating interactive teaching-learning situations:

Engle, R.A. (2006). Training Interactions to Foster Generative Learning: A Situative Explanation of Transfer in a Community of Learners Classroom. The Journal of the Learning Sciences 15(4): 451-498. Concerning interactivity in learning and its effectiveness, periodicals such as The Journal of Interactive Online Learning or The Journal of Interactive Learning Research provide current articles on research in this arena.

Rieber, L.P. (1996). Seriously Considering Play: Designing Interactive Learning Environments Based on the Blending of Microworlds, Simulations, and Games. Educational Technology Research and Development 44(2): 43-58.

Stolovitch, H.D. (1984). Frame Games and Gamechains: A Technology for Interactive Teaching-Learning. In R.K. Bass and C.R. Dills, eds., Instructional Development: The State of the Art, II. Dubuque, IA: Kendall/Hunt Publishing Company.

Meaningfulness in learning interactions

⁶Meaningfulness in learning has been a topic of interest globally for many years. Here are a few references to clarify what is meant by meaningfulness and what its effect on learning can be:

Boettcher, J. (2000). Designing for Learning: What Is Meaningful Learning? Syllabus 14(1): 54-56. Satheim-Smith, A. (1998). Focusing on Active, Meaningful Learning: IDEA Paper No. 34. Manhattan, KS: Kansas State University.

Wilson, A., and L. Burket. (1989). What Makes Learning Meaningful? Proceedings of the Annual Meeting of the American Association for Adult and Continuing Education, Atlantic City, NJ, October 1989. (ERIC Document ED 313 586)

Yasin, M., M.N. Daud, H. Musa, and S.S. Imam. (2007). Effect of Meaningfulness of Learning Materials on Speed of Learning. e-IPRC Proceedings, Srinakharinwirot University, Bangkok, Thailand.

Chapter 3: The Human Learner

Physiology and evolution of human learning

¹The following references on the physiology and evolution of human learning provide an excellent base for understanding how learners came to be who and what they are today:

D’Aracangelo, M. (2000). How Does the Brain Develop? A Conversation With Steven

Peterson. Educational Leadership 58(3): 68-71. Jenkins, M. (1999). Evolution. Lincolnwood, IL: NTC Publishing Group. Kingdon, J. (1993). Self-Made Man. New York: John Wiley & Sons. Ornstein, R. (1991). The Evolution of Consciousness: The Origins of the Way We Think. New

York: Simon & Schuster. Palumbi, S.R. (2001). The Evolution Explosion. New York: W.W. Norton & Company. Sousa, D.A. (2001). How the Brain Learns: A Classroom Teacher’s Guide, 2d ed. Thousand

Oaks, CA: Corwin Press. Sprenger, M. (1999). Learning and Memory: The Brain in Action. Alexandria, VA: Association for Supervision and Curriculum Development.

The senses

²For a great book about the latest research on the senses, read Rosenblum, L.D. (2010). See What I’m Saying: The Extraordinary Powers of Our Five Senses. New York: WW Norton and Company. Neuroscientific and cognitive research have uncovered the amazing capabilities of all our senses—some unimaginable even a few years ago. This is also an inspiring book on overcoming sensory deficiencies.

³Thalheimer, W. (2004). Bells, Whistles, Neon, and Purple Prose: When Interesting Words, Sounds, and Visuals Hurt Learning and Performance—A Review of the Seductive-Augmentation Research. This is a meta-analytic research report examining the impact of “seductive elements” such as music, animation, sound, and other exciting effects in instructional programs. Thalheimer examines 29 studies and concludes that unless absolutely necessary for the instructional objective, these elements, overall, have a decremental effect on learning and retention. Available at http:// www.worklearning.com/seductive_augmentations.htm.

⁴Mayer, R.E. (2001). Multimedia Learning. New York: Cambridge University Press. For more extended information and a review of the research in this area, see also Clark, R.E., and D.F. Feldon. (2005). Five Common but Questionable Principles of Multimedia Learning. In R.E. Mayer, ed., The Cambridge Handbook of Multimedia Learning. New York: Cambridge University Press.

Autonomic nervous system

⁵For a succinct description of the autonomic nervous system, how it functions, and how influential a role it plays in directing our daily activities, read Dr. David H.P. Streeten’s online article, The Autonomic Nervous System, published by the National Disautonomia Research Foundation. Available at http://www.ndrf.org/ans .html#General%20Organization%20of%20the%20Autonomic%20Nervous%20 System).

Three somewhat dated but still useful references are

Backs, R.W. (1995). Going Beyond Heart Rate: Autonomic Space and Cardiovascular Assessment of Mental Workload. International Journal of Aviation Psychology 5(1): 25-48.

Microsoft Encarta Online Encyclopedia 2001. (2001). Alternative Medicine. Available at http://encarta.msn.com. Miyake, S. (1997). Factors Influencing Mental Workload Indexes. Journal of Occupational and Environmental Health 19(4): 313-325.

For a general overview of “arousal,” go to http://en.wikipedia.org/wiki/arousal. At the Rockefeller Institute, a team of researchers, led by Donald W. Pfaff, have uncovered even more specific factors affecting arousal. Consult the Internet for various papers Dr. Pfaff and his team have published in the Proceedings of the National Academy of Sciences. Additional references of interest are:

Microsoft Encarta Online Encyclopedia 2001. (2001). Alternative Medicine. http://encarta.msn.com.

Walsh, V. (1998). Skill Learning: Bringing Cognition to Its Senses. Current Biology 8(16): 572-574.

Short-term memory

⁶Cowan, N. (2001). The Magical Number 4 in Short-Term memory: A Reconsideration of Mental Storage Capacity. Behavioral and Brain Sciences 24: 97-185. Cowan has extended his examination of both short-term and working memory in Cowan, N. (2005). Working Memory (Essays in Cognitive Psychology). New York: Psychology Press. This volume encompasses a wide range of research on both short-term and working memory, with clear distinctions between the two (storage versus working attention).

Oberauer, K. (2002). Access to Information on Working Memory: Exploring the Focus of Attention. Learning, Memory, and Cognition 28(3): 411-421.

Some additional references are

Banikowski, A.K., and T.A. Mehring. (1999). Strategies to Enhance Memory Based on Brain Research. Focus on Exceptional Children 32(2): 1-6.

Gathercole, S.E. (1996). Models of Short-Term Memory. East Sussex, UK: Psychology Press. Microsoft Encarta Online Encyclopedia 2001. (2001). Memory (psychology). Available at http://encarta.msn.com.

Schooler, J. (1998). A Multiplicity of Memory. Exploring 22(1): 4-6.

Sprenger, M. (1999). Learning and Memory: The Brain in Action (chapter 4). Alexandria, VA: Association for Supervision and Curriculum Development.

Chapter 4: Getting Learners to Learn

¹Chi, M.T.H. (2006). Laboratory Methods for Assessing Experts’ and Novices’ Knowledge. In K.A. Ericsson, N. Charness, P.J. Feltovich, and R.R. Hoffman, eds., The Cambridge Handbook of Expertise and Expert Performance. New York: Cambridge University Press, 167-184.

Two other readings that help clarify how experts process information differently from others:

Comptelli, G., and F. Gobet. (2005). The Mind’s Eye in Blindfold Chess. European Journal of Psychology 17: 23-45

Hoffman, R.R., G. Trafton, and P. Roebber. (2006). Minding the Weather: How Expert Forecasters Think. Cambridge, MA: MIT Press.

The greater the expertise, the greater the difference in thinking from novices

²Chase, W.G., and H.A. Simon. (1973). Perception in Chess. Cognitive Psychology 4(1): 55-81.

The following readings are also enlightening for training professionals:

Brandsford, J., A.L. Brown, and R.R. Cocking. (2000). How People Learn: Brain, Mind, Experience, and School, expanded ed. Washington, DC: National Academy Press.

Clark, R.C. (2008). Building Expertise: Cognitive Methods for Training and Performance Improvement, 3d edition. San Francisco: Wiley-Pfeiffer.

Daley, B. (1999). Novice to Expert: An Exploration of How Professionals Learn. Adult Education Quarterly 49(4): 133-147.

Stepich, D. (1991). From Novice to Expert: Implications for Instructional Design. Performance and Instruction 30(6): 13-17.

Declarative and procedural knowledge

³The following two references clarify the differences between declarative and procedural knowledge. In the process, they add extra nuances and layers to the terse distinctions generally made between the two:

Jiamu, C. (2001). The Great Importance of the Distinction Between Declarative and Procedural Knowledge, Analise Psicológica 4(19): 559-566.

ten Berge, T., and R. van Hezewijk. (1999). Procedural and Declarative Knowledge; An Evolutionary Perspective. Theory Psychology 9(5): 605-624.

Also, see

Gagné, R., and K.L. Medsker. (1996). The Conditions of Learning: Training Applications. Orlando, FL: Harcourt Brace and Company. This volume is a classic. Most professionals in the training-learning profession have read it. It is a must-have on your bookshelf.

Smilkstein, R. (1993). Acquiring Knowledge and Using It. Gamut. Seattle: Seattle Community College District. (ERIC Document ED 382 238)

⁴If you want to know more about the indicative and subjunctive moods, here are two brief, but clear references online: For the indicative mood, see http://www.lousywriter.com/verbs_indicative_mood.php; for the subjunctive mood refer to http://englishplus.com/grammar/00000031.htm.

⁵Although somewhat technical, the following reference brings home this point in a programming context: Rosenbloom, P.S. (2010). Combining Procedural and Declarative Knowledge in a Graphical Architecture. In D.D. Salvucci and G. Gunzelmann, eds., Proceedings of the 10th International Conference on Cognitive Modeling, Philadelphia, PA: Drexel University, 205-210.

Ability, prior knowledge, motivation

⁶A recent study by two Turkish university researchers underscores the roles ability, prior knowledge, and motivation play in learning. Although the study used eighth grade science students as the subjects, their findings are in line with past discoveries of the relationships among these three variables and learning outcomes. Araz, G., and S. Sungur. (2007). The Interplay Between Cognitive and Motivational Variables in a Problem-Based Learning Environment. Learning and Individual Differences 17(4): 291-297.

⁷Probably the most cited proponent of multiple intelligences is Howard Gardner of Harvard University. A good reference for learning more about this:

Armstrong, T. (2009). Multiple Intelligences in the Classroom, 3rd ed. Alexandria, VA: Association for Supervision and Curriculum Development.

⁸Not everyone is convinced of this belief concerning the innate abilities model. Read Colvin, G. (2008). Talent Is Overrated: What Really Separates World-Class Performers From Everybody Else. New York: Penguin Group. He presents the results of a great deal of investigation and some convincing research-based arguments against the “born with talent” postulate for outstanding achievement. So, for that matter, does Malcolm Gladwell in a well-argued piece in The New Yorker titled “Talent Myth: Are Smart People Overrated?” (July 22, 2002).

⁹Much has been written on the topic of prior knowledge and its influence on learning. A neat, clean study demonstrates the relationship succinctly. Its reference section is also brief, but to the point:

Halikari, T., N. Kanjavuouri, and S. Lindblom-Ylanne. (2008). The Relevance of Prior Knowledge in Learning and Instructional Design. American Journal of Pharmaceutical Education 72(5): 1-8.

¹⁰The three variables of value, confidence, and mood form the basis for what is known as the CANE model of motivation, developed by R.E. Clark of the University of Southern California. This model can easily be accessed online. A recent study applying the CANE model illustrates how value, confidence, and mood play vital roles in affecting motivation:

Condly, S.J., and R. Di Pietro. (2004). Motivation in the Hospitality Industry. New York: Site Foundation. (A summary of the report is available at http://www.incentivecentral .org/employees/whitepapers/motivation_in_ the_hospitality_industry.1903.html.)

¹¹What affects learning has many facets. The following readings have been around for a number of years, but are still frequently cited. Browse through these to gain more insights into the roles ability, prior knowledge, and motivation play in transforming learners:

Bandura, A., C. Barbaranelli, G.V. Capara, and C. Pastorelli. (1996). Multifaceted Impact of Self-Efficacy Beliefs on Academic Functioning. Child Development 67(3): 1206-1222.

Clark, R.E. (1999). The CANE (Commitment and Necessary Effort) Model of Work Motivation: A Two Stage Process of Goal Commitment and Mental Effort. In J. Lowyck, ed., Trends in Corporate Training. Leuven, Belgium: University of Leuven Press.

Dochy, F.J.R.C. (1988). The “Prior Knowledge State” of Students and Its Facilitating Effect on Learning: Theories and Research. Heerlen, The Netherlands: Open University, Secretartiaat. (ERIC Document ED 387 486)

Dochy, F.J.R.C., et al. (1996). The Importance of Prior Knowledge and Assessment for Increasing Efficiency of the Learning Processes, Especially in “Problem-Based” Learning Environments. European Journal of Agricultural Education and Extension 3(3): 141-166.

Kaplan, A., and C. Midgley. (1997). The Effect of Achievement Goals: Does Level of Perceived Academic Competence Make a Difference? Contemporary Educational Psychology 22(9): 415-435.

Mager, R.F. (1997). Making Instruction Work. Atlanta: Center for Effective Performance.

Stipek, D. (1998). Motivation to Learn—From Theory to Practice. Los Angeles: Allyn & Bacon.

The instructor as a compensation for what the learner lacks

¹²These two readings allude to the notion of the instructor as a flexible support for compensating for what learners lack:

Watson, D.L., and N.A. Stockert. (1987). Ensuring Teaching and Learning Effectiveness. Thought and Action 3(2): 91-104.

Yelon, S.L. (1996). Powerful Principles of Instruction. Reading, MA: Addison-Wesley.

Chapter 5: Adult Learning Principles

¹The field of adult learning or “andragogy” became a popular center of interest and study from the late 1960s into the 1990s. It has continued to have a powerful influence on the training world. The following are some useful readings that summarize key principles and currents within the adult learning domain:

Birkenholz, R.J. (1999). Effective Adult Learning. Danville, IL: Interstate Publishers.

Brookfield, S.D. (1991). Understanding and Facilitating Adult Learning: A Comprehensive Analysis of Principles and Effective Practices. San Francisco: Jossey-Bass Publishers.

Cohen, S.L., D.W. Dove, and E.L. Bachelder. (2001). Time to Treat Learners as Consumers. Training & Development 55(1): 54-57.

Davenport, J., III, and J.A. Davenport. (1985). Knowles or Lindeman: Would the Real Father of American Andragogy Please Stand Up. Lifelong Learning 9(3): 4-5.

Van Houten, C. (2000). Awakening Will: Principles and Processes in Adult Learning. Herndon, VA: Anthroposophic Press.

———. (2000). Practicing Destiny: Principles and Processes of Adult Learning. Herndon, VA: Anthroposophic Press.

Principles of adult learning in action

²There is vast literature on adult learning theory and research and best practices in diverse areas such as adult literacy, adult learning methodologies, English as a second language, and workplace training. Here is an enlightening paper on adult learning:

Thompson, M.A., and M. Deis. (2004). Andragogy for Adult Learners in Higher Education. In Proceedings of the Academy of Accounting and Financial Studies 9(1): 107-111.

³The National Center for the Study of Adult Learning and Literacy (NCSALL) in the Graduate School of Education, Harvard University, maintains an up-to-date database of research in this area and provides links to other organizations that conduct adult learning research.

⁴For information about Malcolm Knowles, his writings, and his life, go to http:// en.wikipedia.org/wiki/Malcolm_Knowles.

⁵An excellent resource for helping to create readiness to learning:

Merriam, S.B., R.S. Caffarella, and L.M. Baumgartner. (2007). Learning in Adulthood: A Comprehensive Guide, 3rd ed. San Francisco: Jossey-Bass.

⁶Tashiro, A., M. Hiroshi, and F.N. Gage. (2007). Experience-Specific Functional Modification of the Dentate Gyrus Through Adult Neurogenesis: A Critical Period During an Immature Stage. The Journal of Neuroscience 27(12): 3252-3259.

⁷It is always exciting to discover studies that actually examine the effects of a phenomenon such as “self-directedness in learning” with adults. Loibl, C., and T.K. Hira. (2005). Self-Directed Financial Learning and Financial Satisfaction. Financial Counseling and Planning 16(1): 11-21 offers such an example. The results demonstrate the effectiveness of self-directed financial learning and the role good financial management practices play in the relationship of financial learning with financial satisfaction and workers’ satisfaction and with their career progression.

The following references from the first edition of Telling Ain’t Training still offer interesting and useful information for training professionals:

Beitler, M.A. (2000). Self-Directed Learning Readiness at General Motors Japan. (ERIC Document ED 447 266)

Brockett, R.G., and R. Hiemstra. (1991). Self-Direction in Adult Learning: Perspectives on Theory, Research, and Practice. Routledge Series on Theory and Practice of Adult Education in North America. New York: Routledge, Chapman and Hall.

Piskurish, G.M. (1994). Self-Directedness, Quality, and the Learning Organization. Performance and Instruction 33(7): 32-35.

Tobias, S. (1993). Interest and Prior Knowledge. Proceedings of the Annual Meeting of the American Education Research Association, Atlanta, April 12-16. (ERIC Document ED 362 480)

Dieting, weight loss programs, and regaining of lost weight

⁸Curioni, C.C., and P.M. Lourenço. (2005). Long-Term Weight Loss After Diet and Exercise: A Systematic Review. International Journal of Obesity 29: 1168–1174.

Chapter 6: A Five-Step Model for Creating Terrific Training Sessions

¹Most of the ways we learn to adapt to our environment are bound up in what is genetically programmed within us. Our cultural and physical circumstances may affect how we engage in learning, but the mechanisms that trigger learning appear to be universal. Researcher Patricia Greenfield has written extensively on research in this area.

Greenfield, P.M., H. Keller, A. Fuligni, and A. Maynard. (2003). Cultural Pathways Through Universal Development. Annual Review of Psychology 54: 461-90.

²We will likely return to the issue of individual differences and learning styles. However, while some insist on the importance of learning styles, ways of measuring these, their classifications (now up to more than 70), and diverse viewpoints on what to do about them, considerable confusion still exists about this shaky construct. What should one do when teaching shapes (triangle, square, polygon) to a child who is classified as “aural”? Is it better to speak and describe, or should one employ pictures? Frequently, the content itself dictates how it should be presented during instruction. A wonderful, highly readable essay on this topic with numerous recent citations is “Do Learning Styles Exist?” by Hugh Lafferty and Keith Bur-ley (2009), easily accessed online at http://www.learningstyles.webs.com/.

³An early experimental study demonstrated that personally perceived value of a topic enhanced desire to learn, even more than curiosity, which was also studied:

Rossing, B.E., and H.B. Long. (1981). Contributions of Curiosity and Relevance to Adult Learning Motivation. Adult Education Quarterly 32(1): 25-36.

⁴To add to the comment, “regardless of the type of learner,” two experimental studies on two different types of learners generated the same consistent results (also found elsewhere) concerning the personally meaningful “why”:

Choi, J., J. Fiszdon, and A. Medalia. (2010). Expectancy-Value Theory in Persistence of Learning Effects in Schizophrenics: Role of Task Value and Perceived Competency. Schizophrenia Bulletin 36(5): 957-965.

Hye-Yeon, L. (2007). Effects of Attributions and Task Values on Foreign Language Use Anxiety. Journal of Education and Human Development 1(2): 1-20.

In the latter case, the more students perceived the value to them of acquiring the foreign language, the lower their anxiety.

⁵Although written more than 35 years ago, the findings of the following article have been sustained to today:

Lawson, T. (1974). Effects of Instructional Objectives on Learning and Retention. Instructional Science 3(1): 1-22. Providing a true sense of where the instruction is heading, what is important, and what to focus on do have a strong impact on learning.

See especially the work of David Ausubel going back to the 1960s. The foundational reading for this is

Ausubel, D.P. (1960). The Use of Advance Organizers in the Learning and Retention of Meaningful Verbal Material. Journal of Educational Psychology 51: 267-272.

Concerning the orienting of learners on what is to be acquired, here are some useful resources. While taking a more general approach to advance organizers, they nevertheless provide excellent guidance on directing learners concerning expectations and on what they should focus.

Mayer, R. (2002). The Promise of Educational Psychology. New Jersey: Pearson Education, Inc.

Mayer, R. (2003). Learning and Instruction. New Jersey: Pearson Education, Inc. Woolfolk, A. (2001). Educational Psychology, 8th ed. Boston: Allyn and Bacon.

⁶Bromley, K., L. Irwin-DeVitis, and M. Modlo. (1995). Graphic Organizers. New York: Scholastic Professional Books. This book deals with the importance of structure in learning and retention.

A good example of how structure of any sort (visual, text, or both) affects learning and retention is presented in

Colaso, V., A. Kaml, P. Saraiya, C. North, S. McCrikard, and C.A. Shaffer. (2002). Learning and Retention in Data Structures: A Comparison of Visualization, Text, and Combined Methods. In P. Barker and S. Rebelsky, eds., Proceedings of the 14th 2002 World Conference on Educational Multimedia, Hypermedia & Telecommunications. Denver, CO, June 24-29. In a series of studies, they demonstrated that all forms of structure positively affected learning and retention, but that the combination of structures worked best.

⁷Excellent examples of this can be found in two more recent studies. Both find a decided edge for the covert response in terms of learning and retention, although learners enjoy the use of overt response mechanisms such as clickers.

Haggas, A.M., and D.A. Hantula. (2002). Think or Click? Student Preference for Overt Versus Covert Responding in Web-Based Instruction. Computers in Human Behavior 18(2): 165-172.

Martyn, M. (2007). Clickers in the Classroom: An Active Learning Approach. Educause Quarterly 30(2) (online publication).

⁸Although somewhat dated, this is our favorite review of the feedback literature going back to the 19th century. It covers practically everything one could ask for about feedback research and findings and offers well-supported keys to practice.

Kluger, A.N., and A. De Nisi. (1996). The Effect of Feedback Interventions on Performance: A Historical Perspective, a Meta-Analysis, and a Preliminary Feedback Intervention Theory. Psychological Bulletin 119(2): 254-284.

Here are two recent studies on this topic. Both present more up-to-date information and variations on what we know about feedback, and both make excellent reading:

Austermann Hula, S.N., D.A. Robin, and E. Maas. (2008). Effect of Feedback Frequency and Timing on Acquisition, Retention, and Transfer of Speech Skills in Acquired Apraxia of Speech. Journal of Speech, Language, and Learning Research 51: 1088-1113.

Smith, T.A., and D.R. Kimball. (2010). Learning From Feedback: Spacing and the Delay-Retention Effect. Journal of Experimental Psychology: Learning, Memory, and Cognition 36(1): 80-95.

⁹A large number of studies on the use of rewards and incentives for learning have shown a link between the two, although it is not as linear and straightforward as some might suggest. Recent examination of this connection has been conducted by neuroscientists using sophisticated electronic means for tracking brain activity when rewards are presented or anticipated. One interesting study found that when tested 24 hours post scan, subjects were significantly more likely to remember scenes that followed cues for high-value rather than low-value reward. The researchers were able to localize areas of the brain that deal with rewards and influence learning and retention. This form of research is not only exciting in and of itself, it also represents a trend toward how newer types of research studies on learning and retention will be conducted over the next decade.

Adcock, R.A., A. Thangaval, S. Whitfield-Gabrielli, B. Knutson, and J.D.E. Gabrielli. (2006). Reward-Motivated Learning: Mesolimbic Activation Precedes Memory Formation. Neuron 50: 507-517.

¹⁰All of the advice provided derives from Kluger and DeNisi’s (1996) review of feedback research cited in endnote 8.

¹¹To close out the five-step model and the six universals of research on which it is founded, here are additional readings:

On the importance of knowing why (reason for learning) and what

Choi, I., and D.H. Jonasson. (2000). Learning Objectives From the Perspective of the Experienced Cognition Framework. Educational Technology 40(6): 36-40.

Keller, J.M. (1987). Strategies for Stimulating the Motivation to Learn. Performance and Instruction 26(8): 1-7.

Klein, J.D., J.C. Cavalier, and C. Jamie. (1999). Using Cooperative Learning and Objectives With Computer-Based Instruction. Proceedings of Selected Research and Development Papers Presented at the 21st National Convention of the Association for Educational Communications and Technology (AECT), Houston, February 10-14. (ERIC Document ED 436 134)

Morse, J.A., and P.K. Morse. (1978). Effects of Instructional Objectives on Relevant and Incidental Learning for Experienced and Inexperienced Possessors. Proceedings of the Annual Meeting of the 62nd American Educational Research Association, Toronto, Ontario, Canada, March 27-31. (ERIC Document ED 173 357)

On structure and learning

Bills, C.G. (1997). Effects of Structure and Interactivity on the Achievement of Students Receiving Internet-Based Instruction. Proceedings of the Interservice/Industry Training, Simulation, and Education Conference, Orlando, FL, December 1-4. (ERIC Document ED 416 317)

Dowaliby, F.J., R. Curwin, and L. Quinsland. (1981). Classroom Structure and Student Participation: An Aptitude by Treatment Interaction Approach to Instructional Research for the Hearing-Impaired. Paper Series #47. Washington, DC: Rochester Institute of Technology, NY National Technical Institute for the Deaf. (ERIC Document ED 224 257)

Gregg, L.W., and S. Farnham-Diggory. (1989). Content and Structure in Learning. Rockville, MD: National Institute of Mental Health.

Inteligen. (2001). Human Memory, Encoding, Storage, Retention, and Retrieval. Available at http://www.brain.web-us.com/memory_encoding.htm.

On overt and covert responding

Dwyer, C., M. Moore, and F. Dwyer. (1992). Covert and Overt Rehearsal Strategies Used to Complement Visualization. Proceedings of the 24th Annual Conference of the International Visual Literacy Association, Pittsburgh, September 30-October 2. (ERIC Document ED 363 290)

Elred, J.P. (1996). Incorporating Experiential Learning in the Teaching of the Nonverbal Communication Course. Proceedings of the 82nd Annual Meeting of the Speech Communication Association, San Diego, CA, November 23-26. (ERIC Document ED 408 627)

Shettel, H.H., and R.H. Lindley. (1961). An Experimental Comparison of Two Types of Self-Instructional Programs for a SAGE System Paired-Associate Task. Pittsburgh: American Institute for Research in Behavioral Sciences. (ERIC Document ED 130 605)

Wilder, L., and D.J. Harvey. (1971). Overt and Covert Verbalization in Problem Solving. Report From the Project on Variables and Processes in Cognitive Learning. Madison, WI: Wisconsin University—Research and Development Center. (ERIC Document ED 062 001)

On meaningful engagement and learning

Ausabel, D.P. (2000). The Acquisition and Retention of Knowledge: A Cognitive View. New York: Kluwer Academic Publishers.

Cavallo, A.L. (1992). Students’ Meaningful Learning Orientation and Their Meaningful Understandings of Meiosis and Genetics. Proceedings of the Annual Conference of the National Association for Research in Science Teaching, Boston, March. (ERIC Document ED 356 140)

Nystrand, M., and A. Gamoran. (1990). Student Engagement: When Recitation Becomes Conversation. Washington, DC: Office of Educational Research and Improvement. (ERIC Document ED 323 581)

On importance of feedback in learning

Kulik, J.A., and C.C. Kulik. (1988). Timing of Feedback and Verbal Learning. Review of Educational Research 58(1): 79-97.

Ovando, M.N. (1992). Constructive Feedback: A Key to Successful Teaching and Learning. (ERIC Document ED 404 291)

On corrective and confirming feedback

Hodes, C.L. (1984). Relative Effectiveness of Corrective and Non-Corrective Computer Feedback on Cognitive Learning of Science. Master’s thesis. Pennsylvania State University. (ERIC Document ED 311 883)

Latham, A.S. (1997). Learning Through Feedback. Educational Leadership 54(8): 86-87.

Silverman, S., L. Tyson, and J. Krampitz. (1992). Teacher Feedback and Achievement in Physical Education: Interaction With Student Practice. Teaching and Teacher Education 8(4): 333-344.

On feedback on performance versus feedback toward self

Geddes, D., and F. Linnehan. (1996). Exploring the Dimensionality of Positive and Negative Performance Feedback. Communication Quarterly 44(3): 326-344.

Tosti, D., and S. Jackson. (1999). Feedback Systems. In H.D. Stolovitch and E.J. Keeps, eds., Handbook of Human Performance Technology: Improving Individual and Organizational Performance Worldwide. San Francisco: Jossey-Bass/Pfeiffer.

On immediate and delayed feedback, effect on learning

Cope, P., and S. Simmons. (1994). Some Effects of Limited Feedback on Performance and Problem-Solving Strategy in a Logo Microworld. Journal of Educational Psychology 86(3): 368-379.

Nishikawa, S. (1988). A Comparison of the Effects of Locus of Control With Feedback Strategies on Factual Information Recall and Retention During Computer-Assisted Instruction. Proceedings of the Annual Meeting of the Association for Educational Communication and Technology, New Orleans, January 14-19. (ERIC Document ED 295 655)

Sturgess, P.T. (1978). Immediate vs. Delayed Feedback in a Computer-Managed Test: Effects on Long-Term Retention. Technical Report #NPRDC-TR-78-15, March-August 1976. Chico, CA: California State University. (ERIC Document ED 160 635)

On frequent, specific feedback, effect on learning

intime (integrating new technologies into methods of education). (2001). Frequent Feedback. Available at http://www.intime.uni.edu/model/learning/freq.html.

Poertner, S., and K. Massetti Miller. (1996). The Art of Giving and Receiving Feedback. AMI: How to Series. Urbandale, IA: Provant Media.

On too-detailed feedback, effect on learning

Sasaki, Y. (1997). Individual Variation in a Japanese Sentence Comprehension Task— Form, Function, and Strategies. Applied Linguistics 18(4): 508-537.

On reinforcement and learning

Lee, K., and J. David. (1998). The Perceived Impacts of Supervisor Reinforcement and Learning Objectives: Importance on Transfer of Training. Performance Improvement Quarterly 11(4): 51-61.

On intrinsic and extrinsic rewards in learning

Bishop, J., S. Kang, and C. Wilson. (1985). Incentives, Learning, and Employability. Washington, DC: National Institute of Education. (ERIC Document ED 268 378)

Ryan, R.M., and E.L. Deci. (1985). Intrinsic and Extrinsic Motivations: Classic Definitions and New Directions. Contemporary Educational Psychology 25(1): 54-67.

Stolovitch, H.D., R.E. Clark, and S.J. Condly. (2001). Incentives, Motivation, and Workplace Performance: Research and Best Practice. New York: Society of Incentives and Travel Executives Research Foundation.

On objectives and learning, positive effects

Choi, J., and D.H. Jonassen. (2000). Learning Objectives From the Perspective of the Experienced Cognition Framework. Educational Technology 40(6): 36-40.

Lee, K., and J. David. (1998). The Perceived Impacts of Supervisor Reinforcement and Learning Objectives: Importance on Transfer of Training. Performance Improvement Quarterly 11(4): 51-61.

Main, R. (1979). Effect of Intended and Incidental Learning From the Use of Learning Objectives With an Audiovisual Presentation. Proceedings of the Annual Convention of the Association for Educational and Communications Technology, New Orleans, March. (ERIC Document ED 172 794)

Chapter 7: Getting Learners to Remember

¹The terms metacognition and metacognitive skills have taken on various meanings over time. Writings about them, however, have two things in common. They tend to agree that knowing about knowing or being aware of your thinking and learning skills helps you strategize more effectively to learn something new. Although the terms may discuss different skills required to learn effectively, they also agree that metacognitive skills are a key differentiator between excellent and poor learners.

²The concept of the mind’s operating system is drawn from Clark, R.C. (2008). Building Expertise. San Francisco, CA: Pfeiffer-Wiley. We suggest the recommendations presented on pages 313-336.

³Even beyond the findings that superior metacognitive skills are highly correlated with learning success of various types, despite equal intellectual capabilities, some researchers have found only a slight correlation between intelligence and metacognitive skills. See for example

Desoete, A., H. Roeyers, and A. Huylebroeck. (2006). Metacognitive Skills in Belgian Third Grade Children (Age 8 to 9) With and Without Mathematical Learning Disabilities. Metacognition and Learning 1(2): 119-135.

Veenman, M.V.J., and M.A. Spaans. (2005). Relation Between Intellectual and Metacognitive Skills: Age and Task Differences. Learning and Individual Differences 15(2): 159-176.

⁴Jean Piaget, the famous psychologist who studied developmental stages of children, indirectly referred to what we now commonly call metacognitive skills. Here is a useful article dealing with metacognitive skills at the third-grade level:

Goldberg, P.D., and W.S. Bush. (2003). Using Metacognitive Skills to Improve 3rd Graders’ Math Problem Solving. Focus on Learning Problems in Mathematics, September 2003, available at http://www.accessmylibrary.com/article-1G1-117322859/ using-metacognitive-skills-improve.html.

⁵This article provides empirical evidence of the successful impact of cueing to help learners develop metacognitive capabilities:

Veenman, M.V.J., R. Kok, and A.W. Blote. (2005). The Relationship Between Intellectual and Metacognitive Skills in Early Adolescence. Instructional Science 33(3): 193-211.

⁶Introduced in the 1980s by researcher Patricia Palincsar, the basic concept and procedure of reciprocal teaching is to “be used to not only help poor comprehenders improve their rate of comprehension, but to enable them to maintain that progress over time and to transfer those critical thinking skills to different learning tasks (p. 169)”:

Palincsar, A.L., and A.L. Brown. (1984). Reciprocal Teaching of Comprehension-Fostering and Comprehension-Monitoring Activities. Cognition and Instruction 1(2): 117-175.

A more recent description of reciprocal teaching is found in Philicia Randolph’s and Jeanice Lewis’s essay, “Reciprocal Teaching,” online at http://red6747.pbworks .com/Reciprocal%20Teaching.

⁷There are many definitions given to the term cognitive strategies. We have selected to use the one that fits with designing and delivering effective training:

West, C.K., J.A. Farmer, and P.M. Wolff. (1991). Instructional Design Implications From Cognitive Science. Englewood Cliffs, NJ: Prentice-Hall.

⁸We have already referred to David Ausubel and his work on advance organizers. Here is a recent study on the use of advance organizers in the accounting arena:

Togo, D.F. (2002). Topical Sequencing of Questions and Advance Organizers Impacting on Students’ Examination Performance. Accounting Education: An International Journal 11(3): 203-216.

⁹The psychological literature is replete with research studies on the use of analogies, metaphors, and the like. Here is an interesting take on their use in science education that offers analogous implications for other disciplines:

Aubusson, P.J., A.G. Harrison, and S.M. Ritchie. (2006). Metaphor and Analogy in Science Education: Serious Thought in Science Education. Science & Technology Education Library 30: 1-9.

¹⁰The 1970s and 1980s were years in which the use of memory aids, especially mnemonic devices and strategies, were studied as means to facilitate retention and recall for a variety of populations from young children and older students to learning-disabled persons, the elderly, and even Alzheimer patients. Overall, memory aids and mnemonic devices and strategies were found to be effective for retention and recall. To read more about these, we suggest three friendly readings you can access online:

Ehren, B.J. (2002). Mnemonic Designs, reading 5. Lawrence, KS: University of Kansas, Center for Research on Learning.

http://elearndesign.org/teachspecialed/modules/ocada304_norm1/15/18_5.html. This online article provides a number of mnemonic illustrations that are highly effective. We informally tested these on random adult subjects and found that retention was significantly higher than for equivalent word mnemonics.

Mnemonic Strategies and Techniques—Components of Mnemonic Techniques, Varieties and Uses of Mnemonic Techniques, Educational Applications of Mnemonic Techniques. Available at http://education.stateuniversity.com/pages/2241/ MnemonicStrategies-Techniques.html.

¹¹These three books are worthwhile examining, if you are interested in mnemonics:

Evans, R.L. (2007). Every Good Boy Deserves Fudge. New York: Perigee Book-Penguin. The volume inundates you with a deluge of mnemonics drawn from an almost endless array of topics. It is a great reference book for finding mnemonics on just about anything.

Holliman, C. (2009). The Mnemonics Book: 30 Ways in 30 Days to Maximize Your Memory. Chapel Hill, NC: Professional Press. We particularly like this book because it teaches the reader how to strengthen memory through the strategic use of mnemonics. The author is a professor at Old Dominion University and tutors students on how to prepare for their SAT exams. The author proposes a Velcro Theory of Memory—the more hooks we put into our associations, the better we will recall them. This publication is almost a training manual and draws heavily from cognitive psychological principles.

Stevens, C. (2008). Thirty Days Has September: Cool Ways to Remember Stuff. New York: Scholastics Inc. This book covers a large number of topics, provides mnemonics for each, and then explains how each mnemonic works.

¹²The following references were included in the first edition of Telling Ain’t Training and still have relevance today:

On metacognition

Hacker, D.J., J. Dunlosky, and A.C. Graesser. (1998). Metacognition in Education: Theory and Practice. Mahwah, NJ: Lawrence Earlbaum Associates.

Metcalfe, J.S., and A.P. Shimamura. (1996). Metacognition. Cambridge, MA: MIT Press.

Rampp, L.C., and J.S. Guffey. (1999). Metacognition: A New Implementation Model for Learning. (ERIC Document ED 440 088)

On metacognitive skills

Clark, R.E., and S. Blake. (1997). Analyzing Cognitive Structures and Processes to Derive Instructional Methods for the Transfer of Problem Solving Expertise. In S. Dijkstra and N.M. Seel, eds., Instructional Design Perspectives. Volume II, Solving Instructional Design Problems. Oxford, UK: Pergamon.

Hall, Y.C., and M. Esposito. (1984). What Does Research on Metacognition Have to Offer Educators? Proceedings of the Annual Meeting of the Northeastern Educational Research Association, Ellenville, NY, October 24-26. (ERIC Document ED 254 552)

Mayer, R.E. (1998). Cognitive, Metacognitive, and Motivational Aspects of Problem Solving. Instructional Science 26(1-2): 49-63.

On well-developed metacognitive skills and learning success

Carr, M., and H. Thompson. (1995). Metacognitive Intervention and Interest as Predictors of Memory for Text. Reading research report No. 35. Athens, GA: National Reading Research Center. (ERIC Document ED 387 791)

El-Hindi, A.E. (1996). Enhancing Metacognitive Awareness of College Learners. Reading Horizons 36(3): 214-230.

On metacognitive skill variation, achievement, and intelligence

Davis, E.A. (1996). Metacognitive Scaffolding to Foster Scientific Explanations. Proceedings of the Annual Meeting of the American Educational Research Association, New York, April 8-14. (ERIC Document ED 394 853)

Gilbert, L.C. (1986). Inducement of Metacognitive Learning Strategies: Task Knowledge, Instruction, and Training. Proceedings of the 70th Annual Meeting of the American Educational Research Association, San Francisco, April 16-20. (ERIC Document ED 271 486)

Romainville, M. (1994). Awareness of Cognitive Strategies: The Relationship Between University Students’ Metacognition and Their Performance. Studies in Higher Education 19(3): 359-366.

Veenan, M.V., J.J. Elshout, and A. Meijer. (1997). The Generality Versus Domain Specificity of Metacognitive Skills in Novice Learning Across Domains. Learning and Instruction 7(2): 187-209.

On reciprocal teaching

Kincannon, J., C. Gleber, and J. Kim. (1999). The Effects of Metacognitive Training on Performance and Use of Metacognitive Skills in Self-Directed Learning Situations. Proceedings of the 21st National Convention of the Association for Educational Communications and Technology, Houston, February 10-14. (ERIC Document ED 436 146)

Rosenshine, B., and C. Muster. (1991). Reciprocal Teaching: A Review of Nineteen Experimental Studies. Proceedings of the 72nd Meeting of the American Educational Research Association, Chicago, April 3-7. (ERIC Document ED 394 683)

West, C.K., J.A. Farmer, and P.M. Wolff. (1991). Instructional Design: Implications From Cognitive Science. Englewood Cliffs, NJ: Prentice Hall.

On cognitive strategies

Clark, R.C. (1994). The Causes and Cures of Learner Overload. Training 31(7): 40-43.

Rabinowitz, M. (1988). On Teaching Cognitive Strategies: The Influence of Accessibility of Conceptual Knowledge. Contemporary Educational Psychology 13(3): 229-235.

Reigeluth, R.M. (1980). Meaningfulness and Instruction: Relating What Is Being Learned to What a Student Knows. New York: Syracuse University, School of Education. (ERIC Document ED 195 263)

On advance organizers

Explorations on Learning and Instruction: The Theory Into Practice Database. Subsumption Theory. (2001). Available at http://www.tip.psychology.org/ausubel.html.

Schwartz, N.H., L.S. Ellsworth, L.S. Graham, and B. Knight. (1998). Accessing Prior Knowledge to Remember Text: A Comparison of Advance Organizers and Maps. Contemporary Educational Psychology 23(1): 65-89.

Stone, C.L. (1983). A Meta-Analysis of Advance Organizer Studies. Journal of Experimental Psychology 54: 194-199.

On image-rich comparisons

Ausubel, D.P. (2000). The Acquisition and Retention of Knowledge: A Cognitive View. New York: Kluwer Academic Publishers.

Clement, J.J. (1998). Expert-Novice Similarities and Instruction Using Analogies. International Journal of Science Education 20(10): 1271-1286.

Duit, R. (1991). On the Role of Analogies and Metaphors in Learning Science. Science Education 75(6): 649-672.

Chapter 8: Training Approaches and a Cornucopia of Learning Activities

¹Clark, R.C. (2008). Building Expertise: Cognitive Methods for Training and Performance Improvement, 3rd ed. San Francisco: Pfeiffer-Wiley. This is an outstanding book for training professionals. See especially chapter 17 on learning architectures.

²A cursory examination of online “awareness” training programs and courses offered in a wide variety of subject-matter areas convinces us even more of the vagueness of what they purport to accomplish. We suggest that the reader examine some of these offerings to determine what the real, verifiable accomplishments might be.

³As an exploratory treat, visit Edmunds.com to simulate purchasing or leasing a new or used car. You will learn which models of vehicles are available; what the best price to pay will be in your zip code; and even, with step-by-step guidance, how to negotiate like an expert in purchasing or leasing the vehicle of your choice. Specific vehicle owners offer you their experiences, what they paid, and can even give you advice as required. What a learning experience! We actually did it with no initial training or guidance and ended up negotiating like pros and leasing a vehicle at an excellent rate, as we later learned from other leasers, through the online information sharing function.

⁴Thiagarajan, S., and T. Tagliati. (2011). Jolts. San Francisco: Wiley-Pfeiffer.

Chapter 9: Testing or Examining—What’s the Difference?

¹There is a vast and growing literature on what is labeled as expectancy-value and achievement motivation. In essence, the theory is that how well one believes she or he will perform on a task and how much the person values successful achievement is related to performance. This has been demonstrated experimentally with young children all the way through adults in the workplace. For an excellent immersion into this interesting arena, read

Wigfield, A., and J.S. Eccles. (2002). Development of Achievement Motivation. San Diego, CA: Academic Press.

Wigfield, A., and J.S. Eccles. (2000). Expectancy-Value Theory of Achievement Motivation. Contemporary Educational Psychology 25: 68-81.

²There are a large number of opinions concerning the use of feedback to improve or maintain performance. The best and most complete resource on what research has found over the past 100-plus years is this wonderful article:

Kluger, A.N., and A. DeNisi. (1996). The Effect of Feedback Intervention on Performance: A Historical Review, a Meta-Analysis, and a Preliminary Feedback Intervention Theory. Psychological Bulletin 119(2): 254-284.

A more recent article on this is

VanDijk, D., and A.V. Kluger. (2010). Task Type as a Moderator of Positive/Negative Feedback on Motivation and Performance: A Regulatory Focus Perspective. Journal of Organizational Behavior. The volume and number and page numbers are not available as the article is in press at this time. An advanced online version was published September 6, 2010.

³Research on test anxiety and its effects both on test performance and individuals who experience this phenomenon has a long history. A recent study, available online, that deals with nurses provides some useful baseline information:

Driscoll, R., G. Evans, G. Ramsey, and S. Wheeler. (2009). High Test Anxiety Among Nursing Students. Available at http://www.eric.ed.gov/PDFS/ED506526.pdf.

A more comprehensive, recent study is a downloadable book containing a vast amount of information on test anxiety and ways of overcoming it. Although conducted at the high-school level, the findings and principles are generalizable across age groups:

Bradley, T.R., R. McCraty, M. Atkinson, L. Arguelles, R.A. Rees, and D. Tomasino. Reducing Test Anxiety and Improving Test Performance in America’s Schools: Results From the TestEdge National Demonstration Study. Available at http://store .heartmath.org/s.nl/c.582612/it.A/id.311/.f.

⁴Oludipe, B.D. (2009). Influence of Test Anxiety on Performance Levels on Numerical tasks of Secondary School Physics Students. Academic Leadership 7(4), available at http://www.academicleadership.org/emprical_research/Influence_of_Test_ Anxiety_on_Performance_Levels_on_Numerical_tasks_of_Secondary_School_ Physics_Students.shtml.

Trudeau, T.L. (2009). Test Anxiety in High-Achieving Students: A Mixed-Methods Study. Doctoral dissertation, University of Alberta; NR55624.

⁵Stolovitch, H.D., and E.J. Keeps. (2003). Engineering Effective Learning Toolkit. San Francisco: Wiley-Pfeiffer.

⁶This is one of the best sources of information on testing and specifically on how to create criterion-referenced learning checks and more formal tests:

Shrock, S.A., and W.C. Coscarelli. (2007). Criterion-Referenced Test Development: Technical and Legal Guidelines for Corporate Training. San Francisco: Pfeiffer-Wiley.

⁷We are not offering any specific references on simulations for learning and assessment as this is such a growing field that new information and tools are continually popping up on the web. The world of Web 2.0 has us dwelling in and interacting with virtual realities. A quick browse on the Internet using terms such as learning, testing, assessment, simulation, and virtual reality will be sufficient to get you started. You can then add your own content interest areas to discover numerous resources and guidance.

⁸There is a wonderful, brief download with guidelines for writing tests. It is free and easy to use or share:

Cohen, A.S., and J.A. Wollock. Handbook on Test Development: Helpful Tips for Creating Reliable and Valid Classroom Tests, available at http://testing.wisc.edu/Handbook%20 on%20Test%20Construction.pdf.

⁹The nature of this book is to provide useful information and guidance on a number of topics related to “telling ain’t training.” However, testing is so important as a means for reinforcing and strengthening learning that we feel it is necessary to provide additional references on the subject:

On the positive impact of testing on learning

Eilertsen, T.V., and O. Valdermo. (2000). Open-Book Assessment: A Contribution to Improved Learning. Studies in Educational Evaluation 26(2): 91-103.

Williams, L.P. (2000). The Effect of Drill and Practice Software on Multiplication Skills: “Multiplication Puzzles ‘Versus’ the Mad Minute.” (ERIC Document ED 443 706)

On test anxiety

Cassady, J.C., J. Budenz-Anders, G. Pavlechko, and W. Mock. (2001). The Effects of Internet-Based Formative and Summative Assessment on Test Anxiety, Perceptions of Threat, and Achievement. Proceedings of the Annual Meeting of the American Educational Research Association, Seattle, April 10-14. (ERIC Document ED 453 815)

Spielberger, C.D., and P.R. Vagg. (1995). Test Anxiety: Theory, Assessment, and Treatment. The Series in Clinical and Community Psychology. Bristol, PA: Taylor & Francis.

Williams, J.E. (1992). Effects of Test Anxiety and Self-Concept on Performance Across Curricular Areas. (ERIC Document ED 317 554)

On criterion-referenced testing

Westgaard, O. (1999). Tests That Work: Designing and Measuring Fair and Practical Measurement Tools in the Workplace. Somerset, NJ: Pfeiffer and Company.

On test validation

Eyres, P.S. (1999). Legal Implications of Human Performance Technology. In H.D. Stolovitch and E.J. Keeps, eds., Handbook of Human Performance Technology: Improving Individual and Organizations Performance Worldwide. San Francisco: Jossey-Bass/ Pfeiffer.

Gibbons, P.L. (1998). Are You Flunking Testing? Inside Technology Training 2(6): 24-27.

Gray, T.B. (1997). Controversies Regarding the Nature of Score Validity: Still Crazy After All These Years. Proceedings of the Annual Meeting of the Southwest Educational Research Association, Austin, TX, January 23-25. (ERIC Document ED 407 414)

On writing good tests

Hacker, D.G. (1998). Testing for Learning. Infoline. Alexandria, VA: American Society for Training & Development.

Osterlind, S.J. (1998). Constructing Test Items: Multiple-Choice, Constructed Response, Performance, and Other Formats (Evaluation in Education and Human Services, 47). New York: Kluwer Academic Publishers.

Wiggins, G.P. (1998). Educative Assessment: Designing Assessments to Inform and Improve Student Performance. San Francisco: Jossey-Bass Publishers.

Chapter 10: Training and Technology

¹De Smet, A., M. McGurk, and E. Schwartz. (2010). Getting More From Your Training Programs. McKinsey Quarterly, October 2010. This report includes the findings of a McKinsey survey in which “only one-quarter of the respondents… said their training programs measurably improved business performance, and most companies don’t even bother to track the returns they get on their investments in training.”

²Arthur, Jr, WA., W. Bennett, Jr., P.S. Edens, and S.T. Bell. (2003). Effectiveness of Training in Organizations: A Meta-Analysis of Design and Evaluation Features. Journal of Applied Psychology 88(2): 234-245.

³Clark, R.E. (2001). Learning From Media. Greenwich, CT: Information Age Publishers. To summarize the author’s position, here is a salient quote: “The best…evidence is that media are mere vehicles that deliver instruction but do not influence student achievement any more than the truck that delivers our groceries causes changes in our nutrition. Basically, the choice of vehicle might influence the cost or extent of distributing instruction, but only the content of the vehicle can influence achievement” (p. 13).

Media and Learning—Definitions and Summary of Research, Do Media Influence the Cost and Access to Instruction? Available at http://education.stateuniversity.com/ pages/2211/Media-Learning.html#ixzz19ieTYCsy.

⁴K. Harman, and A Koohang, eds. (2007). Learning Objects: Standards, Metadata Repositories, and LCMS. Santa Rosa, CA: Informing Science. This book presents solid information on the technicalities of producing e-learning based on reusable learning objects. As you will note, the way training must be designed and produced requires a strong grounding in the technical knowledge and skills of instructional design for reusability.

Chapter 11: Learning With Technology: Making It Work

¹As noted in chapter 10, a plethora of terms refers to the use of various forms of technology to support; present; and, in some instances, drive training-learning in the workplace. Currently, most of the terms imply use of computer technology, online learning, and the web in some form or another. Computer technology has allowed us to integrate video, audio, 3-D, simulation, virtual worlds, community forums, texting, and almost all other media and formats into one melting pot that can be delivered to terminals, telephones, tablets, and an ever-expanding array of devices. In Telling Ain’t Training: Updated, Expanded, and Enhanced, we mostly use “training-learning technologies” as the umbrella term. However, we also use e-learning, collaborative learning, online learning, and other terms to break up the redundancy.

²Fallon, C., and S. Brown. (2003). E-Learning Standards: A Guide to Purchasing, Developing, and Deploying Standards-Conformant E-Learning. Boca Raton, FL: St. Lucie Press.

For a more recent publication that is available online, see

Laselle, J. (2008). 5 Steps to Developing E-Learning Standards. Available at http://www .lr.com/Portals/61190/pdfs/eLearning_Standards.pdf.

³A fun, but informative read on this topic is “Converting Instructor-Led Training to E-learning or Distance Learning: Keys to Success” (2008) produced by Bottom-Line Performance, Inc. The brief article also includes readers’ reactions and comments. Available at http://www.bottomlineperformance.com/lolblog/?p=1343.

⁴For a literature review on trainer resistance to technology-based learning:

Bozarth, J. (2006). Classroom Trainer Resistance to E-Learning: Literature Review. Available at http://www.bozarthzone.com/bozarth_classroom_trainer_ resistance_lit_review_2006.pdf.

⁵This term has caught on like wildfire. See Cammy Bean’s comments in her December 10, 2010, blog, “And a clicky-clicky bling-bling to you.” Cammy Bean’s Learning Visions: Musings on E-Learning, Instructional Design, and Other Stuff. A reference we provided earlier is also relevant here:

Thalheimer, W. (2004). Bells, Whistles, Neon and Purple Prose: When Interesting Words, Sounds, and Visuals Hurt Learning and Performance—A Review of the Seductive-Augmentation Research. Available at http://www.worklearning.com/seductive_augmentations.htm.

⁶Stolovitch, H.D., and E.J. Keeps. (2004). Front-End Analysis and Return on Investment Toolkit. San Francisco, CA: Wiley/Pfeiffer.

⁷Professor Michael Resnick of the MIT Media Laboratory has written a strong and persuasive article rejecting “edutainment,” but favoring “playful learning.” The article is illustrated with colorful photographs and can be accessed at http://web .media.mit.edu/~mres/papers/edutainment.pdf.

⁸This is the standard reference for creating valid tests and assessments for the workplace:

Shrock, S.A., and W.C. Coscarelli. (2007). Criterion-Referenced Test Development: Technical and Legal Guidelines for Corporate training, 3rd ed. San Francisco: Pfeiffer-Wiley.

⁹The University of Adelaide, Australia, Centre for Professional Development, provides a succinct explanation of online, synchronous learning in describing its programs: “Synchronous Learning is often referred to as ‘live’ learning and is used in conjunction with online learning. It means that the communications occur at the same time between individuals and information is accessed instantly. People can communicate in ‘real time’ using their computers to both talk to each other as well as text chat. Presentations can be made using electronic whiteboards and electronic slides. This type of interaction is referred to as a ‘virtual classroom.’ Presentations, conversations, and text can be stored (archived) and made available online as a resource.”

According to The Webinar Blog of February 11, 2009, several terms in the online synchronous domain require definition, including web-conferencing, webcasts, and webinars. Here is what this specialized blog offers:

Web conferencing is often used as an umbrella term for all web-based collaboration between two or more people over the Internet. As such, it may encompass webcasts or webinars depending on the context. But more specifically, web conferencing usually refers to interactive collaboration in work groups, such as brainstorming sessions between employees working on a new business strategy or product design. A typical web conference involves a relatively small number of participants, perhaps 15 as a maximum. Everybody is expected to contribute to the flow of ideas and information. You sometimes see the term “web meeting” used in exactly the same context.

A webcast refers to information dispersed to a large audience via the Internet. It might be just a simple audio stream, or it might include visual aids, such as Power-Point slides, recorded video clips, or live software demonstrations.

A webinar expands the idea of a webcast into a more interactive format. The experience attempts to reproduce the benefits of attending a live seminar. Audience members can ask questions of the presenter, and the speaker can survey or poll the audience and get feedback as he or she delivers the information.

¹⁰This excellent article defines, albeit succinctly, synchronous and asynchronous learning. It goes on, however, to deal with research on both of these modes of online learning. The article contains a highly useful set of endnotes with research references. This is a recommended read for those who are interested in pursuing this topic further:

Hrastinski, S. (2008). Asynchronous and Synchronous E-Learning. EDUCAUSE Quarterly 31(4): 51-55.

¹¹This wonderful expression appears to have had its origin in

King, A. (1993). From Sage on the Stage to Guide on the Side. College Teaching 41(1): 30-35.

¹²According to the American Society for Training & Development’s State of the Industry Report 2010, “Employees in the surveyed organizations accessed an average of 31.9 hours of formal learning content in 2009, down from 36.3 hours in 2008. Although the number of hours of learning decreased, the amount still demonstrates that organizations expect employees to allocate a meaningful amount of time to formal learning and development activities.”

¹³Estimates as to how much on-the-job learning takes place informally range from 58 percent to 86 percent. We examined a number of writings and supposed research and found that the methodologies for estimating the percentages varied widely and that authors criticized others’ approaches. What we have seen leads us to believe that the general consensus is a 70/30 split between informal and formal learning. This reasonable book on the subject (that suggests the split is 80/20) explains the informal workplace learning process and helps capture its benefits:

Cross, J. (2007). Informal Learning: Rediscovering the Natural Pathways That Inspire Innovation and Performance. San Francisco: Wiley-Pfeiffer.

¹⁴In surveying a number of articles on both acceleration and obsolescence of knowledge, two findings emerge. The first is that, overall, the most frequently cited estimates suggest a doubling of knowledge every five years. Some state that the rate is every 10 years. Some even suggest that it occurs annually. The bottom line is that no one knows, but experts agree that it is rapid and increasing. In the second case, with respect to obsolescence of knowledge, the estimates are even vaguer. However, in the new technology specialties such as nanotechnology or stem cell research, the pace of new patents suggests that older ones are being replaced or outmoded by an accelerating stream of new ones.

¹⁵Brown, J.S., and J.P. Alper. (2008). Minds on Fire: Open Education, the Long Tail, and Learning 2.0. EDUCAUSE Review 43(1): 16-32. This article clearly explains the vision John Seely Brown and Ralph Alper have about how social media can and is transforming education and much more. An exciting read. Available at http://www.johnseelybrown.com/mindsonfire.pdf.

¹⁶The American Society for Training & Development has produced a useful report that speaks to this point:

Paradise A., L. Patel, H.B. Thompson, M. Vickera, and D. Wentworth. (2010). The Rise of Social Media: Enhancing Collaboration and Productivity Across Generations. Alexandria, VA: American Society for Training & Development.

Chapter 12: Hit or Myth: What’s the Truth?

¹Chapters 2 and 3 of the following reference explain differences between expert and novice learning, thinking, and information processing:

Bransford, J., A. Brown, and R. Cockey, eds. (2000). How People Learn: Brain, Mind, Experience, and School. Washington, DC: NAS Press.

The best source of information on this whole area of expert versus novice is, as noted earlier:

Ericsson, K.A., N. Charness, N. Hoffman, and R.R. Feltovich. (2006). The Cambridge Handbook of Expertise and Expert Performance. New York: Cambridge University Press.

²Geake, J. (2008). Neuromythologies in Education. Educational Research 50: 123-133. Geake also arrives at the same conclusion based on studies involving neuroimaging and tracking brain activity. He demonstrates, rather, the power of using multiple stimuli.

Massa, L.J., and R.E. Mayer. (2006). Testing the ATI Hypothesis: Should Multimedia Instruction Accommodate Verbalizer-Visualiser Cognitive Style? Learning and Individual Differences 16: 321-336. The authors arrived at the conclusion that the value of matching instruction to a specific style or attribute was not much more than folklore, with no true evidence to support the belief or practice.

³Coffield, F., D. Mosely, E. Hall, and K. Ecclestone. (2004). Should We Be Using Learning Styles? London, UK: Learning and Research Centre. Cited in Bishka, A. (2010). Learning Styles Fray: Brilliant or Batty? Performance Improvement Journal 49(10): 9-14.

⁴A great deal of material based on both research and enthusiastic opinion exists concerning the relationship between enjoyment or satisfaction and learning outcomes. Here is a recent list of more serious studies and works:

Blunsdon, B., K. Reed, and N. McNeil. (2003). Experiential Learning in Social Science Theory: An Investigation of the Relationship Between Student Enjoyment and Learning. Journal of Further and Higher Education 27(1): 3-14.

Fielding, M. (2006). Leadership, Radical Student Engagement, and the Necessity of Person-Centred Education. International Journal of Leadership in Education 9(4): 299-313.

Goetz, T., C. Nathan, B. Hall, C. Anne, A. Frenzel, and R. Pekrun. (2006). A Hierarchical Conceptualization of Enjoyment in Students. Learning and Instruction 16: 323-338.

Hartley, D. (2006). Excellence and Enjoyment: The Logic of a “Contradiction.” British Journal of Educational Studies 54(1): 3-14.

Remedios, R., D.A. Lieberman, and T.G. Benton. (2000). The Effects of Grades on Course Enjoyment: Did You Get the Grade You Wanted? British Journal of Educational Psychology 70: 353-368.

Rieber, L.P., and D. Noah. (2008). Games, Simulations, and Visual Metaphors in Education: Antagonism Between Enjoyment and Learning. Educational Media International 45(2): 77-92.

This is an interesting study showing that law students, generally considered to be high-performing learners and capable of self-directed learning, strongly prefer structured instruction:

Boyle, R.A., and L. Dolle. (2008). Providing Structure to Law Students: Introducing the Programmed Learning Sequence as an Instructional Tool. Legal studies research paper series, paper #08-0113. Queens, NY: St. John’s University School of Law. Available at http://ssrn.com/AbstractID=1103963.

⁵This study was sponsored by the U.S. Office of Education:

Means, B., Y. Toyama, R. Murphy, M. Bakla, and K. Jones. (2010). Evaluation of Evidence-Based Practices in Online Learning: A Meta-Analysis and Review of Online Learning Studies. Washington, DC: U.S. Department of Education, Office of Planning, Evaluation, and Policy Development–Policy and Program Studies Service. Available at http://www2.ed.gov/rschstat/eval/tech/evidence-based-practices/finalreport.pdf.

⁶The following two references are not only informative with respect to the value of worked-out examples for learning, but also are representative of the large number of studies demonstrating how worked-out examples in endless variations can be applied to improve problem-solving skills.

Kim, R.S., R. Weitz, N. Heffernan, and N. Krach. (2009). Tutored Problem Solving vs. “Pure” Worked Examples. In N.A. Taatgen and H. van Rijn, eds., Proceedings of the 31st Annual Conference of the Cognitive Science Society, 2009. Austin, TX: Cognitive Science Society.

Ringenberg, M.A., and K. VanLehn. (2006). Scaffolding Problem-Solving With Annotated, Worked-Out Examples to Promote Deep Learning. In M. Ikeda, K.D. Ashley, and T.W. Chan, eds., Intelligent Tutoring Systems 8th International Conference Proceedings, Jhongli, Taiwan. Berlin/Heidelberg, Germany: Springer-Verlag.

⁷Jennifer Herrod has produced a clear, brief tutorial on cognitive overload, how it affects memory, and what instructional strategies you can use to decrease the burden placed on the learner when there is a lot of content. Try it out: http://www.jchconsulting.com/fall2000/index.htm.

John Sweller also offers an online, brief introduction to cognitive load theory along with a good list of readings on the topic (http://www.scitopics.com/Cognitive_ Load_Theory.html):

Clark, R.C., F. Nguyen, and J. Sweller. (2006). Efficiency in Learning: Evidence-Based Guidelines to Manage Cognitive Load. San Francisco, CA: Pfeiffer.

Geary, D. (2007). Educating the Evolved Mind: Conceptual Foundations for an Evolutionary Educational Psychology. In J.S. Carlson and J.R. Levin, eds., Psychological Perspectives on Contemporary Educational Issues. Greenwich, CT: Information Age Publishing.

Sweller, J. (2003). Evolution of Human Cognitive Architecture. In B. Ross, ed., The Psychology of Learning and Motivation. San Diego, CA: Academic Press.

Sweller, J. (2004). Instructional Design Consequences of an Analogy Between Evolution by Natural Selection and Human Cognitive Architecture. Instructional Science 32: 9-31.

Sweller, J., and S. Sweller. (2006). Natural Information Processing Systems. Evolutionary Psychology 4: 434-458.

Finally, for a short, fun article with a serious theme, read John Naish’s article in the June 2, 2009, The Times (of London) titled “Warning: Brain Overload.” In it he reports, “Scientists fear that a digital flood of 24-hour rolling news and infotainment is putting our primitive grey matter under such stress that we can no longer think wisely or empathise with others.”

⁸Stephen R. Ruth, Martha Sammons, and Lindsey Poulin raise concerns about the quality of online or distance learning programs. In a well-documented argument, they point out that the quantity of e-learning, especially that offered by higher education institutions may be suspect:

Ruth, S.R., M. Sammons, and L. Poulin. (2007). eLearning at a Crossroads: What Price Quality? Educause 30(2): 32-39.

⁹Finding accurate information on costs and use of technology-delivered learning is difficult. Many of the costs are indirect ones and are therefore hard to capture (for example, updating of equipment, security, and maintenance). An interesting report that provides some benefits and concerns for technology-delivered instruction is a survey report of 110 professionals using technology for informal learning. Most of the survey respondents are IT sales and marketing professionals and turn to technology for acquiring product knowledge:

Training Industry Inc. and Intrepid Learning Solutions. (2010). Effectively Using Technology-Enabled Informal Learning. Cary, NC: Training Industry Inc. and Intrepid Learning Solutions.

¹⁰Shaw, S., and S. Sniderman. (2002). Reusable Learning Objects: Critique & Future Directions. In M. Driscoll and T. Reeves, eds., Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2002. Chesapeake, VA: AACE.

¹¹This foundational book in human performance technology, the field concerned with obtaining valued behavior and accomplishment in the workplace details, with numerous examples, the many factors that strongly affect how people perform. It places skills and knowledge within the basket of other individual and environmental influences on obtaining “worthy” results:

Gilbert, T.F. (1996). Human Competence: Engineering Worthy Performance. Washington, DC: International Society for Performance Improvement.

The most up-to-date set of readings on what truly affects performance in workplace or organizational settings is the three-volume series of handbooks that not only deal with what makes for effective performance, but also provides cases, tools, and a great deal of evidence on the impact of individual variables influencing outcomes:

Silber, K.H., W.R. Foshay, R. Watkins, D. Leigh, J.L. Mosely, and J.C. Desinger, eds., Handbook of Improving Performance in the Workplace, volumes 1-3. San Francisco: Pfeiffer-Wiley.

¹²A great deal of controversy exists about the exact amount of training that gets transferred to the workplace. This is a futile argument as a number of factors affect transfer: utility of the training, the workplace environment, inhibitors and facilitators that occur posttraining, and even the attitudes of supervisors and colleagues. What appears to be the consensus of researchers is that the training event is one thing. The application is quite another. Some influencing factors relate to the quality and relevance of the training; most reside outside of the trainee. For a more rounded discussion of the important issue of transfer of training–transfer of learning, explore these publications:

Arthur, W., W. Bennett, P.S. Edens, and S.T. Bell. (2003). Effectiveness of Training in Organizations: A Meta-Analysis of Design and Evaluation Features. Journal of Applied Psychology 88(2): 234-245.

Bhati, D. (2007). Factors That Influence Transfer of Hazardous Material Training: The Perception of Selected Fire-Fighter Trainees and Supervisors. PhD dissertation. Orlando, FL: University of Central Florida.

Lim, D.H., and M.L. Morris. (2006). Influence of Trainee Characteristics, Instructional Satisfaction, and Organizational Climate on Perceived Learning and Training Transfer. Human Resource Development Quarterly 17(1): 85-115.

Morrow, G.M., M.Q. Jarrett, and M. Rupinsky. (1997). An Investigation of the Effect of the Effort and Economic Utility of Corporate-Wide Training. Personnel Psychology 50: 91-119.

Rodríguez, C.M., and S. Gregory. (2005). Qualitative Study of Transfer of Training of Student Employees in a Service Industry. Journal of Hospitality & Tourism Research 29(1): 42-66.

Subedi, B.S. (2006). Cultural Factors and Beliefs Influencing Transfer of Training. International Journal of Training and Development 10(2): 88.

Sugrue, B., and R.J. Rivera. (2005). ASTD 2005 State of the Industry Report. Alexandria, VA: American Society for Training & Development.

Tracey, J.B., S.I. Tannenbaum, and M.J. Kavanagh. (1995). Applying Trained Skills on the Job: The Importance of the Work Environment. Journal of Applied Psychology 80(2): 239-252.

Yamnill, S., and G.N. McLean. (2005). Factors Affecting Transfer of Training in Thailand. Human Resource Development Quarterly 16(3): 323-344.

¹³Dietrich, A., and R. Kanso. (2010). A Review of the EEG, ERP, and Neuroimaging Studies of Creativity and Insight. Psychological Bulletin 136(5): 822-848.

¹⁴Barnett, L. (2006). The Universe and Einstein. Mineola, NY: Dover Publications. The book is a reissue of the original published in 1957.

¹⁵A Discourse on Method was originally published by Descartes in 1637 in Leiden. This quote was drawn from

Descartes, R. (2004). A Discourse on Method: Meditations on the First Philosophy—Principles of Philosophy. London, UK: J.M. Dent. This translation by John Veitch was originally published in 1912. Of particular interest to anyone who would like to dig a little more deeply into what lies behind the quote, read the first part: Discourse on the method of rightly conducting the reason, and seeking truth in the sciences.

¹⁶Here is a list of references that were included in the first edition of Telling Ain’t Training that we believe still make for good reading:

Phrenology

Chevenix, R. (1828). Gall and Spurzheim—Phrenology. Foreign Quarterly Review 2: 1-59.

Pearson, K. (1906). On the Relationship of Intelligence to Size and Shape of the Head. Biometrika 5: 105-146.

Riley, J. (1999). Did They Actually Really Believe This? Authentic Documents as a Window on the Past. Social Studies and the Younger Learner 11(3): 2-5.

Tomlinson, S. (1997). Phrenology, Education, and the Politics of Human Nature: The Thought and Influence of George Combs. History of Education 26(1): 1-22.

The brain as a muscle

Thorndike, E.L., and R.S. Woodworth. (1901). The Influence of Improvement in One Mental Function Upon the Efficiency of Other Functions. Psychology Review 8: 247-261.

Experts and unconscious competence

Bradley, F. (1997). From Unconscious Incompetence to Unconscious Competence. Adults Learning (England) 9(2): 20-21.

Ericsson, K.A., and J. Smith. (1991). Toward General Theory of Expertise: Prospects and Limits. New York: Cambridge University Press.

Specific aptitudes and learning

Snow, R. (1989). Aptitude-Treatment Interaction as a Framework for Research on Individual Differences in Learning. In R. Ackerman, R.J. Sternberg, and R. Glaser, eds., Learning and Individual Differences. New York: W.H. Freeman.

Tobias, S. (1985). Review, Other Macroprocesses, and Individual Differences. Proceedings of the 69th Annual Meeting of the American Educational Research Association, Chicago, March 31-April 4. (ERIC Document ED 258 134)

Enjoyment or satisfaction and learning

Bers, T.H. (1975). The Relationship Between Learning and Enjoyment: A Study of Student Perceptions of Teaching Techniques. Morton Grove, IL: Oakton Community College. (ERIC Document ED 119 764)

Clark, R.E. (1980). Do Students Enjoy the Instructional Method From Which They Learn Least? Antagonism Between Enjoyment and Achievement in ATI Studies. Proceedings of the Annual Meeting of the American Educational Research Association, Boston, April. (ERIC Document ED 188 601)

Media and learning

Clark, R.E. (1999). Media Will Never Influence Learning. Educational Technology Research and Development 47(2): 21-29.

Working out problems versus worked-out problems and learning

Van Morrienboer, J.J.G. (1997). Training Complex Cognitive Skills: A Four Component Instructional Design Model for Technical Training. Englewood Cliffs, NJ: Educational Technology Publications.

Van Morrienboer, J.J.G., and F.G.W. Paas. (1990). Automation and Schema Acquisition in Learning Elementary Computer Programming: Implications for the Design of Practice. Computers in Human Behavior 6(3): 273-289.

Too much content and learning: information overload

Clark, R.C., and D. Taylor. (1994). The Causes and Cures of Learner Overload. Training 31(7): 40-43.

Doring, A. (1999). Information Overload? Adults Learning (England) 10(10): 8-9.

Martin, D.W. (1980). Performance During the Stress of Processing Overload. Proceedings of the 86th Annual Convention of the American Psychological Association, Montreal, Quebec, September 1-5. (ERIC Document ED 195 892)

Sweller, J., and P. Chandler. (1991). Evidence for Cognitive Load Theory. Cognition and Instruction 12(3): 185-233.

Technology-based learning: need for evidence of effectiveness and cost savings

Bork, A. (1991). Is Technology-Based Learning Effective? Contemporary Education 63(1): 6-14.

Whalen, T., and D. Wright. (1999). Methodology for Cost-Benefit Analysis of Web-Based Telelearning: Case Study of the Bell Online Institute. American Journal of Distance Education 13(1): 24-44.

Technology: not necessarily the key to future workplace success

Ellis, R.K. (2000). Technology for Good/Technology for Evil. Training and Development 54(11): 32-33.

McCune, J.C. (1998). The Productivity Paradox. HR Focus 75(4): 4-5.

Weill, M., and L.D. Rosen. (1999). Don’t Let Technology Enslave You. Workforce 78(2): 56-59.

Workplace performance: alternatives to training

Mager, R.F. (1992). What Every Manager Should Know About Training or “I’ve Got a Training Problem”…and Other Odd Ideas. Belmont, CA: Lake Publishing.

Mager, R.F., and P. Pipe. (1997). Analyzing Performance Problems: Or You Really Oughta Wanna. Atlanta: Center for Effective Performance.

Sanders, E.S., and S. Thiagargajan. (2001). Performance Intervention Maps. Alexandria, VA: American Society for Training & Development.

Stolovitch, H.D., and J-G. Maurice. (1998). Calculating the Return on Investment in Training: A Critical Analysis and a Case Study. Performance Improvement 37(8): 9-15, 18-20.

Successful performance during training and lack of transfer to the job

Baldwin, T.T., and J.K. Ford. (1988). Transfer of Training: A Review and Directions for Future Research. Personnel Psychology 41(1): 63-105.

Ford, J.K., and D.A. Weissbein. (1997). Transfer of Training: An Updated Review and Analysis. Performance Improvement Quarterly 10(2): 22-41.

Effect of pretraining activities on transfer of learning

Baldwin, T., and R.J. Magjuka. (1991). Organizational Training and Signals of Importance: Linking Pre-Training Perception to Intentions to Transfer. Human Resource Development Quarterly 2(1): 25-36.

Broad, M.L., and J.W. Newstrom. (1992). Transfer of Training: Action-Packed Strategies to Ensure High Payoff From Training Investments. Reading, MA: Addison-Wesley.

Cohen, D.J. (1990). What Motivates Trainees? Training and Development Journal 44(11): 91, 93.

Common sense versus science

Musgrave, A. (1993). Common Sense, Science and Skepticism: A Historical Introduction to the Theory of Knowledge. Oxford, UK: Oxford University Press.

Chapter 13: Concluding Reflections on Telling Ain’t Training

¹Jung, C.G. (1969). The Psychology of the Transference, 3rd ed. Princeton, NJ: Princeton University Press.

²Eggleston, E. (1943). The Hoosier School-Master: A Story of Backwoods Life in Indiana. New York: Books, Inc. Publishers. (Originally published in 1871.)