SRI Consulting Business Intelligence


Advanced Search                           
Bringing Futures into Focus
Research Programs Consulting Services What's New? About SRIC-BI Contact Us Search (Advanced)
Learning on Demand
Learning-on-Demand European Meeting Summary
Swiss Federal Institute of Technology; Zürich, Switzerland
16 February 2004

Author: Eilif Trondsen
Learning on Demand program logo

Meeting Summary
Presentations


Introduction

This LoD meeting was organized to take advantage of the meeting at the Swiss Federal Institute of Technology (ETH) by the Advanced Distributed Laboratory (ADL), the first ADL Co-lab Plugfest outside the United States. A number of the 25 participants at the LoD meeting thus also participated in the ADL meeting on the following three days (for presentations at International Plugfest, go to the Resource Center at ADL's Web site: http://www.adlnet.org/index.cfm?fuseaction=srchrslt).

In contrast to previous LoD meetings in Europe, where we have had one overarching theme, this meeting had presentations and discussions across three different themes (see headings below) that attendees had expressed interest in discussing.


eLearning Standards: What's Ahead and What Does It Mean for Learners and Vendors?

This theme was addressed first by Wayne Hodgins, strategic futurist and director of Worldwide Learning Strategies, Autodesk, and by Paul Jesukiewicz, director, ADL Co-Lab. Both Wayne and Paul have been active in learning standards for years and are two of the most well-known speakers on the topic of learning standards. But as a futurist who admits he is mostly concerned with issues that lie a number of years ahead and who views issues from a 40 000-foot perspective, Wayne deals more with conceptual issues and likes to frame the discussion. Paul, on the other hand, operates "closer to the ground."

Here is a summary of a few of the main themes that Wayne addressed:
  • A key aspect of learning is to be able to respond to the unexpected. Although a lot of eLearning deals with "learning and transferring what we know," much of today's business deals with responding well and flexibly to the unexpected. It is important that we learn to cope with growing uncertainty, and in our LoD publications we have often talked about the use of scenarios and simulations as tools to explore an uncertain future and prepare for the unexpected (as I discuss below, ADL's future work will focus more on simulations than it has in the past).

  • The next big thing is getting really small. Wayne has long been a proponent for information and learning objects and believes that the use of small, standard components is a way to "dynamically assemble into 'just the right' solutions" (what he calls "me-learning") and that this use applies to software code, equipment manufacturing, and content as well as human competences (that is, testing and certification). Wayne also referred to both the growing use of the Google search engine (the world is "becoming Googleised") and to Amazon's "inside the book" offering (where one can search and see content from inside more than 135 000 books that Amazon has indexed so far) as one example of other fields where the trend of "getting small" is also taking place. Another example is TiVo (the television service that operates a digital-video recorder that is like a videocassette recorder but with a hard drive and enables users to manage recorded television content better), because it gives viewers more options in terms of accessing and managing content at more of a micro level.

  • Context versus reusability. A trade-off is necessarily here, and smaller and smaller learning objects or content components enable greater reusability, which improves the economics of content development. But we also have a continuum with the following extremes: high reusability (with small components) and low context or high context and low reusability (context is an important factor for improving learning). In a presentation during the eLearning International conference in Edinburgh, Scotland, a few days after our LoD meeting, the lead developer in the U.K. eUniversity project at Sheffield Hallam University discussed this issue and noted that researchers there categorized learning objects into different types—core concepts, activities, readings, and self-assessment—to aid students. They also used what they called a "core document," which they believed was necessary as an "educational narrative to provide integrative cohesive framework for stand-alone learning objects. Primary tutor role is to support using online communication" (Stephen Wan, Sheffield Hallam University). The discussion about these issues at our Zürich, meeting referred also to a recent book published by Chuck Barritt, who was a team member of Cisco's reusable learning object team (the book is Creating a Reusable Learning Object Strategy: Leveraging Information and Learning in a Knowledge Economy, by Chuck Barritt and F. Lee Alderman Jr.; publisher: Pfeiffer).

  • Digital rights in the future. Wayne recognized that this business problem is important, but not new. The music industry in particular has been struggling with it for years. Although the situation by no means has resolution at present and will become more important if increasingly granular content will need tracking and transaction, Wayne believes that digital identifiers will likely be in use and will help us address these digital-rights issues.
According to Wayne, Paul Jesukiewics would "ground us" more by dealing more with what ADL has been doing on the shareable content object reference model (SCORM) front and where it is heading next. But even Wayne's presentation described how First Generation of SCORM (now finished) focused on data exchange and data interoperability. We are now in the Second Generation with a greater focus on behaviorist specification ("how it should work and not just a data model"). Paul gave the group a primer on ADL and SCORM, including the ADL model for standards evolution, which was useful because most people do not have a clear understanding of the various processes and organizations (often with acronym names, such as IMS, IEEE, and W3C; see Paul's presentation for more on this subject).

Paul noted that the focus of ADL's work has been on "does it really work?" and "is it interoperable?" and thus ADL has not really been able yet to focus very much on the "advanced" part of distributed learning that its name refers to. As part of this focus on the "here and now," Paul pointed to the fact that 28 LMS/LCMSs have now been certified through the various Certification Testing Centers that ADL has established. Ten content products have also been certified, and 94 vendors have become SCORM adopters. A key element of SCORM 2004 is sequencing (and dynamic sequencing of objects is what Wayne pointed to as the ultimate "me-learning"), which Paul defined as "predictable, consistent ordering and delivery of learning activities in an instructionally meaningful manner regardless of delivery environment." ADL has also been developing tools to assist with content conversion (from SCORM 1.2 to SCORM 2004). These tools include content-packaging converter (for manifests), sharable content object wrappers, meta-data converter, and sequencing rule examples.

Questions about meta data came up during Paul's presentation, and he recognized that meta-data generation is critical for the overall success of the project. "Stamp" functions are under development to do this generation automatically. Various products are on the market, and a number of them were on display at the Plugfest exhibition. The available products are very technical at the moment, but they are the first generation. More user-friendly products will doubtlessly emerge. A parallel exists with Web-page construction: Authors no longer need to be proficient in HTML to do this it.

The last slide that Paul presented showed two key areas of future activities that ADL will focus on as SCORM evolves. Besides maintaining and extending SCORM 2004, the following two areas will gain growing attention:
  • Expanded scope of SCORM: Integration with other architectures. This area will include simulation (to enable tracking of simulations-based learning in the LMS, for instance), performance support, mobile systems, intelligent tutoring, and knowledge-based systems.

  • New architecture R&D. This area will likely involve Web services, the Semantic Web, new knowledge representation approaches, "next generation" platform, and DAML + OIL. (According to Marcelo Hoffmann of SRI Consulting Business Intelligence, "Currently, the leading ontology system for resource-description framework [RDF] is the DARPA Agent Markup Language [DAML], the result of research and development efforts by the U.S. Defense Advanced Research Projects Agency [DARPA]. DAML now includes concepts that originated in Ontology Inference Layer [OIL], a European project to construct some low-level rules in RDF form. The resulting language is DAML + OIL. More detailed information is available at www.daml.org.")

eLearning and Knowledge Management: What Is Truth and What Is Hype about Convergence and Integration?

A number of our LoD reports—especially eLearning in the Life-Sciences Industry and the Third Quarter 2003 LoD Bulletin, "Integration of eLearning, Knowledge, and Digital-Content Management in Next Generation Learning"—have noted that a growing number of organizations at least now recognize and want to integrate eLearning and knowledge management (KM) but that relatively few organizations have come very far, at least until now. We wanted to use this opportunity to hear from the Europeans attending the meeting and the speakers about what the situation was in Europe on this front, and to what extent the situation in Europe is different from that in the United States.

Andrea Back, who has done both research and consulting in this area for a number of years as part of her work at University of St. Gallen, noted that KM and Educational Management (EM) in Europe are two distinct and separate domains (with different roles and positioning) under Organizational Development, even though they have much the same goals (Andrea sees KM and EM as "two hemispheres in companies"). In most U.S. organizations, KM often involves the chief knowledge officer and an internal consulting group, as well as business-unit business managers. EM, on the other hand, is often led by a chief learning officer and involves people from personnel or corporate universities and training units and the HR units focusing on personnel development. Although Wayne Hodgins noted that the focus of either camp should be on "solving specific business problems" and that the labels really do/should not matter, Andrea noted that the reality of most organizations is that they tend to deal in boxes or silos—and this approach often causes problems and inefficiencies. For instance, in the case of one large European bank, successful eLearning activities in the KM group were uprooted and moved to the EM part of the organization because the manager of this group made it clear that there is "where this learning program belonged."

Andrea presented a "business engineering perspective on eLearning and KM" consisting of three key central layers of strategies, processes and methods, and systems and technologies (at the technical level, Andrea thinks that the convergence between KM and EM is most prominent, but in other areas much wider gaps exist, as the discussion below shows)—with a management layer on either side (representing KM and EM). This model focuses on formal learning—or education and training—and thus leaves out a major part of learning that is more informal and may in fact bridge the KM and eLearning. eCollaboration, which Andrea places in EM or eLearning, could also be a process or activity that tends to fall within both EM and KM and thus can be a bridge between them.

Andrea ended her presentation by contrasting EM and KM in the following four areas:
  • Strategy integration. Learning has only a loose or weak coupling to business strategy (this characterization of the current state of affairs is probably correct, although most organizations recognize and are attempting to rectify it and make the connection and linkage stronger between business strategy and learning), whereas a close coupling between business strategy and KM exists. (In eLearning in the Life Sciences Industry we noted that a number of executives in pharmaceuticals companies have KM in their titles but few have learning, and omission may be indicative of the situation that Andrea referred to.)

  • Place, time, and way of acquiring knowledge. Learning—again the formal part of learning that takes the form of "education" or "training"—tends to be more in the form of seminars and conferences and takes place at home, in large blocks, and with rather weak personalization (the last is, of course, the area where the "me-learning" of Wayne Hodgins and the use of dynamic assembly of learning objects could dramatically change the situation). KM, on the other hand, takes place more in short units, and integrates more with work (and takes place more at work) and with a higher degree of personalization than learning.

  • Cultural aspects. Andrea believes that the target groups for learning are more skeptical toward technology than are KM users and that the latter work more in groups and communities, whereas learners tend to be less group oriented. Learning uses more of a "push principle," whereas KM is more "pull oriented." (These characterizations are less true of more sophisticated learning organizations in the United States that use a variety of tools, technologies, processes and approaches in their learning operations.)

  • Measurement. Measurement tends to be more institutionalized through assessments, exams, and certification processes, whereas KM tends to have less formally structured and weaker measurement and often is more part of the regular employee assessment (that is, it is not separate but integrated overall into the regularly scheduled employee evaluation).
Although we did not have the opportunity to discuss in depth what specific European companies are doing in the area of eLearning and KM integration, the overall thrust of Andrea's presentation implied that European organizations are probably not more advanced than U.S. organizations in integrating eLearning and KM. In fact, North American organizations—Cisco, the Bank of Montreal, and others that I referred to in my presentation (and that we have discussed in our LoD reports)—may have come further along, although most U.S. organizations also have a long way to go before they reach the point of ignoring labels and organizational silos and just focusing on solving the business problems at hand using the best tools and approaches, irrespective of where they reside in the organization.

In my presentation I pointed to factors both favoring and preventing integration, but I also provided a broader framework for eLearning (which includes formal and informal learning and connects them to the value chain; see the figure in my presentation) within which we can see a greater degree of integration between eLearning and KM than we may see in the more formal learning framework presented by Andrea.


Quality Issues in eLearning: What Do They Mean for Industry and Academia?

This last topic has been gaining increasing interest in Europe, and a number of initiatives and research projects, many funded by the EU Commission, are now under way to try to provide some clarity about what "quality in learning" means and what steps are necessary to improve it. Some of the European initiatives include the eLearning Initiative of DG Education and Culture, a number of national initiatives (including in Germany and the United Kingdom), Learning Regions initiative, and European Standardization work. Issues in many of these projects and initiatives include localization/contextualization (dealing with culture, language, and didactics), harmonization versus localization, regional aspects, and integration/inclusion (of new countries joining the EU). Clearly, the "(e)Learning quality puzzle" contains a large number of pieces and will likely continue to gain considerable attention for years to come.

Sabine Seufert of the Swiss Center for Innovations in Learning, and Jan Pawlowski of the University of Duisburg and the European Quality Observatory presented different perspectives on the topic. Sabine's discussion focused on a framework of four key questions about quality and related these questions to University of St. Gallen's collaboration with the European Foundation for Management Development, which has an accreditation program for universities (EQUIS) and an accreditation for corporate universities (CLIP) on eLearning Quality Improvement Program (ELIP). The quality framework and criteria to be used in ELIP have emerged from interviews with 25 experts and a Delphi study of 37 experts and two roundtable discussions. These processes have resulted in 47 quality criteria and a self-assessment guide and review guide for auditor teams.

The four questions and their implications for ELIP are the following:
  • Goals. Why should we evaluate and what are the norms? ELIP has a marketing strategy to "promote the advanced quality of eLearning" with transparency for end customers. From a research perspective ELIP wants a theoretically sound quality framework supported by empirical evidence. ELIP is also emerging because no systematic approach exists today for programs that focus on quality improvement (and the European Quality Observatory lists more than 100 different quality approaches).

  • Objects. What are the targeted objects to evaluate? In ELIP's case, the result will be an eLearning Program for Management Education.

  • Process. How should the evaluation take place, and what are the methods? ELIP's processes focus on the following five elements (see Sabine's presentation for more details with definition and examples of each element): pedagogy, organization, technology, resources and culture.

  • People. Who are the internal and external evaluators? ELIP will rely on both self-assessment and an auditor team that will undertake evaluations.
The project involves a pilot phase with participation by two universities and one company as well as a Ph.D. dissertation focusing on validation of the quality criteria to be used in ELIP. One of the goals is to shift from static quality assurance toward dynamic quality development of eLearning.

Jan Pawlowski followed Sabine and started with a figure showing a number of different factors that one might want to include in an individual quality profile, such as individualization/personalization, interoperability, accessibility, flexibility, ergonomics, and organizational success. He discussed eLearning quality issues within a more general and conceptual framework, but with specific reference to the European Quality Observatory (EQO). The latter is still under development but will become a major repository of information on quality in learning. Jan showed one graphic that showed EQO in the center of a circulatory flow between the following four phases of a quality approach: First, identify requirements and options (input into this phase might include quality-management approaches, quality-assurance approaches, and specific standards and criteria). Second, analyze and compare quality approaches (this step could be done through the use of an EQO model that Jan's presentation shows in slide 8). Third, make a decision for an approach, and fourth, localize and adapt harmonized model (this model could address country and company specific needs).

Both Sabine's and Jan's presentations gave the attendees a good picture of the complexity and diversity of approaches (and needs) that one will have to cope with in the area of learning quality. Although work on many of these issues will very likely continue both at the EU and the national level, we did had insufficient time to discuss how eLearning vendors will likely be affected by this ongoing work and continuing concern about quality. Companies adopting eLearning will have very specific quality needs, and the bottom line for them is: Will eLearning have a significant and cost-effective impact on business performance? If it does, it is likely that the quality of the various components of the eLearning program—consisting of a multitude of formal and informal learning elements and processes—is reasonably good. But the question still remains: How can the quality of the program improve further to ensure maximum performance impact? Thus, the discussion we started in Zürich on these issues will continue, both in future meetings and in our future research publications.




Presentations:
  Mass Customization of eLearning Standards: Oxymoron or Secret to Meeting
European and Global Needs and Diversities
(2.25 MB)
Wayne Hodgins, Strategic Futurist, Director of Worldwide Learning Strategies, Autodesk Inc.
  Future Plans for SCORM and Implications for Learners and Vendors (1.3 MB)
Paul Jesukiewicz, Director, ADL Co-lab
  Corporate eLearning and Knowledge Management: Will convergence bridge the gaps? (1.2 MB)
Prof. Dr. Andrea Back, Institute of Information Management, University of St. Gallen
  Current Status and Future Perspectives on KM and eLearning Integration in U.S.
Corporations
(0.25 MB)
Eilif Trondsen, Director, Learning on Demand Program, SRIC-BI
  What Current Research May Mean for the Future of Quality Management Systems (1.0 MB)
Dr. Sabine Seufert, Swiss Centre for Innovations in Learning, University of St. Gallen
  Perspectives on European Quality Initiatives and Programs and their Impacts
and Implications
(0.4 MB)
Dr. Jan Pawlowski, University of Duisburg and European Quality Observatory



SRI Consulting Business Intelligence -- An SRI International Business Partner
Contact Us / Become a Client Korean   (Korean Inquiries) Japanese web site   (Japanese site)
Privacy Policy Sign up for SRIC-BI News, a free newsletter!
© 2001–08 by SRI Consulting Business Intelligence. All rights reserved. Unauthorized use or reproduction of all or any part of this document is prohibited. webmaster@sric-bi.com.