Research, Development and Evaluation Methods for Scaling Workplace Learning

Introducing learning technology in workplace settings requires a radically different approach than in formal educational settings. The Layers project has explored a number of different approaches along the research and development process. These approaches have been closely interlinked, serving to support our central aim of developing applications to support informal learning in Small and Medium Enterprises and to scale up those applications beyond initial test users to wider sectors and companies. Such an undertaking has had major implications for project planning and methodology.

Thus research, development and evaluation serve to both advance our understandings of the challenges for SMEs in adopting digital technologies and to shape design processes for new digital tools for learning. All methods we have employed share common aspects:

  • They are participatory in that they involve all stakeholders in the research process from the outset and treat them as equal participants.
  • They are iterative in that they start with smaller scale solutions or good enough prototypes and then develop more optimal and scalable solutions over time.
  • They seek to contribute to true innovation in that they combine a research-driven approach with an attempt to take seriously the complexities and messiness of real-world contexts.

Figure 1

Design

The project has been designed around design-based research and co-design processes, bringing together researchers, developers and SME organisations, as well as other stakeholders. Organisations from the health sector in north east England and from the construction sector in north west Germany participated in the project from the outset. The main project work was organised around a number of design teams involving researchers, developers and applications partners in each group. While design processes followed slightly different pathways in each of the two sectors, end users from the two sectors were closely involved in the design process, through testing and providing feedback on iterative versions of prototypes. In both sectors, a series of meetings with key stakeholders, at local, regional and national levels were also organised.

Read more:

Capacity Building

The challenge of digital transformations was identified early on as a key challenge for the project in both sectors. In construction the research partners organised a series of workshops for construction trainers in using different digital media for teaching and training and disseminating these ideas through the Bau ABC workforce.

Read more:

Development

Stakeholder-driven research also requires an agile approch to developing and deploying software. DevOps focuses on supporting deployability, modifiability, testability, and monitor ability. The provision of DevOps services has reduced the need for a big upfront design and allowed for releasing software early and continuously. One particular finding in Layers was the importance of end-user participation in requirements elicitation, early design, beta-testing of prototypes, evaluation, etc. Therefore, we extended DevOps by this important end-user perspective, thus coining the term DevOpsUse. Tools have supported social requirements engineering, installation and maintenance of applications and analytic services. One core aim for these services is to ensure the scalability of the project’s research, development, and evaluation methods from a methodological viewpoint as well as the scalability of project applications from a technical viewpoint.

The DevOpsUse tools and infrastructure also support the release and sustainability of Learning Layers tools and products as Open Source Software.

Read more:

Evaluation

Evaluation has been seen as a central and ongoing activity in the project, once more increasing our theoretical and research driven understandings of the challenges of scaling up learning in SMEs and also feeding back into iterative development and improvement of Learning Layers applications and tools. The approach adopted has been a Stakeholder driven evaluation including extensive surveys and interviews with a wide ranging group of stakeholders in both sectors.

In order to scale up evaluation, our observations and experiences supplemented the development of digital tools for Community Evaluation & Learning Analytics, based on data generated through the use of Learning Layers infrastructure and online end-user feedback. These tools have been integrated in Layers Box.

Read more:

Sustainability, Scaling and Exploitation

A key aim for the Learning layers project has been to develop pedagogies, tools and applications for learning that are sustainable in the longer term than project funding and which can be scaled up, both within the construction and health sectors but also transferred to other sectors. Inevitably different tools are at different levels of maturity and implementation and have different models for sustainability. The project partners have produced a series of exploitation journeys, building on a Business Model Canvas produced for each application.

Read more: