Layers Adapter
Purpose
Layers Adapter is a reverse proxy component with integrated usage data collection and awareness for OpenID Connect-compliant authentication, bundling all Layers services under one virtual service endpoint. The Layers adapter plays a very important role in the integration strands existing in Layers. Built on the Reverse Proxy design pattern [1], Layers Adapter serves as highly scalable and configurable solution to serve as central endpoint to all services in a Layers Box, thereby fostering easy service integration with secure authentication and integrated usage data collection with the MobSOS framework.
Description
In general, Layers Adapter consists of two parts integrated with each other: a reverse proxy and a MobSOS Monitoring Pipeline. Main function of the reverse proxy is to effectively expose a set of RESTful APIs, bi-directionally routing access from Layers Apps to Layers Services and vice versa. OpenID Connect thereby serves as single sign-on solution for secure authentication and authorization to all Layers services. In order to support community evaluation and learning analytics, the reverse proxy allows to collect rich logs from user-to-service interactions in order to make them accessible to subsequent enrichment with contextual metadata and persistence in a database, ultimately enabling community evaluation and learning analytics. This collection of enriched usage data is realized with MobSOS Monitoring Pipeline, being part of the MobSOS framework for community information systems success awareness.
Reverse Proxy
While the reverse proxy concept of Layers Adapter remained stable and proved as solid scaling solution, our choice for a reverse proxy Open Source Software product was subject to incremental improvements during the project for the sake of improving scalability, integration, and overall conceptual match with RESTful microservice architectures (missing reference)[2] and the DevOpsUse methodology [3] developed in Learning Layers. In its final version, Layers Adapter principally builds upon openresty as an Open Source Software bundle including the very lightweight nginx Web server and a set of essential 3rd-party modules, integrated LuaJIT support, SSL termination, load balancing, improved logging, etc. The following aspects were the main drivers for our final choice for a reverse proxy product.
Compliance with RESTful Microservice Paradigm
Any truly RESTful microservice architecture requires a reverse proxy product built upon the same principles. nginx fulfills this requirement, as it was designed for hosting applications close to basic Web standards directly originating from REST design principles. With all services in a Layers Box complying to the RESTful service paradigm, nginx is thus an appropriate choice.
Scalable RESTful Service Management
Every RESTful service to be hosted by Layers Adapter requires explicit mappings of services to resources, including supported methods, parameter sets, etc. as part of its configuration. Following REST design principles of loose coupling and separation of concern, nginx merely requires a single proxy directive of a service’s physical endpoint to a top-level resource in the Layers Adapter URL domain. The details of resource mapping are left to the services themselves, thus providing a low constant configuration effort per service that can be automated with Docker. nginx thus supports a scalable solution to efficiently manage emergent and dynamic sets of services hosted by any Layers Box.
Low Resource Footprint
Following the separation of concern argument, a reverse proxy solution should exactly fit its purpose, be light-weight and keep resource footprints low. nginx is known for being an extremely light-weight, yet powerful and scalable reverse proxy solution exactly fitting our purposes and exhibiting a low resource footprint. A typical nginx installation uses less than 5MB disk space and approximately 15MB random access memory at peak load. With a switch from an enterprise service bus ESB product to nginx, we could drastically reduce disk space usage by 99% and random access memory use by 97%.
Support for Automated Continuous Deployment
With particular respect to our DevOpsUse methodology and the modular containerization of Layers Box services with Docker under the governance of Layers Adapter, nginx’s modular file-based configuration approach allows for automated continuous deployments and load balancing.
Automated Security & Privacy
In the same fashion as for other configuration options, nginx provides simple configuration file-based SSL termination for encrypted data exchange with Layers Boxes, thus guaranteeing automated deployments of security and privacy measures.
Rich & Easily Configurable Logging
nginx provides a single well-documented directive for including rich logging of Layers Box-wide community interaction with Layers services on the application protocol level. The availability of rich log data allows for standard auditing, but also for sophisticated evaluation and learning analytics, as it is realized with the MobSOS framework. In particular, logging integrates seamlessly with MobSOS Monitoring Pipeline.
MobSOS Monitoring Pipeline
As part of the MobSOS framework for community information systems success awareness, MobSOS Monitoring Pipeline [4][5] is dedicated to the collection of context and metadata-enriched usage data. The principle is shown in the figure below. The entry point to the pipeline is a buffered stream of real-time usage data from the reverse proxy’s log module. Raw log data is then piped through an arbitrary number of linked elements. The initial pre-processing elements realize non-idempotent data transformation functions such as tokenization, cleaning, filtering and anonymization. All subsequent elements are idempotent. The first of these elements persists the pre-processed log data to the MobSOS database within Layers Box premises. Subsequent enrichment elements derive further context information from pre-processed data with the help of data enrichment functions and persist the resulting data in the MobSOS database, thereby guaranteeing referential integrity to the original pre-processed data. The underlying database engine can then easily join pre-processed log data and all enrichments in one usage data vector. In Layers, enrichment functions extract learners and the artifacts they used from OpenID Connect access tokens contained in the log data, thus allowing to follow the digital traces of individual learner interactions with Layers Box and its services, applications, and other resources. Further enrichments include geo-location, i.e. deriving geospatial client location from IP addresses, or interaction clustering, i.e. assigning individual interactions to categories of interaction types.
Provided Services
Layers Adapter provides no own services functionality. It rather provides facilities to configure and manage the ecosystem of services hosted in a Layers Box and serves as main endpoint, under which all Layers Box services can be accessed with a uniform resource locator scheme.
Scaling and Integration
All parts of Layers Adapter, including MobSOS Monitoring Pipeline, are integral parts of any Layers Box. In the context of our DevOpsUse strategy, Layers Adapter thus serves as main entrypoint for service use, supporting monitoring, evaluation, and learning analytics for improved reflection and awareness on overall success of Layers Box and its hosted services. The default configuration in Docker guarantees automatic data collection without any further efforts besides basic configuration.
The Layers Adapter allows horizontal as well as vertical integration. Some Layers services want to use the full capacity of scaling offers of the Layers infrastructure, including cloud-based hosting, cloud-based storage, cloud-based identification and cloud-based multimedia services. This is known as vertical integration. Horizontal integration means to offer a minimal set of core integration services, which consist in the moment of the OpenID Connect cloud-based identification service that gives every informal learner in the Layers digital ecosystem a unique identification for using informal learning solutions.
As for scalability, we carefully selected the most fit and at the same time parsimonious reverse proxy product available, thus guaranteeing compliance to the RESTful microservice paradigm, scalable service management, low resource footprint, support for automated continuous deployment, support for security and privacy, as well as rich and easily configurable collection of usage data for large-scale community evaluation and learning analytics.
Material
Links
Developers and Contributors
References
- P. Sommerlad, “Reverse Proxy Patterns,” in Proceedings of the 8th European Conference on Pattern Languages of Programms (EuroPLoP ’2003), Irsee, Germany, June 25-29, 2003., 2003, pp. 431–458.
- P. de Lange, P. Nicolaescu, M. Derntl, M. Jarke, and R. Klamma, “Community Application Editor: Collaborative Near Real-Time Modeling and Composition of Microservice-based Web Applications,” in Modellierung 2016 Workshopband, 2016, pp. 123–127.
- R. Klamma, I. Koren, P. Nicolaescu, D. Renzel, M. Kravčík, M. Shahriari, M. Derntl, G. Peffer, and R. Elferink, “DevOpsUse - Scaling Continuous Innovation,” Learning Layers Project, Deliverable D6.3/Report 4, 2015.
- D. Renzel and R. Klamma, “From Micro to Macro: Analyzing Activity in the ROLE Sandbox,” in Proceedings of the Third International Conference on Learning Analytics and Knowledge, 2013, pp. 250–254. DOI: 10.1145/2460296.2460347
- D. Renzel, R. Klamma, M. Kravcik, and A. Nussbaumer, “Tracing Self-Regulated Learning in Responsive Open Learning Environments,” in Advances in Web-Based Learning - ICWL 2015, Berlin-Heidelberg, 2015, vol. 9412, pp. 155–164. DOI: 10.1007/978-3-319-25515-6_14