Integration Readiness levels Evaluation and Systems Architecture: A Literature Review

— The success of complex systems projects is strongly influenced by their architecture. A key role of a system architect is to decide whether and how to integrate new technologies in a system architecture. Technology readiness levels (TRL) scale has been used for decades to support decision making regarding the technology infusion in complex systems, but it still faces challenges related to the integration of technologies to a system architecture. Integration Readiness Levels (IRL) scale has been elaborated in the last decade to face these challenges, representing the integration maturity between the technological elements of a system. The aim of this theoretical article is to perform a literature review on IRL scale evaluation and on systems architecture, through bibliographic research. Results show the review organized in five topics that surrounds the research objective, presenting the IRL and TRL scales evolution, comparing their evaluation practices, and exploring the architecture complexity of systems. Suggestions for future research are proposed based on these results.


I. INTRODUCTION
Systems architecture has strong influence on the success or failure of complex systems (Maier & Rechtin, 2000), and one of the key roles of a system architect is to decide whether and how to integrate new technologies in a system architecture (Crawley, Cameron, & Selva, 2016). Complex Products and Systems (CoPS) are defined (Hobday, 1998) as high cost and engineering intensive products, systems, networks and buildings. The Technology Readiness Levels (TRL) scale was developed to support decision making in relation to the introduction of technologies during the development of complex systems (Mankins, 2009). Although this scale has been used for decades, it does not reflect well the integration of technological elements into the system architecture, and its application has other challenges related to system complexity, project planning, subjectivity and imprecision of the scale (Olechowski, Eppinger, & Joglekar, 2015).
In the space systems industry, the current global scenario presents notable factors such as intense technological innovation, growing globalization, entrepreneurship, proliferation of increasingly smaller satellites, and product modularization (Futron, 2014). Many techniques used for space systems development were conceived at the time of the space race, where the projects had large budget and greater continuity in planning (D. Hastings, 2004;Ross, Hastings, Warmkessel, & Diller, 2004). Given the current scenario of greater technological, commercial, political and application uncertainties, spacesystem architectures must face such uncertainties (D. E. Hastings, Weigel, & Walton, 2003), and technology readiness assessment methodologies should be updated (Olechowski et al., 2015). The Integration Readiness Levels (IRL) scale was proposed to represent the maturity of the integration between technological elements of a system (Sauser, Verma, Ramirez-Marquez, & Gove, 2006) and has been evolving over the last decade. The objective of this research is to perform a literature review on IRL scale evaluation and systems architecture. The literature review aims to compare the incipient IRL evolution and evaluation practices to the more consolidated TRL literature, and to explore the systems complexity environment where both scales are used, by reviewing concepts related to systems architecture, integration and their representation.

II.
METHODOLOGY Research methodology consisted in bibliographic research with qualitative analysis, comprising five topics. The first topic presents the TRL scale fundamentals and current limitations. In the second topic, the IRL scale is analyzed through an historical perspective and according to topics of interest for this research. The third topic presents methodologies and best practices related to TRL assessment process and the equivalent IRL assessment process. The fourth topic presents concepts about systems architecture and integration. The last topic shows selected concepts about the representation of dependencies in complex systems.
According to Cornford and Sarsfield (2004), the TRL scale is focused on a particular technology evaluation, and significant integration challenges may occur when a technology is included in a space system. So even that the technology is mature, using this technology in new applications may be challenging, and the TRL scale usually does not represent the challenge of integrating the technology in a space system. Olechowski, Eppinger and Joglekar (2015) investigated the use of the TRL scale in different industrial sectors through interviews and analyzes on industry standards and organizational guidelines. These authors (Olechowski et al., 2015) found that the TRL scale is widely used in different complex systems industries and identified fifteen challenges to improve the TRL scale utilization, categorized in three topics: system complexity, planning and review, and assessment validity. Subsequently, Tomaschek, Olechowski, Eppinger and Joglekar (2016) conducted a survey with TRL scale practitioners in different industries worldwide to identify, among the fifteen identified challenges, which were the most priority challenges. The survey results show that the four highest priority challenges are related to the systems complexity, and they are: representation of the integration between technologies, interfaces maturity, modifications in the system and system overall maturity.

INTEGRATION AND SYSTEM READINESS LEVELS
Research initiated at the Stevens Institute of Technology, led by the researcher Brian J. Sauser, proposed two new readiness levels scales (Sauser et al., 2006) to complement the TRL scale, as options to overcome the TRL scale challenges related to the systems complexity, the same challenges identified by Tomaschek et al. (2016). The two new scales proposed were: Integration Readiness Levels (IRL) and System Readiness Levels (SRL). The research about the integration maturity between components in a system is also justified by the fact that failures of many space systems are related to the integration of components (Sauser, Reilly, & Shenhar, 2009). For Sauser, Ramirez-Marquez, Henry and DiMarzio (2008), technology integration is part of the systems engineering effort and demands a quantitative assessment tool to evaluate the risk of technology integration in a complex system. Sauser, Gove, Forbes and Ramirez-Marquez (2010) consider the systems integration definition proposed by Buede (2000), as the aggregation process from components that need to be aggregated from the system configuration items.  also consider the integration process as the upward slope in a "V" model commonly used in systems engineering. Subsequently, the scale was modified to also represent the architecture definition activities, the downward slope in "V" model in systems engineering.  propose that the IRL scale should be able to be applied to different hierarchical levels from configuration items to the system level, and that the scale should represent the integration in sufficiently general terms, but specific enough to be useful.  suggested that scales have been extensively used to support the integration between components in the computer industry, but scales to support more general systems integration are less developed. The main methodologies used to perform a TRA have been qualitative analysis from experts to establish the adequate TRL level, and analysis supported by structured interviews and quantitative methods (Hueter & Tyson, 2010). These quantitative methods may be supported by probability and statistics analyses (Ristinen, 2010 Bilbro (2009). Cornford and Sarsfield (2004) argue that: the TRL assessment techniques at that moment were qualitative; and that the importance of the language and culture involved in the technology transfer process between laboratories and their integration in space systems was generally underestimated. In this way, the TRL assessments were subjective due to factors such as: the team carrying out the assessment was usually the same development team and not a third party with more impartial view; the original settings of the TRL scale could be interpreted in different ways; even though quantitative methods were used, the database was still subjective due to limitations in the TRL scale definitions objectivity. A list of supporting information for each TRL level was developed by the DoD (2011) in order to make more objective assessments, containing information related to systems engineering, verification, technical requirements and circumstances relevant to each TRL level. The ISO 16290:2013 standard (ISO, 2013) presents a list with the work performed and documented as evidences required to achieve each level TRL. This list of evidences presents predominantly elements of space systems verification discipline, accompanied by their respective performance requirements and technology definition documents (ECSS, 2017a). GAO (2016) published a preliminary report in order to establish a methodology based on best practices that could be used by the USA federal government to perform technology readiness assessments (TRA), aiming mainly to support decision making on programs and projects which involve large commitments of financial resources. Some of the TRA best practices described in this document (GAO, 2016) are:


The responsible for the TRA should understand which evidences would be needed for the TRL scale and understand the operational environment in which the technology should operate, depending on whether the assessment is performed to meet government agencies requirements or it is performed as an internal exercise to monitor the technology readiness;  Reliable assessments are supported by artifacts and clear information, such as requirements documents, analyzes, test reports and environmental testing considerations;  Supporting information and evidence needed for each TRL level are best practices. They are not exhaustive and may vary according to the technology or application, so it is necessary to adapt these definitions to better reflect the technology and its application;  The quality of a TRA depends on the accuracy and relevance of the artifacts, test data, analysis reports, and other supporting information. This information can be dependent on other technologies or activities that are outside the assessment scope, but may need to be included to better assess the technology readiness. Changes or refinements in requirements, technology parameters or other factors may affect the TRA, in which case the TRA should be updated. ECSS published the handbook ECSS-E-HB-11A "Technology readiness level (TRL) guidelines" (ECSS, 2017a) as a guide to the TRL scale application in space missions and programs, offering guidelines for the interpretation of the TRL scale and best practices related to the technology readiness assessment process. ECSS  The highest hierarchical level of this mapping is based on operational requirements and activities. Then the functions supporting these operational activities are mapped. After that, the system components that perform these functions are identified. In turn, the components are composed by technologies. Connection diagrams between the components help to understand the system architecture and integration. Fig. 1 illustrates an example of SRA application (Austin & York, 2015).   /dx.doi.org/10.22161/ijaers.5.4.12  ISSN: 2349-6495(P) | 2456-1908(O) environmental interface requirements, data structure and protocols, the system becomes more complex. Blanchard and Fabrycky (2006) proposed the following main system life cycle phases: Conceptual Design, Preliminary Design, Detail Design and Development, Production/Construction, Operational use and system support, Retirement. The activities of testing, evaluation and validation of the system are progressively carried out throughout its development. According to Blanchard and Fabrycky (2006), models are created to represent a system under study and can be classified as physical, analogue, schematic, and mathematical types. Physical models look like what they represent, analogue models behave like the system, schematic models describe graphically a process or situation, and mathematical models represent symbolically the principles of a situation under study. During the early phases of detail design, breadboards, bench-test models, engineering models, engineering software, and service test models are built aiming to verify specific performance or physical design characteristics. Formal tests and demonstrations are carried out during the latter activities of the detailed design phase when pre-production prototype equipment, software, and similar formal procedures are available. Also, according to Blanchard and Fabrycky (2006) the basic architecture of the system is established with the definition of system operational requirements, the concept of maintenance and support and the identification and prioritization of technical performance measures (TPMs). A system architecture represents the system high-level design or configuration, its operational interfaces, anticipated usage profiles (mission scenarios) and the environment in which it must operate, describing how these various requirements should interact for the system. Next, the functional architecture describes the system in functional terms. From this analysis, through the requirements allocation process and the definition of the various resource requirements necessary for the system to reach its mission, the physical architecture is defined. Maier and Rechtin (2000) proposed that the progressive refinement of the design is one of the most basic patterns for the engineering practice. The process of systems architecting is performed through the progression, or gradual reduction of the abstraction, modeling, evaluation criteria, heuristics and purposes, from the initial ideas up to reach the most formal and detailed processes in different fields of engineering. Thus, the evolution and development of models are threated as the core process of systems architecting. The models represent and control the system specification, its design and its production plan. Even after the system delivery, the modeling will be the mechanism to evaluate the system behavior and plan its evolution. Rechtin (2000) proposed that decisions related to the evolution or creation of new system architectures influence directly the competitiveness of organizations to meet the demands of their customers, and therefore influence directly the success of these organizations. Regarding the product development process, Ulrich and Eppinger (2012) proposed that product architecture is the allocation of functional elements to physical elements of the product. The purpose of the product architecture is to define the basic building blocks of the system physical elements in terms of what they do and what are their interfaces with the rest of the product. After completing the architecture definition, it is possible to perform the detailed design and testing of these building blocks, allocate them to teams or suppliers, so that the development of different parts of the product could be done simultaneously. Decisions on the product architecture and modularity influence directly important aspects of the organization success, such as future changes in the product range, components standardization, product performance, manufacturing, and product development management (Ulrich, 1995;Ulrich & Eppinger, 2012). Crawley, Cameron and Selva (2016) proposed that a simple definition for system architecture is the abstract description of the system entities and the relationships between them, and proposed that in systems engineering the architecture can be represented as a set of decisions. These authors (Crawley et al., 2016) proposed that system architecting is a composition of science and art, with the rationalization of decisions through the formulation of how these decisions can impact the system performance. These authors (Crawley et al., 2016) also suggested that the system architecture can be used to map and analyze an existing system, in a reverse engineering method, or be used in the synthesis of a new system, in a direct engineering method. Crawley, Cameron and Selva (2016) considered that a new technology is often at the heart of a new product, and a change in the technology is often a major motivation for a new architecture development. So, according to these authors (Crawley et al., 2016), one of the key roles of the system architect is to decide whether and how to infuse a new technology into a system architecture. This infusion would require deep knowledge of the available technology and its maturity, the process to integrate the technology into the system, and the value that this integration would create. Still according to these authors (Crawley et al., 2016), the TRL scale is useful to support system architects to take these decisions. Ross, et al. (2004) and Hastings (2004) suggested that many techniques to support space systems development were conceived during the space race, where the projects had large budgets and great planning continuity.

International Journal of Advanced Engineering Research and Science (IJAERS)
[ Vol-5, Issue-4, Apr-2018]  https://dx.doi.org/10.22161/ijaers.5.4.12  ISSN: 2349-6495(P) | 2456-1908(O) Nowadays, in addition to face the technical challenges in building such complex systems, engineers must also deal with changes in the political and economic context that influence the design and development of space systems. Hastings, et al. (2003) proposed that decisions in system architectures should help to address these uncertainties, with a focus on strategies such as flexibility and robustness that can lead designers to different optimization solutions to meet specifications or other specific criteria. Crawley, et al. (2004) suggested that system architectures are not static, but they evolve for long periods as technologies mature. They also evolve during the natural process of designing a system. These evolutionary patterns are useful for understanding the importance of the representation and the decisions involved in a system architecture.
In line with the previously described context that systems and their architectures face, Olechowski, Eppinger and Joglekar (2015) proposed that is important to update the TRL scale application methods. The argument is that since the current context of growing systems complexity, greater dynamics of innovation, the current use of TRL in decision-making and the current use in different organizational processes, are significantly different from the context experienced by NASA in the 1970s when the original TRL scale was created.
Regarding the definition for systems integration, the systems engineering literature and standards propose a common notion for systems integration as the process of assembling and integrating elements of smaller hierarchical levels, successively into larger hierarchical levels, until the system and its desired functionalities are realized (Buede, 2000 , 2007) added that to perform the integration of elements into a system, the design and interface logic of these elements must be met. For Sage and Lynch (1998), the integration of technical parameters and interface compatibility are generally assumed as a technical effort carried out during the latter parts of the system development lifecycle. However, if this effort is not planned properly in the definition phases and in the early parts of the development phase, integration is likely to be difficult at the end of the development phase. To perform concurrent engineering and simultaneously develop different parts of the system, initial efforts must be made in system architecture and system partitioning, and then an explicit system integration stage is required.
In this way, Sage and Lynch (1998) suggested that if we consider that the activities of analysis, definition, design, requirements and control of interfaces are all integration efforts, then the system integration occurs at almost every stage of the systems development lifecycle, not just in its later stages. In his proposal for a general system integration theory, Langford (2013) proposed that integration is the approach of building or creating a whole from parts, and is more than simply combining or assembling these parts. Many system integration efforts undergo changing requirements for many reasons, and while there are numerous strategies for solving these system integration problems, it is time consuming to plan and integrate any part of the system as part-by-part, because the problems persist (Ramamoorthy, Chandra, Kim, Shim, & Vij, 1992). The integration concept proposed by Langford (2013) expresses integration of artifacts as "part-to-all-expected" rather than "part-to-part". For example, to integrate parts A and B to reach system C: Plan and integrate Part A in the way that system C should behave, and then Part B in the way that system C should behave, is more effective than integrating Part A to Part B to reach system C. If Part A is not available, Part B can still be integrated with the behaviors of C to show how Part A would (and should) behave. According to this author (Langford, 2013), the application of this "part-to-all-expected" system integration concept illustrates the power of thought and theoretical planning, that is, most situations that need to be addressed during integration planning could be threated regardless of the individual parts situation. Zandi (1986) suggested that science and engineering should make use of systemic thinking, considering that a system is more than the sum of its parts, possessing emergent properties. Crawley, Cameron, and Selva (2016) proposed that emergence refers to what appears, materializes, or surfaces when a system operates, or in other words the functions that emerge when a system operates. Emerging functions can be classified as desired or undesirable, and anticipated or unanticipated (Crawley et al., 2016). Sillitto (2005) proposed that systems engineering could be defined as the management of emerging properties, such as the importance of emerging properties in the management of a system.

REPRESENTATION OF DEPENDENCIES IN COMPLEX SYSTEMS
According to Browning (2016), the Design Structure Matrix (DSM), also called the dependency structure matrix, became a modeling framework widely used in many areas of research and practice. DSM has advantages of simplicity and conciseness in its representation and, supported by appropriate analysis, may also highlight important patterns in system architectures (design . The diagram, in a matrix form, is used to represent the interfaces of a system. The components or functions of the system are placed diagonally, while the other cells in the NxN matrix represent the inputs and outputs of the interfaces, the outputs being represented in rows and the entries in columns. Alternatively, the interfaces may be represented without polarization, that is, without distinction between inputs and outputs. The N2 diagram may be applied at successively lower hierarchical system levels, and may also represent external interfaces to the system. The N2 diagram application is similar to the DSM (Eppinger & Browning, 2012).

IV. CONCLUSION
This paper presented a literature review on IRL scale evaluation and systems architecture, covering five topics that surrounded the main topics of interest. The fundamentals of the TRL scale and its assessment methods were analyzed in order to understand the IRL origin and be able to compare both scales literatures. Systems architecture and systems integration concepts were presented in order to highlight the complexity of systems and explore the context where these scales are used. Selected concepts about the representation of dependencies in complex systems are shown in order to identify methodologies being practiced in the literature. Through the literature review analysis, it is possible to suggest future researches about the IRL scale, such as: Evaluate the scale definition and the evidences needed for each level, towards a more discipline neutral approach, as the TRL assessment is being consolidated as interdisciplinary and relies on verification and documentation practices, and considering that IRL scale is evolving from a data integration focus to a multidisciplinary approach; Explore complementary methods to analyze the integration readiness through additional systems architecture analysis approaches, given that systems architecture is a vast topic, reflecting the complexity of systems; Analyze a system with multiple interface types between its elements is also a topic not yet explored in IRL scale literature. This paper may support researchers and practitioners to better understand the IRL scale evolution, opportunities to the scale evaluation process, and IRL scale relations to the system architecture and integration concepts. IRL scale complements TRL scale as a tool to support decision making in the development of complex systems. System architects need to decide whether and how to integrate new technologies in a system architecture, being the system architecture a critical factor for the system success.