8. Impact, excellence and beyond: understanding complexity in analysing and assessing socio-technical impacts

Convenors: Mika Nieminen, Kirsi Hyytinen, Peter Biegelbauer, Michael Dinges and Faïz Gallouj.

Session type: Full paper and speed talk sessions.

During the last years, the need for policies, which are systemic by nature and consider the increasing societal complexity, has become evident. Innovation policy cannot be limited to a specific administrative silo or to sector specific issues. Rather, innovation policy makes an effort to cross-sectorally link various fields of policy-making (Biegelbauer & Borrás 2003), necessitating “policy-mixes” and a fully renewed understanding of the role, aims and logic of innovation policy (Diercks et al. 2019; Schot & Steinmuller 2018). Finally, the very basis of policy analysis and impact evaluation is under critique, not only because policy-making does not neatly follow sequential steps (Hoppe 2010), but also because policy processes, when faced with complex challenges, necessitate frequent adaptations (Kuhlmann et al 2019).

These dynamics call for more systemic approaches, holistic and hybrid ways to understand and analyse the complex interaction and social, environmental and economic impacts in networked socio-technical systems. In practice, impact evaluation necessitates new kinds of methods, which take into account the increasing complexity and pace of change in order to guarantee robust and real-time information in decision-making (e.g. Hyytinen 2017; Nieminen et al. 2020). While there is an increasing number of individual tools to generate and handle information and to use it in strategy and management, there is an apparent need for more integrative evaluation approaches.

Recently, new approaches and supplementary methods have been sought for impact evaluation (Reynolds et al. 2016). In general, systemic approaches, foresight exercises and participatory approaches have been recommended to analyze the prospects of innovations and to feed into strategy development. Furthermore, in line with systemic approaches, service innovation scholars have highlighted the need for ‘pluralistic and flexible evaluation’ with multiple criteria (Djellal & Gallouj, 2010; 2013).

This track seeks to contribute to this debate by providing a platform for papers and presentations dealing with theoretical as well as methodological developments, empirical cases and studies on the need, practice and impacts of systemic evaluation in the field of innovation studies and innovation policy.

The topic relates to the overall theme of the conference by asking how does the claimed bipolar relationship between science and innovation appear from the perspective of innovation and knowledge, if they are understood as systemic, interdependent, and intertwined with various societal dimensions?

We welcome abstracts for full paper and speed talk sessions as well as proposals for roundtables.



Biegelbauer, P. and Borrás, S. Eds. (2003). Innovation Policies in Europe and the US. The New Agenda. Aldershot, UK, Ashgate. 

Diercks G., Larsen H., Steward F., (2019) Transformative innovation policy: Addressing variety in an emerging policy paradigm, Research Policy, Volume 48, Issue 4, 2019, https://doi.org/10.1016/j.respol.2018.10.028. 

Faridah D. and Gallouj, F. (2010), The Innovation gap and the performance gap in the service economies: a problem for public policy. In Gallouj, F. and Djellal, F. (Eds.), The Handbook of Innovation in Services. A Multi-disciplinary Perspective. Edward Elgar, Cheltenham, pp. 653-673. 

Faridah D. and Gallouj, F. (2013), The Productivity in services: measurement and strategic perspectives. The Service Industries Journal, Vol. 33, No. 3-4, 282-299. 

Hyytinen K. (2017) Supporting service innovation via evaluation: a future oriented, systemic and multi-actor approachAalto University publication series DOCTORAL DISSERTATIONS, 14/2017, VTT Science, 146https://aaltodoc.aalto.fi/bitstream/handle/123456789/24403/isbn9789526072609.pdf?sequence=1&isAllowed=y  

Kuhlmann, S., et al. (2019). “The tentative governance of emerging science and technology—Aconceptual introduction.” Research Policy 48(5): 1091-1097. https://doi.org/10.1016/j.respol.2019.01.006 

Reynolds M., Gates, E., Hummelbrunner, R., Marra, M. and Williams, B. (2016) Towards Systemic Evaluation. Systems Research and Behavioral Science, 33(5) pp. 662–673.DOI: https://doi.org/10.1002/sres.2423   

Nieminen M., Hyytinen K., Salminen V., Ruutu S. (2020) “Systemic Evaluation in the making: A Case Study” In Lehtimäki H., Uusikylä P., Smedlund A. (eds.) (2020) Society as an Interaction Space. Springer, https://link.springer.com/chapter/10.1007/978-981-15-0069-5_5  

Schot J., W. E. Steinmueller (2018) Three frames for innovation policy: R&D, systems of innovation and transformative change, Research Policy, Volume 47, Issue 9, 2018, https://doi.org/10.1016/j.respol.2018.08.011.