Multimodal transcription as academic practice: A social semiotic perspective – RCA Reading Group 2017 – Wk4, 15/2/17

Below are the details for the text under discussion, I will update this post with notes post the session.

Multimodal transcription as academic practice: A social semiotic perspective
Bezemer, J., & Mavers, D. (2011). Multimodal Transcription as Academic Practice: A Social Semiotic Perspective.
International Journal of Social Research Methodology 14, 3, 191-207.
Online here.

Technology and Embodiment in Ihde and Merleau-Ponty – RCA Reading Group 2017 – Wk3, 1/2/17

Below are the details for the text under discussion, I will update this post with notes post the session.

Brey, P. (2000). ‘Technology and Embodiment in Ihde and Merleau-Ponty.’ Metaphysics, Epistemology, and Technology. Research in Philosophy and Technology, vol 19. ed. C. Mitcham. London: Elsevier/JAI Press.
Online here.

Research through Design as a Method for Interaction Design Research in HCI – RCA Reading Group 2017 – Wk2, 25/1/17

Below are the details for the text under discussion, I will update this post with notes post the session.

Research through Design as a Method for Interaction Design Research in HCI – John Zimmerman, Jodi Forlizzi, Shelley Evensong. Carnegie Mellon University
online here:

Projection, Place, and Point-of-View in Research Through Design – RCA Reading Group 2017 – Wk1, 18/1/17

Below are the details for the text under discussion, I will update this post with notes post the session.

Chapter 18: Projection, Place, and Point-of-View in Research Through Design Steven Dow, Wendy Ju & Wendy MackayIn: The SAGE Handbook of Digital Technology Research
Edited by: Sara Price, Carey Jewitt & Barry Brown

Ben Stopher & Tobias Revell : Reflections on the SCD Summer School at LCC, UAL.

This article was originally posted on Medium, here.

Between July 4th and 14th 2016 at the London College of Communication, UAL we led a Summer School looking specifically at examining the practices of Speculative and Critical Design.



Speculative and Critical Design is a contentious term. Critical Design was first coined by Tony Dunne in Hertzian Tales (1999) as an approach to design ‘…challenge narrow assumptions, preconceptions and givens about the role products play in everyday life. It is more of an attitude than anything else, a position rather than a method. This ideology was explored under Tony Dunne’s leadership of the now defunct MA Design Interactions at the Royal College of Art. James Auger developed the term speculative design during his time with Dunne at the RCA saying that it ‘…combines informed, hypothetical extrapolations of an emerging technology’s development with a deep consideration of the cultural landscape into which it might be deployed, to speculate on future products, systems and services.’ Along the way these two terms became conflated and combined to form the umbrella term ‘Speculative and Critical Design’ which has since become the cover-all for a set of inquisitive, future-orientated and critical design practices, methods and ideals. It’s not without its academic detractors and critics point to a euro-centrism, colonial attitude and elitism as flaws. Nonetheless, as a ‘field’ it has radically challenged ideas of what design is, what design is for and what the role of the designer is in a world riddled with uncertainty and instability. It now sits in curriculums of design schools around the world and is gaining acceptance in normally industry-focused design events such as the Milan Design Salone. It’s been used as an applied tool of policy research and development and as part of the iterative process in product development.

Teaser for the SCD Summer School

SCD in the Studio

All this is to say that even if SCD cannot be not a part of mainstream design, it’s certainly more than a fad. At the London College of Communication, UAL we’ve been working it into the curriculum of various courses (Specifically MA Interaction Design Communication & BA Interaction Design Arts) as a way to enhance and supplement courses where human experience, technology and social critique play a vital role. SCD invites these students to think wider and deeper about their design process and practice, how they integrate with the needs of people for whom they are designing and provides a radical framework for iterative concept testing. Most importantly, it invites them to consider the nature of a design problem, question the standard approach of just adding technology and actually think about when not designing is the best solution.

As educators, we don’t often get the chance to test the ideas and tools we’re working with in an intensive context. Usually, and in order to provide for the overall design education experience, these techniques are used lightly and over an extended period. Part of the reason for running the Summer School was to stress test the existing tools we’ve used over the last few years and to invent and try out some new ones with a group of design-literate students.

Building the School

The participants came from all over the world; the US, East Asia, the Middle East, South America and Europe and from a wide range of backgrounds — architecture, design, technology, social science, activism, policy and art. Most of them had a post-graduate education in their field or were otherwise established professionals. This lead to a great energy and some challenging debate and discussion. Most participants were also well versed in the texts and projects of the field and a lot of the criticism.

Each day began with a talk from a thinker or practitioner on an aspect of the subject. Rather than simply bringing in ten people who saw themselves as open SCD practitioners, we invited most speakers from a range of ‘border’ fields that provided insight on a specific aspect of the practice.

Paul Graham Raven (writer, theorist, critical futurist) kicked off by introducing concepts of narrative, scenarios and world building. He also began introducing the vexed problems of futuring, utopia and dystopia which often come up in the borderlands of thinking speculatively about how design and technology might proceed.

Dr Georgina Voss continued this trail by picking up on ideas of imaginaries and intention in imagining and then building the future, talking about how worlds are often unintentionally willed into being by rendering and policy.

Seamlessly, Dr Dan Lockton continued with this line of how intent is embodied through design in a talk around design for behavioural change and it’s history in designing the future.

Dr Catharine Rossi then took us back to a historical line for radical approaches to design through the activities of the Italian Radicals, drawing parallels with the institutional and political contexts of design then and now.

Ai Hasegawa (MIT Media Lab) Anab Jain (founder Superflux) and David Benque (Microsoft Research, Cambridge UK) came to talk about their practices. Although all RCA Design Interactions graduates who use a form of speculative and critical design (no caps) in their practice, they showed the participants different approaches of bringing this practice to research and industry and different models for operating in the ‘attitude’ of critical thinking.

Ai Hasegawa — MIT Deisgn Fictions Group

Anan Jain — Founder SuperFlux

David Benque — Microsoft Research, Cambridge UK

Finally, Nicolas Nova came to talk about design fiction — another often affiliated term — and his practice with Near Future Laboratory, providing another insight into how these practices interlock with the existing design establishment.

Week 1 ‘People and Perspectives’

The first week of the Summer School was structured around a series of rapid workshops in core techniques and ideas while the second week was focused on a participant-driven brief which they undertook in groups.

The idea for the first week was to do something we’ve found to be successful in other contexts; an early stage of bombardment and ‘keeping it moving, keeping it weird’ (as Justin Pickard would say) to open up the field and get the students in a productive mode — quickly iterating based on new ideas and constraints with tangible outputs. The workshops of the first week were also structured around introducing material ideas of speculative and critical design practice. We borrowed elements from Matt Jones’ Hopeful Monsters to kick off with, getting the students to forensically take apart familiar objects and think about the social, cultural, material and political constraints that had forced their construction. The students were then asked to switch these constraints around and redesign the object as if from another world.

Workshop output from week 1

We introduced the Strange Telemetry version of Future Tarot to get the students to invent new futures and then systems, services and objects that might exist in these futures. We also introduced the idea of perverse incentive and evil users, asking the students to consider how their new systems might be broken, hacked or hurt people unintentionally — a kind of reverse user-journey for corrupt systems.

Taking James C. Scott’s Seeing Like a State as a starting point we also turned the participants into top-down planners. Similar to Hopeful Monsters, we asked them to analyse the construction of historical cities for indicators as to the intentions behind them and then redesign new cities based on alternative political frameworks. Finally, we ended with a rapid-fire science-fiction film day that allowed the students to get a bit looser with the ideas of story and narrative they’d been picking up.

Week 2 ‘the Brief’

The second week introduced a brief that was student-led. The brief asked the participants to think about the conflicts between mass manufacturing, technological customization and the political drive of individuality. A note on the brief put it like this: “This brief asks you explore trajectories of extreme personalisation looking at how increasingly individual approaches to design enabled by ubiquitous technology and distributed production methods, will change our ecologies of design.”

A critical idea in this brief is that normative design is generous at a wide scale — as its serves a wide group — and that a move from mass manufacturing to distributed manufacturing may undermine this generosity and asks what might this mean for concepts of accessibility. It also asks students to consider where these imperatives come from i.e. the political economies that are involved and where these emanate from. A simple example in the context of this brief is that much of the discourse around distributed manufacturing stems from a digital paradigm of modularity and customisation that has specific ideological connection with what — in 1995 — Dr Richard Barbrook and the late Andy Cameron called The Californian Ideology.

The “ecologies of design” as referenced in the brief was borrowed in this instance from a reading of various texts from Nordes 2015 “Challenging anthropocentrism in the design of sustainable futures Konstfack — University College of Arts, Crafts and Design. Stockholm, Sweden Sunday 7 — Wednesday 10 June 2015” — and in particularly thoughts whilst reading Cameron Tonkinwise’s paper Designing in an Era of Xenophobia.

These ideas — amongst others — were embedded in the brief as coordinates to frame a body of practice based enquiry that connects design practice with critical ideas pertaining to design futures. This approach also offers an opportunity to engage with ideas emanating from the design research community though design practice, which we think is one of the more vital aspects of what can be called SCD practice. Simply put, SCD for us offers one way to develop what in the UK is called “research informed teaching”.

Project work

We divided the students into groups based on their initial responses to the brief. Some were interested in looking at algorithmically-mediated systems, others at climate, another group was interested in mental wellbeing. These were only loose beginnings but meant to jumpstart the process of getting the students quickly into productive conversation. With only four days to go, it was vital that the usual philosophising and interpersonal negotiation at the beginning of a project were cut through.

We went largely hands-off with the students. Remaining in the room to encourage discussion and help them move beyond sticking points but the next real deadline was a halfway presentation of design proposals with two days to go. By this point, most of the groups had settled on their concept and story but were struggling with the hardest part of this kind of project — how best to materialise their things into design outcomes. This is often the hardest point of the project because it inevitably means letting go of concepts and ideas that you’ve come up with and find personally important. The difference between an SCD project and a lengthy text is that a good SCD project often gives very little away to the audience, it acts as a lure to bring an audience into the story and allows the audience to imagine the rest of the world around the object or artefact. To do this effectively often means cutting things that might prove problematic when trying to develop a good relationship with an audience such as ideas or concepts that only appeal to a limited range of people or things that are just too complex for people to grasp easily. Faith in your audience is important here.

By this stage we had the beginnings of four strong projects. One project dealt with a ‘normalised’ consumer AI that might inhabit personal devices making simple decisions for you such as accepting calendar invites or auto-replying to messages but with long ranging effects. Another was still thinking about how emotions were quantified by systems to make them legible (think here of Facebook’s reactions). The third team was looking at the interlocking of ‘open’ data and welfare systems and the fourth proposed replacing the Intergovernmental Panel on Climate Change with an international sport in wake of a reaction of ambivalence to the climate crisis and a public mood of anti-intellectualism.

The next stage was to consider the material practicalities of these outcomes. Most groups gravitated towards videos and short films. The viral qualities of short films as a media and the control you have as a creator over the story has a lot of appeal for short deadlines but we worked to encourage them to explore other options.

Design Responses

The Intergovernmental Game of Climate Change team produced a series of illustrations exploring different key positions that could be taken on their invented pitch as well as a rulebook and a short ‘live’ radio broadcast of a game. At one point they were drawn towards producing more ‘photo-real’ outcomes such as renderings but after discussion we encouraged them to produce slightly more ‘distanced’ objects to avoid destroying the affective humour of the concept by making it too real and to allow the audience to imagine the game themselves based on their experience of international sport.

IGCC Presentation

The Emotional Devolution team produced a series of images that explored how reductive interpretations of emotions through systems like social networks might influence human relations as we see the world more through these systems. They also, in video, made nods to behavioural change systems like the advertising reaction patent that demand more and more explicit emotional labour out of us in order to function. They considered how our physiognomy might change with all this repetitive emoting behaviour inculcated by computer vison, is the web cam the new fork?

Emotional Devolution Presentation

The Update team imagined a coercive personal assistant that crossed the creepy line using a combination of AI and personal information which was delivered by automatic update to your phone. They produced a journalistic timeline showing the evolution of this product through reports in tabloid media. As a medium for showing technological development, this was particularly interesting. Looking at how pulp publications and pop culture deal with and assimilate technology is a great way of studying how stories build around it. The influence of Near Future Laboratory was very clear here. They also produced a short piece of motion graphics showing how a fictional conversation between a couple after a first date was being manipulated subtly by their phones’ AIs to influence the relationship based on metrics the AIs had grabbed from elsewhere.

‘Update’ Presentation

Finally, the algorithmic welfare Got Milk? team adopted a documentary style of storytelling to show how one individual was first struggling to deal with the system and then how they might hack it to their benefit. For instance, avoiding penalties on milk consumption by not putting it in the smart fridge but in a bucket of ice. As a response to the time constraints, the idea of a documentary format was clever as well. This allowed the group to be looser with the filming and scripting as they could treat the ‘actor’ more candidly.

‘Got Milk? Presentation’ Presentation


The projects that emerged were in no way meant to be an outcome from the two weeks. They were intended as part of the continuing exploration from the first week of intense workshop but played out more responsively. Less — ‘here’s some tools, now go and use them’ than ‘here’s some tools, look how they’re made, see if you can build your own.’ So the projects were by no means slick, exhibition ready pieces but rough and ready sketches of a grab bag of stuff thrown at the students over the two weeks. It’s worth mentioning again the sheer energy and intellectual strength and design chops of the students. Not only us but every speaker was probed and dissected for reasons, stories, ideas and justification. The conversations I overheard between the students both in the school and at the pub afterwards were sharp and intense, driven by an open-minded embrace of the subject. The criticisms that SCD faces were repeatedly raised and new ones levelled but all discussed maturely and with a broad, ranging look at the future of SCD in a wider design context than academia.

The audience for SCD is broadening significantly as it gains traction in research, policy and industry. The discrete set of tools and principles that the field has developed so far have room to expand into the practices of those without a design-academic heavy background and the summer school was to us as educators and practitioners, an important testing ground.

There are still things to be learnt, for example it may well be too much over two weeks, it was tough f0r studnets to keep the required energy for the full two time and the length precluded the attendance of those who couldn’t find two weeks to spare or afford it. Looking at projects like Dan Hill, Joseph Grima and Marco Ferrari’s Incomplete City Studio there may be space for a SCD pedagogy at a summer school level that is more participative. Rather than a linear lecture and workshop based model, a way of teaching that evolves skills and understanding in an ongoing project.

Students reflect on the SCD Summer School

This said, as a first experiment on whether it was even possible to introduce and engage students knowledgeably with such a contentious and counter-intuitive field it was a success. As you can see from the video above the indications point to the students grasping the concepts and tools skilfully and with energy. The intesive experince offered by a summer school meant that as design educators we learnt new ways of introducing SCD ideas and new ways of talking about how we integrate them into student experience.

Words by,

Tobias Revell
Senior Lecturer Digital & Critical Design

Ben Stopher
Programme Director: Interactive & Visual Communication

Ways of Seeing

Wide-scale collaborative systems of digital culture have acted specifically on the ways designers see. How designers see has been long discussed as a point of distinction in determining what designers do and what their particular skill set is. For example Flusser (2013), building on a conception of seeing through the eyes of the soul, splits the act of seeing in ‘two eyes’ (Flusser, 2013 p. 39). The first eye looks at things that have been – i.e. things; and the second eye is concerned with things that may be (i.e. the future) and the things that may exist. He goes on to say that the first eye has been augmented by apparatus of perception such as the telescope and microscope, and proposes that this has fundamentally changed our sense of place in the world and our sense of the scope of things in general. Regarding the second eye – which he regards to be of specific relevance to design – he discusses how looking forward involves inventing the world object by object, possibility by possibility, all based on the knowledge you currently have. This, for him, is the creative eye – one that in earlier times would belong only to God – and it is this eye that is core to the practice of bringing things into being through the process of design. Bringing us more into the contemporary world, he discusses this second eye as undergoing the same radical augmentation at the hands of computer technology as the first underwent due to the telescope and the microscope.

“This could be the beginning of a technology of the soul’s second eye. All eternal forms, all immutable Ideas, can be formulated as equations, and these equations can be translated from the numerical code into computer codes and fed into computers. The computer for its part can display these algorithms as lines, areas and (a bit later on) volumes on the screen and in holograms, out of which it can create ‘numerically generated’ artificial images.” (Flusser, 2013 p. 41)

This sentiment is remarkably prescient when viewed more than 30 years later. Computers have indeed allowed us to model and then remake the world in remarkable ways, and in the case of designers, they have, in some areas, completely remade the vision available from our so-called second eye. It is important to consider that this idea is based on a visual metaphor, and Flusser does not consider the network effects that have since been incorporated into any contemporary experience of computing, and how these have augmented our ability as designers to see. Flusser considers the second eye as describing the dominant mode of vision of the designer, and goes on to say ‘The soul’s second eye continues to look into eternity, but this is now an eternity that it can manipulate. This is the designer’s way of seeing…’ (Flusser, 2013 p. 42).

My research explicitly seeks to expand this idea, and interrogate what effects collaborative vision may have had on the way designers see and act, both on a macro and micro scale. Flusser is clearly struggling to develop a sophisticated lexicon for the visual properties of computing – ‘lines, areas and (a bit later on) volumes on the screen and in holograms’ (Flusser, 2013 p. 41), even though his insight was clear in terms of the abilities computers would acquire in terms of display. My research attempts to describe other factors of digital culture act on the designer’s ability to see, and how these might be built upon from the perspective of a designer’s concern with ideas and things.


Flusser, V. (2013) Shape of Things: A Philosophy of Design. Reaktion Books. Available at:

Ontology and The Semantic Web

The wide-scale systems of the web are seen as providing an ontological infrastructure for digital information. The semantic web attempts to enact this through technical specifications (Semantic Web – W3C, no date) of the World Wide Web Consortium (W3C). Shirky (2005) describes the applied ontology from different perspectives:

‘The main thread of ontology in the philosophical sense is the study of entities and their relations. The question ontology asks is: What kinds of things exist or can exist in the world, and what manner of relations can those things have to each other? Ontology is less concerned with what is than with what is possible.’


‘The knowledge management and AI communities have a related definition — they’ve taken the word “ontology” and applied it more directly to their problem. The sense of ontology there is something like “an explicit specification of a conceptualization.”’

(Shirky: Ontology is Overrated — Categories, Links, and Tags, 2005)

As Shirky identifies, the idea of ontology from a technical engineering perspective is that the explicit categorisation that wide-scale digital information structures require, provides an opportunity to develop a schema of knowledge that will help with the task of correlating information from different domains, thereby providing a powerful basis for new tools and insights (Schema ref RB?). This again is the powerful idea that the collective endeavour of distributed users gives rise to new metadata which has value is describing relationships in information structures. 2005 could be considered the early years of what was called Web 2.0, which defined the web as more of a platform for user generated content than an infrastructure for the one-way delivery of static information. In this context Shirky goes on to develop the idea that categorisation by ontologically ascribing information hierarchies misses the real nature and potential of digital information by reinforcing the ontological biases of, for example, history, power and control that are at work in our analogue repositories of knowledge. If this new era of user generated content is freed from this hierarchical determinism and part of the task of categorisation is left to users to ascribe en masse through user action (tagging, reassigning, commenting, rating etc.), a crowd-based ontological schema could emerge that is richer and more useful, in that it does not pre judge its relationships to other things. Shirky specifically discuss the practice of tagging in this context, as below;

‘The strategy of tagging — free-form labeling, without regard to categorical constraints — seems like a recipe for disaster, but as the Web has shown us, you can extract a surprising amount of value from big messy data sets.’ (Shirky: Ontology is Overrated — Categories, Links, and Tags,)

This shows that at the heart of systems built on top of the idea of the semantic web also lies the idea that such systems create extra value by capturing the interactions of users and developing new data sets accordingly. In this context, my research explores what kind of data are created by designers when considered as a subgroup of specialist users, and specifically any useful user generated ontology in the design domain.

In the 10 years since Shirky’s talk, these ideas have been further explored by subgroups of the HCI community. A good example of this was Eighth International Conference on Design Science Research in Information Systems and Technology whose abstract below shows a mature context for some of the issues from an HCI perspective:

‘Theme: Design Science at the Intersection of Physical and Virtual Design

There has been a surge of interest in design science research in Information Systems in the last few years and we can say that this is now a mature field. The goal of the design science research paradigm is to extend the boundaries of human and organizational capabilities by designing new and innovative constructs, models, methods, processes, and systems. Scholars from different backgrounds – such as information systems, computer science, software engineering, energy informatics and medical informatics – are actively engaged in generating novel solutions to interesting design problems in Information Systems.’ (DESRIST 2013 | Eighth International Conference on Design Science Research in Information Systems and Technology)

Within the published conference proceedings (Brocke et al., 2013) ‘Ontological Explorations’ are specifically discussed in the context of developing a specific model of cross-domain knowledge queries. This again reinforces the context of my research in developing ontological descriptions around distributed and collaborative systems of the design process built on top of the wide-scale systems of the semantic web.


Brocke, J., Hekkala, R., Ram, S. and Rossi, M. (2013) Design Science at the Intersection of Physical and Virtual Design: 8th International Conference, DESRIST 2013, Helsinki, Finland, June 11-12,2013, Proceedings. Springer Berlin Heidelberg (Lecture Notes in Computer Science).
Available at:

Shirky: Ontology is Overrated — Categories, Links, and Tags (no date). Available at:

Generative & Participatory Design

Participatory Design as a term has come to mean many different situations of design process and design thinking that share some common traits. Chief among these is that end-users particularly, as well as other stakeholders contribute in some meaningful way to the final – or more accurately – used design. With the advent of digital design culture and designers working with both code and coders directly participatory design strategies have been extended to include areas of generative design. This is where user input in the form of data (i.e. movement, location etc.) or content (i.e. text, image etc.) is taken into a system of generative design, then, processed and mixed in a pre-determined way according to a designed algorithm. Generative designers do this for a variety of reasons and benefits dependent on the situation at hand. When describing this in their book about participatory design and user–generated content the design educators Helen Armstrong and Zvezdana Stojmirovic (Armstrong, 2011 p.121) describe this particular situation:

“Designers develop algorithmic structures, users feed those structures with content, and out of the process emerges something entirely new and sometimes unforeseen.”

What is described here is not fully the traditional model of participatory design, i.e. of stakeholder engagement etc. but it alludes to what the practice of generative design can bring to participatory design, i.e. mass-collaboration and the generation of potentially highly personal or formally varied design outcomes. There seems to be two distinct types of participation, one extends a more traditional model i.e. stakeholders using digital systems to contribute content for inclusion where they can potentially re-mix and edit and thus are genuinely participating willfully in the design act. Where as the second type of participation is not as consensual or structured on the users side. For example Armstrong and Stojmirovic cite the interactive designer Johnathan Harris’ ‘We Feel Fine’ project that aggregates blog posts and displays them in various ways that explore the purported happiness of participants in different data visualizations. This situation leads to a kind-of de-facto participation that is more and more common in our highly networked environment and one that is highly attractive to generative designers, as this kind of user activity is readily feedable into the algorithmic structures mentioned above. It is worth asking what this somewhat denuded second type of participation offers the designer apart from a ready stream of data, what is the medium specific value? Armstrong and Stojmirovic (Armstrong, 2011 p.117) also discuss this indirectly when saying:

“It is the unpredictable nature of the content and its constant reference to the larger culture that draws the user into the experience.”

What stands out for me in this sentence is “its constant reference to the larger culture” and I think that this is the gold for the generative designer. The consumption of design just like the creation of design is a more and more communal act and I think there is an innate desire from people (i.e. designers, commissioners of design and users of design) that design reflects the world it inhabits in a material way. In the ephemeral world of networked digital design, connectedness can be thought of as a material property of design and one that has value and can be explored and used and more importantly a property that commissioners of design want embodied for them in their design and communications.

Armstrong and Stojmirovic also interview the designer, academic and director of the Walker Art Center Design Studio Andrew Blauvelt (Armstrong, 2011 p.101) about how these new material concerns of designers are shaping design practice:

“Perhaps I think the role of the Designer is a Designer. But instead of designing a fixed, final thing they are more likely to be designing things people use to create or design other things. Designing Design. Designing Systems. Designing Processes.”

Again here you can feel the weight of connectedness and evolution as things that that have become fundamental material concerns for designers. This in some way explains the explosion in frameworks for designers i.e. API’s for all kinds of data around peoples interaction online and content sharing etc. These frameworks made possible by digital technologies provide ways for designers to mix and re-mix parts of our collective experience in a live and fluid way into design systems, processes and outcomes.


Helen Armstrong, 2011. Participate: Designing with User-Generated Content. Edition. Princeton Architectural Press.