880419 :Multimodality & Communication (BDM/CD)


Voertaal Engels
Werkvorm: 2 small group seminars per week, one literature discussion seminar, one assignment seminar (Collegerooster)
Tentamenvorm: tentamen (50%) + paper (50%) (Tentamenrooster)
Studielast:6 ECTS credits
Inschrijving:Enrollment via Blackboard before start of lectures
Blackboard informatieLink to Blackboard (Als u de melding 'Guest are not allowed in this course' krijgt, dient u nog bij Blackboard in te loggen)


dr. J. Schilperoord (coordinator)

dr. N.T. Cohn

Doel van de cursus (alleen in het Engels beschikbaar)

People communicate in ways beyond just language. They use images, sounds, gestures as well. The combination of this is called multimodality. The overal aim of the course is for students to gain insight into the ways people process and understand information that is coded in different modalities, such as combinations of text and image, or speech and gesture. This information can relate to each other in different ways: complementary, similar, contrastive. To meet these ends, we study several papers on multimodality and the ways meaning is expressed beyond language alone.

After the course the students are able to: 

  • Describe, analyze and compare relevant literature on multimodality
  • Interpret and evaluate relevant experimental research on multimodality. 
  • Describe and evaluate main stream theoretical accounts of multimodality in cognition and communication
  • Analyze and create multimodal messages
  • Apply and evaluate different research methods
  • Design a research project in the field of multimodality
  • Interpret and analyze experimental data 

This course contributes to student’s skills concerning

  • Processing, interpreting and evaluating scientific literature
  • Autonomous acquisition of new knowledge and broadening existing knowledge
  • Designing new research questions on the basis of existing literature, application of knowledge to other fields (transfer knowledge).

This course contributes to student’s skills to:

  • Motivate research questions on the basis of literature and preceding research.
  • Discuss theories and apply them to relevant data
  • Analyze and search for appropriate real world material
  • Write a scientific paper

Inhoud van de cursus (alleen in het Engels beschikbaar)

All student activities during this course will center on a scientific research project, which you carry out in small groups (4 or 5 students).

Each week contains a 2 hour literature seminar, and a 2 hour seminar on applying the theories real world cases of multimodal messages and designing the experiment. Attending this course requires willingness and eagerness to engage in the following activities: 

Literature study

  • The individual study of research articles (essays, experimental papers, corpus research papers), contemplating its merits and reflecting on it with fellow participants. 
  • Participation in weekly literature seminars where literature (1, 2 or 3 articles) will be discussed. In the first seminar(s) we will focus on the different types of questions which can be asked about literature. Gradually, we make a shift from questions prepared by supervisors to self-imposed questions which guide studying papers. So, questions of various types, from comprehension over application to evaluation and finally to synthesis, will be prepared by students and discussed in groups under the supervision of the teacher. 


  • Participation in weekly seminars where analytical issues are discussed and practiced, and all matters having to do with designing and conducting the group experiment.
  •  Preparation of seminars also includes meetings with colleague students to prepare the seminars.  

Bijzonderheden (alleen in het Engels beschikbaar)

The final mark for the course is based on: (1)  an individual final exam on the required reading (50%); (2) the group paper about the experiment. To complete the course, both the mark for the final exam and the paper must be minimally a pass.

This course also aims at raising your practical skills in designing multimodal messages for different audiences, serving a range of purposes, both informative and persuasive. 

Verplichte literatuur

  1. Selection of about 12 articles and tutorials.

Aanbevolen literatuur

  1. Some assignments require also studying and searching additional literature

Gewenste voorkennis

Apart form an analytical mindset and interest in the wonders of multimodal communication, it is highly recommended for participants to be intrigued by and attracted to comics and all kinds of other forms of visual communication.

Vereiste voorkennis


Mogelijk interessant voor

  • Bedrijfscommunicatie en Digitale Media ( 2015, 2016, 2017 )
  • Communicatie-Design ( 2015, 2016, 2017 )
  • Human aspects of Information Technology ( 2015, 2016 )
  • Data Journalism ( 2015, 2016 )
  • Communication and Information Sciences ( 2015, 2016, 2017 )
  • Cognitive Science and Artificial Intelligence ( 2017 )