Virtually Reconstructing Musical Numbers: The Modeller’s Perspective

Earlier this year I had the opportunity to get involved in a unique inter-disciplinary project, one that has taken me outside of my usual research area, as part of the SotonDH small grants. The project is a collaboration with Beth Carroll, who for her film studies PhD thesis is analysing the genre of musicals – specifically the phenomenon of the ‘musical number’.

An Unlikely Collaboration

The fact that Beth wanted my help as part of her analysis of the likes of Fred Astaire and Ginger Rogers may seem surprising, considering I am primarily a medieval archaeologist. What was needed was not my expertise in the past, but in the use of 3D visualisation, which I use as part of my own doctoral work on reconstructing the medieval city of Fustat, Egypt.

Beth is exploring the use of space in musical numbers, moving away from more conventional narrative based approaches (see her previous blog post), and looking at the interaction of camera, performers, scene, props and sound throughout the number. My task is to create animations to represent these spatial interactions for three musical numbers.

For each number I watch the film scene (repeatedly) and re-create the apparent arrangement of the scenery in 3D modelling software. I then place objects representing the camera, props and actors into the scene and animate their movements in time with the film. By keeping the ‘virtual’ animation camera stationary and observing these movements from a detached perspective, along with the score, one should be able to gain a better understanding of the use of space and sound.

3D Visualisation as an Interpretative Tool in Humanities

What struck me as interesting about this project was not only that, as Beth has described, such a visualisation is wholly innovative for the analysis of film, but that it has both interesting parallels with, and differences from, the way we use 3D visualisation within archaeology.

Similar to its utility in archaeology, 3D visualisation is being used as an interpretative aid. It is not used simply to demonstrate that which we already have decided upon, but as a way of exploring possibilities, highlighting undiscovered correlations and unanticipated inconsistencies in our interpretation. In archaeology, this would usually be in terms of reconstructing how a site or building may have looked in the past – where throughout the modelling process we reflect on the plausibility and interaction of different elements.

During the creation of the musical number animations, that process of reflection and revision was evident from the beginning of the task. I was surprised how hard it was in some cases to reconstruct aspects of the space, having to link props and characters that may only appear for less than a second in multiple shots in order to gain a clear picture of the position of some elements. Sometimes my first attempts resulted in impossible situations, and changes were made as I went along. It was only though repetitive and close viewing of the scene, along with continuing attempts at 3D representation, that an accurate picture of space emerged.

I would suggest that all this implies that while space is being used to great effect to create a certain sensation or impression on the viewer, how this is being done is less obvious than one might expect – and therein lies the strength of Beth’s spatial analysis from a detached and analytical perspective.

However, in contrast to its use in archaeology where most often 3D visualisation strives towards a ‘realistic’ representation of a (past) actuality – the animations of musical numbers had to be as simple and abstract as possible. The task was one of representing complexity and interaction in a way that could be interpreted by the viewer with ease, not of recreating a film in an aesthetically impressive format. As the project has continued Beth and I have discussed ways of representing factors that she considered important to the use of space – interaction with props, direction of view of actors, activity of actors, cut-away scenes, etc. These have been represented through changes in colour, light, movement and addition of objects, in an attempt to create a readable scene. In a sense this is a ‘data visualisation’ approach to 3D modelling.