4.7: Accessibility Through Sonification

What is Sonification?

Throughout this chapter we've focused on the visible aspects of spectra and their appearance using our eyes. Another important and rapidly emerging branch of astronomical research is through the use of sound, or Sonification. We can think of this research as "hearing" data collected from space. 

Before getting into the details, it's helpful to see/hear a few examples of Sonification. NASA has created a full repository of Sonifications, using data from James Webb and Hubble Space Telescopes, as well as Spitzer and Chandra X-ray observatories on a number of astronomical objects. You can view NASA's main Sonifications Webpage to view some examples.

The linked YouTube video below is a composite image/sound from the center of the Milky Way galaxy. The data includes x-ray, optical, and infrared light both visually and using sound. 

Sounds from Around the Milky Way

Video Length: 1 minute 4 seconds

Context: This video allows users to "listen" to the center of the Milky Way as observed in X-ray, optical, and infrared light.

("Sounds from Around the Milky Way," NASA JPL Education, CC BY 4.0, Youtube)

Using visual data alone, we normally analyze data from images by peaks on a graph. In other words, a bright spot on an image is represented by a sharp peak on the graph. The NASA sonifications we see above also provide an audio representation of this data by "reading" the image from left to right. As the video plays, we see the peaks of the graph change as bright or dim objects pass by, matched by the audible sounds being played.

Sonification takes a visual signal and translates it into corresponding frequencies that can be heard by the human ear. To be clear, the sound you hear in the video is NOT one you would actually hear from space. Think of it like the difference between reading a physical book versus listening to an audiobook. An audiobook contains the same information as the printed words, but has been translated into sounds so that we no longer need to rely on our eyes to read.

To help understand the motivation for using sound instead of light, we turn to the many astronomers that currently perform this research. According to Zanella et al., there are several limitations of analyzing astronomical data using solely visual methods. In their Nature Astronomy article, the authors state, "datasets are becoming extremely large and complex, often containing many dimensions. This makes it difficult to effectively and comprehensively display the data using standard visualization techniques and it is often required to prioritize and filter the data to only visualize the information that is believed to be relevant."(p. 1241). This complexity can also be observed in the sonification examples from above. Composite images that contain multiple types of electromagnetic waves become nearly impossible to display all of the meaningful data at the same time (consider what it would look like seeing all of the peaks from the video displayed on a single graph. Yikes, that's a lot!).

The authors also point out several benefits to using sound for data analysis, "Sound is inherently multidimensional because it is characterized by various parameters (for example, pitch, volume, tempo, location in a stereo field, timbre). We can also perceive several sounds simultaneously, meaning that we can listen to different sonified streams in parallel and our hearing can focus on one of many audio streams" (p. 1241). Returning to the audiobook analogy, one extra dimension you might consider is how the reader of the audiobook uses their voice to convey extra dimensions of tone, volume, and pace in their reading to add additional meaning than if you were silently reading the words on the page.

Sonification and Accessibility

Dr. Wanda Díaz-Merced, pictured below, has played a major role in the advancement of sonification research in Astronomy, calling particular attention to elements of accessibility and access to Astronomy data for the visually impaired. She began her journey into sonification during college, having lost her sight, but still maintained her desire to pursue astronomy as a career. Since then, her work has been a major source of advancement in analyzing large data sets in the form of auditory sound.

Figure \(\PageIndex{1}\): Wanda Díaz-Merced. Wanda Díaz-Merced at the inauguration of the the Inspiring Stars exhibition at the IAU General Assembly in Vienna, Austria. (CC BY 4.0,  IAU/M. Zamani via Wikimedia Commons).

Throughout her career, Dr. Díaz-Merced has collaborated with other astronomers on numerous applications of sonification in astronomy, particularly with accessibility in mind. Some of her most notable research has involved the development of advanced sonification software, making the process easier and more useful for modern research applications (Díaz-Merced et al., 2011). In one application, her team analyzed data from the Chandra X-Ray telescope, identifying frequencies of a cataclysmic variable star. (Change to link in textbook when ready) She has also studied solar phenomena such as major solar flares, coronal mass ejections (CMEs), and solar wind plasma using this technique. In short, all of this work has significantly advanced the field of sonification and the software required to utilize this technique.

Futher Exploration: Interactive Activity

• Watch the TED Talk How a Blind Astronomer Found a Way to Hear the Stars to learn how Dr. Wanda Díaz-Merced uses sonification to analyze astronomical data and advance accessibility in astronomy research.

References

Diaz-Merced, W. L., Candey, R. M., Brickhouse, N., Schneps, M., Mannone, J. C., Brewster, S., & Kolenberg, K. (2011). Sonification of Astronomical Data. Proceedings of the International Astronomical Union, 7(S285), 133–136. https://doi.org/10.1017/S1743921312000440

Zanella, A., Harrison, C. M., Lenzi, S., Cooke, J., Damsma, P., & Fleming, S. W. (2022). Sonification and sound design for astronomy research, education and public engagement. Nature Astronomy, 6(11), 1241-1248. https://doi.org/10.1038/s41550-022-01721-z