Next Generation Visualization and Pharma

Mark Hoffman, Chief Research Information Officer, Children’s Mercy Hospital

content-image

If you have been a gamer in your life or have seen somebody moving their head in awe with their cell phone attached to goggles, you’ve probably already been exposed to the next generation of data visualization technologies. Virtual reality (VR) is an immersive experience in which the user wears a device and occludes the rest of the environment. Although comprehensive, this technology can cause vertigo in some users. Augmented reality (AR) inserts virtual objects into the “real” world. PokéMon Go is one of the most successful recent AR efforts, requiring only a cell phone with a camera to chase digital creatures. Mixed Reality (MR) is an extension of AR with the additional capability of anchoring the visualizations to objects using surface mapping and other spatial technologies. Mixed reality visualizations can be pinned to a wall, counter or fixed position in space and persist across multiple user sessions.

Life happens in 3D but we use two dimensional, flat monitors to understand everything from protein structures to cellular dynamics. In the 1990s I had my first exposure to molecular visualization with a Silicon Graphics workstation and the Biosym package for visualizing protein structures. This server required a dedicated room and UNIX knowledge to work with the files. When I attended a training workshop, I was struck by how many users were from pharma but then realized that the earliest practical applications of molecular visualization would be in evaluating small molecule docking with target structures. Indeed, later in the 1990s the development of HIV protease inhibitors demonstrated the power of rational drug design.

In his recent book tour, the CEO of Microsoft, Satya Nadella identified mixed reality and cloud technologies as the two technologies that he thinks will have transformational effects. Facebook spent two billion dollars to acquire the Oculus VR technology. Cell phone companies are investing in inexpensive kits to turn their devices into VR goggles and are competing aggressively to provide the best VR experience.

Check Out: Life Sciences Review
 

In his recent book tour, the CEO of Microsoft, Satya Nadella identified mixed reality and cloud technologies as the two technologies that he thinks will have transformational effects


What does this mean for the pharma industry? Here are a few thoughts:

• Leave the monitor and mouse behind: Wearable MR and VR devices can free the user from the flat two dimensional screen and tethered controllers. We have used the Microsoft Hololens to demonstrate the feasibility of straightforward data pipelines and processes to import small molecule or more complex protein structures into a mixed reality device. The Hololens is a standalone computing device and does not require tethering to a PC, laptop or cell phone. The wearer of the device can invoke a molecule and see the structure floating in the room. A surface view can be overlaid on a ribbon view and the user can peer into the inner structure of the protein. The freedom of a wearable visualization device allows the user to walk around the visual, look under it or above it. With modest development effort, controls can be added to the visualization enabling gesture or voice driven rotation, resizing and other modifications.

• Change collaboration forever: Multi-user VR, AR, and MR experiences will transform collaboration. Through emerging capabilities multiple users of an MR device can see the same visualization. The users can be thousands of miles apart. With one user acting as the “driver,” a drug development team could see and discuss the molecular interactions between a ligand and its target in great detail.

• More than molecules: Other complex data that is relevant to drug development is often visualized in two dimensions. For example, biological pathways are traditionally presented as flowchart-like diagrams. Perhaps immersive representations of these pathways could convey deeper meaning. As multi-billion dollar entities pharma companies also aggregate massive amounts of business data. The dashboards of the future are likely to incorporate elements of these new visualization technologies to improve understanding of market dynamics, sales performance and other operational data.

• Expand the “mixed” in mixed reality: The integration of complex visualizations with Application Programming Interfaces (APIs) will be the next step in this journey. Connecting visualizations to databases of compound libraries will promote new dialogue and analysis. One exciting application of Mixed Reality is the ability to pin a map to a table surface. If the map is connected through an API to a sales or customer database, MR can assist in strategic planning.

While the benefits of these transformative approaches seem intuitively obvious, they also open new opportunities for research into the cognitive responses to different display modalities. As with most new technologies, it is tempting for developers to overload creative applications with features and details. Innovations in human factors and design are critical for the long term success of these applications. In the near term, next generation visualizations are likely to supplement rather than replace current technologies but have immense potential to transform both the scientific and business activities of pharma.

Weekly Brief