Campus news, Research, Technology & engineering, Science & environment

Soaring through Peru: Berkeley student turns summer research into augmented reality app

By Allison Lin

With her eyes locked on the Inca ruins in front of her, UC Berkeley student Natalea Schager gently nudged a switch. A white DJI Phantom 3 Professional drone hovered on the horizon nearly 1,000 feet away and scanned the vast Sacsayhuamán Archaeological Park below.

Flying a drone here in Cusco, Peru, is tricky business. The archaeological park is in the Andes, 11,600 feet above sea level, where the Incas built some of their most monumental and important buildings. Flying a drone at such high altitude quickly drains the battery. If the battery dies mid-flight, the drone will try to land itself without knowing what is beneath it. It could hit something on its way down and crash, or it could damage an irreplaceable relic.

All of this ran through Schager’s mind as her drone floated above trees and disappeared into the sky like a helium balloon. Schager had spent hours honing her drone skills before this research mission, so with a few taps on her remote, she guided the drone back down towards Earth, to the park’s edge where she was perched on a stone bench.

The drone flight was one of dozens that Schager piloted last summer as part of an undergraduate research experience. Hands-on research experiences like this, where students like Schager are in the driver’s seat, are a hallmark of the Berkeley undergraduate education.

Schager’s drone flights produced so much information on Inca archaeology in Peru, and video footage, that she was able to launch an augmented reality app of Sacsayhuamán. The app is used as an educational tool aimed at giving students a 3D tour of the archaeological park from their classroom.

Drone footage from above the Sacsayhuamán Archaeological site.

Schager had never flown a drone before spring 2017, but the idea of it had always intrigued her. She took a particular interest when a friend invited her to a drone-flying session as a program sponsored by the unmanned aerial vehicle, or UAV, club on campus. When she RSVP’d for the Facebook event to fly drones, her interest caught the eye of archaeologist Alexei Vranich, who had been a resident professor at Bowles Hall where Schager lived as a sophomore.

Vranich specializes in field research in South America. He told Schager about his research in Peru and that he was recruiting students who wanted to experiment with drones. The research opportunity coincided with a field class Vranich was teaching through Berkeley Study Abroad, where students would live and study in Cusco, the original capital of the Inca Empire and a major archaeological hotspot.

“It sounded like a great way to spend my summer, so I was really interested in the opportunity,” Schager said.

She packed her bags and headed south along with 20 other students. In the first three weeks of the five-week program, Vranich taught the group field methods in archaeology and conservation, and he demonstrated excavation, computer modeling and field catalog methods. In the last two weeks, students branched off into different research projects based on their interests.

“Professor Vranich is incredibly knowledgeable and a go-with-the-flow kind of person, which is something I really liked,” Schager said. “The Cusco course wasn’t structured in the way that Berkeley classes are, and we had no idea what to expect each day of class.”

The class was taught outdoors every weekday and used the surrounding historical sites as a living classroom. Vranich brought the class to visit different ruins, waterfalls and even Machu Picchu.

Schager at Lake Humantay in Peru.

“He would point out many hidden parts to the architecture, like how the Inca architecture was built right into the mountains and landscapes,” Schager said.

With an enhanced knowledge of Inca religion and the ways the Incas built their temples and communities, Schager was equipped with the tools to begin her project. She based her research at nearby Sacsayhuamán Archaeological Park.

A drone’s view of the archaeological site.

Originally from Apple Valley, California, Schager studies cognitive science. She had taken courses in artificial intelligence, and had a background in 3D modeling and 3D graphics. What she lacked was extensive drone-flying experience.

Before leaving for Peru, Schager’s friend in the Berkeley UAV club taught her how to fly a drone. She first had to get a drone-flying license, register as a drone-flyer and then pass safety training in order to practice in the United States. These steps are not required in Peru, but the practice proved invaluable. She conducted test flights and familiarized herself with the DJI Phantom 3 Professional model, which is the same model Vranich had for her in Peru.

Schager flies the drone through a remote that connects to an app displayed on her smartphone. The drone’s camera takes pictures and records video, which Schager can view in real time. Through the app, she can also keep tabs on the drone’s altitude, speed and height, and see a map of the drone’s path during that flight. All of the information is saved for future analysis.

As the only person in her team with drone-flying experience, she also taught her peers the basics.

Natalea’s drone captures a candid shot of her and archaeologist Alexei Vranich.

Back home, Schager set out to transform her drone video into a learning tool, specifically an augmented reality app, an increasingly popular tool used in classrooms.

“Archaeologists have seen firsthand how accessible augmented reality technology could be, and how they are able to communicate the experiential aspects of a dramatic landscape and awe-inspiring architecture,” Vranich said.

With the drone footage, Schager used photogrammetry to see the differences between photos, which allows her to measure depths of the locations in the photos and then create a 3D model. Because the drone captures the location of each photo, the 3D model is georeferenced, meaning it knows exactly where in the world these photos were taken. Using Unity, a game development platform known for its user-generated 3D creations, Schager converted the 3D model into the new virtual space.

In the final product, users hold a tablet over a 2D map of Sacsayhuamán, which prompts 3D models of the Inca ruins to appear on the screen. As the tablet is moved around above the map, users can view different areas and select additional features to showcase possible aqueducts and landscaping methods.

“The Incas built directly into the rocks and landscape, so it’s really hard to point out detailed features when looking at a 2D map,” Schager said. “This project is important because it brings new technology to archaeology, and people can have a better understanding of what they’re looking at.”

After finishing a rough draft in just two months, Schager presented her project at the Association for Computing Machinery Multimedia conference. After a few more months of work, Schager presented an updated version in January at the Institute of Andean Studies conference, a coveted opportunity for an undergraduate researcher.

“Schager focused her research into a manageable task and successfully and diligently used the resources in the field and back at the university to create a comprehensible and accessible presentation at the conference,” Vranich said. “She was also able to enlist the help of other students and specialists.”

Looking back on her time in Peru, Schager has many fond memories spent at Sacsayhuamán, Cusco and other areas that Vranich exposed them to. But it’s the llamas that make her smile the most.

“When we were in Machu Picchu, everyone was trying to take pictures with the llamas, so I took some too,” Schager said. “When I came back from Peru, my friends and family showered me with llama gifts, like llama mugs and stuffed animals.”

Natalea Schager with a llama