3 Dimensional Technology Is Helping Athletes' Performance At The 2020 Olympics, Here's The Explanation

The world's athletes, can be helped by the 3D video. (photo: unsplash)

JAKARTA - This year's Olympics may be closed to most spectators due to COVID-19. But the eyes of the world are still on the athletes thanks to dozens of cameras that record their every action such as jumps, dives, and punches.

Among all the broadcasting equipment, the competition in athletics this time saw five additional cameras. This is the first step in a detailed 3-D tracking system that gives spectators near-instant insight into every step of the race.

This tracking is just the beginning. The technology on display in Tokyo shows that the future of elite athlete training lies not only in collecting data about the human body. But it also uses that data to create a digital replica of it. These avatars could someday be run through hypothetical scenarios to help athletes decide which option will produce the best results.

The tracking system used in Tokyo, an Intel product called 3DAT, feeds footage directly into the cloud. There, an artificial intelligence program using deep learning analyzes the athlete's movements and identifies key performance characteristics such as top speed and deceleration.

The system shares that information with viewers by displaying a slow-motion graphic representation of the action, and highlighting key moments. The entire process, from recording the motion to broadcasting the analysis, takes less than 30 seconds.

For example, during an NBC broadcast of the 100-meter test run in Eugene, Oregon, AI showed how Sha'Carri Richardson reached a speed of 38.7 km per hour at her peak and slowed to 32.0 km per hour by the time she reached the finish line. It was enough to win the race.

“It's like having your own personal commentator showing you things in a race,” said Jonathan Lee, director of sports performance technology in the Olympic technology group at Intel.

To train their Olympic AI through machine learning, Lee and his team had to capture as much footage of elite athletic athletes in motion as possible. They needed recordings of the human body performing certain movements, but pre-existing footage was used for similar studies showing the average person moving, which would confuse the algorithm.

"People usually don't fully lift the body horizontally seven feet in the air, but world-class high jumpers can reach that height on a regular basis," Lee said.

In the footage, the team at Intel manually annotated every part of the body, from the eyes, nose, shoulders, and more, pixel by pixel. Once these key points are identified, the model can begin to relate them in three dimensions until they have a simplified rendering of the athlete's form.

Tracking this "skeleton" allows the program to perform 3-D pose estimation (a computer vision technique that tracks an object and tries to predict changes it might experience in space) on an athlete's body as it moves through an event.

This tracking system is limited to athletics at this year's event. But similar technology could become standard in a variety of sports. This is suggested by Barbara Rita Barricelli, researcher in human-computer interaction and assistant professor at the Italian University of Brescia. But he was not involved with this Intel project.

“The real big shift is when technology is not only used for entertainment or research, but is accepted by the community of practice,” Barricelli said.

For example, when the video assistant referee (VAR) was first used in football. This technology is very popular on television broadcast networks today, but some (human) referees simply refuse to rely on them in making game-changing decisions.

This technology remains controversial in football, but now many referees routinely use VAR to assist in making decisions. Barricelli suggests 3DAT's Olympic debut may be a "big step for the practice of research meetings—or better, the practice of embracing research outcomes."