Siemens AP Assistant
Augmented Reality, UX/UI Design
2019 - 3 Months
Unity, ARKIt 2, Sketch, Rhino, KeyShot, After Effect, Premiere Pro, Illustrator, Photoshop
My Role: I worked with Siemens as a research assistant at my university's lab. I conducted UX/UI design on my own and did AR research with the other research assistants.
This project consists of two parts.
AP Assistant is an AR app using ARKit2 and iPad Pro to help engineers decided where to place Siemens's radio transmitters (AP) at factories. Besides, we also researched
AR spatial data visualization to show the strength of the radio signal.
Improve engineers' working efficiency at factories using AR app
AR Spatial Data Visualization
Siemens was doing a research on their AP, which can transmit a special kind of radio signal at factories (similar to routers that can create WIFI signal). Usually, a couple of APs are needed in a factory. Since the radio signal is invisible, it's hard for engineers to know where to place the APs in order to cover the whole factory with radio signal. Siemens already had the algorithm to calculate the strength of the signal at any point. We were asked to research on how to visualize the signal in a 3D space using AR technology and the computational model. Besides, we also need to design an AR app on iPad pro to help engineers decide where to place the AP.
How might we design an AR app to help engineers decide where to place Siemens AP at factories.
How might we visualize the strength of the radio signal at the factories using AR technology.
Process
Information Architecture
We made an information architecture to meet the requirements from Siemens.
AR Testing
Because of AR technology limitations, we need to build AR prototypes to test every feature I designed.
Following is an example of the limitations we found
View ExampleThe AR app needs to detect the space where users can interact with virtual objects first. ARKit 2 can recognize the space by detecting different planes in this space. However, there are some limitations as follows:
The planes built by ARKit 2 automatically are very messy. It would become worse at factories.
Those planes without enough feature points such as white walls and ceilings are hard to be detected.
The ceilings at factories are too high to be detected.
Low-Fidelity WireFrame
I modified my design according to our test findings. We did many iterations using this process.
High-Fidelity Wireframe
We presented our video prototype to Siemens in our weekly meeting. Sometimes they demonstrated our prototype to engineers for evaluation. Then we iterated our design according to their feedback. Here's the final version of our high-fidelity wireframe.
Development
The engineers in my team built the interactive AR prototype using Unity in the end.
Final App Design
Create A Space
According to our AR testing, it's better to let users create a space manually where they can interact with the virtual objects instead of adding ARPlaneAnchor with world tracking session automatically. Meanwhile, a 2D map will also be generated when the space is created.
Add And Move AP
Users will need to move the AP not only on a single plane but also in the space or transfer it to another plane. So I designed a three coordinate axes to help users move AP.
2D Map
Because factories are usually very large. 2D map can help users check the layout of all the AP and place them easily.
2D Signal Visualization
This is a simple 2D signal visualization we used in the flow of this app. Besides, we did more research on both 2D and 3D spatial data visualization later.
Signal At specific points
Sometimes users also need to check the strength of radio signal from different AP at a specific point.
AR Visualization Research
Because radio signal is invisible, we were asked to visualize the strength of the signal at factories to help engineer decide where to place Siemens AP. We were able to use Siemens's algorithm to calculate the strength of the signal and we need to figure out the best way to visualize the spatial data efficiently. Finally, we designed both 2D and 3D spatial data visualization using AR technology and tested the AR prototype in real world environment to compare them with each other.
2D Spatial Data Visualization
3D Spatial Data Visualization
View ConclusionConClusion
Problem
According to the tests of our AR prototype, we found that the virtual objects we designed look like a single layer on the screen. Therefore, it's hard for users to tell the strength of the signal in the space according to these virtual objects' location.
Causes
Solution
1. The virtual objects which should be obstructed by walls, floors ceilings or obstacles still appear on the screen
2. The virtual objects float in the air which makes it hard for users to tell their relative position in the real world.
3. The lighting isn't perfectly simulated which makes it hard for users to tell the relative position of the virtual objects in the real world.
4. The cluttered visualization makes users feel confused.
1. Create connections between virtual objects and the real world.
2. Improve the rendering, especially lighting.
3. Make visualization as simple as possible.
