I moved to Barcelona in September 2018 and that month I started working at Crisalix where I stayed until January 2024. They were years of quite intense work where I expanded my horizons from strictly a VFX career to the one of mobile graphics apps developer. I worked both on iOS and Android devices, but mainly on iPhones and iPads.

Galderma™ Face AR

This is the AR app I worked on for years together with a very strong, dedicated team. We built a real-time AR pre-visualization engine for aesthetics products: users in clinics or specialized kiosks could see the result on their faces of products that influence the shape of cheeks, lips, forehead, nose, jaw. Some of the effects were not only deformation but included skin texture processing. The app was developed for iPhones and iPads using XCode, Swift  and Metal.

Some of the tasks and features I worked on:

• Renderer: we designed a Metal renderer and integrated it with the AR workflow of ARKit
• Warping shaders: blendshapes activations warped the camera image in real-time
• Skin filters shaders: effects like skin smoothing or wrinkles removal
• Blendshapes management: the engine contains many blendshapes and I often supervised the creation of them
• Before and After tool: given two AR recordings align in time and space the videos for a before&after comparison
• Backend integration: many tasks where I had to collaborate with the Web team to save/replace/delete/retrieve data on server
• Backup system: upload and recovery system for before&after data

Unity AR

After the launch and success of the Face AR app, I worked on a version compatible with Android as well.
This AR app was built with Unity and used the Banuba SDK for the AR tracking. I did lead a small team for this project and presented various challenges. I was in charge of the app design, SDK integrations, warping shader integration, UI implementation and strict collaboration with the Deep Learning team to integrate their wrinkles detection/removal model.

I also worked and contributed to the new Three JS Engine built at Crisalix and used in their flagship product, this time there was no AR involved but pure 3D computer graphics.
The team was spectacular and very talented while the project presented big challenges as everything was ported from Unity to a browser essentially.
In particular I created modules and shaders for the heatmap deformation visualization and the scars shader, just some of the tools that doctors use during their consultations to plan with patients an aesthetics procedure.

Other project and tasks

The tasks  I worked on and the teams I collaborated with at Crisalix were very diverse and I could touch multiple disciplines while learning new skills. Some of those tasks are resumed below:

• Morphing Tool: a direct editing tool to deform at will on the iPad the frontal and profile photos of a face and obtain a 3D model that can be used in AR mode.
• Deep Learning models integration: I worked on the integration of custom in-house DL models for body tracking.
• Synthetic Data: using Blender and its Python API I generated face renders for the internal training of experimental DL models.

As a fun fact, I won in team the Crisalix Hackathon twice.