3D Blood Pulsation Maps

AI summary

Overview

Research area: Camera-based physiological sensing (photoplethysmography imaging, PPGI), 3D face reconstruction, and computer vision dataset construction.

Technical level: Intermediate. The paper is readable with a general computer vision background, but assumes familiarity with terms like structure-from-motion, 3D morphable models, and blood volume pulse signal processing.

Scope: The paper introduces Pulse3DFace, a multi-view RGB video dataset of 15 subjects from 23 viewpoints, together with a pipeline that lifts 2D camera-based blood pulsation measurements onto the FLAME 3D head model to produce high-resolution 3D blood pulsation maps.

What This Paper Is About

Contact-free pulse measurement with cameras works well in controlled conditions, but almost all prior work measures pulsation from a single viewpoint and ignores how uneven lighting biases which facial regions look "good" for pulse extraction. The authors build a dataset and processing pipeline that capture facial blood pulsation from many angles and project the results onto a shared 3D face model, so that pulsation can be studied as a property of the skin surface rather than of one particular camera view. The goal is a principled data basis for synthetic data generation, region-of-interest selection, and study of how blood pulse amplitude, phase, and signal quality are distributed across the face and neck.

Key Contributions

1. A pipeline for creating 3D skin perfusion models built on FLAME, a morphable head model that is animatable in pose, expression, and viewpoint and compatible with most 3D software. The authors state its main purpose is creating artificial training data, improving on prior work that assumed a constant BVP phase across the face.
2. Pulse3DFace, a multi-view pulsation dataset with videos from 23 angles for 15 subjects, designed to mitigate the illumination biases of earlier single-view datasets.
3. Processed pulsation maps in FLAME texture space covering signal-to-noise ratio (SNR), per-color-channel pulse amplitude and phase, POS-derived phase, and heart rate, plus skin masks.
4. A first attempt at quantitative evaluation of 3D pulse maps, including a proposed measure that compares reprojection error of pulse maps against the reprojection error of the diffuse RGB skin texture to isolate illumination effects.

Main Findings

• Signal quality is highest on bare skin: SNR is high in skin regions and low in regions covered by facial hair and at the eyes. The green channel amplitude is mostly uniform over the face with the exception of the eyes.
• Nasal bone shows reduced signal: At the nasal bone, where skin thickness is minimal, both SNR and amplitude are decreased.
• Phase is largely uniform, with exceptions: Phase (green channel and POS-based) is uniform over most facial skin. A phase shift appears at the lips and eye regions, and a phase inversion of ±180° appears at the neck in the vicinity of the carotid artery, attributed to a different physical signal source (mechanical skin micro-motion from the artery wall rather than skin color change).
• Skin perturbations are visible in the maps: A slight scratch induced on subject S02's forehead appeared in all maps, with the strongest change in the SNR map. A blemish barely visible in RGB became clearly distinguishable in the 3D SNR texture on the right cheek.
• FLAME fitting accuracy: The mesh-to-scan error stays below 1 mm in most of the facial skin region, with larger errors at facial boundaries, the ears, and the neck where the setup provides no or low-quality data.
• Illumination strongly affects SNR-based ROI choice: Diffuse light from large softboxes gives homogeneous but low-SNR maps requiring larger window size (a usable SNR distribution was observed at k=17); the ring light gives strong illumination but local highlights in the pulse maps.
• 3D lifting reduces highlight bias: Based on Pearson correlation, the 3D lifted map from the ring-light condition approximates the diffuse illumination condition more closely than the 2D ring-light map directly. The effect is most pronounced in SNR and RGB maps; phase and amplitude errors are similar between the two lighting conditions and unaffected by highlights due to normalization.
• Reprojection error correlations: A significant correlation (p < 0.05 and |r| > 0.2) was found between SNR and phase reprojection error and the diffuse texture reprojection error for the skin region. Amplitude reprojection error showed no significant correlation. Subjects S04 and S11 were excluded due to strong motion artifacts.
• Between-subject variability: Averaged over all 15 subjects, decreased SNR and amplitude appear in the nasal bone region, and the largest between-subject differences appear in the beard and eyebrow regions. Maximum SNR per subject varied strongly despite controlled, identical illumination and camera settings.

Methodology in Plain English

Each participant lies in a supine position and is asked to minimize head motion. A robot arm carrying an RGB camera and an LED ring light moves around the head. First, 113 photographs are taken across a hemisphere (6° steps) using only softbox lighting to approximate diffuse illumination; these are used to build a detailed 3D scan via structure-from-motion. Then 70-second videos are recorded at 23 positions (15° steps) with the ring light added for strong illumination.

For the pulse maps, the authors compute a reference pulse signal from the whole face (skin-segmented, averaged, then processed with the plane-orthogonal-to-skin algorithm) instead of relying only on the contact sensors. Pixel-wise signal parameters are computed in sliding windows of size k ∈ [3, 5, 7, 9, 13, 17] and across seven overlapping 20-second segments of each 70-second video, then averaged to reduce noise. These 2D maps include SNR, per-channel amplitude and phase, POS-based phase, and heart rate.

To lift the maps to 3D, they fit the FLAME morphable model to the structure-from-motion scan using landmark initialization (MediaPipe Blazeface) and a non-rigid optimization with weighted terms for vertex distance, landmark distance, shape, expression and pose regularization, plus a Geman-McClure robustifier. Camera poses and FLAME parameters are then used to build a diffuse texture for the model, and the pulse maps are reprojected into that same texture space, with spatial averaging across views to combine maps from different angles.

Two contact sensors (an ear-clip and a finger BVP sensor at 128 Hz) provide reference pulse measurements synchronized by UNIX timestamps.

Why This Matters

Impact on research: Existing facial pulse datasets are single-view, do not account for lighting bias, and lack quantitative metrics for pulse map quality. Pulse3DFace offers a shared texture space in FLAME, which the authors argue enables computing statistics across subjects and comparing maps, and supports multi-view approaches to mitigating illumination effects.

Real-world applications (as discussed or implied by the paper):

• Generating synthetic video data of avatars with plausible blood volume pulse to train end-to-end pulse extraction models.
• Remote heart rate and heart rate variability estimation, and blood pressure research, where phase distribution matters for signal morphology.
• Spatial perfusion assessment, including in-surgery skin flap monitoring where contact sensors cannot be used.
• Region-of-interest selection for robust camera-based pulse measurement in real-world, non-uniform lighting.

Industry relevance: Relevant to contactless vital-sign monitoring for ICU and hospital settings, sports monitoring, and consumer camera-based health sensing, as well as to graphics and avatar pipelines that model dynamic skin appearance.

Future Directions

• Develop an independent ground truth system for pulsation maps, which the authors state does not currently exist and which limits quantitative validation.
• Improve capture with a dedicated multi-camera setup to reduce motion and alignment problems (subjects S04 and S11 showed excessive motion), and evaluate polarization filters for specular reflection handling.
• Improve FLAME fitting for eyelid position (e.g., an eye closure loss) and for non-rigid motion of nostrils, mouth, and neck, potentially using analysis-by-synthesis morphable models.
• Refine texture fusion so that averaging across views does not smooth phase inversions toward 0°, and postprocess amplitude and phase maps to remove ballistocardiographic artifacts near edges and boundaries.

Target Audience

Researchers and engineers working on remote photoplethysmography, camera-based vital sign monitoring, facial 3D reconstruction and morphable models, and synthetic facial data generation. It is also useful for practitioners building contactless health monitoring systems who need to understand how illumination and viewpoint bias pulse measurement region selection.

Authors’ abstract

We present Pulse3DFace, the first dataset of its kind for estimating 3D blood pulsation maps. These maps can be used to develop models of dynamic facial blood pulsation, enabling the creation of synthetic video data to improve and validate remote pulse estimation methods via photoplethysmography imaging. Additionally, the dataset facilitates research into novel multi-view-based approaches for mitigating illumination effects in blood pulsation analysis. Pulse3DFace consists of raw videos from 15 subjects recorded at 30 Hz with an RGB camera from 23 viewpoints, blood pulse reference measurements, and facial 3D scans generated using monocular structure-from-motion techniques. It also includes processed 3D pulsation maps compatible with the texture space of the 3D head model FLAME. These maps provide signal-to-noise ratio, local pulse amplitude, phase information, and supplementary data. We offer a comprehensive evaluation of the dataset's illumination conditions, map consistency, and its ability to capture physiologically meaningful features in the facial and neck skin regions.