AneDet System Walkthrough

A complete guide to how the non-invasive anemia detection system works.
View on GitHub

System in Simple Terms

What is Anemia?

Anemia is a condition where your blood doesn’t have enough hemoglobin (Hb) — the protein in red blood cells that carries oxygen around your body. When hemoglobin is too low, you may feel tired, weak, or dizzy.

Normally, to check for anemia, a nurse draws blood from your arm and sends it to a laboratory. This takes time, costs money, and requires a needle. AneDet tries to do the same check without any needles or blood — just by looking at your fingernail with a camera.

How Can a Camera Detect Anemia?

The colour of your fingernail bed (the pink area under your nail) changes depending on how much hemoglobin is in your blood:

• Healthy blood (high hemoglobin) → the nail bed looks pink or reddish
• Anemic blood (low hemoglobin) → the nail bed looks pale or whitish

This is actually the same thing doctors check during a physical exam — they press your nail and look at the colour. AneDet does this automatically with a camera and artificial intelligence.

What Does the System Look Like?

The system is a small device with three parts:

1. A small computer (Raspberry Pi 5) — about the size of a credit card
2. A camera (Raspberry Pi Camera Module 3) — points at the patient’s finger
3. A touchscreen (5 inches) — shows the results and lets the operator control the system

Everything runs on this small device. It does not need an internet connection — all the intelligence is built in.

How Does a Test Work?

A typical test takes about 26 seconds and works like this:

1. Place your finger — The patient holds their index finger or including middle and ring finger in front of the camera, about 10–15 cm away
2. The system finds the nail — The first AI automatically locates the fingernail in the camera image (like how your phone camera finds faces)
3. The system reads the colour — The second AI examines the nail colour and the skin colour next to it. It compares them to hundreds of examples it has seen before
4. It takes many readings — Over 16 seconds, the system takes about 20 separate readings and combines them to get a reliable result
5. The result appears on screen — A hemoglobin number (like “13.2 g/dL”) and a status (“Normal”, “Borderline”, or “Anemia”)

What Do the Results Mean?

How Accurate Is It?

Based on testing with 673 patients:

• Sensitivity: ~82% — Out of 100 anemic patients, the system correctly identifies about 82 of them
• Specificity: ~80% — Out of 100 healthy patients, the system correctly identifies about 80 of them
• Average error: about 1.3 g/dL — The system’s estimate is typically within 1.3 g/dL of the true lab value

This means the system is useful for screening (quickly checking many people) but is not a replacement for laboratory blood tests. Some healthy people may be flagged as anemic, and some anemic people may be missed.

How Does the AI “Learn” to Read Nail Colour?

Think of it like training a new doctor:

1. Show many examples — The AI was shown 673 photos of fingernails from different patients, along with their actual hemoglobin levels from lab blood tests
2. Let it find patterns — The AI gradually learned which nail colours correspond to which hemoglobin levels. Paler nails generally mean lower hemoglobin
3. Test it on new patients — After training, the AI was tested on patients it had never seen before to make sure it works on strangers, not just the people it practised on

The AI also looks at the skin colour next to the nail as a reference. This helps compensate for different lighting conditions — just like how a human would compare “is the nail paler than the surrounding skin?” rather than judging nail colour in isolation.

Why Does It Take 20 Readings Instead of Just One?

Imagine you’re trying to weigh yourself on a shaky scale. One reading might say 70 kg, the next might say 72 kg, and the next 69 kg. None of them are perfectly accurate. But if you take 20 readings and use the middle value, you get a much more reliable answer.

The AI works the same way — each individual prediction has some uncertainty, so the system takes many readings, throws out the obvious outliers, and uses the stable middle value as the final result.

What Are the Safety Features?

The system has several built-in safety measures:

• Focus check — If the camera image is blurry, the system waits until it’s sharp before taking readings
• Confidence check — If the AI isn’t confident it found the nail, it won’t take a reading
• Outlier removal — If one reading is wildly different from the others, it’s automatically discarded
• Minimum samples — If fewer than 6 good readings were collected, the system says “No Valid Reading” instead of guessing
• Borderline band — Results very close to the anemia boundary (12.0–12.2 g/dL) are labelled “Borderline” instead of being forced into Normal or Anemia

1. What the System Does

AneDet is a non-invasive anemia screening tool. Instead of drawing blood, it:

1. Points a camera at the patient's fingernail (index finger)
2. Detects the nail automatically using a YOLO object detection model
3. Analyses the nail colour using a MobileNetV3 regression model
4. Estimates the hemoglobin (Hb) level in g/dL
5. Classifies the result as Normal, Mild Anemia, Moderate Anemia, or Severe Anemia

The system runs entirely on-device — no internet connection is needed. The whole process takes about 26 seconds per reading.

Clinical Thresholds (g/dL)

2. Hardware Overview

The system runs on three physical components:

Camera Configuration

The Raspberry Pi Camera Module 3 has autofocus, which is critical for close-up nail imaging at 10–15 cm distance.

3. Software Architecture

The system is a Flask web application that runs locally on the Pi. The user interacts with it through a Chromium browser (typically launched in kiosk mode on the 5-inch display).

Thread Architecture

There are 3 concurrent execution contexts:

Shared State: InferenceState

The InferenceState dataclass acts as the communication channel between the inference worker and frame generator:

4. File Inventory

Core Application Files

Training & Evaluation Files

Data

5. The Two AI Models

The system uses two models in sequence: one to find the nail, one to predict hemoglobin.

Model 1: YOLO Nail Detector (best.pt)

Every 0.4 seconds, the inference worker runs YOLO on the latest camera frame. It finds all nail bounding boxes, picks the one with the highest confidence, and smooths its position using an exponential moving average (EMA) to reduce jitter.

Model 2: Hemoglobin Regressor (hb_regressor.onnx)

How the 6 Skin Features Are Computed

def compute_nail_skin_ratio(nail_patch, skin_patch):
    nail_lab = cv2.cvtColor(nail_patch, cv2.COLOR_BGR2Lab)
    skin_lab = cv2.cvtColor(skin_patch, cv2.COLOR_BGR2Lab)
    nail_mean = nail_lab.reshape(-1, 3).mean(axis=0)   # → 3 values (L, a, b)
    skin_mean = skin_lab.reshape(-1, 3).mean(axis=0)   # → 3 values (L, a, b)
    return [nail_L, nail_a, nail_b, skin_L, skin_a, skin_b]  # → 6 total

These features are then z-normalised using training-set statistics from hb_regressor_config.json:

6. How a Measurement Works (User Perspective)

A measurement has 3 phases that the user sees on screen:

Phase 1: Positioning (at least 10 seconds)

1. User taps “Start Reading”
2. Camera feed appears on screen
3. User places their index fingernail in front of the camera (~10–15 cm away)
4. The Placement Quality bar fills up as the system detects the nail and confirms focus
5. Green bounding box appears around the detected nail
6. User guidance text updates: “Place index fingernail in the center” → “Hold still” → “Great position”

Phase 2: Acquiring (16 seconds)

1. Automatically starts after positioning quality is confirmed
2. The system takes ~20 hemoglobin readings over 16 seconds
3. Status shows “ANALYZING” with a countdown timer
4. Each reading is a separate ONNX model inference
5. Intermediate values are smoothed so the display doesn't jump around

Phase 3: Done

1. The system computes the final Hb value from all collected samples
2. The Estimated Hemoglobin Level appears (e.g., “11.7 g/dL”)
3. A status badge shows the classification (e.g., “MILD ANEMIA”)
4. User can tap “Save Reading Data” to store the result in the database

What Can Go Wrong

7. How a Measurement Works (Technical Deep-Dive)

Startup Sequence

When app2.py starts, it performs these steps in order:

Phase Transitions (State Machine)

Positioning Phase: What the Inference Worker Does

Every cycle (~20ms sleep + processing time), the worker thread does:

1. Captures a frame from Picamera2 (thread-safe via _camera_lock)
2. Focus evaluation (every 0.25s):
   • Crops a region around the detected nail (or center 50% of frame)
   • Computes Laplacian variance (sharpness metric)
   • Maintains a rolling window of 6 sharpness scores
   • focus_ok = median(sharpness) ≥ 25.0
3. YOLO detection (every 0.4s):
   • Runs YOLO at 416px input size
   • Extracts all nail bounding boxes (class ID 0)
   • Picks the highest-confidence detection
   • EMA-smooths the bounding box position (α=0.25) to reduce jitter
4. Quality assessment:
   • quality_ok = nail_detected AND nail_conf ≥ 0.18 AND focus_ok
   • Computes a 0–100 quality score: 60 points for nail confidence + 40 points for focus
   • Generates a user-facing tip string

Acquiring Phase: Hb Inference Loop

Once quality is confirmed and 10 seconds have elapsed, the system transitions to acquiring. Every 0.8 seconds, the worker checks: Is there a smoothed nail box? Is confidence ≥ 0.34?

If yes, it runs the full Hb inference pipeline:

Key Details:

• Skin patch derivation: Cropped from below the nail — centered horizontally under the nail box, offset 4 pixels below the bottom edge. Size: 96×96 pixels.
• Test-Time Augmentation (TTA): Runs the model 5 times on slightly shifted crops (±5 pixels). The median of the 5 predictions reduces sensitivity to exact crop positioning.
• Confidence gating: Only samples with confidence ≥ 0.34 are accepted. Only samples where the skin patch was derived from the actual detection box (not a fallback) are kept.

Done Phase: Final Result Computation

When 16 seconds of acquisition have elapsed:

Key safety features:

• MIN_VALID_SAMPLES = 6: If fewer than 6 good samples were collected, the result is “No Valid Reading”
• IQR filtering: Removes outlier predictions (e.g., if the user moved their finger during one frame)
• Anemia Guard: If the median says “Normal” but ≥40% of readings say “Anemic,” the result is gently nudged downward (70/30 blend with the 30th percentile)
• Borderline detection: If Hb is within ±0.3 g/dL of the threshold and anemia ratio is 35–65%, the result is “Borderline - Recheck”

8. Signal Stabilisation Pipeline

Raw model predictions are noisy. The system applies four layers of smoothing:

Layer 1: TTA Median (Per-Frame)

• 5 shifted crops → median prediction
• Removes: random crop-position sensitivity

Layer 2: Rolling History Median (Across Frames)

• Maintains a deque of the last 15 raw Hb values
• Each new prediction is compared to the median of the history, not the raw value
• Removes: frame-to-frame noise

Layer 3: EMA Smoothing with Asymmetric Step Caps (Display)

delta = history_median - smoothed_hb
if delta >= 0:
    delta = min(delta, 0.05)    # slow upward moves (conservative)
else:
    delta = -min(abs(delta), 0.18)  # faster downward correction (safety)
candidate = smoothed_hb + 0.10 * delta
if abs(candidate - smoothed_hb) >= 0.10:   # deadband
    smoothed_hb = candidate

Layer 4: Session Aggregation (Final Result)

• Takes the last 10 samples
• IQR filtering removes outliers
• Median of filtered samples = final Hb
• Anemia Guard may pull the result lower

9. Unit Conversion & Calibration

The Unit Problem

The model is trained on data labelled in g/L (grams per litre), but clinical Hb is reported in g/dL (grams per decilitre):

g/dL = g/L ÷ 10

So a model output of 120 g/L becomes 12.0 g/dL.

Calibration Pipeline

The system has four calibration stages:

Default runtime: The ÷10 conversion, slope correction (1.3× stretch), and final clip are active. The config calibration and legacy curve are both disabled by default.

10. The Web Interface

The UI is served as a single-page Flask template (index.html) with CSS and JavaScript.

API Endpoints

Frontend Polling

The JavaScript polls /get_session_state continuously while a session is active. Example response:

{
    "phase": "acquiring",
    "remaining": 8.3,
    "quality_ok": true,
    "quality_reason": "Quality OK",
    "quality_score": 100,
    "quality_tip": "Great position. Keep still until analysis completes.",
    "samples": 12,
    "hb": 11.7,
    "status": "Mild Anemia"
}

11. Database & Data Privacy

Storage

Results are stored in a local SQLite database (data/readings.db):

Encryption

Patient names are encrypted at rest using a custom stream cipher with HMAC-SHA256 integrity:

Plaintext → XOR with SHA-256 keystream → prepend 16-byte nonce
→ append 32-byte HMAC → Base64-encode

• The encryption key is generated once and stored in data/.anedet_secure.key (permissions 0600)
• Can also be set via ANEDET_DATA_KEY environment variable
• Decryption requires the same key — without it, patient names are unrecoverable
• Designed for compliance with the Philippine Data Privacy Act (RA 10173)

Admin Functions (Password-Protected)

All admin operations require authentication:

• Export Records: Decrypts all patient names and writes a CSV to databaseRecords/
• Reset Database: Deletes all readings (irreversible)
• Exit to Desktop: Kills the browser and exits the app cleanly

12. How the Model Was Trained

The training pipeline (train_hb_regressor.py) produces the ONNX model that runs on the Pi.

Training Data

• 673 patients, each with a hand photograph
• 2,019 samples (multiple nail patches per patient, matched with skin patches)
• Hb labels from laboratory blood tests in g/L
• Split: 572 patients for train/val, 101 patients for test (patient-level split prevents leakage)

Model Architecture: HBRegressor

class HBRegressor(nn.Module):
    # Backbone: MobileNetV3-Small (ImageNet pretrained)
    features = mobilenet_v3_small().features  # → 576-dim feature vector
    pool = AdaptiveAvgPool2d(1, 1)

    # Regression head: concatenate 576 CNN features + 6 skin features
    regressor = Sequential(
        Linear(576 + 6, 128),   # fusion layer
        ReLU(),
        Dropout(0.3),
        Linear(128, 1),         # → single Hb prediction (g/L)
    )

Training Strategy

• Loss: ThresholdAwareHuberLoss — Huber loss (δ=10) with extra weight near the 120 g/L anemia threshold
• Optimiser: AdamW (weight decay 1e-4)
• Scheduler: Cosine annealing per phase
• Early stopping: Patience of 8 epochs
• Data augmentation: Horizontal flip, ±10° rotation, colour jitter, bounding box jitter

Achieved Performance (Best Seed = 42)

Cross-Validation (3-Fold)

13. Configuration & Tuning Reference

All runtime parameters can be overridden via environment variables without changing code:

Core Inference

Session Timing

Confidence Thresholds

Camera

Calibration Toggles

14. Starting the System

On the Raspberry Pi

cd /path/to/oneModel
python app2.py

The Flask server starts on http://0.0.0.0:5000. On the Pi's display, Chromium should open in kiosk mode pointing to http://localhost:5000.

Typical Boot Sequence Console Output

load models
loaded hb config: /path/to/hb_regressor_config.json
runtime hb config | skin_norm=True | cfg_cal=False (slope=0.7000,
    intercept=20.00) | legacy_curve=False | anemia_threshold=12.20
models loaded
camera initialized (Picamera2, Camera Module 3, size=640x480)
autofocus: Continuous + Fast (Camera Module 3)
 * Running on http://0.0.0.0:5000

Complete System Data Flow (End to End)