AGENTS.md — Face Attendance Kiosk (Kotlin / Android, Offline-First)

An Android-only kiosk app that records employee time-in / time-out punches using face recognition.

Core flow

1. Register: an employee's face is enrolled and linked to their employee ID.
2. Punch: the kiosk detects a face, verifies it is a live person (anti-spoofing), matches it against enrolled employees, and automatically records employee ID + date + time.
3. Offline-first: everything is saved on the phone first. When internet returns, the app syncs to the server (MySQL via a PHP API).

Non-goals: iOS, cloud face APIs, paid SDKs, computing payroll hours on the device.

"Save in the phone" means a local SQLite (Room) database that mirrors the data model of the server's MySQL database. They are two databases kept in sync, not one.

app/
  camera/       CameraX setup, frame analyzer, lifecycle
  face/         detector wrapper, quality gate, embedder, matcher
  liveness/     passive model (MiniFASNet), challenge manager, consistency check
  data/
    local/      Room entities, DAOs, migrations
    remote/     Retrofit API, DTOs, auth interceptor
    repo/       repositories (single source of truth = Room)
  sync/         WorkManager workers, sync state, conflict rules
  ui/           register, punch, admin, settings screens (Compose)
  di/           Hilt modules
  util/         time provider, crypto, logging

Rules

• UI never touches the DAO, camera, or model directly. It talks to ViewModels.
• Room is the single source of truth. The UI reads from Room, and the network only syncs into/out of Room.
• All ML inference runs off the main thread (Dispatchers.Default). Never block ImageAnalysis; use STRATEGY_KEEP_ONLY_LATEST.
• Wrap time in an injectable TimeProvider so it can be tested.

1. Input employee ID. Validate against the locally synced employee list; if online, confirm with the server.
2. Capture at least 5 samples: front, slight left, slight right, slight up, slight down.
3. Every sample must pass the quality gate (Section 6) and a liveness check.
4. Generate one embedding per sample. Reject the batch if samples disagree with each other (pairwise similarity below ENROLL_INTRA_MIN).
5. Duplicate check: compare against all enrolled employees. If the new face matches a different employee above DUPLICATE_THRESHOLD, block enrollment and flag it.
6. Save templates to Room (face_templates) with model_version, mark sync_status = PENDING.
7. Sync templates to the server so other devices and backups have them.

Re-enrollment: revoke old templates (revoked_at), never hard-delete locally until synced.

1. Camera runs continuously. Wait for a face that passes the quality gate.
2. Run liveness (passive + random active challenge). Fail closed.
3. Generate an embedding from the same frames that passed liveness (consistency: no swapping faces mid-process).
4. Match 1-to-many against active templates.
5. Accept only if best_score >= MATCH_THRESHOLD and best_score - runner_up_score >= MATCH_MARGIN.
6. Record a punch (Section 7) with employee ID, timestamps, scores, and an audit snapshot.
7. Show a confirmation (name, time). Ignore repeat punches from the same employee within DEBOUNCE_SECONDS.
8. Never auto-accept a low-confidence match. Offer a fallback (PIN or admin override) that is logged as such.

The device records raw events only. It does not compute hours, lateness, or undertime, and it does not make the final IN/OUT decision. It may show a suggested IN/OUT label as a hint.

Reject a frame before matching unless all pass (tune the constants in one FaceConfig object):

• Face box width >= MIN_FACE_RATIO of frame width
• Face roughly centered
• Head pose (Euler X/Y/Z) within MAX_YAW, MAX_PITCH, MAX_ROLL
• Eyes open (probability >= MIN_EYE_OPEN)
• Sharpness (e.g., variance of Laplacian) >= MIN_SHARPNESS
• Brightness within [MIN_LUMA, MAX_LUMA]
• Exactly one face (ignore or reject if multiple faces compete)

Use all layers; no single check is trusted alone.

1. Passive model: MiniFASNet (open-source Silent-Face-Anti-Spoofing) converted to LiteRT. Score each frame; require a rolling average above LIVENESS_MIN.
2. Active challenge: a random action each time (blink, turn left/right, look up). Verify with ML Kit eye-open probability and head Euler angles. Randomization defeats pre-recorded video.
3. Motion/parallax cue: natural face-size and landmark changes during the head turn; flat photos move rigidly.
4. Optional screen-flash challenge: flash colors and check reflection response.
5. Consistency: the face that passes liveness must be the same identity (embedding similarity across challenge frames) that is matched.
6. Behavioral limits: cap retries, lock out after repeated failures, flag unusual patterns.

Honest limits: RGB-only anti-spoofing reduces risk but is not perfect against high-quality masks or 3D attacks. Always store an audit snapshot so HR can review disputes.

Licensing: before bundling any pretrained model, check its license (code and weights separately). Some popular weights are non-commercial. Record each model's name, source, license, and model_version in docs/MODELS.md. If a license is unclear, stop and ask.

#8.1 Phone (Room / SQLite)

@Entity(tableName = "employees")
data class EmployeeEntity(
    @PrimaryKey val employeeId: String,
    val name: String,
    val isActive: Boolean,
    val updatedAt: Long            // epoch ms
)

@Entity(tableName = "face_templates")
data class FaceTemplateEntity(
    @PrimaryKey val templateId: String,   // UUID
    val employeeId: String,
    val embedding: ByteArray,             // encrypted at rest
    val modelVersion: String,
    val qualityScore: Float,
    val createdAt: Long,
    val revokedAt: Long?,
    val syncStatus: Int                   // 0 pending, 1 synced
)

@Entity(
    tableName = "punches",
    indices = [Index("syncStatus", "punchedAtEpochMs"), Index("employeeId", "punchedAtEpochMs")]
)
data class PunchEntity(
    @PrimaryKey val punchUuid: String,    // generated on device, used for idempotency
    val employeeId: String,
    val deviceId: String,
    val punchedAtEpochMs: Long,           // wall clock (UTC epoch ms)
    val tzOffsetMinutes: Int,             // device tz offset at punch time
    val elapsedRealtimeMs: Long,          // monotonic clock, for drift detection
    val suggestedType: String?,           // "IN" | "OUT" hint only
    val matchScore: Float,
    val runnerUpScore: Float,
    val livenessScore: Float,
    val snapshotPath: String?,
    val source: String,                   // "FACE" | "FALLBACK_PIN" | "ADMIN"
    val syncStatus: Int,                  // 0 pending, 1 synced, 2 rejected
    val syncedAt: Long?,
    val rejectReason: String?
)

Storage rules: encrypt embeddings (Android Keystore-backed key, or SQLCipher). Store snapshots in app-private storage and purge them after sync + retention period.

#8.2 Server (MySQL / MariaDB)

Align with the existing timekeeping tables. Before creating tables, inspect the existing tk_ tables and reuse naming and conventions. Map accepted punches into the existing raw-log table instead of duplicating logic. Mark unknowns TODO(decision).

CREATE TABLE kiosk_devices (
  device_id     VARCHAR(64) PRIMARY KEY,
  name          VARCHAR(100) NOT NULL,
  token_hash    CHAR(64) NOT NULL,
  last_seen_at  DATETIME NULL,
  is_active     TINYINT(1) NOT NULL DEFAULT 1
) ENGINE=InnoDB;

CREATE TABLE face_templates (
  template_id   CHAR(36) PRIMARY KEY,
  employee_id   VARCHAR(32) NOT NULL,
  embedding     BLOB NOT NULL,
  model_version VARCHAR(32) NOT NULL,
  quality_score FLOAT NULL,
  created_at    DATETIME NOT NULL,
  revoked_at    DATETIME NULL,
  INDEX idx_emp (employee_id),
  INDEX idx_model (model_version)
) ENGINE=InnoDB;

CREATE TABLE face_punches (
  punch_uuid        CHAR(36) PRIMARY KEY,   -- idempotency key
  employee_id       VARCHAR(32) NOT NULL,
  device_id         VARCHAR(64) NOT NULL,
  punched_at_utc    DATETIME(3) NOT NULL,
  tz_offset_minutes SMALLINT NOT NULL,
  server_received_at DATETIME(3) NOT NULL,
  clock_drift_ms    BIGINT NULL,
  suggested_type    ENUM('IN','OUT') NULL,
  match_score       FLOAT NULL,
  runner_up_score   FLOAT NULL,
  liveness_score    FLOAT NULL,
  source            ENUM('FACE','FALLBACK_PIN','ADMIN') NOT NULL,
  snapshot_ref      VARCHAR(255) NULL,
  status            ENUM('ACCEPTED','FLAGGED','REJECTED') NOT NULL DEFAULT 'ACCEPTED',
  INDEX idx_emp_time (employee_id, punched_at_utc),
  INDEX idx_device (device_id, punched_at_utc)
) ENGINE=InnoDB;

Time rules

• Device sends UTC epoch ms + tz offset. Server stores UTC and derives local date/time.
• Never pair IN/OUT by calendar date. Overnight shifts mean a punch after midnight can belong to the previous shift. Pairing and shift assignment happen on the server using shift rules. Check the existing overnight-shift handling before changing it.
• Server computes clock_drift_ms = server_now - device_time at sync and flags large drift.

Principles

• Write to Room first, always. Never block a punch on the network.
• Every punch has a client-generated punch_uuid; the server must treat repeated uploads as no-ops.
• Sync is batch-based, resumable, and safe to retry.

Workers (WorkManager)

• PunchUploadWorker: constraints NetworkType.CONNECTED, exponential backoff, batches of ~50, oldest first. Mark each punch by the per-item result.
• TemplateSyncWorker: upload pending templates, download new/updated/revoked templates using a since cursor.
• EmployeeSyncWorker: pull employee list (active/inactive) using a since cursor.
• Also trigger a sync when connectivity returns and on a periodic schedule (e.g., every 15 min).

Conflict rules

• Punches are append-only. No updates; corrections are done on the server by authorized users.
• Templates: server is authoritative for revocation. If a template is revoked server-side, remove it from local matching on next sync.
• If model_version changes, templates must be re-enrolled; do not mix embedding versions in one matching pass.

The backend runs PHP 7.3 on Windows with MySQL/MariaDB. Do not use PHP 8-only features (no match, no named arguments, no union types, no str_contains, no constructor property promotion, no nullsafe ?->). Use prepared statements (PDO) everywhere.

Base: /api/v1/kiosk. All requests send Authorization: Bearer <device_token> and X-Device-Id. HTTPS only.

POST /punches/sync request:

{
  "deviceId": "KIOSK-01",
  "sentAtEpochMs": 1760000000000,
  "punches": [
    {
      "punchUuid": "…",
      "employeeId": "E1234",
      "punchedAtEpochMs": 1759999990000,
      "tzOffsetMinutes": 480,
      "suggestedType": "IN",
      "matchScore": 0.71,
      "runnerUpScore": 0.32,
      "livenessScore": 0.93,
      "source": "FACE"
    }
  ]
}

Response (per item, never fail the whole batch for one bad row):

{
  "results": [
    { "punchUuid": "…", "status": "ACCEPTED" },
    { "punchUuid": "…", "status": "DUPLICATE" },
    { "punchUuid": "…", "status": "REJECTED", "reason": "UNKNOWN_EMPLOYEE" }
  ],
  "serverTimeEpochMs": 1760000001000
}

Treat ACCEPTED and DUPLICATE as synced. Treat REJECTED as terminal (keep for review, do not retry forever).

• Face embeddings are sensitive biometric data. Encrypt at rest on the phone and on the server; never log embeddings or raw images.
• Transport over HTTPS; per-device tokens that can be revoked.
• Kiosk hardening: lock-task / kiosk mode, disable screenshots (FLAG_SECURE) on admin screens, protect admin and registration with auth.
• Limit snapshot retention; document the retention period.
• Keep employee consent and a privacy notice in the rollout checklist (Philippine Data Privacy Act treats biometrics as sensitive personal information).
• Never hardcode secrets, tokens, or API URLs; use build config / secure storage.

• Kotlin official style; ktlint + detekt must pass.
• Immutable data classes, sealed classes for UI/result states, no !! in production code.
• Coroutines with structured concurrency; inject dispatchers.
• Public functions in face/, liveness/, and sync/ have KDoc and unit tests.
• Threshold constants live only in FaceConfig; never scatter magic numbers.
• Log with a wrapper (AppLog) that can be disabled and never prints PII or embeddings.
• Database changes require a Room migration and a migration test.

Unit: matcher (threshold, margin, tie cases), quality gate, debounce logic, sync result handling, time/timezone conversion, overnight punch timestamps.

Instrumented: Room DAOs + migrations, WorkManager workers with a fake API (success, partial failure, timeout, duplicate responses).

Biometric evaluation (required before pilot): collect a small labeled set from real employees under varied lighting. Report:

• FAR (false accepts) and FRR (false rejects) at the chosen threshold
• Spoof test results: printed photo, phone replay, tablet replay, mask. Report attack success rate for each
• Median and p95 time from face in view to recorded punch

Do not raise thresholds or lower margins just to make demo punches succeed.

Offline scenarios: punch with airplane mode on; restart app/device while offline; sync after hours or days offline; same punch uploaded twice; server returns 500 mid-batch.

1. M1 Camera & detection: CameraX preview, ML Kit detection, quality gate, on-screen debug overlay.
2. M2 Recognition: LiteRT embedder, matcher, docs/MODELS.md; verify on a handful of people.
3. M3 Registration: guided capture, intra-sample check, duplicate check, Room storage.
4. M4 Punch: full punch flow, debounce, audit snapshot, local punch log screen.
5. M5 Anti-spoofing: active challenge first, then MiniFASNet passive model, then consistency check.
6. M6 Sync: Room-first repos, WorkManager workers, PHP endpoints on MySQL, idempotency tests.
7. M7 Hardening & tuning: kiosk mode, encryption, threshold tuning, spoof test report.
8. M8 Pilot: one site, supervised, compare against existing logs.

Complete and verify one milestone before starting the next. After each milestone, update docs/PROGRESS.md with what was done, what is untested, and open decisions.

• Make small, reviewable changes; one concern per commit.
• Before adding a dependency, state why, its license, and its size. Prefer what is already in the stack.
• Never introduce paid SDKs or network calls to third-party face services.
• Never store or transmit raw face images except the audit snapshot to our own server.
• Do not silently change thresholds, schemas, or API shapes. Propose changes and note them in docs/PROGRESS.md.
• If a requirement is ambiguous (especially around existing tk_ tables, shift rules, or model licensing), stop and ask rather than guess.
• When unsure about accuracy impact, favor rejecting a punch (with a clear retry message) over accepting a wrong one.

• Works fully offline and syncs correctly afterward
• Unit/instrumented tests added and passing; lint clean
• No PII or embeddings in logs
• Thresholds read from FaceConfig
• Docs updated (PROGRESS.md, MODELS.md if models changed)
• Tested on the actual target kiosk device, not only an emulator

• TODO(decision): exact column mapping from face_punches into the existing tk_ raw-log table
• TODO(decision): shift/overnight rules the server applies when pairing IN/OUT
• TODO(decision): target kiosk device model (camera quality, IR/depth sensor availability)
• TODO(decision): number of employees per kiosk (affects matching speed and thresholds)
• TODO(decision): snapshot retention period and fallback (PIN/admin) policy