Adversarial Gameplay

An end-to-end evaluation harness that turns a black-box reinforcement-learning adversary into a system you can measure, compare, and explain.

Senior design project sponsored by Lockheed Martin. A human player commands three mech units in a turn-based tactical game on an 8×8 grid, while a multi-headed reinforcement-learning model controls the enemy. Around the game we built a complete evaluation pipeline. I owned the evaluation system and the LLM grading integration.

Problem

Win rate and cumulative reward tell you whether an adversarial agent succeeded, not how or why. Two agents can post the same win rate with very different decision quality, and a high reward can hide poor tactics masked by luck. For a sponsor whose agents may inform real strategic systems, that opacity makes iteration guesswork, especially when a classifier routes decisions to a specialist policy.

Process

A FastAPI server sits between the game and the models, translating game state into the 2,372-dimensional observation tensor the policy networks expect and decoding action logits back into legal moves. A VQ-VAE transformer classifier detects which of five strategies the player is using each turn and routes the PPO adversary to the matching head. On top of that I designed a deterministic grader that scores every enemy decision across nine dimensions, then hands the metrics to Llama 3.1 on Groq for a short written analysis after each game.

Solution

• REST inference API with hard action masking for 65 discrete moves, under a 500ms latency budget.
• Nine-dimension deterministic grading: target hit rate, threat coverage, kill potential, damage threatened, position quality, mech HP lost, mechs killed, buildings destroyed, and vek lost, combined into a 0–10 score per turn and 0–100 per game.
• LLM analyst layer that turns the numbers into a two-to-three sentence narrative, such as a perfect hit rate with zero kills pointing to cautious positioning.
• React dashboard that persists every game to JSON, charts trends across model versions, and expands any mission into its per-turn breakdown.

Results

• 9 Grading dimensions per turn
• 2,372 Dimensions in the observation tensor
• 5 Trained RL model checkpoints
• <500ms API latency budget

Justin Moran · UI/UX Designer · Chicago, IL · LinkedIn · Resume (PDF)