---
title: "Taking AI Agents to the Race Track: Lessons from the High-Velocity Field Test"
description: "What happens when you strap a laptop into a race car, connect GPS devices, and run Gemini Nano at 150mph - a GDE field test."
url: "https://h3manth.com/scribe/high-velocity-ai-field-test/"
canonical_url: "https://h3manth.com/scribe/high-velocity-ai-field-test/"
markdown_url: "https://h3manth.com/scribe/blog/high-velocity-ai-field-test.md"
publishDate: "2025-12-18T08:00:00.000Z"
author: "Hemanth HM"
category: "ai"
tags: ["ai"]
word_count: 1607
reading_time_minutes: 9
x_markdown_tokens: 3275
content_signal: "ai-train=yes, search=yes, ai-input=yes"
---

# Taking AI Agents to the Race Track: Lessons from the High-Velocity Field Test

> What happens when you strap a laptop into a race car, connect GPS devices, and run Gemini Nano at 150mph - a GDE field test.

What an experience. Taking AI agents from notebooks to race tracks was one of the most incredible collaborative efforts I've been part of. GDE++ indeed. 🤓

This is just the beginning, and I'm really looking forward to more experiments like this.

## The Setup

Picture this: a laptop strapped into a race car, GPS devices wired up, real-time telemetry rendering on a UI, and Gemini Nano running inference-all while the car is doing 150mph around the track.

That was our field test.

<video controls playsinline preload="metadata" width="100%" style="border-radius: 8px; margin: 1rem 0;">
  <source src="/scribe/blog/koru/race-track-v3.mp4" type="video/mp4" />
  Your browser does not support the video tag.
</video>

We talked to the CEO of the race track. We got expert pro driver assessments to improve our agents. We debugged code between sessions while mechanics prepped the car. This wasn't a hackathon in a conference room, this was AI meeting reality.

## The Split-Brain Architecture

Real-time racing coaching needs two very different capabilities:

1. **Instant reactions** - "BRAKE!" can't wait 500ms
2. **Strategic advice** - "You're lifting too early compared to the pro"

We built a dual-path system:

```
┌────────────────────────────────────────────────────────────┐
│              10Hz GPS TELEMETRY STREAM                     │
│   (lat, lon, speed, heading, timestamp) @ 100ms intervals  │
└─────────────────────────┬──────────────────────────────────┘
                          │
              ┌───────────┴───────────┐
              ▼                       ▼
┌──────────────────────┐    ┌─────────────────────────┐
│    GEMINI NANO       │    │    GEMINI FLASH 3.0     │
│    Chrome Browser    │    │    Cloud REST API       │
├──────────────────────┤    ├─────────────────────────┤
│ Inference: <50ms     │    │ Inference: 500-1500ms   │
│ Context: 1KB         │    │ Context: 32KB           │
│ Output: 1 token      │    │ Output: Full analysis   │
└──────────────────────┘    └─────────────────────────┘
```

### Nano: The Reflexes

Gemini Nano runs entirely in Chrome via the `window.LanguageModel` API. No network round-trip-the model lives on-device.

```typescript
const getInstantFeedback = async (telemetry: TelemetryData): Promise<HotAction> => {
  const session = await window.LanguageModel.create({
    systemPrompt: `Racing copilot. Respond with ONE action word.
      
      DECISION MATRIX:
      | Condition                        | Action      |
      |----------------------------------|-------------|
      | brakePos > 50 AND longG < -0.8   | THRESHOLD   |
      | brakePos > 10 AND latG > 0.4     | TRAIL_BRAKE |
      | latG > 1.0 AND throttle < 20     | COMMIT      |
      | latG reducing AND throttle < 50  | THROTTLE    |
      | throttle > 80 AND latG < 0.3     | PUSH        |
      | throttle < 10 AND brakePos < 10  | NO_COAST    |`
  });

  const result = await session.prompt(
    `Telemetry: ${JSON.stringify(telemetry)}`,
    { 
      responseConstraint: {
        type: "object",
        properties: {
          action: {
            type: "string",
            enum: ["THRESHOLD", "TRAIL_BRAKE", "COMMIT", "THROTTLE", "PUSH", "NO_COAST", "MAINTAIN"]
          }
        }
      }
    }
  );

  session.destroy();
  return JSON.parse(result);
};
```

The `responseConstraint` schema forces valid output-no parsing failures, no hallucinations, guaranteed enum values.

### Flash: The Strategist

For lap analysis and technique comparison, we use Gemini Flash via REST:

```typescript
const BASE_URL = "https://generativelanguage.googleapis.com/v1beta/models";

const getCoachingAdvice = async (
  currentTelemetry: TelemetryData,
  referenceLap: Lap,
  coach: CoachPersona
): Promise<ColdAdvice> => {
  
  const response = await fetch(
    `${BASE_URL}/gemini-flash-3.0:generateContent?key=${API_KEY}`,
    {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({
        contents: [{ 
          parts: [{ text: buildPrompt(currentTelemetry, referenceLap, coach) }] 
        }],
        generationConfig: { 
          responseMimeType: "application/json",
          maxOutputTokens: 500 
        }
      })
    }
  );

  const result = await response.json();
  return parseAdvice(result.candidates[0].content.parts[0].text);
};
```

## GPS-Derived Virtual Sensors

Professional telemetry requires OBD-II hardware to read brake pressure and throttle position. We made it work with GPS alone by deriving these values from physics:

```typescript
interface TelemetryPoint {
  lat: number;
  lng: number;
  speed: number;      // GPS-derived m/s
  gLat: number;       // Lateral G-force (calculated)
  gLong: number;      // Longitudinal G-force (calculated)
  brake: number;      // Virtual sensor: 0-100
  throttle: number;   // Virtual sensor: 0-100
  time: number;
}

const calculateVirtualSensors = (
  current: GPSPoint, 
  previous: GPSPoint, 
  dt: number
): TelemetryPoint => {
  // Longitudinal G from velocity derivative
  const longG = (current.speed - previous.speed) / dt / 9.81;
  
  // Lateral G from heading change rate and speed
  const headingDelta = normalizeAngle(current.heading - previous.heading);
  const angularVelocity = headingDelta / dt;
  const latG = (current.speed * angularVelocity) / 9.81;

  // Virtual sensors inferred from G-forces
  const isBraking = longG < -0.5;  // Significant deceleration
  const isThrottle = longG > 0.1;   // Positive acceleration

  return {
    lat: current.lat,
    lng: current.lng,
    speed: current.speed * 2.237,  // Convert to mph
    gLat: latG,
    gLong: longG,
    brake: isBraking ? Math.min(100, Math.abs(longG) * 120) : 0,
    throttle: isThrottle ? Math.min(100, longG * 200) : 0,
    time: current.timestamp
  };
};
```

## The CAN Bus Discovery

One of the biggest takeaways: **AI can now interface with hardware effortlessly.**

And then there was CAN bus (Controller Area Network)-the protocol that connects everything in a modern vehicle. Every sensor, every ECU, every control module talks over this two-wire bus.

```
┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│   Engine    │     │   Brakes    │     │  Dashboard  │
│    ECU      │     │    ABS      │     │   Cluster   │
└──────┬──────┘     └──────┬──────┘     └──────┬──────┘
       │                   │                   │
       └───────────────────┴───────────────────┘
                     CAN Bus (500kbps)
                           │
                    ┌──────┴──────┐
                    │  OBD-II     │
                    │  Adapter    │
                    └──────┬──────┘
                           │
                    ┌──────┴──────┐
                    │   Laptop    │
                    │  + AI Agent │
                    └─────────────┘
```

With a $20 OBD-II adapter, you can read:
- Real brake pressure (not GPS-derived)
- Actual throttle position
- Engine RPM, coolant temp, fuel level
- Wheel speeds (individual!)
- Steering angle

The next iteration will tap directly into CAN for ground-truth telemetry.

## Predictive Geofencing

Best coaching comes *before* you need it. We defined geofences around every corner using Haversine distance:

```typescript
const GEOFENCE_RADIUS = 200; // meters

const getFeedForwardAdvice = (
  currentPos: { lat: number; lon: number },
  corners: Corner[]
): string | null => {
  
  for (const corner of corners) {
    const distance = haversineDistance(
      currentPos.lat, currentPos.lon,
      corner.lat, corner.lon
    );

    // Trigger in 150-200m approach band
    if (distance <= GEOFENCE_RADIUS && distance > (GEOFENCE_RADIUS - 50)) {
      return `APPROACHING ${corner.name}: ${corner.advice}`;
    }
  }
  return null;
};

const haversineDistance = (lat1: number, lon1: number, lat2: number, lon2: number): number => {
  const R = 6371e3; // Earth radius in meters
  const φ1 = lat1 * Math.PI / 180;
  const φ2 = lat2 * Math.PI / 180;
  const Δφ = (lat2 - lat1) * Math.PI / 180;
  const Δλ = (lon2 - lon1) * Math.PI / 180;

  const a = Math.sin(Δφ/2) * Math.sin(Δφ/2) +
            Math.cos(φ1) * Math.cos(φ2) *
            Math.sin(Δλ/2) * Math.sin(Δλ/2);
            
  return R * 2 * Math.atan2(Math.sqrt(a), Math.sqrt(1-a));
};
```

## Low-Latency Audio

A 50ms AI decision is worthless if audio synthesis takes 200ms. We pre-cache all command audio:

```typescript
class CoachAudioService {
  private audioContext: AudioContext;
  private buffers = new Map<string, AudioBuffer>();
  private minRepeatInterval = 1500; // Prevent spam
  
  async initialize(): Promise<void> {
    this.audioContext = new AudioContext();
    
    const commands = [
      'THRESHOLD', 'TRAIL_BRAKE', 'COMMIT', 'THROTTLE',
      'PUSH', 'NO_COAST', 'GOOD', 'SEND_IT'
    ];
    
    await Promise.all(
      commands.map(async cmd => {
        const response = await fetch(`/audio/${cmd}.mp3`);
        const buffer = await this.audioContext.decodeAudioData(
          await response.arrayBuffer()
        );
        this.buffers.set(cmd, buffer);
      })
    );
  }

  play(action: string): void {
    const buffer = this.buffers.get(action);
    if (!buffer) return;

    const source = this.audioContext.createBufferSource();
    source.buffer = buffer;
    source.connect(this.audioContext.destination);
    source.start(0);  // Instant - no network, no TTS synthesis
  }
}
```

Result: decision-to-audio in **under 10ms**.

## Coach Personas

Different drivers respond differently. Tony needs encouragement. Rachel wants physics. Garmin wants data.

```typescript
type CoachPersona = 'TONY' | 'RACHEL' | 'AJ' | 'GARMIN' | 'SUPER_AJ';

const COACH_PROMPTS: Record<CoachPersona, string> = {
  TONY: `Encouraging, feel-based. "Commit!", "Good hustle!", "Send it!"`,
  
  RACHEL: `Physics teacher. Focus on weight transfer, platform balance.
           "Smooth brake release", "Settle the rear"`,
           
  AJ: `Direct commands. Connect vehicle state to driver input.
       "Lat G stable-hammer throttle", "Rotation complete-unwind"`,
       
  GARMIN: `Data robot. Delta times only. "Brake 0.2s later", "-0.3s sector"`,
  
  SUPER_AJ: `Adaptive. Switch persona based on error type:
             Safety → Imperative ("STABILIZE!")
             Technique → Physics ("Smooth release")
             Confidence → Encouragement ("Commit now!")
             Optimization → Delta ("Brake later")`
};
```

## Heuristic Fallback

Not everyone runs Chrome Canary with Nano flags. So there's always a rule-based fallback:

```typescript
const getHeuristicAction = (data: TelemetryData): HotAction => {
  const { speedMph, throttle, brakePos, latG, longG } = data;

  if (brakePos > 30 && longG < -0.2) return { action: "THRESHOLD", color: "#ef4444" };
  if (brakePos > 5 && Math.abs(latG) > 0.1) return { action: "TRAIL_BRAKE", color: "#ef4444" };
  if (Math.abs(latG) > 0.3 && throttle < 40) return { action: "COMMIT", color: "#3b82f6" };
  if (Math.abs(latG) > 0.15 && throttle < 60) return { action: "THROTTLE", color: "#f97316" };
  if (throttle > 60 && Math.abs(latG) < 0.1) return { action: "PUSH", color: "#f97316" };
  if (throttle < 15 && brakePos < 5 && speedMph > 50) return { action: "NO_COAST", color: "#eab308" };
  
  return { action: "MAINTAIN", color: "#a855f7" };
};
```

Tuned over hundreds of simulated laps. Not as smart as Nano, but works everywhere.

## Designing for the Real World

Racing pushes hardware to the limit-and that made it the perfect stress test:

- **High G-forces** taught us to secure every connection
- **Thermal extremes** forced us to optimize for efficiency
- **Remote locations** validated our offline-first edge architecture
- **Split-second timing** proved the value of pre-cached responses

Every constraint became a design feature. No connectivity? Edge inference. Synthesis latency? Pre-cached audio. No Nano support? Heuristic fallback. The environment forced good architecture.

## What's Next

This was a fantastic AI field test, and there's definitely more to come.

The pattern we validated-edge AI for reflexes, cloud AI for strategy, human expertise as ground truth-applies beyond racing:

- Sports coaching
- Surgical guidance  
- Head up display feedback
- Agent with persona of the coach we meet in the room

Any domain with real-time feedback and verified human expertise to learn from.

## Thank You

Huge thanks to Ajeet Mirwani for organizing this and giving us the thrill of being in a race car. Taking agents to the track was unforgettable.

And to the entire GDE team who collaborated across domains-AI, hardware, UX, racing-to make this happen. What an incredible effort.

This is just the beginning. 🏎️

---

*Built as part of the GDE High-Velocity AI Field Test.*