> ## Documentation Index
> Fetch the complete documentation index at: https://docs.protonverse.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Running AI Inference with TFLite Micro on Proton AI Core

> Run TensorFlow Lite Micro models on Proton AI Core's ESP32-S3 for on-device image classification, object detection, and other edge AI tasks.

The ESP32-S3-WROOM-1 at the heart of Proton AI Core includes hardware acceleration for neural network operations via its vector instructions (PIE — Processor Instruction Extensions), making it well suited for edge AI inference without any cloud dependency. You can run quantized TensorFlow Lite models directly on-device, processing camera frames or sensor data in real time.

## Supported Frameworks

Proton AI Core works with the following edge AI frameworks:

* **TensorFlow Lite for Microcontrollers (TFLM)** — the primary framework for on-device inference on the ESP32-S3. It supports INT8 quantized models and integrates directly with the ESP-IDF and Arduino ecosystems.
* **Edge Impulse** — a cloud-based platform for model training, optimization, and deployment. Edge Impulse can generate a ready-to-flash Arduino or PlatformIO library for your trained model.

<Note>
  Additional framework support is planned with the Proton AI Platform.
</Note>

## Installing TensorFlow Lite Micro

<Tabs>
  <Tab title="Arduino IDE">
    Open the **Library Manager** (**Sketch → Include Library → Manage Libraries**), search for `Arduino_TensorFlowLite`, and install the latest available version.

    <Note>
      Library version availability varies between registries. For the latest ESP32-S3-specific optimizations, refer to the Espressif TFLite Micro fork: [https://github.com/espressif/tflite-micro-esp-examples](https://github.com/espressif/tflite-micro-esp-examples)
    </Note>
  </Tab>

  <Tab title="PlatformIO">
    Add the following to your `platformio.ini`:

    ```ini theme={null}
    lib_deps =
        tensorflow/lite-micro @ ^1.0.0
    ```

    <Note>
      Library version availability varies. Check the Espressif TFLite Micro fork for latest ESP32-S3 optimizations: [https://github.com/espressif/tflite-micro-esp-examples](https://github.com/espressif/tflite-micro-esp-examples)
    </Note>
  </Tab>
</Tabs>

## Model Preparation

Before you can run inference on-device, you need to convert your trained model into a C byte array that can be compiled into your firmware.

<Steps>
  <Step title="Obtain a .tflite model">
    Train or download a `.tflite` model for your target task — for example, image classification or person detection. Espressif's [tflite-micro-esp-examples](https://github.com/espressif/tflite-micro-esp-examples) repository includes pre-built models to get you started.
  </Step>

  <Step title="Convert the model to a C byte array">
    Use the `xxd` command-line tool to generate a header file from your model file:

    ```bash theme={null}
    xxd -i model.tflite > model_data.h
    ```

    This produces a `model_data.h` file containing a `unsigned char` array. When the input file is named `model.tflite`, `xxd` names the array `model_tflite`. Update the array name in your code if your filename differs.
  </Step>

  <Step title="Include the header in your sketch">
    Add `#include "model_data.h"` at the top of your main source file. Reference the array by the name `xxd` generated — by default, `model_tflite` for a file called `model.tflite`.
  </Step>
</Steps>

## Running Inference

The snippet below shows a minimal TFLite Micro inference loop. It loads the model, allocates tensors, copies image data into the input tensor, invokes the interpreter, and reads the output score. This example assumes a float32 model; see the note below for INT8 quantized models.

```cpp inference.cpp theme={null}
#include "tensorflow/lite/micro/all_ops_resolver.h"
#include "tensorflow/lite/micro/micro_interpreter.h"
#include "tensorflow/lite/schema/schema_generated.h"
#include "model_data.h"

constexpr int kTensorArenaSize = 100 * 1024;  // Adjust as needed
uint8_t tensor_arena[kTensorArenaSize];

void runInference(float* image_data, size_t num_elements) {
  const tflite::Model* model =
      tflite::GetModel(model_tflite);

  tflite::AllOpsResolver resolver;
  tflite::MicroInterpreter interpreter(
      model, resolver, tensor_arena, kTensorArenaSize);
  interpreter.AllocateTensors();

  TfLiteTensor* input = interpreter.input(0);
  // Copy pre-processed image data into the input tensor
  memcpy(input->data.f, image_data, num_elements * sizeof(float));

  interpreter.Invoke();

  TfLiteTensor* output = interpreter.output(0);
  float score = output->data.f[0];
  Serial.printf("Inference score: %.3f\n", score);
}
```

<Note>
  For INT8 quantized models, the input tensor uses `input->data.int8` (or `input->data.uint8` for unsigned quantization) and the output tensor uses `output->data.int8`. You must also apply the tensor's quantization scale and zero-point to convert raw INT8 values to meaningful scores. INT8 models run significantly faster on the ESP32-S3 and are recommended for production use.
</Note>

## Performance Tips

Getting the best inference speed on the ESP32-S3 requires a few deliberate choices at both the model and firmware levels:

* **Allocate the tensor arena in PSRAM.** Proton AI Core includes 8 MB of PSRAM. For large models, allocate the tensor arena dynamically using `heap_caps_malloc` to avoid exhausting the 512 KB of internal SRAM:

  ```cpp theme={null}
  uint8_t* tensor_arena = (uint8_t*) heap_caps_malloc(
      kTensorArenaSize, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
  ```

  For small models that fit comfortably in internal SRAM, a static declaration (`uint8_t tensor_arena[kTensorArenaSize]`) is simpler and avoids heap fragmentation.

* **Quantize your models to INT8.** INT8 quantization typically reduces model size by 4× and speeds up inference significantly compared to float32, with minimal accuracy loss.

* **Use `FRAMESIZE_QVGA` or smaller for camera-based inference.** Smaller input images reduce both pre-processing time and input tensor memory requirements.

* **Run the CPU at maximum frequency.** Enable the `CONFIG_ESP32S3_DEFAULT_CPU_FREQ_240` option in your `sdkconfig` (ESP-IDF) or set `board_build.f_cpu = 240000000L` in `platformio.ini` to clock the dual-core Xtensa LX7 at 240 MHz.

<Tip>
  The Proton AI Platform (coming soon) will include a model management dashboard for deploying and updating models OTA — no manual `xxd` conversion needed.
</Tip>