Skip to main content
The ESP32-S3-WROOM-1 at the heart of Proton AI Core includes hardware acceleration for neural network operations via its vector instructions (PIE — Processor Instruction Extensions), making it well suited for edge AI inference without any cloud dependency. You can run quantized TensorFlow Lite models directly on-device, processing camera frames or sensor data in real time.

​
Supported Frameworks

Proton AI Core works with the following edge AI frameworks:
  • TensorFlow Lite for Microcontrollers (TFLM) — the primary framework for on-device inference on the ESP32-S3. It supports INT8 quantized models and integrates directly with the ESP-IDF and Arduino ecosystems.
  • Edge Impulse — a cloud-based platform for model training, optimization, and deployment. Edge Impulse can generate a ready-to-flash Arduino or PlatformIO library for your trained model.
Additional framework support is planned with the Proton AI Platform.

​
Installing TensorFlow Lite Micro

Open the Library Manager (Sketch → Include Library → Manage Libraries), search for Arduino_TensorFlowLite, and install the latest available version.
Library version availability varies between registries. For the latest ESP32-S3-specific optimizations, refer to the Espressif TFLite Micro fork: https://github.com/espressif/tflite-micro-esp-examples

​
Model Preparation

Before you can run inference on-device, you need to convert your trained model into a C byte array that can be compiled into your firmware.
1

Obtain a .tflite model

Train or download a .tflite model for your target task — for example, image classification or person detection. Espressif’s tflite-micro-esp-examples repository includes pre-built models to get you started.
2

Convert the model to a C byte array

Use the xxd command-line tool to generate a header file from your model file:
xxd -i model.tflite > model_data.h
This produces a model_data.h file containing a unsigned char array. When the input file is named model.tflite, xxd names the array model_tflite. Update the array name in your code if your filename differs.
3

Include the header in your sketch

Add #include "model_data.h" at the top of your main source file. Reference the array by the name xxd generated — by default, model_tflite for a file called model.tflite.

​
Running Inference

The snippet below shows a minimal TFLite Micro inference loop. It loads the model, allocates tensors, copies image data into the input tensor, invokes the interpreter, and reads the output score. This example assumes a float32 model; see the note below for INT8 quantized models.
inference.cpp
#include "tensorflow/lite/micro/all_ops_resolver.h"
#include "tensorflow/lite/micro/micro_interpreter.h"
#include "tensorflow/lite/schema/schema_generated.h"
#include "model_data.h"

constexpr int kTensorArenaSize = 100 * 1024;  // Adjust as needed
uint8_t tensor_arena[kTensorArenaSize];

void runInference(float* image_data, size_t num_elements) {
  const tflite::Model* model =
      tflite::GetModel(model_tflite);

  tflite::AllOpsResolver resolver;
  tflite::MicroInterpreter interpreter(
      model, resolver, tensor_arena, kTensorArenaSize);
  interpreter.AllocateTensors();

  TfLiteTensor* input = interpreter.input(0);
  // Copy pre-processed image data into the input tensor
  memcpy(input->data.f, image_data, num_elements * sizeof(float));

  interpreter.Invoke();

  TfLiteTensor* output = interpreter.output(0);
  float score = output->data.f[0];
  Serial.printf("Inference score: %.3f\n", score);
}
For INT8 quantized models, the input tensor uses input->data.int8 (or input->data.uint8 for unsigned quantization) and the output tensor uses output->data.int8. You must also apply the tensor’s quantization scale and zero-point to convert raw INT8 values to meaningful scores. INT8 models run significantly faster on the ESP32-S3 and are recommended for production use.

​
Performance Tips

Getting the best inference speed on the ESP32-S3 requires a few deliberate choices at both the model and firmware levels:
  • Allocate the tensor arena in PSRAM. Proton AI Core includes 8 MB of PSRAM. For large models, allocate the tensor arena dynamically using heap_caps_malloc to avoid exhausting the 512 KB of internal SRAM: 
    uint8_t* tensor_arena = (uint8_t*) heap_caps_malloc(
    kTensorArenaSize, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
    For small models that fit comfortably in internal SRAM, a static declaration (uint8_t tensor_arena[kTensorArenaSize]) is simpler and avoids heap fragmentation.
  • Quantize your models to INT8. INT8 quantization typically reduces model size by 4× and speeds up inference significantly compared to float32, with minimal accuracy loss.
  • Use FRAMESIZE_QVGA or smaller for camera-based inference. Smaller input images reduce both pre-processing time and input tensor memory requirements.
  • Run the CPU at maximum frequency. Enable the CONFIG_ESP32S3_DEFAULT_CPU_FREQ_240 option in your sdkconfig (ESP-IDF) or set board_build.f_cpu = 240000000L in platformio.ini to clock the dual-core Xtensa LX7 at 240 MHz.
The Proton AI Platform (coming soon) will include a model management dashboard for deploying and updating models OTA — no manual xxd conversion needed.