Lecture 1: Course Introduction and Programming Environment Setup

This lecture is devoted to the software suite install so that everybody can follow the other lectures from Insa or from his home if it needs to be done in distant work.

Course outline

All Lecture (2h on a computer) are labs on the LyraT board

Part 1 : Board introduction and Audio Signal Processing Basics

  • Lecture 1 (2h): Course Introduction, programming env setup
  • Lecture 2: Basic signal processing,
  • Lecture 3: Audio systems architecture, audio callback
  • Lecture 4: Hardware Audio codec configuration
  • Lecture 5 & 6: Audio signal processing basics synthesis
  • Lecture 7: Faust Language tutorial https://faust.grame.fr/

Part 2: Embedded Audio System Architecture

  • Lecture 8: RTone comference on embedded systems in industry https://rtone.fr/
  • Lecture 9: Embedded System Peripherals
  • Lecture 10: embedded OS, free-RTOS, embedded Audio linux devices

Part 3: LyraT programming

  • Lecture 11-14: mini project
  • Lecture 15-16: demonstrations

Introduction to AUD2020 and ESP3

ESP32 die shot, taken from zeptobars, its functional block and its pin layout, taken from esp32 datasheet

The development in AUD are performed on LyraT which is developed by the espressif company. The programming environment used is esp-idf and esp-adf (esp-adf is a wrapper around esp-idf so as to enable easy audio configuration).

Espressif Systems is a fabless IC design company, founded in 2008 in Shanghai, China (~200 employees in 2018). Espressif is designing and manufacturing low power wireless sensor chips.

Their main product today is the ESP32 microcontroler, based on an old Xtensa processor architecture, that is sold for less the 5€ and that is strongly oriented toward Internet of Things (IoT). Before that was ESP8089 and ESP8266 wifi chip

ESP32 and LyraT

The ESP32 is a Single 2.4 GHz Wi-Fi and Bluetooth combo chip (i.e. radio basebands and processor integrated into the same chip). The processor is and Xtensa Single or Dual-core 32-bit LX6 microprocessor that can reach 600 MIPS. It comes with a ROM of 448 KB, and a SRAM of 520KB and, as any micro-controller, with a bunch of peripherals (ADC, DAC, Touch pad, SPI, I2S, I2C, PWM, SDIO, Ethernet, UART, etc.), see its functionnal blocks.

The two CPUs are named “PRO_CPU” and “APP_CPU” (for “protocol” and “application”), however, for most purposes the two CPUs are interchangeable.

Espressif proposes three documentations for the ESP32:

We use ESP32-LyraT v4.3 evaluation board, all available documentation is present on espressif web site. You will need to access to the hardware reference and the schematics of the board.

LyraT simple diagram and board layout (from espressive webite)

The ESP32-LyraT v3.4 is a hardware platform designed for the dual-core ESP32 audio applications, e.g., Wi-Fi or Bluetooth audio speakers, speech-based remote controllers, connected smart-home appliances with one or more audio functionality, etc.

The components are quite clearly shown on Figure above, here are some precision:

  • Output socket to connect headphones use a 3.5 mm stereo jack. The socket may be used with mobile phone headsets and is compatible with OMPT standard headsets only. It does not work with CTIA headsets.
  • When programming (i.e. flashing) the board , the following actions must be performed: hold down the Boot button and simultaneously momentarily press the Reset button. This initiates the firmware upload mode. Then user can upload firmware through the serial port (using the flash program on the host computer).
  • once the board is programmed (i.e. flashed), pressing the Reset button is necessary for the new program to start.
  • The audio chip used is the ES8388 from Everest. It is quite important because performance and properties of audio codec vary a lot from one to another. It is connected to I2C and I2S busses of the ESP32.
  • The USB-UART port is used to have a serial communication between the ESP32 and the host computer as well as for flashing/programming the ESP32 with JTAG protocol using openocd tool.
  • The Green 'Standby/Charging' LED indicates that the board is powered from USB. The red 'Power On' LED indicates that the board is on (there is a switch to cut it off). The 'Green' LED can be used by the user program.

ESP32 developpement framework: ESP32 IDF

IDF stands for IOT Development Framework, it is relatively straightforward to install it on your computer. It has been installed on TC dept machines, in /opt/esp-adf/esp-idf. In order to use it, you have to run the file export.sh located in the directory where you have installed IDF.

Installing ESP32 IDF on your computer

Note that IDF installation uses more than 3GB of disk space. Note also you will need to have Python3 (and not Python2.7), you can handle different version of Python on Linux using update-alternatives

Follow the instruction on the espressive IDF getting started page and install IDF on your computer (The installation is quite long 10-20 minutes depending on the quality of your connection).

Although it is not mandatory, it is recommended to add the IDF_PATH (which is the location where you installed ESP32-IDF) in your environment. When you have install ESP32-IDF in directory ''ESP32-IDF'', you have to source the export.sh file before building a project:

source ESP32-IDF/export.sh

it is not recommended to perform the source of ''export.sh'' in the profile script because it might invalidate other tool using python. Instead you can define a command for performing the source:

alias get_idf='. $HOME/esp/esp-idf/export.sh'

Different compilation tools used

ESP32-IDF projects supports several build/compilation tools: make, cmake and idf.py (we recommend the use of idf.py tool)

  • make, using a generic and quite complex ID_PATH/make/project.mk Makefile for all existing project. This was the original only tool used IDF, however it is being progressively replaced by the cmake compilation tools.
  • cmake which is the recommended toolchain as make might not be supported anymore in further version. Here is an example of project compilation with cmake:

    mkdir build;
    cd build;
    cmake ../
    make 
    make flash 
    make monitor
    

  • idf.py is a top-level python config/build command line tool for ESP-IDF provided by espressive build. Here is an example of project compilation with idf.py:

    idf.py all
    idf.py flash 
    idf.py monitor
    

Getting Started on TC Machines

Launching the Compilation Flow on TC Machines

The idf tool chain is installed on TC machine in /opt/esp-idf/. In order to use this tool chain, do the following commands:

export IDF_TOOLS_PATH=/opt/idf_tools
source /opt/esp-idf/export.sh

You should get the following message (if you do not, it do not go further):

[...]
Go to the project directory and run:

  idf.py build

then copy de /opt/esp-idf/examples/get-started/ to your home directory:

cp /opt/esp-idf/examples/get-started/ ~/my-esp/

Go in the ``get-started/hello_world and build the project

cd ~/my-esp/hello_world/
idf.py build

Once the binary program built, connect the lyraT board with the usb cables, make sure that the central switch is in position 'on'.

Load the program: execute the following command and do the 'flash' manipulation: push continuously on the boot button, press (shortly, but not too shortly) on the reset button, release the boot button:

idf.py flash

you should see the load executing..

Open the serial port on /dev/ttyUSB0 using IDF monitor facility:

idf.py monitor

Do not forget to reset the board once again (push on the reset button) after each flash operation otherwise your program is not started. Then you should see the board booting every 10 second. Kill the serial monitor by using the command:

Ctrl-alt gr-]

Flashing the LED.

Go in the get-started/blink directory

this program blink the LED, but the port in not configured correctly as it is an information that depends on the experimental board in which the ESP32 is used. launche the menuconfig interface, select example configuration and choose 22 for Blink_GPIO_number

Known Problems

Requirements are not satisfied: gdbgui>=0.13.2.0

On ubuntu, this message sometimes occurs when sourcing export.sh. We did not completely understood the problem but two solutions seemed to work:

  1. Remove explicitely the faulty dependance in ${IDF}/requirement.txt. The faulty dependance is not gdbgui but pugdbmi: comment the line mentionning pugdbmi
  2. Use the reddit solution: https://www.reddit.com/r/esp32/comments/ifgfy9/why_am_i_getting_this_gdbgui01320_error/

USB driver on MAC platforms

It occurs on some MAC computers that the USB driver are not installed, you have to install it expicitely, it is explained here for instance: https://www.amstramgrame.fr/gramophone/loader/

Pyhton (build) problem on MAC platforms

Not solved yet