How to add custom peripherals to qemu-esp?

[64-core CPU]

(Please note that English is not my native language)

Hi all! I am developing my own device with esp32 chip as CPU. This device has various peripherals on different buses such as i2c and spi. Some of the peripherals also have GPIO connections for interrupts.

I want to make an emulator of my device using QEMU. I am new to QEMU programming and have a vague understanding of the QEMU architecture and the ESP32 virtual machine in particular.

For example, I wrote a simple device for the i2c bus and connected it to an esp32 machine:

Code: Select all

static void esp32_machine_init_bic_peripherals( Esp32SocState* s ){
	
	DeviceState* i2c_master = DEVICE(&s->i2c[0]);
	I2CBus* i2c_bus = I2C_BUS( qdev_get_child_bus(i2c_master, "i2c") );
	
	I2CSlave* bicPhoneKeyboard;
	
	qemu_irq irq;
	DeviceState* deviceState;
	
	bicPhoneKeyboard = i2c_slave_create_simple( i2c_bus, "bic-phone-keyboard", 0x58 );
	deviceState = DEVICE( bicPhoneKeyboard );
	irq = qdev_get_gpio_in( DEVICE(&s->intmatrix), ETS_GPIO_INTR_SOURCE );
	
	qdev_connect_gpio_out( deviceState, 0, irq );
}

I added a call to this function from esp32_machine_init in /hw/xtensa/esp32.c. The device is successfully added to the i2c bus and I can work with it inside the virtual machine. But the interrupts aren't working. As I understand it, intmatrix isn't the place where I should be connecting the GPIO.

I also looked into the file /hw/gpio/esp32_gpio.c. Read and write functions do not appear to be implemented. https://github.com/espressif/qemu/blob/ ... gpio.c#L24 Now I'm completely confused and don't know which direction to go next. Ideally, I'd like to see documentation, but as far as I understand, there simply isn't any. If anyone has any information on how to properly connect devices to the machine using GPIO, i2c and SPI, please share the information. Ideally, I would like to be able to connect devices using qemu launch options.

[64-core CPU]

I did a little digging through the source code and concluded that my assumption about the gpio subsystem was correct—it's not implemented. The only thing it does implement is reading bootstrap. Obviously, this isn't enough for gpio to work in any form, so I tinkered with the code a bit. Here is a link to my implementation of the gpio subsystem https://pastee.dev/p/uhW37KEF

I implemented the virtual registers from the "esp32 technical reference manual," section 4.12.1—the code can write data to and from the GPIO matrix registers, and they will retain their states. I added 40 qemu_irq input lines, which are responsible for emulating the GPIO lines.When signals arrive on these lines, virtual registers will be updated and edge-triggered interrupts will be generated. Level-triggered interrupts will also be generated, but they will work incorrectly—they will be cleared even if the signal remains active. Frankly, I'm just too lazy to implement the correct logic for their operation—I need a mental break from digging through the qemu source code and do something else. I didn't implement the GPIO output lines at all, but you can do that yourself if you need them. The logic is very simple: when the ESP firmware wants to set a certain GPIO level, it simply writes the required values ​​to the appropriate registers. You simply look at the data written to the registers and modify the corresponding qemu_irq lines.

A few words about how to apply my code. First, completely replace the old files /hw/gpio/esp32_gpio.c and /include/hw/gpio/esp32_gpio.h. Next, in the file /hw/xtensa/esp32.c, find the function esp32_soc_realize. It contains these lines:

Code: Select all

    qdev_realize(DEVICE(&s->gpio), &s->periph_bus, &error_fatal);
    esp32_soc_add_periph_device(sys_mem, &s->gpio, DR_REG_GPIO_BASE);

Immediately after these lines, add the following code

Code: Select all

qdev_connect_gpio_out_named(
		DEVICE( &s->gpio ), 
		ESP32_GPIO_IRQ_GPIO,
		0,
		qdev_get_gpio_in( DEVICE(&s->intmatrix), ETS_GPIO_INTR_SOURCE )
	);
	
	qdev_connect_gpio_out_named(
		DEVICE( &s->gpio ),
		ESP32_GPIO_NMI_IRQ_GPIO,
		0,
		qdev_get_gpio_in( DEVICE(&s->intmatrix), ETS_GPIO_NMI_SOURCE )
	);

These two functions connect 2 interrupt lines "irq" and "nmi irq" from the gpio matrix to the interrupt matrix. It works like this: When you configure an interrupt on any GPIO, the GPIO matrix begins monitoring the GPIO states. When a GPIO interrupt occurs, the GPIO matrix sets the interrupt active level on the IRQ and NMI IRQ lines depending on the settings. The firmware then accesses the GPIO matrix, determines which pins the interrupt occurred on, and runs the appropriate interrupt handler.

Next, if you want to use the GPIO input, you should add code like this to esp32.c

Code: Select all

	DeviceState* i2c_master = DEVICE(&s->i2c[0]);
	I2CBus* i2c_bus = I2C_BUS( qdev_get_child_bus(i2c_master, "i2c") );
	
	I2CSlave* bicPhoneKeyboard;
	DeviceState* keyboardDeviceState;
	qemu_irq gpioMatrixInGpio;
	
	bicPhoneKeyboard = i2c_slave_create_simple( i2c_bus, "bic-phone-keyboard", 0x58 );
	keyboardDeviceState = DEVICE( bicPhoneKeyboard );
	
	gpioMatrixInGpio = qdev_get_gpio_in_named( DEVICE(&s->gpio), "gpio_in", 25 );
	qdev_connect_gpio_out( keyboardDeviceState, 0, gpioMatrixInGpio );

I'm new to qemu programming, so I'm not sure exactly where to add it. I moved my code into a separate function and run it from the esp32_machine_init function immediately after the line esp32_machine_init_i2c(ss); This code, for example, connects my keyboard's output GPIO to input GPIO 25 on the ESP32 GPIO matrix. Further in my esp32 firmware code, I configure the interrupt for gpio25, and it works as usual.

I hope this information will be useful to someone and will help them achieve their goal.

[64-core CPU]

OK, let's continue. I was able to connect the devices via the i2c bus and GPIO lines without any problems - they work as expected and without any issues.

Now when I tried to connect devices via SPI bus, I had problems with it. Data simply doesn't pass through SPI. The emulated device reads only zeros. The ESP32 firmware reads the same. My research led me here https://github.com/espressif/qemu/blob/ ... spi.c#L276

When the esp32 firmware wants to send something over the spi bus, it first writes various values ​​into the esp32 spi registers and then sends a command to start the transaction.This code doesn't handle any DMA registers at all. It assumes that data is transferred through the SPI data buffer (registers W0-W15).

I decided to write my own implementation in order to understand and figure out how everything works.In the process of writing, I realized that the addresses of the first elements of the DMA linked list are written to the SPI_DMA_IN_LINK_REG and SPI_DMA_OUT_LINK_REG registers. Obviously, the addresses are written relative to the guest address space. If I try to dereference them in the host space, a segfault occurs. As expected.

I wrote some code that prints the addresses written to the registers. Here's one of them: 0x000E05E4.This is a rather strange address. According to the documentation, addresses 0x00000000 - 0x3F3FFFFF are reserved.Can someone explain 2 things:

1) How should addresses written to the SPI_DMA_OUT_LINK_REG and SPI_DMA_IN_LINK_REG registers be interpreted? These registers only allocate 20 bits for the address itself—not enough to cover the entire address space.

2) I can't find any information on how to access the QEMU guest's memory. Does anyone have any examples of how to do this?

[64-core CPU]

It seems that I have found the answer to the first question. I assumed that this was an offset relative to something and started going through the starting addresses of the regions specified in the documentation.It looks like the address 0x3FF00000 is used as a base - this is the starting address in the peripheral address space. Thus, it turns out that the offset relative to 0x3FF00000 is written to the SPI_DMA_OUT_LINK_REG register. By adding the base address and the offset, a pointer to the DMA linked list structure is obtained.I tried transmitting different ones from different locations in memory using spi_device_polling_transmit() and after the call I read the SPI_DMA_OUT_LINK_REG register.After that, I added the base address to the value contained in that register and got a pointer to the DMA linked list structure. The buffer address pointer value always pointed to the data that I transmitted via spi.

If I have made a mistake in my research, I will be glad if my mistake is pointed out to me.

[64-core CPU]

The solution was found here https://stackoverflow.com/questions/654 ... pplication Using cpu_physical_memory_read you can read the guest's memory.

So, let's summarize SPI emulation using DMA channels:
1) The ESP firmware writes various registers, including SPI_DMA_OUT_LINK_REG and SPI_DMA_IN_LINK_REG.
2) 0x3FF00000 is added to the values ​​in these registers. This results in the address within the guest system.
3) The cpu_physical_memory_read function is used to read the DMA linked list structure, which is described in the ESP32 technical reference manual, section 6.3.2 DMA Linked list.
4) The buffer address pointer values ​​from the linked list structures are used in the cpu_physical_memory_read() function to read the data that the emulated system is trying to send over the SPI bus.
5) The read data is used to send data over the SPI bus using ssi_transfer().
6) Data received from the SPI device can be written to the guest memory using cpu_physical_memory_write(). However, I haven't tested this—my code only sends data over the bus to the display, but doesn't read it.
7) At the end of the transfer, do not forget to set the SPI_TRANS_DONE bit in the SPI_SLAVE_REG register, otherwise the guest system will go into an infinite loop waiting for this bit to appear there, indicating the completion of the transfer.

Actually, I'm not sure I did everything correctly, but my method works. That's all, I think. Thank you for your attention.