Future Electronics – Smartphone UI quality in an embedded system: the capability of new graphics hardware in today’s microcontrollers

By John Robins
Business Development Manager for Embedded Solutions Future Electronics (EMEA)

Only a few years ago, a microcontroller’s graphics capability was limited to rendering monochrome lines or patterns and simple images on a plain background for display on a small, low-resolution display.

A typical example is the set of small display screens in the centre of an instrument cluster in a high-end car, the Lexus LS460, dating from 2010, as shown in Figure 1.


Fig. 1: A graphical user interface from 2010: the instrument cluster of the Lexus LS460 passenger car. (Image credit: Maryland Pride under Creative Commons 3.0 licence)

Today, this type of Human-Machine Interface (HMI) appears crude and old-fashioned. A generation of users of smartphones, accustomed to their large and slick touchscreens featuring attractive icons rendered in gorgeous high definition, have much higher expectations of the HMI of any device that they interact with.

How can this expectation for a sophisticated graphics display-based HMI be met by design engineers who work with microcontrollers? For many years, the embedded design community worked under the assumption that a graphics display of any sophistication called for an applications processor running on a full-featured operating system such as the Linux® platform.

In fact, this is no longer the case. Very attractive and intuitive graphics displays supporting touch-sensing controls, and including streaming video, are capable of being implemented today by engineers who use an MCU. This is thanks to a combination of dedicated hardware features provided in the MCU architecture, and a new generation of graphics design tools that provide performance optimisations for the high-end graphics-oriented MCUs.

The effect is to make the implementation of high-performance, sophisticated graphics on a 32-bit MCU platform easier and more effective than it has ever been.

The constraint of the typical MCU architecture
In an embedded device, the HMI and display tend to be peripheral functions. This means that the core application might demand the majority of the CPU’s bandwidth most of the time. Even in an MCU that has a high-end CPU such as an ARM® Cortex®-M7 core, the HMI’s processing overhead must be kept below a certain limit. In the end, rendering graphics is a matter of running the C code compiled from a graphics display design tool. The more sophisticated the graphics, the higher the resolution and the larger the display, the more code must be executed.

The MCU’s processor usage, then, imposes a constraint on the HMI designer. The obvious solution might appear to be to migrate to an applications processor. But this is hugely disruptive to design teams that have previously worked in an MCU environment, and few are prepared to cope with the consequences of such an abrupt change.

And in fact the MCU manufacturers have taken steps to improve the effective amount of graphics processing that the CPU’s bandwidth supports, and thus to enable a much more attractive graphics implementation using CPUs with which MCU users are already familiar.

They have done this in one of two ways.

The offload method: STMicroelectronics in its STM32F and STM32L4 series of MCUs implements a hardware graphics unit called the Chrom-ARTTM accelerator. This implements common repetitive graphics processing functions such as blending, pixel-format conversion and anti-aliasing in a hardware block separate from the MCU’s Cortex-M7 or Cortex-M4 core. Chrom-ART also implements a 2D Direct Memory Access (DMA) to memory resources. In the STM32F7, as well as the STM32H7 series introduced in the second half of 2017, ST has also added a hardware JPEG codec, which can render still or moving pictures, in addition to the Chrom-ART graphics accelerator.

The effect is to offload the burden of graphics and image rendering from the CPU, leaving it free to implement the core application’s real-time processing functions.

In the PIC32MZ DA series, Microchip has also taken the offload approach. It calls its hardware block the 2D Graphics Processing Unit. It has a dedicated bus access to memory. It also performs programmable raster operations, offers fixed functions such as ‘line draw’, ‘rectangle fill’ and ‘rectangle clear’, and implements dithering, rotation, clipping and text rendering.

The pipeline method: NXP Semiconductors in its high-end LPC546xx series of MCUs features an SPI Flash Interface (SPIFI), a dedicated channel between the MCU and an external serial Flash device which allows the external memory to appear in the MCU’s memory map and to be read as though it were internal memory, as shown in Figure 2. Giving around 70% of the bandwidth that internal memory provides, the SPIFI accelerates the delivery of images and other graphics data to the CPU for faster execution. So, rather than offloading certain graphics processing tasks from the CPU, the NXP method shortens the periods in which the CPU is dedicated to graphics processing, leaving more time free for the CPU to run the core application.


Fig. 2: NXP’s SPI Flash Interface enables external Flash to appear in the MCU’s internal memory map

Practical experience shows that both these techniques produce good performance even in demanding graphics display applications.

Products such as the device shown in Figure 5 are good examples of what is possible when using an advanced MCU. Making a precise comparison of the graphics capabilities of different MCUs is, however, somewhat difficult.

The market for Graphics Processing Units (GPUs) supports standard performance benchmarks which allow users to compare the speed with which different chips perform identical graphics processing and video processing tasks. These benchmarks do have some value, particularly as a means to compare the relative performance of different models of GPU from a single manufacturer.

In the high-end MCU market, however, there is no benchmark to which designers can refer. GPU manufacturers address a limited range of applications in the fields of gaming and media playback, and the benchmarks, measuring parameters such as frame rate, are intended to represent the kind of functions that these devices typically implement.

But in the embedded devices that MCUs support, these parameters fail to helpfully reflect the performance of the processor in the functions most commonly performed in a graphics HMI.

Experienced graphics HMI designers will say that the only benchmark that really matters is whether the displayed content is good enough. If the design team first, and then the end-user, feel that the HMI is responsive and it functions appropriately for the application, then the MCU meets the requirements of the product. And by monitoring CPU usage in the application, the designer can ensure that the system has sufficient resources to run the application code while simultaneously presenting the data to the user.

To help designers choose an appropriate device, each MCU manufacturer provides general guidelines on the capabilities of their products. ST, for instance, maintains an STM32 Advanced GUI-Enabled MCU Portfolio, devices which feature the Chrom-ART accelerator and display interfaces. Parts which have a TFT-LCD controller interface, it suggests, support ‘mid-sized’ displays at up to XGA resolution. The STM32F469 and STM32F769, which also have a MIPI-DSI® controller, are for the latest displays with a higher pixel density.

The use of evaluation kits that the manufacturers provide for their graphics portfolios of MCUs provides the best way to evaluate the capabilities of the MCU, as shown in Figure 3: these are supplied with a display of a size and resolution that best illustrates the MCU’s capabilities.


Fig. 3: The LPCXpresso development board for LPC5460x MCUs from NXP includes application examples displayed on its 4.3”, 272 x 480px capacitive touchscreen

Drawing on these indications of what is possible with any given MCU part, the designer’s best method of performance evaluation is to develop a proof of concept on the MCU development board or on a custom hardware prototype, and to see whether the graphics performance in the intended application meets the requirements of the HMI.

Fortunately, this has become much easier to do in recent times thanks to the introduction of new embedded graphics design tools that work closely with many graphics-enabled MCUs.

Graphics design achieves high productivity
At the same time as MCU manufacturers have developed new MCUs with specific graphics hardware features, a new generation of tools has emerged which are optimised to take advantage of these features. The TouchGFX toolset offers a sleek user interface, a WYSIWYG canvas for development with a huge library of widgets, and support for the creation of custom graphics content and images, as shown in Figure 4. Intended for use with displays in sizes ranging from 1” to 10” and in resolutions up to XGA, the TouchGFX tool produces a C++ output that, when compiled, is highly optimised for the performance, Flash size and RAM constraints of graphics MCUs. The tightly optimised tool supports nine MCUs from NXP, and six from ST.


Fig. 4: The TouchGFX tool’s user interface provides the ability to drag and drop ready-made graphics widgets on to the design canvas

This is a premium tool that makes it easy for the user to quickly produce smartphone-quality displays, as shown in Figure 5. This is thanks to features such as:

• Advanced rendering algorithms – optimised visible surface determination algorithm and customised invalidation techniques minimise the number of drawn pixels.
• Creation of custom control graphics, enabling the user to extend or modify existing widgets or combine existing widgets with custom functionality.
• Creation of complex graphical objects: the user can draw lines, circles, custom shapes and graphics, or apply scaling and 3D rotation to images at runtime with memory-efficient algorithms.


Fig. 5: The UI for the OneBreath ventilator was created with the TouchGFX tool. The product runs on NXP’s LPC4357 MCU. (Image credit: OneBreath at www.onebreathventilators.com)

For first-time users, the tool’s GUI, called TouchGFX Designer, is intuitive, providing on one side easy access to the library of ready- made widgets and controls such as buttons and sliders, and on the other various editing tools to allow the user to quickly customise these graphic elements or create custom ones. The code generated from the TouchGFX designer then integrates into the software development tools of the MCU provider, making for a seamless development environment.

The emWin GUI builder from Segger provides a similar range of advanced graphics design features. Like TouchGFX, emWin provides a rich graphics library, a drag-and-drop canvas for quick and efficient UI building, and a wide range of customisation tools for the creation of original graphics elements.

The emWin tool supports a wider range of hardware targets. According to Segger, emWin supports any CPU, and has a low memory footprint of less than 60kbytes of ROM and RAM. MCUs from NXP, ST and Microchip all support emWin, and the tool is well integrated with these MCU manufacturers’ Integrated Development Environments (IDEs). Microchip also provides its own graphics design tool built into its MPLAB® Harmony IDE. The Graphics Composer tool offers a feature set which will be familiar to any HMI designer.

Setting new limits for MCU graphics performance
The limits to what an MCU is able to implement in a graphics display are mirrored in the capabilities of tools such as TouchGFX and emWin: if a capability is not provided in the tool, then it is beyond the performance limits of today’s MCUs.

These limits, however, are constantly being reset. The current benchmark for MCU graphics performance is probably set by the new STM32H7 parts from ST: because of the hardware JPEG accelerator, these MCUs extend the range of video streaming capabilities open to embedded designers without compromising the performance of the core application. In end-products such as security cameras and door- entry systems, for instance, the streaming video capability provided by the STM32H7 could dramatically enhance the value of today’s microcontroller-based designs. The roadmaps of manufacturers such as NXP, Microchip and others promise similar and other new capabilities.

Already today, design teams using an MCU can produce designs with a touchscreen display rivalling a smartphone for both attractiveness and usability. The MCU architecture is no longer an impediment to the implementation of a highly sophisticated UI, and with the promised introduction of improved graphics features in forthcoming MCUs, the HMIs of MCU-based embedded products are set to rapidly get better and better.