Over the last few months I’ve been lucky to work with two very talented people on an interesting project for multi-camera machine vision applications: Gianluca Filippini and Mario Bergeron. Back in 2022, I was contacted by Gianluca, an engineer from EBV Elektronik. He was using our FPGA Drive FMC, an adapter for connecting Solid-state drives to FPGAs - but he wasn’t interested in SSDs. He wanted to use it for a new and exciting AI accelerator from a company called Hailo. The idea was to take the Zynq UltraScale+, a powerful MPSoC geared for video applications, and arm it with a top performing AI accelerator to run intelligent vision algorithms on the edge.

When working with PetaLinux tools, we need to be wary of the often very sizeable temporary files generated during builds. These temporary files can often take up more than 10GB, and they are crucial for the build process.

By default, PetaLinux tools create a temporary folder internal to the project directory. Having it set this way is useful because our projects are well contained and we have clear insight into the total disk space used by a project. However, it’s not uncommon to encounter configurations where the temporary folder is directed to /tmp. I just came across an example of this in the Vitis Video Analytics SDK repo.

In certain scenarios, you might want to build PetaLinux without relying on an active internet connection. This offline building feature can be useful for various reasons. To build PetaLinux without a network connection, you obviously need pre-downloaded source packages and dependencies. Xilinx allows you to pre-download all of these dependencies (called sstate cache artefacts) and then perform the build offline.

Why You Would Use It

The reason to use offline mode really depends on your situation and it’s definitely not something that everyone will get benefit from. However there are some reasons why you might want to consider it:

When designing custom PetaLinux builds, we often end up with a bunch of commands that we have to run after login to set things up and make initializations. Having a script built into the root filesystem can make this process easier, so in this post I’m going to show you how you can set that up in PetaLinux 2022.1.

These instructions assume that you already have a PetaLinux project created.

In this post we’re going to build a smart camera using a Raspberry Pi camera, the ZCU104 board and PetaLinux. We’re going to do this by leveraging the Smartcam app which was originally designed for the AMD Xilinx Kria SoM. Our version of the Smartcam app can take video inputs from a Raspberry Pi camera, a USB camera or a file, and it can output video to a DisplayPort monitor, a file or via Ethernet over Real-time Transport Protocol. The application can run three different AI models:

Many smart vision applications need to make fast decisions: autonomous vehicles, drones, surveillance and industrial robotics are only a few examples. When developing these kinds of AI vision systems, understanding performance-affecting factors is critical.

In this post, we’ll explore two such factors: the operating system and camera type. We’ll measure and compare the performance of the NLP-SmartVision app on the ZCU104 board. The setups we’ll use are:

• Operating system:
  • Docker container on Certified Ubuntu 22.04 LTS for Xilinx devices
  • Certified Ubuntu 22.04 LTS for Xilinx devices
  • PetaLinux
• Camera type:
  • Raspberry Pi camera v2 (2-lane MIPI, 1024x768 resolution 30fps)
  • Logitech USB Camera C930e (USB, 800x600 resolution 24fps)

Defining performance

To assess performance, we’ll measure two aspects: frame rate and glass-to-glass latency. For the frame rate, we are going to use on the application’s own ability to measure frame rate. For the glass-to-glass latency, we are going to use a method that involves placing an LED in front of the camera and a detector in front of the monitor. The latency is measured as the time lag between the input and output signals of this system.

In the last post we looked at how to run the Smartcam and NLP-SmartVision apps on the ZCU106 and Certified Ubuntu 22.04 LTS. One reader mentioned that running these apps in a Docker container on Ubuntu probably comes with a performance penalty when compared to running it on a lean PetaLinux build. This piqued my curiosity so in this post, we’re going to get the NLP-Smartvision app running in PetaLinux on the ZCU104 and then in the next post we’ll measure whatever penalty there may be to the throughput (in frames per second) and/or the glass-to-glass latency (in milliseconds).

If you want to develop smart vision applications using Raspberry Pi cameras, the Zynq UltraScale+ MPSoC and Ubuntu 22.04 LTS, we’ve just released some code and the hardware to help you get started. For the launch of our new RPi Camera FMC, we’ve ported the Kria Vitis Platforms and Overlays over to the ZCU106 armed with an RPi Camera FMC. The result is that you can now build and develop the Smartcam and NLP-Smartvision applications in Certified Ubuntu 22.04 LTS for the ZCU106 with Raspberry Pi cameras connected via the FMC connector.

This post is a continuation of the previous post where we setup our ZCU106 board for Ubuntu 20.04 LTS. In this post we’re going to build the hardware platform that is built into the Certified Ubuntu 20.04 LTS images for the ZCU106 board. The reason that we would want to be able to do this is so that we can customize the hardware platform - perhaps add functionality, add external connections, or add accelerator IP. In this post we wont actually modify the design, we’ll just build it and then verify that it works on the hardware. In a later post we will actually modify the platforms. My objective at that time will be to add support for the RPi Camera FMC.

In this post we’re going to download the Certified Ubuntu 20.04 LTS image for the ZCU106 board and get it running on the hardware. This is a fairly simple post but I’ve got to write it up in preparation for future posts where we will dig deeper into this exciting new way of developing applications for Zynq UltraScale+ devices.

Here’s what you’ll need:

• ZCU106 Zynq UltraScale+ Development board
• SanDisk 32GB microSD card (or one with a fairly good capacity)
• USB keyboard and mouse connected to the ZCU106
• USB webcam (if you want to try a video application at the end)
• USB hub (since ZCU106 has only one USB connector)
• DisplayPort cable connecting ZCU106 to a monitor
• Ethernet cable connecting to your network router

Burn the ZCU10x Ubuntu image to an SD card

Start by downloading Ubuntu 20.04 LTS for ZCU10x. You will see that there is already a more recent version available (22.04), but in this post we will use the older version (20.04). At the time of writing, the newer version doesn’t yet have support for some of the interesting applications that we would like to test on it such as the Vitis AI Library Samples snap.