Raspberry Pi 4B: Natively Built 64 Bit Fully Preemptible Kernel (Real-Time) Gets Overridden
Running PnCconf on the natively built 64-bit Fully Preemptible Kernel (Real-Time) kernel, as previously discussed, resulted in the following message: You are using a simulated-realtime version of LinuxCNC, so testing / tuning of hardware is unavailable. In this post, we will examine what has happened, prove that the finding is correct, and present the results of the jitter or latency test we conducted on the Raspberry Pi OS.
🐧 Index of the Complete Series.
 |
|---|
| Raspberry Pi 4B: Natively Built 64 Bit Fully Preemptible Kernel (Real-Time) Gets Overridden |
Accepting the Raspberry Pi OS prompt to update the system results in my natively built 64-bit Fully Preemptible Kernel (Real-Time) being removed and the original kernel reinstalled! 🤬
❶ Toward the end of October 2024, I finished building a 64-bit Fully Preemptible
Kernel (Real-Time) for use with LinuxCNC,
as documented. I received the
7I96S STEP/IO Step & dir plus I/O card on 22/01/2025.
On 05/02/2025, I was able to run the Mesa Configuration Wizard,
PnCconf.
When clicking on the Test / Tune Axis, I got the above message as illustrated in the following screenshot:
I actually built LinuxCNC from source and installed the locally built package. I followed the instructions to the letter but thought my build had issues. Unsure of what had happened, I sought help from the forum. I was advised to use the official image: https://linuxcnc.org/iso/rpi-4-debian-bookworm-6.12.11-arm64-ext4-2025-01-27-0404.img.xz. I heeded the advice, as seen in previous posts.
I still had another SD card with the real-time kernel and LinuxCNC mentioned above. In an effort to understand the issue, on 17th February 2025, I removed my original build and installation of LinuxCNC 2.9.3, downloaded, and installed the linuxcnc-uspace_2.9.4_arm64.deb package. The commands to remove and install are:
$ sudo apt remove linuxcnc-uspace
$ sudo dpkg -i /home/behai/Public/linuxcnc-uspace_2.9.4_arm64.deb
This time, I was able to run
PnCconf successfully, as
mentioned in previous posts. I was convinced that my LinuxCNC 2.9.3 build was
the problem 😂. I shut down PnCconf to run the latency test.
Raspberry Pi OS notified me of pending updates, which I accepted. Then I started the
HAL Latency Test application. With only three instances of
glxgears running, the Max Jitter for the Base Thread
reached nearly 500,000 nanoseconds!
I was quite taken aback. The previous latency
test results
were nowhere near this bad: the tests were conducted on a different OS, but the hardware was
the same. When I looked at the Kernel-version field on the HAL Latency
Test screen, it showed PREEMPT not PREEMPT_RT.
💥 That was when I realised that accepting the Raspberry Pi OS prompt to update the system led to the custom-built 64-bit Fully Preemptible Kernel (Real-Time) being removed and the original kernel reinstalled!
Just to be doubly certain, I ran PnCconf again and expected the above message to appear — and it did. I accepted the prompted update on the first SD card too.
❷ Three months have passed since November 2024, and it’s time for a new updated real-time kernel.
Following the
first post, I successfully built a real-time kernel version
6.6.77-rt50:
With this version, we need to install ifupdown, which is a set of high-level scripts to configure network interfaces. The command to install it is:
$ sudo apt install ifupdown
Before we can install the downloaded LinuxCNC 2.9.4 linuxcnc-uspace_2.9.4_arm64.deb package, we need to install the missing packages. The command is:
$ sudo apt --fix-broken install
Then, finally:
$ sudo dpkg -i /home/behai/Public/linuxcnc-uspace_2.9.4_arm64.deb
PnCconf now runs, and LinuxCNC also “runs”. I am able to move along the X-axis, and the single stepper motor (please refer to this post) also turns. Please see the screenshot below:
❸ We will not describe the test sequence in detail. Briefly, the test ran for more than an hour. With 12 instances of glxgears, the Max Jitter for the two threads remained constant at:
- Servo Thread: 98,571 nanoseconds (98.571 microseconds)
- Base Thread: 100,776 nanoseconds (100.776 microseconds)
The raw recording of the test is included below:
The environment:
FileManager
Terminal Window
Text Editor
Chromium opened with Google tab
USB stick plugged-in
15:26 -- trying to capture screen, moving windows etc.
15:32
Servo-Thread: 54,080 ns
Base-Thread: 56,944 ns
15:34: applied cooling.
15:35: Change to YouTube run a 1:20:00 clip.
15:37:
Servo-Thread: 365,375 ns
Base-Thread: 610,162 ns
15:40 -- Shutdown Chromium.
15:41:
Servo-Thread: 369,687 ns
Base-Thread: 610,162 ns
15:42: Reset Statistics
15:42 -- 15:46
Servo-Thread: 31,981 ns
Base-Thread: 57,369 ns
15:46: run 3 glxgears in succession.
Servo-Thread: 67,487 ns
Base-Thread: 60,221 ns
15:49: run Chromium single Google tab search for something.
15:53:
Servo-Thread: 93,932 ns
Base-Thread: 100,776 ns
15:55:
run another 3 glxgears in succession. 6 instances now.
Google something else.
16:04: Archive /home/behai/linux: 2.7GB
On the first terminal window, under /home/behai, run:
$ tar -zcvf 6-6-77-rt50-v8-behai-rt-build-home-behai-linux.tar.gz /home/behai/linux
-rw-r--r-- 1 behai behai 748173752 Feb 18 16:08 6-6-77-rt50-v8-behai-rt-build-home-behai-linux.tar.gz
16:12:
Servo-Thread: 98,571 ns
Base-Thread: 100,776 ns
16:14:
run another 3 glxgears in succession. 9 instances now.
16:22
Servo-Thread: 98,571 ns
Base-Thread: 100,776 ns
16:23:
run another 3 glxgears in succession. 12 instances now.
16:36
Servo-Thread: 98,571 ns
Base-Thread: 100,776 ns
16:37: using a preinstalled UI app to open 6-6-77-rt50-v8-behai-rt-build-home-behai-linux.tar.gz.
16:44: finished browsing the above *.gz file.
16:45: using File Manager moved 6-6-77-rt50-v8-behai-rt-build-home-behai-linux.tar.gz to /home/behai/Public/.
16:46
Servo-Thread: 98,571 ns
Base-Thread: 100,776 ns
16:51: ran behai@picnc:~ $ rm -rf linux/
16:52
Servo-Thread: 98,571 ns
Base-Thread: 100,776 ns
16:53: shutdown 3 instances (first) glxgears. 9 remain.
16:55: shutdown 3 instances (first) glxgears. 6 remain.
16:57: shutdown 3 instances (first) glxgears. 3 remain.
16:59: copied around 740 MB to the USB stick using File Manager.
17:01: shutdown the last 3 glxgears instances.
17:02
Servo-Thread: 98,571 ns
Base-Thread: 100,776 ns
17:03: went to YouTube using Chromium:
Servo-Thread: 98,571 ns
Base-Thread: 140,405 ns
17:04: played a 1:36:00 clip.
17:05:
Servo-Thread: 281,238 ns
Base-Thread: 375,605 ns
17:05:
Servo-Thread: 355,255 ns
Base-Thread: 453,918 ns
17:06: Reset Statistics.
Servo-Thread: 470,420 ns
Base-Thread: 498,306 ns
The screenshot below illustrates a result of the test:
❹ I had no plan to write this post, but what I have found is rather unexpected and quite interesting. Hopefully, this post will be helpful to someone in the future.
If you happen to read this post, thank you for your time, and I hope I did not waste it. Stay safe, as always.
✿✿✿
Feature image source: