Phil's Occasional Blog

Converting CentOS 8 Stream to AlmaLinux 8.4

There's no better time to migrate CentOS 8 Stream to AlmaLinux 8.4. Because RedHat Enterprise Linux 8.4 has only just been released, the differences between Stream and 8.4 are minor.

First off, make sure your box is up to date:

dnf distro-sync

Then back it up or snapshot it. Some packages may revert to older versions and break things.

Then download the AlamLinux conversion script

curl -O https://raw.githubusercontent.com/AlmaLinux/almalinux-deploy/master/almalinux-deploy.sh

edit almalinux-deploy.sh

MINIMAL_SUPPORTED_VERSION='8'

In REMOVE_PKGS add "centos-stream-release" "centos-stream-repos"

Amend the grep command in get_os_version:

        if ! os_version="$(grep -oP 'CentOS\s+(Linux|Stream)\s+release\s+\K(\d+(\.\d+)?)' \

                                    "${REDHAT_RELEASE_PATH}" 2>/dev/null)"; then

Or alternatively, download my modified install script from my repo:

curl -O https://raw.githubusercontent.com/philrandal/almalinux-deploy/master/almalinux-deploy.sh

Then you're set to go. The script works with CentOS 8.3 and CentOS 8 Stream.

bash almalinux-deploy.sh

and if it succeeds

reboot

After the reboot, check for orphaned packages: I got this from migrating a full CentOS 8 Stream install to AlmaLinux 8.4

dnf list extras
Extra Packages

llvm-compat-libs.x86_64 10.0.1-1.module_el8.4.0+533+50191577

They can safely be removed.

When you reboot, you'll see that the original CentOS rescue kernel is still there.

Fix that with:

rm -f /boot/vmlinuz-0-rescue-* /boot/initramfs-0-rescue-*.img
/usr/lib/kernel/install.d/51-dracut-rescue.install add $(uname -r) "" /lib/modules/$(uname -r)/vmlinuz

To remove old CentOS kernels,

rpm -qa | grep kernel-core

Then use yum remove to get rid of any superfluous ones.

Test, test, and test again.

Remove any snapshots once you're happy.

Postscript:

So you just want to convert back to CentOS 8.x from CentOS Stream?

No worries (except for the caveat that downgrades may break things).

curl -O http://mirror.centos.org/centos-8/8.3.2011/BaseOS/x86_64/os/Packages/centos-linux-release-8.3-1.2011.el8.noarch.rpm
curl -O http://mirror.centos.org/centos-8/8.3.2011/BaseOS/x86_64/os/Packages/centos-linux-repos-8-2.el8.noarch.rpm

rpm -e --nodeps centos-stream-repos centos-stream-release

rpm -ivh centos-linux-release-8.3-1.2011.el8.noarch.rpm centos-linux-repos-8-2.el8.noarch.rpm

dnf distro-sync

reboot

Posted by Phil at 5:28 PM
Edited on: Wednesday, June 09, 2021 12:55 PM
Categories: IT, Software

Lenovo IdeaPad S340-15API

I've just purchased a Lenovo IdeaPad S340-15API From Argos to replace our aging and very slow Acer Aspire E5-511 laptop.

The spec is good. For £550 you get a 15.6 inch Full HD (1920 x 1080) display, AMD Ryzen 7 3700U quad-core 2.3GHz processor (with Simultaneous Multi-Threading giving 8 virtual cores) with Radeon Vega Mobile graphics, 8GB of RAM, of which 2GB is taken as display RAM, and 512GB SSD (WDC PC SN520 SDAPMUW-512G-1101).

Windows 10 1809 Home pre-installed (in S mode), but it is a simple fix to switch to full Windows 10 Home.

It didn't offer me Windows 10 1903, so I went and fetched it using the Microsoft Update Assistant.

It took a while to get that on, and the successive updates.

Lenovo Vantage offered offered a few driver updates, but I went to the Lenovo Support website and manually downloaded the latest BIOS and assorted drivers.

Power settings are never right by default, as I discovered the hard way. I could boot once, close the lid to put it into sleep mode, open it again to wake up, but the second time I closed the lid it would not behave and effectively locked up.

Powering off and on brought it back to life.

Eventually, after running powercfg /a I discovered a workaround:

C:\Windows\System32>powercfg /a

The following sleep states are available on this system:

Standby (S3)

Hibernate

Hybrid Sleep

Fast Startup

The following sleep states are not available on this system:

Standby (S1)

The system firmware does not support this standby state.

Standby (S2)

The system firmware does not support this standby state.

Standby (S0 Low Power Idle)

The system firmware does not support this standby state.

In the Power Options page in Control Panel, Change Plan Settings, then Change advanced power settings

In the Sleep section of the Advanced settings, turn Allow hybrid sleep on for both On battery and Plugged in

Then it all works as expected.

Postscript, September 28, 2019

The cause of the Sleep problem was the Realtek Audio Driver, and the fix was found on the Lenovo Support Forum here.

If you're not averse to extracting .cab files and manually updating drivers, give it a go.

Posted by Phil at 5:25 PM
Edited on: Saturday, September 28, 2019 8:32 PM
Categories: IT

About to update to RedHat Enterprise Linux or CentOS 7.7? Beware!

There's a couple of caveats before upgrading to RHEL / CentOS 7.7.

1: Upgrading to latest RHEL 7 PCP package pcp-4.3.2-2.el7 gives several issues.

During installation/upgrade of pcp-selinux:

    Updating / installing...
    1:pcp-selinux-4.3.2-2.el7 ################################# [100%]
    Failed to resolve allow statement at /etc/selinux/targeted/tmp/modules/400/pcpupstream/cil:83
    semodule: Failed!

Followed by lots of selinux errors logged.

Workaround is to install 7.7's selinux-policy first

    yum update selinux-policy

or, if you've already updated,

    yum reinstall pcp-selinux

Documented in RedHat Bugzilla entry 1714101

2: bind is updated from version 9.9 to 9.11. This breaks configurations in which the same zone file is defined in several views.

The solution is to use bind 9.11's "in-view xxx" in subsequent definitions of the same zone in different views.

The fix is described in detail here.

Posted by Phil at 7:32 PM
Edited on: Saturday, September 14, 2019 3:47 PM
Categories: IT, Software

Using the Pimoroni Fan Shim with LibreElec

Trying out the latest LibreElec alpha on a Raspberry Pi 4 on the hottest day of the year so far I found the CPU temperature reaching 70ºC. A bit too warm for my liking, so I've added a Pimoroni Fan Shim.

There's no way you can install the Pimoroni python library on LibreElec, but there is an alternative.

First, you need to install the Raspberry Pi Tools addon in LibreElec.

Addons / install from repository / libreelec add-ons / program add-ons / Raspberry Pi Tools.

I found a script on https://forum-raspberrypi.de/forum/thread/43568-fan-shim-steuern/, and edited it slightly. Change the min and max temperatures to suit yourself.

ssh into your LibreElec box as root and

nano /storage/fanshim.py

#!/usr/bin/env python
# https://forum-raspberrypi.de/forum/thread/43568-fan-shim-steuern/
# place command below in /storage/.config/autostart.sh
#   nohup /storage/fanshim.py &
import os
import time
import signal
import sys
sys.path.append('/storage/.kodi/addons/virtual.rpi-tools/lib')
import RPi.GPIO as GPIO
import subprocess
Pause = 30
CoreTempMax = 57
CoreTempMin = 46
GPIO_Pin = 18
Run_Fan_function = False
def init():
    GPIO.setwarnings(False)
    GPIO.setmode(GPIO.BCM)
    GPIO.setup(GPIO_Pin, GPIO.OUT)
    return()
def Set_Fan_ON():
    GPIO.output(GPIO_Pin, True)
    return()
    
def Set_Fan_OFF():
    GPIO.output(GPIO_Pin, False)
    return()
def get_CPU_Temp():
    temp = subprocess.check_output(['vcgencmd', 'measure_temp'])[5:-3]
    return temp
    
def Watch_Temp():
    global Run_Fan_function
    CPU_Temp = float(get_CPU_Temp())
    if Run_Fan_function==False and CPU_Temp>=CoreTempMax:
        Run_Fan_function = True
        Set_Fan_ON()
    if Run_Fan_function==True and CPU_Temp<=CoreTempMin:
        Run_Fan_function = False
        Set_Fan_OFF()
    return();
try:
    init() 
    while True:
        Watch_Temp()
        time.sleep(Pause)
except KeyboardInterrupt:
    GPIO.cleanup()

Save the edited file, and then:

chmod +x /storage/fanshim.py

Then edit LibreElec's autostart.sh

nano /storage/.config/autostart.sh so it contains the line:

nohup /storage/fanshim.py &

Reboot your Pi 4 for it to take effect.

If you're using a TV Hat, you'll need an extra tall stacking header to give a good gap between the fan and the TV Hat.

Enjoy.

Postscript, October 26, 2019

Someone has done a bit of work on this and turned it into a proper installable Kodi addon:

https://github.com/jane-t/rpi-fanshim

Thanks!

Posted by Phil at 10:06 PM
Edited on: Saturday, October 26, 2019 4:27 PM
Categories: IT, Raspberry Pi

A Raspberry Pi Stratum 1 NTP Server

Overview

Updated for the Pi 5 and Raspberry Pi OS Bookworm

The following instructions are how to make a cheap Pulse Per Second (PPS) disciplined Stratum 1 NTP Time server using one of the Raspberry Pi U-blox M8Q based GPS boards sold by Uputronics.

Our basic requirement is for an NTP server which will work standalone without connectivity to other NTP servers, so if a datacentre loses connectivity its servers will still have a stable time source.

As we’re only interested in time and not height, motion, or location, all we’re interested in is the NMEA xxZDA and xxRMC sentences and the highly accurate PPS signal from the GPS module, both of which we feed straight into the ntp daemon. The xxZDA sentences (which give us full 4-digit years) are not output by default by the M8Q so we have to enable them at boot time.

Many guides on the net use gpsd to feed ntp via shared memory. That’s an additional overhead and complexity, especially if we want to enable Galileo reception (gpsd initialises the U-blox the way it sees fit). I prefer to let the NTP daemon read the NMEA stream from the GPS receiver itself, avoiding the middle man.

In this guide we’ve chosen to use NMEA output from the GPS module via the ntpsec Generic GPS Receiver driver 20 ( https://docs.ntpsec.org/latest/driver_nmea.html ) and the PPS driver ( https://docs.ntpsec.org/latest/driver_pps.html ).

You will need a Raspberry Pi 4B, the Uputronics Raspberry Pi+ GPS Expansion Board and a suitable GPS antenna.

This guide assumes that you’re using Raspberry Pi OS Bookworm, 64-bit Lite edition. Download and write this to an SD card.

See https://www.raspberrypi.org/documentation/installation/installing-images/windows.md

You'll need a keyboard and display attached for first boot configuration.

Create a user (mine was ntpadmin, use whatever you like) and set its password.

Attach the Uputronics Raspberry Pi+ GPS Expansion Board to the Pi, insert the SD card, connect the antenna and network cable and boot the Pi up. Either connect locally or via SSH to the Pi. If you can’t SSH in and don’t have a monitor see this

https://www.raspberrypi.org/documentation/configuration/wireless/headless.md

Follow the instructions carefully if you miss steps things won’t work.

The Uputronics board has u-blox firmware 3.01 on it, dated 2016.

The week number rollover is set to 1867 (October 2015). All transmitted week numbers are mapped to the ~19.5 year period between week 1867 and week 2990 (April 2035).

A Note About Accuracy

In theory, GPS-based time receivers can give a very high accuracy, with the PPS (Pulse per Second) signal being accurate to within 10ns.

However, fix data from gpsctl shows:

pi@ntp2:~ $ /usr/local/bin/gpsctl -Q fix

Time (UTC): 2019-04-18 15:31:27 (yyyy-mm-dd hh:mm:ss)

Latitude: 52.05994980 N

Longitude: 2.72698960 W

Altitude: 198.789 feet

Motion: 0.338 mph at 53.114 degrees heading

Satellites: 5 used for computing this fix

Accuracy: time (39 ns), height (+/-18.199 feet), position (+/-103.911 feet), heading(+/-8.230 degrees), speed(+/-0.132 mph)

Note the 39ns time accuracy from that fix. The more satellites, the better.

Errors are also caused by signal delay (4ns per foot) in the cable from the aerial to the GPS receiver, etc.

So we’d be lucky to get 100ns accuracy from the PPS pulse.

Add to that the processing overheads of the PPS interrupt, and processor clock jitter in the Raspberry Pi, and overheads transferring and decoding the NMEA sentences. The jitter on the PPS signal is less than 5 microseconds.

The output from the ntp NMEA driver without PPS correction can have a jitter of a few milliseconds, and an offset from real time which has to be tweaked manually to get within 5ms of the correct time.

NTP sources on the internet can show offsets / jitter of over 5ms, local LAN 50us or more.

On the Pi 4B we can expect an average jitter of less than 1 microsecond.

Required Components

I sourced my components from the Pi Hut unless noted elsewhere:

1 Raspberry Pi Model 4B 1GB RAM

1 8GB or larger micro SD card

(a Transcend High Endurance 32GB micro SD card would be better choice than a generic one for longevity)

1 Uputronics GPS Hat

1 Pi Hut Pi 4 GPS case

1 Raspberry Pi 4 power supply

1 GPS SMA Antenna

(optional) 1 SMA-male to TNC-female (or BNC, as needed) adaptor to connect to existing GPS aerial (from Amazon or eBay)

Prerequisite Settings

Login as pi / raspberry, and immediately change the password from the default

passwd

sudo raspi-config

Advanced Options

Expand filesystem (this is automatically done on first boot)

Interfacing Options

SSH -> Would you like the SSH server to be enabled – Yes (Recommended)

I2C -> Would you like the ARM I2C interface to be enabled? - Yes

Serial -> Login Shell (no) Hardware (yes) (Optional)

Quit but no need to reboot at this point.

for i in systemd-timesyncd avahi-daemon alsa-state bluetooth triggerhappy hciuart rng-tools autologin getty;

do

sudo systemctl disable $i.service; sudo systemctl stop $i.service;

done

sudo systemctl disable serial-getty@ttyAMA0.service

sudo systemctl mask serial-getty@ttyAMA0.service

sudo apt install gpsd-clients pps-tools ntpsec ntpsec-ntpviz i2c-tools linux-cpupower mlocate

sudo systemctl enable ntpsec

sudo apt purge bluez bluez-firmware modemmanager wpasupplicant (assuming no wifi required)

sudo apt update

sudo apt full-upgrade

If you want your Pi to run at full clock speed (active cooling is recommended),

sudo nano /etc/systemd/system/cpu-governor.service

[Unit]
Description=CPU Governor Service
After=raspi-config.service
[Service] 
Type=oneshot
ExecStart=/usr/bin/cpupower frequency-set -g performance
[Install] 
WantedBy=multi-user.target

Then

sudo systemctl enable cpu-governor.service --now

sudo nano /boot/firmware/config.txt and add at the bottom :

dtoverlay=disable-bt
dtoverlay=disable-wifi
dtoverlay=pps-gpio,gpiopin=18
dtparam=eee=off
dtparam=audio=off

On a Pi 4 or below, make sure that enable_uart=1 is set in /boot/config.txt

On a Pi 5, make sure that dtparam=uart0=on is set in boot/config.txt

sudo nano /boot/firmware/cmdline.txt and add at the end

nohz=off

sudo nano /etc/modules and add at the bottom :

pps-gpio

Save and quit nano, then sudo reboot

Enabling Galileo Satellites and Setting Stationary Mode

Stationary mode gives a faster GPS fix.

See Tom Dilatush’s gpsctl:

http://www.jamulblog.com/2017/11/paradise-ponders-gpsctl-functionally.html

I’ve customised it further to enable the use of a config file and put it in my GitHub repo:

https://github.com/philrandal/gpsctl

Download and build gpsctl.

cd ~
wget https://github.com/philrandal/gpsctl/archive/master.zip
unzip master.zip
mv gpsctl-master gpsctl
cd gpsctl
./build.sh
sudo cp ~/gpsctl/gpsctl /usr/local/bin
sudo cp ~/gpsctl/etc/gpsctl.conf /etc/gpsctl.conf

Individual settings can be tweaked in /etc/gpsctl.conf.

To set port speed to 115200 baud, enable Galileo satellites, set stationary mode, tweak antenna delays, PPS timing, etc:

/usr/local/bin/gpsctl -a -B 115200 --configure_for_timing -vv

To reset the device to its defaults

/usr/local/bin/gpsctl -a --reset -vv

To view info:

/usr/local/bin/gpsctl -a -Q satellites

/usr/local/bin/gpsctl -a -Q config

/usr/local/bin/gpsctl -a -Q fix

Note that these commands can only be run when gpsd / ntp are not using /dev/ttyAMA0

Example /etc/gpsctl.conf which configures the U-blox M8Q for this environment:

    
#
# example gpsctl.conf which enables Galileo as in --galileo parameter
#
[gpsctl]
port = /dev/ttyAMA0
# sync method: ASCII = 1, NMEA = 2, UBX = 3
sync method = 3
verbosity = 0
[NMEA]
enabled = true
version = 41
GGA = off
GLL = off
GSA = off
GSV = off
RMC = on
VTG = off
GRS = off
GST = off
ZDA = on
[GPS]
enabled = yes
minimum channels=8
maximum channels=16
[SBAS]
enabled = no
minimum channels=1
maximum channels=3
[Galileo]
enabled = yes
minimum channels=4
maximum channels=8
[Beidou]enabled = no
minimum channels=8
maximum channels=16
[IMES]
enabled = no
minimum channels=0
maximum channels=8
[QZSS]
enabled = no
minimum channels=0
maximum channels=3
[GLONASS]
enabled = yes
minimum channels=8
maximum channels=14
[Navigation Engine]
# Dynamic model: Portable = 0, Stationary = 2, Pedestrian = 3, Automotive = 4,
# Sea = 5, Air1G = 6, Air2G = 7, Air4G = 8, Watch = 9
Dynamic model = 2
# Fix mode: 2D only = 1, 3D only = 2, auto 2D/3D = 3
Fix mode = 3
Fixed altitude (2D) = 0.00 meters
Fixed altitude variance (2D) = 1.0000 meters^2
Minimum elevation = 5 degrees
Position DoP mask = 10.0
Time DoP mask = 10.0
Position accuracy mask = 100 meters
Time accuracy mask = 300 meters
Static hold threshold = 0 cm/s
Dynamic GNSS timeout = 60 seconds
Threshold above C/No = 0 satellites
C/No threshold = 0 dBHz
Static hold max distance = 0 meters
# UTC Standard: AutoUTC = 0, USNO_UTC = 3, GLONASS_UTC = 6, BeiDou_UTC = 7
UTC standard = 3
[Time Pulse]
# the nanoseconds / microseconds after the numbers are just reminders,
# they don't mean anything to the config parserAntenna cable delay = 56 nanoseconds
RF group delay = 20 ns
Unlocked pulse period = 1000000 microseconds
Unlocked pulse length = 0
Locked pulse period = 1000000 microseconds
Locked pulse length = 500000 microseconds
User configurable delay = 0

To run gpsctl at system startup,

sudo cp ~/gpsctl/etc/systemd/system/ublox-init.service /etc/systemd/system/ublox-init.service

or

nano /etc/systemd/system/ublox-init.service

and add the following contents

[Unit]
Description=u-blox initialisation
Before=gpsd.service
Before=ntp.service
[Service]
Type=oneshot
Environment="PARAMS=-q -a -B 115200 --configure_for_timing"
EnvironmentFile=-/etc/default/gpsctl
ExecStart=/bin/bash '/usr/local/bin/gpsctl ${PARAMS}'
[Install]
WantedBy=multi-user.target

This will enable Galileo satellites, set stationary mode, configure comms at 115200 baud, and restrict NMEA output to RMC and ZDA records.

To use the default baud rate of 9600, create /etc/default/gpsctl with the contents

PARAMS=-q --configure-for-timing

Then

sudo systemctl enable ublox-init.service

sudo systemctl daemon-reload

Verifying that PPS Is Working

Ensure the GPS has a lock and the Green PPS LED on the Uputronics Pi+ GPS Expansion Board is blinking once a second.

lsmod | grep pps

Output should be similar to :

pps_gpio                3089  1
pps_core                8606  4 pps_gpio

dmesg | grep pps

Output should be similar to :

[    2.735586] pps_core: LinuxPPS API ver. 1 registered
[ 2.738121] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[ 2.763842] pps pps0: new PPS source pps@12.-1
[ 2.766361] pps pps0: Registered IRQ 169 as PPS source

This indicates that the PPS Module is loaded.

sudo ppstest /dev/pps0

Output should be similar to:

trying PPS source "/dev/pps0"
found PPS source "/dev/pps0"
ok, found 1 source(s), now start fetching data...
source 0 - assert 1418933982.998042450, sequence: 970 - clear  0.000000000, sequence: 0
source 0 - assert 1418933983.998045441, sequence: 971 - clear  0.000000000, sequence: 0

(Press CTRL+C to quit). This indicates that the PPS Module is working.

You need to pick a few local NTP servers to use. The easiest way to do this is pick your region:

https://support.ntp.org/bin/view/Servers/NTPPoolServers

Select your region then you get a list of the country servers. E.g for the UK its uk.pool.ntp.org.

Type:

dig uk.pool.ntp.org

You will get four IP’s back:

;; &lt;&lt;&gt;&gt; DiG 9.10.3-P4-Raspbian &lt;&lt;&gt;&gt; +answer uk.pool.ntp.org
;; global options: +cmd
;; Got answer:
;; -&gt;&gt;HEADER&lt;&lt;- opcode: QUERY, status: NOERROR, id: 51647
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
;; EDNS: version: 0, flags:; udp: 4000
;; QUESTION SECTION:
;uk.pool.ntp.org. IN A
;; ANSWER SECTION:
uk.pool.ntp.org. 24 IN A 193.150.34.2
uk.pool.ntp.org. 24 IN A 176.58.109.199
uk.pool.ntp.org. 24 IN A 195.195.221.100
uk.pool.ntp.org. 24 IN A 185.53.93.157

sudo nano /etc/ntpsec/ntp.conf

Amend the start of ntp.conf to look like this

## nts enable

# You must create /var/log/ntpsec (owned by ntpsec:ntpsec) to enable logging.
#statsdir /var/log/ntpsec/
#statistics loopstats peerstats clockstats
#filegen loopstats file loopstats type day enable
#filegen peerstats file peerstats type day enable
#filegen clockstats file clockstats type day enable

# This should be maxclock 7, but the pool entries count towards maxclock.
#tos maxclock 11

# Comment this out if you have a refclock and want it to be able to discipline
# the clock by itself (e.g. if the system is not connected to the network).
#tos minclock 4 minsane 3

# Specify one or more NTP servers.

# Public NTP servers supporting Network Time Security:
# server time.cloudflare.com nts

#--------------------------
# PPS Driver
refclock pps unit 0 minpoll 4 maxpoll 4 ppspath /dev/pps0
#flag3 Controls the kernel PPS discipline: 0 for disable (default), 1 for enable.
#time1 PPS time offset
# use local clock at stratum 10 if no others available
tos orphan 10
tos mindist 0.002
# NMEA driver (/dev/gps0 and /dev/gpspps0)
# process $GPZDA only (mode 8)
refclock nmea prefer minpoll 4 maxpoll 4 mode 8 stratum 2 baud 115200 time2 0.060 path /dev/ttyAMA0
#time2 NMEA time offset
#mode bit 0 - process $GPMRC (value = 1)
# bit 1 - process $GPGGA (value = 2)
# bit 2 - process $GPGLL (value = 4)
# bit 4 - process $GPZDA or $GPZDG (value = 8)
# mode 9 = process only xxRMC and xxZDA NMEA records
#---------------------------
# pool.ntp.org maps to about 1000 low-stratum NTP servers.  Your server will
# pick a different set every time it starts up.  Please consider joining the
# pool: <https://www.pool.ntp.org/join.html>
server 0.uk.pool.ntp.org iburst
server 1.uk.pool.ntp.org iburst
server 2.uk.pool.ntp.org iburst
#pool 0.debian.pool.ntp.org iburst
#pool 1.debian.pool.ntp.org iburst
#pool 2.debian.pool.ntp.org iburst
#pool 3.debian.pool.ntp.org iburst

(Note that I used “time2 0.060 …” to adjust the GPS time according to the other NTP servers. You can try some higher/lower values to have the offset of your NMEA driver compared to the offsets of those other NTP servers very small. We’ll cover that later.)

We want to make sure that the second reported by the NMEA driver is the second that the last PPS pulse referred to, so it needs to be within a few hundred milliseconds of the correct time so that NTP does the right thing. This is why it’s desirable to mark another NTP server as “prefer” to help the NTP daemon get it right.

Note that we use NTP PPS discipline, not kernel PPS (which in my testing results in warnings from ntp that kernel PPS discipline isn’t supported).

Comment out all the pool lines.

Add the servers from the dig command, or use servers of your choice) with the top one saying prefer on it (example only, don't all use these IP addresses):

server 176.58.109.199 iburst prefer
server 195.195.221.100 iburst
server 185.53.193.157 iburst
# By default, exchange time with everybody, but don't allow configuration.
restrict default kod nomodify notrap nopeer noquery
restrict -6 default kod nomodify notrap nopeer noquery
# Local users may interrogate the ntp server more closely.
restrict 127.0.0.1
restrict -6 ::1
restrict 10.0.0.0 mask 255.0.0.0
restrict 172.16.0.0 mask 255.240.0.0
restrict 192.168.0.0 mask 255.255.0.0
# Drift file etc.
driftfile /var/lib/ntpsec/ntp.drift

Note You MUST add a preferred server or PPS doesn’t work.

Save and close nano.

sudo systemctl restart ntpsec.service

After a few minutes run

ntpq –p

If you get oPPS(0) this indicates source selected, Pulse Per Second (PPS) used and everything is working.

pi@ntp2:~ $ ntpq -pn
remote refid st t when poll reach delay offset jitter
==============================================================================
o127.127.22.0 .PPS. 0 l 6 16 377 0.000 -0.002 0.001
*127.127.20.0 .GPS. 1 l 5 16 377 0.000 -0.323 1.052
+195.195.221.100 .GPS. 1 u 49 64 377 18.307 0.190 0.127
+193.150.34.2 85.199.214.101 2 u 56 64 377 7.953 0.067 0.114

And then

pi@ntp2:~ $ ntpq -csysinfo
associd=0 status=0115 leap_none, sync_pps, 1 event, clock_sync,
system peer: PPS(0)
system peer mode: client
leap indicator: 00
stratum: 1
log2 precision: -21
root delay: 0.000
root dispersion: 2.015
reference ID: PPS
reference time: e0601d81.248b4b37 Tue, Apr 16 2019 10:23:13.142
system jitter: 0.000477
clock jitter: 0.003
clock wander: 0.001
broadcast delay: -50.000
symm. auth. delay: 0.000

If you aren’t seeing the settings its possible the NTP server is picking up the NTP information via DHCP which is overriding your settings above. Do this :

rm /etc/dhcp/dhclient-exit-hooks.d/ntp

rm /var/lib/ntp/ntp.conf.dhcp

At this point you have a NTP server which will use an external time source and use your local PPS to discipline it.

Note: Once you've configured ntp to use your GPS source, you'll need to stop the ntp service before running gpsctl, otherwise they'll both be trying to access ttyAMA0 at the same time.

GPS Offset Tuning

Your PPS time is going to be more accurate than NTP pool servers. Unless you have specialised equipment and local LAN GPS / DCF77 / MSF PPS sources to calibrate against, it’s not really possible to determine the appropriate PPS offset. It’s likely to be in the order of a few microseconds at most ( htt://lists.ntp.org/pipermail/questions/2011-September/030338.html ).

The PPS is a precise signal with around 10 ns of jitter. On the other hand, the offset of the GPS serial data output (GPSD) generally will have much more variation than the PPS because of the variables involved in sending data over an asynchronous serial port. However, the GPSD offset can be reduced somewhat by adjusting the GPSD reference clock fudge parameter time2 in the ntp.conf file.

The initial value used in our ntp.conf file is 60ms (0.060s). This means that the GPS ZDA packet arrives in the NTP daemon approximately 60ms after the PPS pulse corresponding to it. On our test device, we see a jitter of under 3ms on the GPS_NMEA clock.

We might need to adjust the GPS ntp.conf time2 (offset) value to get the NMEA time offset low enough that the offset isn’t great enough to confuse NTP as to which second the PPS pulse referred to.

This option can be used to compensate for a constant error. The specified offset (in seconds) is applied to all samples produced by the reference clock. The default is 0.000s.

Start with these ntp.conf settings, they should get you close enough to get everything working properly:

sudo nano /etc/ntpsec/ntp.conf

# PPS Driver
refclock pps unit 0 minpoll 4 maxpoll 4 ppspath /dev/pps0
#flag3 Controls the kernel PPS discipline: 0 for disable (default), 1 for enable.
#time1 PPS time offset
# use local clock at stratum 10 if no others available
tos orphan 10
tos mindist 0.002
# NMEA driver (/dev/gps0 and /dev/gpspps0)
refclock nmea prefer minpoll 4 maxpoll 4 mode 8 stratum 2 baud 115200 time2 0.060 path /dev/ttyAMA0
#flag1 Disable PPS signal processing if 0 (default); enable PPS signal processing if 1.
#flag2 If PPS signal processing is enabled, capture the pulse on the rising edge if 0 (default); capture on the falling edge if 1.
#flag3 If PPS signal processing is enabled, use the ntpd clock discipline if 0 (default); use the kernel discipline if 1.
#time1 PPS time offset
#time2 NMEA time offset
#mode bit 0 - process $GPMRC (value = 1)
# bit 1 - process $GPGGA (value = 2)
# bit 2 - process $GPGLL (value = 4)
# bit 4 - process $GPZDA or $GPZDG (value = 8)
# mode 9 = process only xxRMC and xxZDA NMEA records at 115200 baud
#---------------------------
# pool.ntp.org maps to about 1000 low-stratum NTP servers.  Your server will
# pick a different set every time it starts up.  Please consider joining the
# pool: <https://www.pool.ntp.org/join.html>
server 0.uk.pool.ntp.org iburst
server 1.uk.pool.ntp.org iburst
server 2.uk.pool.ntp.org iburst

To calculate the GPS offset we must disable GPS by placing a noselect in the ntp.conf "refclock nmea" line. We'll run the time server for a few hours and then compare the ntpq -p GPS offset to the average public time server offset. For accurate tuning use a bunch of known-good Stratum 1 servers in ntp.conf. Remember to remove the "noselect" after tweaking.

sudo systemctl stop ntpsec.service

sudo rm –f /var/log/ntpsec/*

sudo systemctl start ntpsec.service

Start ntpd and let it run for at least four hours. Periodically check progress with "ntpq -p" and wait until change has settled out.

Look for the row with GPS_NMEA(0) and refid GPSD. The offset values probably will be different for each

query. Note that the ntpq offsets are in milliseconds, but the peerstats file offsets and NTP’s time2 parameter are in seconds.

Calculate the average GPS offset (in seconds) using this script:

sudo nano ~/nmeaoffset

#!/bin/sh
#
# Generate an estimate of your GPS's offset from a peerstats file
#

awk '
/NMEA/ { sum -= $5 ; cnt++; }
END { printf("%.6f\n", sum / cnt); }
' </var/log/ntpsec/peerstats

Then sudo chmod +x ./nmeaoffset

Run by typing

pi@ntp2:~ $ ./nmeaoffset

-0.001212

That’s within a few milliseconds, close enough for anyone as PPS is going to do its magic to give us accuracy within a few microseconds.

Adjust the "time2" value for the GPS source of your ntp.conf by adding the average offset from above.

sudo systemctl stop ntpsec.service

sudo rm –f /var/log/ntpsec/*

sudo systemctl start ntpsec.service

Repeat the procedure above until -5ms < offset < +5ms (or under 10ms, if that’s OK with you). When you’re done, remove the noselect from the "refclock nmea" line in ntp.conf.

If you decide to recalculate the average offset using the above procedures, wait at least another day or two.

Avoid unnecessarily changing the time2 value. A typical value for the Adafruit GPSD driver is +0.534 s when using its default 4800 baud interface and in the HAB Supplies/Uputronics GPSD it’s +0.060 s when using 115200 baud and the gpsctl --config_for_timing tweaks.

Automatically updating GPS leap seconds

These semi-annual changes will be made no later than 1 June and 1 December of each year to indicate what action (if any) is to be taken on 30th June and 31st December, respectively.

In Buster, the default ntp.conf points LeapFile at the tzdata-provided leap-seconds.list. That's fine, as long as you auto apply system updates via cron or manually on a regular basis.

Automatically updating Raspbian

To automatically update Raspbian on a schedule, follow these steps:

sudo apt install unattended-upgrades mailutils

sudo nano /etc/apt/apt.conf.d/50unattended-upgrades

Edit to include these sources (remove all others)

    
"origin=Raspbian,codename=${distro_codename},label=Raspbian";
"origin=Raspberry Pi Foundation,codename=${distro_codename},label=Raspberry Pi Foundation";

and add

Unattended-Upgrade::Automatic-Reboot "true";

sudo nano /etc/apt/apt.conf.d/20auto-upgrades

APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Download-Upgradeable-Packages "1";
APT::Periodic::Unattended-Upgrade "1";
APT::Periodic::Verbose "1";
APT::Periodic::AutocleanInterval "7";

sudo dpkg-reconfigure --priority=low unattended-upgrades

Set the systemctl apt timer to fire at 2AM daily

sudo systemctl edit apt-daily-upgrade.timer

[Timer]
OnCalendar=
# OnCalendar=DayOfWeek Year-Month-Day Hour:Minute:Second
# see https://wiki.archlinux.org/index.php/Systemd/Timers
OnCalendar=02:00
RandomizedDelaySec=0

To test,

sudo unattended-upgrade -d -v --dry-run

Static IP and Hostname

If you want to fix your LAN IP you do it by running nmtui to configure your network.

Amend your hostname via raspi-config.

In the MoOde for streaming internet radio on the Raspberry Pi Zero (updated)

When I saw the announcement of the Raspberry Pi Zero my first thought was that it would be a good basis for a headless Internet Radio Streamer for my hifi system.

So when I got my Pi Zero, I looked around for a cost-effective Digital to Audio Converter and found Pimoroni's pHAT DAC for the Pi Zero.

After a bit of web searching and reading, I settled on Tim Curtis' MoOde as my Media Player.

I've updated the instructions which follow for Moode 2.6 TR1 and later.

So, I downloaded MoOde 2.6 TR1 and flashed it to a MicroSD card.

I had a spare TP-Link wifi USB stick which worked with the Pi, so to make that work I booted up MoOde with the TV and USB keyboard plugged in, and edited /etc/network/interfaces so it looked something like this:

auto lo
iface lo inet loopback
#allow-hotplug eth0
#iface eth0 inet dhcp lan0 inet dhcp
allow-hotplug wlan0
iface wlan0 inet dhcp
wpa-ssid your-ssid
wpa-psk your-pre-shared-key

Change the ssid and PSK to suit your environment.

After a reboot, I could SSH into moode.local and had a web interface to play with.

For the pHAT DAC, I selected the hifiberry DAC as it uses the same chipset, and lo, I soon had internet radio streaming into my hifi.

But gosh, the low quality BBC radio streams sounded just awful.

A bit more web searching revealed the solution.

And so I installed minimserver and minimstreamer and access the Beeb's HLS streams via minimstreamer.

The process is:

1: install JDK 8.

    sudo apt-get install oracle-java8-jdk

2: download MinimServer-0.8.4-linux-armhf.tar.gz and MinimWatch-0.8.4-linux-armhf.tar.gz from the minimserver download page.

3: extract as follows:

    sudo tar zxvf MinimServer-0.8.4-linux-armhf.tar.gz -C /opt
    sudo tar zxvf MinimWatch-0.8.4-linux-armhf.tar.gz -C /opt

4: configure minimserver and minimwatch

    sudo /opt/minimserver/bin/setup

change the settings to not run at startup, and do the same for minimwatch

5: configure minimserver's media directory and enable the minimstreamer module in minimserver:

    sudo /opt/minimserver/bin/startc
    packages
    install minimstreamer-0.5.24
    exit

6: create a new file minimserver.service in /etc/systemd/system

    sudo nano /etc/systemd/system/minimserver.service

with the following contents

[Unit]
Description=MinimServer

[Service]
RemainAfterExit=true
ExecStart=/opt/minimserver/bin/startd
ExecStop=/opt/minimserver/bin/stopall

[Install]
WantedBy=network.target

7: Enable the newly created minimserver service

    sudo systemctl enable minimserver.service

8: Create a BBCRadio.m3u file in minimplayer's media directory with contents as per here

9: Reboot your Raspberry Pi

10: Add individual stations via Moode's interface using urls like http://127.0.0.1:9790/minimstreamer/*/R1/

And then enjoy the BBC's 320kbps HLS streams via your Pi Zero and pHAT DAC.

MoOde has excellent support over on the DIYAudio forum

Posted by Phil at 3:39 PM
Edited on: Monday, May 02, 2016 12:01 PM
Categories: Raspberry Pi

H G Wells on Technological Unemployment

The concept of "Technological Unemployment" seems to be in fashion once again.

The sanest words I've read on the subject come from H. G. Wells in his 1914 work "The World Set Free":

He asked a passing stroller, and was told that the men had struck that day against the use of an atomic riveter that would have doubled the individual efficiency and halved the number of steel workers.

'Shouldn't wonder if they didn't get chucking bombs,' said Barnet's informant, hovered for a moment, and then went on his way to the Alhambra music hall.

Barnet became aware of an excitement in the newspaper kiosks at the corners of the square. Something very sensational had been flashed upon the transparencies. Forgetting for a moment his penniless condition, he made his way over a bridge to buy a paper, for in those days the papers, which were printed upon thin sheets of metallic foil, were sold at determinate points by specially licensed purveyors. Half over, he stopped short at a change in the traffic below; and was astonished to see that the police signals were restricting vehicles to the half roadway. When presently he got within sight of the transparencies that had replaced the placards of Victorian times, he read of the Great March of the Unemployed that was already in progress through the West End, and so without expenditure he was able to understand what was coming.

He watched, and his book describes this procession which the police had considered it unwise to prevent and which had been spontaneously organised in imitation of the Unemployed Processions of earlier times. He had expected a mob but there was a kind of sullen discipline about the procession when at last it arrived. What seemed for a time an unending column of men marched wearily, marched with a kind of implacable futility, along the roadway underneath him. He was, he says, moved to join them, but instead he remained watching. They were a dingy, shabby, ineffective-looking multitude, for the most part incapable of any but obsolete and superseded types of labour. They bore a few banners with the time-honoured inscription: 'Work, not Charity,' but otherwise their ranks were unadorned.

They were not singing, they were not even talking, there was nothing truculent nor aggressive in their bearing, they had no definite objective they were just marching and showing themselves in the more prosperous parts of London. They were a sample of that great mass of unskilled cheap labour which the now still cheaper mechanical powers had superseded for evermore. They were being 'scrapped'—as horses had been 'scrapped.'

Barnet leant over the parapet watching them, his mind quickened by his own precarious condition. For a time, he says, he felt nothing but despair at the sight; what should be done, what could be done for this gathering surplus of humanity? They were so manifestly useless—and incapable—and pitiful.

What were they asking for?

They had been overtaken by unexpected things. Nobody had foreseen——

It flashed suddenly into his mind just what the multitudinous shambling enigma below meant. It was an appeal against the unexpected, an appeal to those others who, more fortunate, seemed wiser and more powerful, for something—for INTELLIGENCE. This mute mass, weary footed, rank following rank, protested its persuasion that some of these others must have foreseen these dislocations—that anyhow they ought to have foreseen—and arranged.

That was what this crowd of wreckage was feeling and seeking so dumbly to assert.

'Things came to me like the turning on of a light in a darkened room,' he says. 'These men were praying to their fellow creatures as once they prayed to God! The last thing that men will realise about anything is that it is inanimate. They had transferred their animation to mankind. They still believed there was intelligence somewhere, even if it was careless or malignant.... It had only to be aroused to be conscience-stricken, to be moved to exertion.... And I saw, too, that as yet THERE WAS NO SUCH INTELLIGENCE. The world waits for intelligence. That intelligence has still to be made, that will for good and order has still to be gathered together, out of scraps of impulse and wandering seeds of benevolence and whatever is fine and creative in our souls, into a common purpose. It's something still to come....'

Posted by Phil at 7:28 PM
Edited on: Sunday, January 18, 2015 7:41 PM
Categories: Comment

Deja Vu, Again

The Beeb's been at it again. Their finest journos have rehashed a press release from Premier Oil into yet another gushing news item about fossil fuels.

Just as in my "Here We Go Again" blog post from last June, there's no mention of climate change, not one word, apart from the "Climate" in the Department for Energy and (for) Climate Change's title.

Read it and weep.

Again.

Posted by Phil at 8:44 PM
Edited on: Wednesday, April 01, 2015 9:53 PM
Categories: Environment

Domestic LED lighting is Cost-effective Right Now

Of late I've been looking in the lighting sections of supermarkets in search of usable LED replacements for compact fluorescent and incandescent lamps for normal room lighting, but to no avail.

Until yesterday, that is, when I stumbled across this little beauty in my local Aldi store.

This is a Medion MD14536 11W LED lamp (equivalent to 75W incandescent, 18W compact fluorescent), 1055 lumens, 2700K colour temperature, rated lifetime of 35000 hours. Their light output is around 100 lumens per watt. And all for a princely £9.99. The only drawback is that they are not dimmable.

An equivalent Philips 18W compact fluorescent with a 6000 hour lifetime sells for a fiver on Amazon and gives around 60 lumens per watt.

10W (60W incandescent, 810 lumens) and 5.5W (40W incandescent, 470 lumens) are also available.

My experience of modern CFLs is that they rarely last their rated lifetime. I'm lucky if they get to half of it in my residence. CFLs I purchased over a decade ago were much more reliable.

LED lighting at this price is cost-effective right now. A real bargain!

Postscript, October 14th

My local Asda now has 12W (60W incandescent, 810 lumens, 25000 hour rated lifetime) LED lamps for £16. Not such a bargain.

Posted by Phil at 8:45 PM
Edited on: Wednesday, April 01, 2015 9:20 PM
Categories: Comment, Environment

Scientific Illiteracy, Propaganda, or Both?

The Beeb are are being sloppy, ignorant, incompetent, or just plain malicious propaganda-spreaders once again.

A tweet by Greenpeace Nuclear @nukereaction alerted me this morning

Here it is:

Back in November, I commented on this nonsense:

The irony is that nuclear energy is the ultimate non-renewable, actually destroying matter.

Posted by Phil at 11:53 AM
Edited on: Wednesday, April 01, 2015 9:53 PM
Categories: Comment, Environment

Phil's Occasional Blog

Thursday, May 27, 2021

Converting CentOS 8 Stream to AlmaLinux 8.4

Monday, September 16, 2019

Lenovo IdeaPad S340-15API

Saturday, August 17, 2019

About to update to RedHat Enterprise Linux or CentOS 7.7? Beware!

Sunday, July 14, 2019