Thursday, May 27, 2021
Converting CentOS 8 Stream to AlmaLinux 8.4
There's no better time to migrate CentOS 8 Stream to AlmaLinux 8.4. Because RedHat Enterprise Linux 8.4 has only just been released, the differences between Stream and 8.4 are minor.
First off, make sure your box is up to date:
dnf distro-sync
Then back it up or snapshot it. Some packages may revert to older versions and break things.
Then download the AlamLinux conversion script
curl -O https://raw.githubusercontent.com/AlmaLinux/almalinux-deploy/master/almalinux-deploy.sh
edit almalinux-deploy.sh
MINIMAL_SUPPORTED_VERSION='8'
In REMOVE_PKGS add "centos-stream-release" "centos-stream-repos"
Amend the grep command in get_os_version:
if ! os_version="$(grep -oP 'CentOS\s+(Linux|Stream)\s+release\s+\K(\d+(\.\d+)?)' \
"${REDHAT_RELEASE_PATH}" 2>/dev/null)"; then
Or alternatively, download my modified install script from my repo:
curl -O https://raw.githubusercontent.com/philrandal/almalinux-deploy/master/almalinux-deploy.sh
Then you're set to go. The script works with CentOS 8.3 and CentOS 8 Stream.
bash almalinux-deploy.sh
and if it succeeds
reboot
After the reboot, check for orphaned packages: I got this from migrating a full CentOS 8 Stream install to AlmaLinux 8.4
dnf list extras Extra Packages
llvm-compat-libs.x86_64 10.0.1-1.module_el8.4.0+533+50191577
They can safely be removed.
When you reboot, you'll see that the original CentOS rescue kernel is still there.
Fix that with:
rm -f /boot/vmlinuz-0-rescue-* /boot/initramfs-0-rescue-*.img
/usr/lib/kernel/install.d/51-dracut-rescue.install add $(uname -r) "" /lib/modules/$(uname -r)/vmlinuz
To remove old CentOS kernels,
rpm -qa | grep kernel-core
Then use yum remove to get rid of any superfluous ones.
Test, test, and test again.
Remove any snapshots once you're happy.
Postscript:
So you just want to convert back to CentOS 8.x from CentOS Stream?
No worries (except for the caveat that downgrades may break things).
curl -O http://mirror.centos.org/centos-8/8.3.2011/BaseOS/x86_64/os/Packages/centos-linux-release-8.3-1.2011.el8.noarch.rpm curl -O http://mirror.centos.org/centos-8/8.3.2011/BaseOS/x86_64/os/Packages/centos-linux-repos-8-2.el8.noarch.rpm
rpm -e --nodeps centos-stream-repos centos-stream-release
rpm -ivh centos-linux-release-8.3-1.2011.el8.noarch.rpm centos-linux-repos-8-2.el8.noarch.rpm
dnf distro-sync
reboot
Monday, September 16, 2019
Lenovo IdeaPad S340-15API
I've just purchased a Lenovo IdeaPad S340-15API From Argos to replace our aging and very slow Acer Aspire E5-511 laptop.
The spec is good. For £550 you get a 15.6 inch Full HD (1920 x 1080) display, AMD Ryzen 7 3700U quad-core 2.3GHz processor (with Simultaneous Multi-Threading giving 8 virtual cores) with Radeon Vega Mobile graphics, 8GB of RAM, of which 2GB is taken as display RAM, and 512GB SSD (WDC PC SN520 SDAPMUW-512G-1101).
Windows 10 1809 Home pre-installed (in S mode), but it is a simple fix to switch to full Windows 10 Home.
It didn't offer me Windows 10 1903, so I went and fetched it using the Microsoft Update Assistant.
It took a while to get that on, and the successive updates.
Lenovo Vantage offered offered a few driver updates, but I went to the Lenovo Support website and manually downloaded the latest BIOS and assorted drivers.
Power settings are never right by default, as I discovered the hard way. I could boot once, close the lid to put it into sleep mode, open it again to wake up, but the second time I closed the lid it would not behave and effectively locked up.
Powering off and on brought it back to life.
Eventually, after running powercfg /a I discovered a workaround:
C:\Windows\System32>powercfg /a
The following sleep states are available on this system:
Standby (S3)
Hibernate
Hybrid Sleep
Fast Startup
The following sleep states are not available on this system:
Standby (S1)
The system firmware does not support this standby state.
Standby (S2)
The system firmware does not support this standby state.
Standby (S0 Low Power Idle)
The system firmware does not support this standby state.
In the Power Options page in Control Panel, Change Plan Settings, then Change advanced power settings
In the Sleep section of the Advanced settings, turn Allow hybrid sleep on for both On battery and Plugged in
Then it all works as expected.
Postscript, September 28, 2019
The cause of the Sleep problem was the Realtek Audio Driver, and the fix was found on the Lenovo Support Forum here.
If you're not averse to extracting .cab files and manually updating drivers, give it a go.
Saturday, August 17, 2019
About to update to RedHat Enterprise Linux or CentOS 7.7? Beware!
There's a couple of caveats before upgrading to RHEL / CentOS 7.7.
1: Upgrading to latest RHEL 7 PCP package pcp-4.3.2-2.el7 gives several issues.
During installation/upgrade of pcp-selinux:
Updating / installing... 1:pcp-selinux-4.3.2-2.el7 ################################# [100%]
Failed to resolve allow statement at /etc/selinux/targeted/tmp/modules/400/pcpupstream/cil:83
semodule: Failed!
Followed by lots of selinux errors logged.
Workaround is to install 7.7's selinux-policy first
yum update selinux-policy
or, if you've already updated,
yum reinstall pcp-selinux
Documented in RedHat Bugzilla entry 1714101
2: bind is updated from version 9.9 to 9.11. This breaks configurations in which the same zone file is defined in several views.
The solution is to use bind 9.11's "in-view xxx" in subsequent definitions of the same zone in different views.
The fix is described in detail here.
Sunday, July 14, 2019
Using the Pimoroni Fan Shim with LibreElec
Trying out the latest LibreElec alpha on a Raspberry Pi 4 on the hottest day of the year so far I found the CPU temperature reaching 70ºC. A bit too warm for my liking, so I've added a Pimoroni Fan Shim.
There's no way you can install the Pimoroni python library on LibreElec, but there is an alternative.
First, you need to install the Raspberry Pi Tools addon in LibreElec.
Addons / install from repository / libreelec add-ons / program add-ons / Raspberry Pi Tools.
I found a script on https://forum-raspberrypi.de/forum/thread/43568-fan-shim-steuern/, and edited it slightly. Change the min and max temperatures to suit yourself.
ssh into your LibreElec box as root and
nano /storage/fanshim.py
#!/usr/bin/env python # https://forum-raspberrypi.de/forum/thread/43568-fan-shim-steuern/ # place command below in /storage/.config/autostart.sh # nohup /storage/fanshim.py & import os import time import signal import sys sys.path.append('/storage/.kodi/addons/virtual.rpi-tools/lib') import RPi.GPIO as GPIO import subprocess Pause = 30 CoreTempMax = 57 CoreTempMin = 46 GPIO_Pin = 18 Run_Fan_function = False def init(): GPIO.setwarnings(False) GPIO.setmode(GPIO.BCM) GPIO.setup(GPIO_Pin, GPIO.OUT) return() def Set_Fan_ON(): GPIO.output(GPIO_Pin, True) return() def Set_Fan_OFF(): GPIO.output(GPIO_Pin, False) return() def get_CPU_Temp(): temp = subprocess.check_output(['vcgencmd', 'measure_temp'])[5:-3] return temp def Watch_Temp(): global Run_Fan_function CPU_Temp = float(get_CPU_Temp()) if Run_Fan_function==False and CPU_Temp>=CoreTempMax: Run_Fan_function = True Set_Fan_ON() if Run_Fan_function==True and CPU_Temp<=CoreTempMin: Run_Fan_function = False Set_Fan_OFF() return(); try: init() while True: Watch_Temp() time.sleep(Pause) except KeyboardInterrupt: GPIO.cleanup()
Save the edited file, and then:
chmod +x /storage/fanshim.py
Then edit LibreElec's autostart.sh
nano /storage/.config/autostart.sh so it contains the line:
nohup /storage/fanshim.py &
Reboot your Pi 4 for it to take effect.
If you're using a TV Hat, you'll need an extra tall stacking header to give a good gap between the fan and the TV Hat.
Enjoy.
Postscript, October 26, 2019
Someone has done a bit of work on this and turned it into a proper installable Kodi addon:
https://github.com/jane-t/rpi-fanshim
Thanks!
Edited on: Saturday, October 26, 2019 4:27 PM
Categories: IT, Raspberry Pi
Tweet
Saturday, April 27, 2019
A Raspberry Pi Stratum 1 NTP Server
Overview
Updated for the Pi 5 and Raspberry Pi OS Bookworm
The following instructions are how to make a cheap Pulse Per Second (PPS) disciplined Stratum 1 NTP Time server using one of the Raspberry Pi U-blox M8Q based GPS boards sold by Uputronics.
Our basic requirement is for an NTP server which will work standalone without connectivity to other NTP servers, so if a datacentre loses connectivity its servers will still have a stable time source.
As we’re only interested in time and not height, motion, or location, all we’re interested in is the NMEA xxZDA and xxRMC sentences and the highly accurate PPS signal from the GPS module, both of which we feed straight into the ntp daemon. The xxZDA sentences (which give us full 4-digit years) are not output by default by the M8Q so we have to enable them at boot time.
Many guides on the net use gpsd to feed ntp via shared memory. That’s an additional overhead and complexity, especially if we want to enable Galileo reception (gpsd initialises the U-blox the way it sees fit). I prefer to let the NTP daemon read the NMEA stream from the GPS receiver itself, avoiding the middle man.
In this guide we’ve chosen to use NMEA output from the GPS module via the ntpsec Generic GPS Receiver driver 20 ( https://docs.ntpsec.org/latest/driver_nmea.html ) and the PPS driver ( https://docs.ntpsec.org/latest/driver_pps.html ).
You will need a Raspberry Pi 4B, the Uputronics Raspberry Pi+ GPS Expansion Board and a suitable GPS antenna.
This guide assumes that you’re using Raspberry Pi OS Bookworm, 64-bit Lite edition. Download and write this to an SD card.
See https://www.raspberrypi.org/documentation/installation/installing-images/windows.md
You'll need a keyboard and display attached for first boot configuration.
Create a user (mine was ntpadmin, use whatever you like) and set its password.
Attach the Uputronics Raspberry Pi+ GPS Expansion Board to the Pi, insert the SD card, connect the antenna and network cable and boot the Pi up. Either connect locally or via SSH to the Pi. If you can’t SSH in and don’t have a monitor see this
https://www.raspberrypi.org/documentation/configuration/wireless/headless.md
Follow the instructions carefully if you miss steps things won’t work.
The Uputronics board has u-blox firmware 3.01 on it, dated 2016.
The week number rollover is set to 1867 (October 2015). All transmitted week numbers are mapped to the ~19.5 year period between week 1867 and week 2990 (April 2035).
A Note About Accuracy
In theory, GPS-based time receivers can give a very high accuracy, with the PPS (Pulse per Second) signal being accurate to within 10ns.
However, fix data from gpsctl shows:
pi@ntp2:~ $ /usr/local/bin/gpsctl -Q fix
Time (UTC): 2019-04-18 15:31:27 (yyyy-mm-dd hh:mm:ss)
Latitude: 52.05994980 N
Longitude: 2.72698960 W
Altitude: 198.789 feet
Motion: 0.338 mph at 53.114 degrees heading
Satellites: 5 used for computing this fix
Accuracy: time (39 ns), height (+/-18.199 feet), position (+/-103.911 feet), heading(+/-8.230 degrees), speed(+/-0.132 mph)
Note the 39ns time accuracy from that fix. The more satellites, the better.
Errors are also caused by signal delay (4ns per foot) in the cable from the aerial to the GPS receiver, etc.
So we’d be lucky to get 100ns accuracy from the PPS pulse.
Add to that the processing overheads of the PPS interrupt, and processor clock jitter in the Raspberry Pi, and overheads transferring and decoding the NMEA sentences. The jitter on the PPS signal is less than 5 microseconds.
The output from the ntp NMEA driver without PPS correction can have a jitter of a few milliseconds, and an offset from real time which has to be tweaked manually to get within 5ms of the correct time.
NTP sources on the internet can show offsets / jitter of over 5ms, local LAN 50us or more.
On the Pi 4B we can expect an average jitter of less than 1 microsecond.
Required Components
I sourced my components from the Pi Hut unless noted elsewhere:
1 Raspberry Pi Model 4B 1GB RAM
1 8GB or larger micro SD card
(a Transcend High Endurance 32GB micro SD card would be better choice than a generic one for longevity)
1 Uputronics GPS Hat
1 Pi Hut Pi 4 GPS case
1 Raspberry Pi 4 power supply
1 GPS SMA Antenna
(optional) 1 SMA-male to TNC-female (or BNC, as needed) adaptor to connect to existing GPS aerial (from Amazon or eBay)
Prerequisite Settings
Login as pi / raspberry, and immediately change the password from the default
passwd
sudo raspi-config
Advanced Options
Expand filesystem (this is automatically done on first boot)
Interfacing Options
SSH -> Would you like the SSH server to be enabled – Yes (Recommended)
I2C -> Would you like the ARM I2C interface to be enabled? - Yes
Serial -> Login Shell (no) Hardware (yes) (Optional)
Quit but no need to reboot at this point.
for i in systemd-timesyncd avahi-daemon alsa-state bluetooth triggerhappy hciuart rng-tools autologin getty;
do
sudo systemctl disable $i.service; sudo systemctl stop $i.service;
done
sudo systemctl disable serial-getty@ttyAMA0.service
sudo systemctl mask serial-getty@ttyAMA0.service
sudo apt install gpsd-clients pps-tools ntpsec ntpsec-ntpviz i2c-tools linux-cpupower mlocate
sudo systemctl enable ntpsec
sudo apt purge bluez bluez-firmware modemmanager wpasupplicant (assuming no wifi required)
sudo apt update
sudo apt full-upgrade
If you want your Pi to run at full clock speed (active cooling is recommended),
sudo nano /etc/systemd/system/cpu-governor.service
[Unit] Description=CPU Governor Service After=raspi-config.service [Service] Type=oneshot ExecStart=/usr/bin/cpupower frequency-set -g performance [Install] WantedBy=multi-user.target
Then
sudo systemctl enable cpu-governor.service --now
sudo nano /boot/firmware/config.txt and add at the bottom :
dtoverlay=disable-bt
dtoverlay=disable-wifi
dtoverlay=pps-gpio,gpiopin=18
dtparam=eee=off
dtparam=audio=off
On a Pi 4 or below, make sure that enable_uart=1 is set in /boot/config.txt
On a Pi 5, make sure that dtparam=uart0=on is set in boot/config.txt
sudo nano /boot/firmware/cmdline.txt and add at the end
nohz=off
sudo nano /etc/modules and add at the bottom :
pps-gpio
Save and quit nano, then sudo reboot
Enabling Galileo Satellites and Setting Stationary Mode
Stationary mode gives a faster GPS fix.
See Tom Dilatush’s gpsctl:
http://www.jamulblog.com/2017/11/paradise-ponders-gpsctl-functionally.html
I’ve customised it further to enable the use of a config file and put it in my GitHub repo:
https://github.com/philrandal/gpsctl
Download and build gpsctl.
cd ~
wget https://github.com/philrandal/gpsctl/archive/master.zip
unzip master.zip
mv gpsctl-master gpsctl
cd gpsctl
./build.sh
sudo cp ~/gpsctl/gpsctl /usr/local/bin
sudo cp ~/gpsctl/etc/gpsctl.conf /etc/gpsctl.conf
Individual settings can be tweaked in /etc/gpsctl.conf.
To set port speed to 115200 baud, enable Galileo satellites, set stationary mode, tweak antenna delays, PPS timing, etc:
/usr/local/bin/gpsctl -a -B 115200 --configure_for_timing -vv
To reset the device to its defaults
/usr/local/bin/gpsctl -a --reset -vv
To view info:
/usr/local/bin/gpsctl -a -Q satellites
/usr/local/bin/gpsctl -a -Q config
/usr/local/bin/gpsctl -a -Q fix
Note that these commands can only be run when gpsd / ntp are not using /dev/ttyAMA0
Example /etc/gpsctl.conf which configures the U-blox M8Q for this environment:
# # example gpsctl.conf which enables Galileo as in --galileo parameter
#
[gpsctl]
port = /dev/ttyAMA0
# sync method: ASCII = 1, NMEA = 2, UBX = 3
sync method = 3
verbosity = 0
[NMEA]
enabled = true
version = 41
GGA = off
GLL = off
GSA = off
GSV = off
RMC = on
VTG = off
GRS = off
GST = off
ZDA = on
[GPS]
enabled = yes
minimum channels=8
maximum channels=16
[SBAS]
enabled = no
minimum channels=1
maximum channels=3
[Galileo]
enabled = yes
minimum channels=4
maximum channels=8
[Beidou]enabled = no
minimum channels=8
maximum channels=16
[IMES]
enabled = no
minimum channels=0
maximum channels=8
[QZSS]
enabled = no
minimum channels=0
maximum channels=3
[GLONASS]
enabled = yes
minimum channels=8
maximum channels=14
[Navigation Engine]
# Dynamic model: Portable = 0, Stationary = 2, Pedestrian = 3, Automotive = 4,
# Sea = 5, Air1G = 6, Air2G = 7, Air4G = 8, Watch = 9
Dynamic model = 2
# Fix mode: 2D only = 1, 3D only = 2, auto 2D/3D = 3
Fix mode = 3
Fixed altitude (2D) = 0.00 meters
Fixed altitude variance (2D) = 1.0000 meters^2
Minimum elevation = 5 degrees
Position DoP mask = 10.0
Time DoP mask = 10.0
Position accuracy mask = 100 meters
Time accuracy mask = 300 meters
Static hold threshold = 0 cm/s
Dynamic GNSS timeout = 60 seconds
Threshold above C/No = 0 satellites
C/No threshold = 0 dBHz
Static hold max distance = 0 meters
# UTC Standard: AutoUTC = 0, USNO_UTC = 3, GLONASS_UTC = 6, BeiDou_UTC = 7
UTC standard = 3
[Time Pulse]
# the nanoseconds / microseconds after the numbers are just reminders,
# they don't mean anything to the config parserAntenna cable delay = 56 nanoseconds
RF group delay = 20 ns
Unlocked pulse period = 1000000 microseconds
Unlocked pulse length = 0
Locked pulse period = 1000000 microseconds
Locked pulse length = 500000 microseconds
User configurable delay = 0
To run gpsctl at system startup,
sudo cp ~/gpsctl/etc/systemd/system/ublox-init.service /etc/systemd/system/ublox-init.service
or
nano /etc/systemd/system/ublox-init.service
and add the following contents
[Unit] Description=u-blox initialisation
Before=gpsd.service
Before=ntp.service
[Service]
Type=oneshot
Environment="PARAMS=-q -a -B 115200 --configure_for_timing"
EnvironmentFile=-/etc/default/gpsctl
ExecStart=/bin/bash '/usr/local/bin/gpsctl ${PARAMS}'
[Install]
WantedBy=multi-user.target
This will enable Galileo satellites, set stationary mode, configure comms at 115200 baud, and restrict NMEA output to RMC and ZDA records.
To use the default baud rate of 9600, create /etc/default/gpsctl with the contents
PARAMS=-q --configure-for-timing
Then
sudo systemctl enable ublox-init.service
sudo systemctl daemon-reload
Verifying that PPS Is Working
Ensure the GPS has a lock and the Green PPS LED on the Uputronics Pi+ GPS Expansion Board is blinking once a second.
lsmod | grep pps
Output should be similar to :
pps_gpio 3089 1
pps_core 8606 4 pps_gpio
dmesg | grep pps
Output should be similar to :
[ 2.735586] pps_core: LinuxPPS API ver. 1 registered [ 2.738121] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[ 2.763842] pps pps0: new PPS source pps@12.-1
[ 2.766361] pps pps0: Registered IRQ 169 as PPS source
This indicates that the PPS Module is loaded.
sudo ppstest /dev/pps0
Output should be similar to:
trying PPS source "/dev/pps0" found PPS source "/dev/pps0" ok, found 1 source(s), now start fetching data...
source 0 - assert 1418933982.998042450, sequence: 970 - clear 0.000000000, sequence: 0
source 0 - assert 1418933983.998045441, sequence: 971 - clear 0.000000000, sequence: 0
(Press CTRL+C to quit). This indicates that the PPS Module is working.
You need to pick a few local NTP servers to use. The easiest way to do this is pick your region:
https://support.ntp.org/bin/view/Servers/NTPPoolServers
Select your region then you get a list of the country servers. E.g for the UK its uk.pool.ntp.org.
Type:
dig uk.pool.ntp.org
You will get four IP’s back:
;; <<>> DiG 9.10.3-P4-Raspbian <<>> +answer uk.pool.ntp.org ;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 51647
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
;; EDNS: version: 0, flags:; udp: 4000
;; QUESTION SECTION:
;uk.pool.ntp.org. IN A
;; ANSWER SECTION:
uk.pool.ntp.org. 24 IN A 193.150.34.2
uk.pool.ntp.org. 24 IN A 176.58.109.199
uk.pool.ntp.org. 24 IN A 195.195.221.100
uk.pool.ntp.org. 24 IN A 185.53.93.157
sudo nano /etc/ntpsec/ntp.conf
Amend the start of ntp.conf to look like this
## nts enable # You must create /var/log/ntpsec (owned by ntpsec:ntpsec) to enable logging. #statsdir /var/log/ntpsec/ #statistics loopstats peerstats clockstats #filegen loopstats file loopstats type day enable #filegen peerstats file peerstats type day enable #filegen clockstats file clockstats type day enable # This should be maxclock 7, but the pool entries count towards maxclock. #tos maxclock 11 # Comment this out if you have a refclock and want it to be able to discipline # the clock by itself (e.g. if the system is not connected to the network). #tos minclock 4 minsane 3 # Specify one or more NTP servers. # Public NTP servers supporting Network Time Security: # server time.cloudflare.com nts #-------------------------- # PPS Driver refclock pps unit 0 minpoll 4 maxpoll 4 ppspath /dev/pps0 #flag3 Controls the kernel PPS discipline: 0 for disable (default), 1 for enable. #time1 PPS time offset # use local clock at stratum 10 if no others available tos orphan 10 tos mindist 0.002 # NMEA driver (/dev/gps0 and /dev/gpspps0) # process $GPZDA only (mode 8)
refclock nmea prefer minpoll 4 maxpoll 4 mode 8 stratum 2 baud 115200 time2 0.060 path /dev/ttyAMA0 #time2 NMEA time offset #mode bit 0 - process $GPMRC (value = 1) # bit 1 - process $GPGGA (value = 2) # bit 2 - process $GPGLL (value = 4) # bit 4 - process $GPZDA or $GPZDG (value = 8) # mode 9 = process only xxRMC and xxZDA NMEA records #--------------------------- # pool.ntp.org maps to about 1000 low-stratum NTP servers. Your server will # pick a different set every time it starts up. Please consider joining the # pool: <https://www.pool.ntp.org/join.html> server 0.uk.pool.ntp.org iburst server 1.uk.pool.ntp.org iburst server 2.uk.pool.ntp.org iburst #pool 0.debian.pool.ntp.org iburst #pool 1.debian.pool.ntp.org iburst #pool 2.debian.pool.ntp.org iburst #pool 3.debian.pool.ntp.org iburst
(Note that I used “time2 0.060 …” to adjust the GPS time according to the other NTP servers. You can try some higher/lower values to have the offset of your NMEA driver compared to the offsets of those other NTP servers very small. We’ll cover that later.)
We want to make sure that the second reported by the NMEA driver is the second that the last PPS pulse referred to, so it needs to be within a few hundred milliseconds of the correct time so that NTP does the right thing. This is why it’s desirable to mark another NTP server as “prefer” to help the NTP daemon get it right.
Note that we use NTP PPS discipline, not kernel PPS (which in my testing results in warnings from ntp that kernel PPS discipline isn’t supported).
Comment out all the pool lines.
Add the servers from the dig command, or use servers of your choice) with the top one saying prefer on it (example only, don't all use these IP addresses):
server 176.58.109.199 iburst prefer server 195.195.221.100 iburst
server 185.53.193.157 iburst
# By default, exchange time with everybody, but don't allow configuration.
restrict default kod nomodify notrap nopeer noquery
restrict -6 default kod nomodify notrap nopeer noquery
# Local users may interrogate the ntp server more closely.
restrict 127.0.0.1
restrict -6 ::1
restrict 10.0.0.0 mask 255.0.0.0
restrict 172.16.0.0 mask 255.240.0.0
restrict 192.168.0.0 mask 255.255.0.0
# Drift file etc.
driftfile /var/lib/ntpsec/ntp.drift
Note You MUST add a preferred server or PPS doesn’t work.
Save and close nano.
sudo systemctl restart ntpsec.service
After a few minutes run
ntpq –p
If you get oPPS(0) this indicates source selected, Pulse Per Second (PPS) used and everything is working.
pi@ntp2:~ $ ntpq -pn remote refid st t when poll reach delay offset jitter
==============================================================================
o127.127.22.0 .PPS. 0 l 6 16 377 0.000 -0.002 0.001
*127.127.20.0 .GPS. 1 l 5 16 377 0.000 -0.323 1.052
+195.195.221.100 .GPS. 1 u 49 64 377 18.307 0.190 0.127
+193.150.34.2 85.199.214.101 2 u 56 64 377 7.953 0.067 0.114
And then
pi@ntp2:~ $ ntpq -csysinfo associd=0 status=0115 leap_none, sync_pps, 1 event, clock_sync,
system peer: PPS(0)
system peer mode: client
leap indicator: 00
stratum: 1
log2 precision: -21
root delay: 0.000
root dispersion: 2.015
reference ID: PPS
reference time: e0601d81.248b4b37 Tue, Apr 16 2019 10:23:13.142
system jitter: 0.000477
clock jitter: 0.003
clock wander: 0.001
broadcast delay: -50.000
symm. auth. delay: 0.000
If you aren’t seeing the settings its possible the NTP server is picking up the NTP information via DHCP which is overriding your settings above. Do this :
rm /etc/dhcp/dhclient-exit-hooks.d/ntp
rm /var/lib/ntp/ntp.conf.dhcp
At this point you have a NTP server which will use an external time source and use your local PPS to discipline it.
Note: Once you've configured ntp to use your GPS source, you'll need to stop the ntp service before running gpsctl, otherwise they'll both be trying to access ttyAMA0 at the same time.
GPS Offset Tuning
Your PPS time is going to be more accurate than NTP pool servers. Unless you have specialised equipment and local LAN GPS / DCF77 / MSF PPS sources to calibrate against, it’s not really possible to determine the appropriate PPS offset. It’s likely to be in the order of a few microseconds at most ( htt://lists.ntp.org/pipermail/questions/2011-September/030338.html ).
The PPS is a precise signal with around 10 ns of jitter. On the other hand, the offset of the GPS serial data output (GPSD) generally will have much more variation than the PPS because of the variables involved in sending data over an asynchronous serial port. However, the GPSD offset can be reduced somewhat by adjusting the GPSD reference clock fudge parameter time2 in the ntp.conf file.
The initial value used in our ntp.conf file is 60ms (0.060s). This means that the GPS ZDA packet arrives in the NTP daemon approximately 60ms after the PPS pulse corresponding to it. On our test device, we see a jitter of under 3ms on the GPS_NMEA clock.
We might need to adjust the GPS ntp.conf time2 (offset) value to get the NMEA time offset low enough that the offset isn’t great enough to confuse NTP as to which second the PPS pulse referred to.
This option can be used to compensate for a constant error. The specified offset (in seconds) is applied to all samples produced by the reference clock. The default is 0.000s.
Start with these ntp.conf settings, they should get you close enough to get everything working properly:
sudo nano /etc/ntpsec/ntp.conf
# PPS Driver refclock pps unit 0 minpoll 4 maxpoll 4 ppspath /dev/pps0 #flag3 Controls the kernel PPS discipline: 0 for disable (default), 1 for enable. #time1 PPS time offset # use local clock at stratum 10 if no others available tos orphan 10 tos mindist 0.002 # NMEA driver (/dev/gps0 and /dev/gpspps0) refclock nmea prefer minpoll 4 maxpoll 4 mode 8 stratum 2 baud 115200 time2 0.060 path /dev/ttyAMA0 #flag1 Disable PPS signal processing if 0 (default); enable PPS signal processing if 1. #flag2 If PPS signal processing is enabled, capture the pulse on the rising edge if 0 (default); capture on the falling edge if 1. #flag3 If PPS signal processing is enabled, use the ntpd clock discipline if 0 (default); use the kernel discipline if 1. #time1 PPS time offset #time2 NMEA time offset #mode bit 0 - process $GPMRC (value = 1) # bit 1 - process $GPGGA (value = 2) # bit 2 - process $GPGLL (value = 4) # bit 4 - process $GPZDA or $GPZDG (value = 8) # mode 9 = process only xxRMC and xxZDA NMEA records at 115200 baud #--------------------------- # pool.ntp.org maps to about 1000 low-stratum NTP servers. Your server will # pick a different set every time it starts up. Please consider joining the # pool: <https://www.pool.ntp.org/join.html> server 0.uk.pool.ntp.org iburst server 1.uk.pool.ntp.org iburst server 2.uk.pool.ntp.org iburst
To calculate the GPS offset we must disable GPS by placing a noselect in the ntp.conf "refclock nmea" line. We'll run the time server for a few hours and then compare the ntpq -p GPS offset to the average public time server offset. For accurate tuning use a bunch of known-good Stratum 1 servers in ntp.conf. Remember to remove the "noselect" after tweaking.
sudo systemctl stop ntpsec.service
sudo rm –f /var/log/ntpsec/*
sudo systemctl start ntpsec.service
Start ntpd and let it run for at least four hours. Periodically check progress with "ntpq -p" and wait until change has settled out.
Look for the row with GPS_NMEA(0) and refid GPSD. The offset values probably will be different for each
query. Note that the ntpq offsets are in milliseconds, but the peerstats file offsets and NTP’s time2 parameter are in seconds.
Calculate the average GPS offset (in seconds) using this script:
sudo nano ~/nmeaoffset
#!/bin/sh #
# Generate an estimate of your GPS's offset from a peerstats file
#
awk '
/NMEA/ { sum -= $5 ; cnt++; }
END { printf("%.6f\n", sum / cnt); }
' </var/log/ntpsec/peerstats
Then sudo chmod +x ./nmeaoffset
Run by typing
pi@ntp2:~ $ ./nmeaoffset
-0.001212
That’s within a few milliseconds, close enough for anyone as PPS is going to do its magic to give us accuracy within a few microseconds.
Adjust the "time2" value for the GPS source of your ntp.conf by adding the average offset from above.
sudo systemctl stop ntpsec.service
sudo rm –f /var/log/ntpsec/*
sudo systemctl start ntpsec.service
Repeat the procedure above until -5ms < offset < +5ms (or under 10ms, if that’s OK with you). When you’re done, remove the noselect from the "refclock nmea" line in ntp.conf.
If you decide to recalculate the average offset using the above procedures, wait at least another day or two.
Avoid unnecessarily changing the time2 value. A typical value for the Adafruit GPSD driver is +0.534 s when using its default 4800 baud interface and in the HAB Supplies/Uputronics GPSD it’s +0.060 s when using 115200 baud and the gpsctl --config_for_timing tweaks.
Automatically updating GPS leap seconds
These semi-annual changes will be made no later than 1 June and 1 December of each year to indicate what action (if any) is to be taken on 30th June and 31st December, respectively.
In Buster, the default ntp.conf points LeapFile at the tzdata-provided leap-seconds.list. That's fine, as long as you auto apply system updates via cron or manually on a regular basis.
Automatically updating Raspbian
To automatically update Raspbian on a schedule, follow these steps:
sudo apt install unattended-upgrades mailutils
sudo nano /etc/apt/apt.conf.d/50unattended-upgrades
Edit to include these sources (remove all others)
"origin=Raspbian,codename=${distro_codename},label=Raspbian"; "origin=Raspberry Pi Foundation,codename=${distro_codename},label=Raspberry Pi Foundation";
Unattended-Upgrade::Automatic-Reboot "true";
APT::Periodic::Update-Package-Lists "1"; APT::Periodic::Download-Upgradeable-Packages "1"; APT::Periodic::Unattended-Upgrade "1"; APT::Periodic::Verbose "1"; APT::Periodic::AutocleanInterval "7";
sudo dpkg-reconfigure --priority=low unattended-upgrades
Set the systemctl apt timer to fire at 2AM daily
[Timer] OnCalendar= # OnCalendar=DayOfWeek Year-Month-Day Hour:Minute:Second # see https://wiki.archlinux.org/index.php/Systemd/Timers OnCalendar=02:00 RandomizedDelaySec=0
Static IP and Hostname
If you want to fix your LAN IP you do it by running nmtui to configure your network.
Amend your hostname via raspi-config.
Further Reading
The original text this document was based on can be found at https://ava.upuaut.net/?p=951
Updated to include information from comments at https://blog.webernetz.net/ntp-server-via-gps-on-a-raspberry-pi/
David Taylor’s website https://satsignal.eu/ntp/Raspberry-Pi-NTP.html goes into much further detail about the process above and covers graphing, remote access monitoring.
Rob Robinette has a good write up on Pi-based NTP servers
https://robrobinette.com/pi_GPS_PPS_Time_Server.htm
Whitham D. Reeve’s GpsNtp-Pi Installation and Operation Guide
http://www.reeve.com/RadioScience/Raspberry%20Pi/GpsNtp-Pi.htm
Rich Laager’s Raspberry Pi 3 Stratum 1 NTP Server has the best stuff on offsets
https://coderich.net/2016/11/21/raspberry-pi-3-stratum-1-ntp-server/
Tech Solvency’s similar setup
https://www.techsolvency.com/ntp/systems/tackleberry-uputronics/
Jack Zimmerman’s Raspberry PI NTP Server LCD Display
https://github.com/jacken/Raspberry-Pi-ntp-server-LCD-display
Jack Zimmerman’s Pi U-Blox Stationary Mode
https://github.com/jacken/Raspberry-Pi-U-Blox-Stationary-Mode
GPSD Time Service HOWTO
https://gpsd.gitlab.io/gpsd/gpsd-time-service-howto.html
Tom Dilatush’s gpsctl
http://www.jamulblog.com/2017/11/paradise-ponders-gpsctl-functionally.html
My improved gpsctl with added .conf file support GitHub repo
https://github.com/philrandal/gpsctl
u-blox M8 Receiver Description (the bible)
GNSS Firmware 3.01 for u-blox M8
https://www.u-blox.com/sites/default/files/GNSS-FW3.01_ReleaseNotes_%28UBX-16000319%29_Public.pdf
Postscript, July 15th, 2019
You can enable either BeiDou satellites, or Glonass, but not both with gpsctl. You'll get an error if you try. Section 4.2 of the u-blox m8 receiver description manual has a hint as to why.
From version 1.5 of gpsctl, enabling either BeiDou or Galileo automatically enables NMEA 4.1.
Postscript, January 30th, 2020
Updated for the Raspberry Pi 4, used cpufrequtils to set processor speed, added nohz=off, fixed numerous typos, added automatic updating howto, and added a note about making sure that ntpd is not running when you run gpsctl.
Postscript, June 21st, 2020
Released gpsctl v1.9. Tried to fix an issue with gpsctl not initialising the u-blox module via systemd at startup. Still not convinced that it is right.
Added configure-rv3028.sh script to properly configure the rv3028 RTC chip on the new (June 2020) Uputronics boards.
Without this step I got lots of
kernel: [ 156.227408] rtc-rv3028 1-0052: Voltage low, data is invalid.
messages in /var/log/messages on boot.
To configure Raspberry Pi OS to use the rv3028,
sudo nano /boot/firmware/config.txt
Add the line
dtoverlay=i2c-rtc,rv3028
at the end.
Remove the fake hardware clock:
sudo apt-get -y remove fake-hwclock
sudo update-rc.d -f fake-hwclock remove
sudo systemctl disable fake-hwclock
Run sudo nano /lib/udev/hwclock-set and comment out these three lines:
#if [ -e /run/systemd/system ] ; then # exit 0 #fi
Also comment out the two lines
/sbin/hwclock --rtc=$dev --systz --badyear
and
/sbin/hwclock --rtc=$dev --systz
Configure the rv3028 by running
sudo configure-rv3028.sh
from the gpsctl archive.
Then,
sudo hwclock -w -v
to set the clock for the first time.
To debug, sudo systemctl disable ntp, then sudo reboot
After the reboot, sudo hwclock -r -v should show the retained time.
Then sudo systemctl enable ntp --now to enable and start ntp.
Occasionally, run
sudo hwclock --update-drift -w -v
to calibrate the RTC accuracy.
Postscript, October 17th, 2020
Due to the repeated unreliabilty of gpsctl's automatic baud rate synchronisation, I've reverted to using 9600 baud and not messing with the baud rate at all. The latest version in the github master branch allows you to specify the gpsctl parameters for ublox-init.service in /etc/default/gpsctl. e.g.
PARAMS=-q --configure-for-timing
If /etc/default/gpsctl doesn't exist, the behaviour will be as before.
Any edits of ublox-init.service or /etc/default/gpsctl require you to run sudo systemctl daemon-reload
This comment about unreliability of baud rate sync no longer applies - it all seems to work fine for me with current builds of Raspi OS and current kernels. Also tested on bookworm.
Postscript, October 19th, 2023
This blog post has been revised for Raspberry Pi OS Bookworm and ntpsec. The old udev rules are no longer needed as the PPS and GPS devices are now specified explicitly in ntp.conf.
Postscript, January 23rd, 2024
I've just found Conor Robinson's excellent blog series on how to build a Raspberry Pi NTP Server using a Uputronics HAT and gpsd. Worth a read and a revisit when Conor has filled in the blanks in Part 6 (optimisation).
Postscript, February 3rd, 2024
Another excellent build recipe, but for the Adafruit Ultimate GPS HAT etc: Raspberry Pi 3B+ NTP Server - Stratum 1 (with Adafruit Ultimate GPS HAT)
And for the Uputronics HAT: Raspberry Pi 5B NTP Server - Stratum 1 (with Uputronics GPS HAT)
Postscript, February 18th, 2024
Sad news. The original author of gpsctl, Tom Dilatush, passed away on February 14th.
It is with a heavy heart and deep sadness that we inform you of the death of our dear friend, Tom Dilatush, who passed away on Wednesday, February 14. Tom had suffered from ALS for more than a year and is now finally at peace. We will all miss him more than words can express.…
— Tom Dilatush, Sr. Wizard (@SlightlyLoony1) February 16, 2024
Friday, January 29, 2016
In the MoOde for streaming internet radio on the Raspberry Pi Zero (updated)
When I saw the announcement of the Raspberry Pi Zero my first thought was that it would be a good basis for a headless Internet Radio Streamer for my hifi system.
So when I got my Pi Zero, I looked around for a cost-effective Digital to Audio Converter and found Pimoroni's pHAT DAC for the Pi Zero.
After a bit of web searching and reading, I settled on Tim Curtis' MoOde as my Media Player.
I've updated the instructions which follow for Moode 2.6 TR1 and later.
So, I downloaded MoOde 2.6 TR1 and flashed it to a MicroSD card.
I had a spare TP-Link wifi USB stick which worked with the Pi, so to make that work I booted up MoOde with the TV and USB keyboard plugged in, and edited /etc/network/interfaces so it looked something like this:
auto lo iface lo inet loopback
#allow-hotplug eth0
#iface eth0 inet dhcp lan0 inet dhcp
allow-hotplug wlan0
iface wlan0 inet dhcp
wpa-ssid your-ssid
wpa-psk your-pre-shared-key
Change the ssid and PSK to suit your environment.
After a reboot, I could SSH into moode.local and had a web interface to play with.
For the pHAT DAC, I selected the hifiberry DAC as it uses the same chipset, and lo, I soon had internet radio streaming into my hifi.
But gosh, the low quality BBC radio streams sounded just awful.
A bit more web searching revealed the solution.
And so I installed minimserver and minimstreamer and access the Beeb's HLS streams via minimstreamer.
The process is:
1: install JDK 8.
sudo apt-get install oracle-java8-jdk
2: download MinimServer-0.8.4-linux-armhf.tar.gz and MinimWatch-0.8.4-linux-armhf.tar.gz from the minimserver download page.
3: extract as follows:
sudo tar zxvf MinimServer-0.8.4-linux-armhf.tar.gz -C /opt
sudo tar zxvf MinimWatch-0.8.4-linux-armhf.tar.gz -C /opt
4: configure minimserver and minimwatch
sudo /opt/minimserver/bin/setup
change the settings to not run at startup, and do the same for minimwatch
5: configure minimserver's media directory and enable the minimstreamer module in minimserver:
sudo /opt/minimserver/bin/startc
packages
install minimstreamer-0.5.24
exit
6: create a new file minimserver.service in /etc/systemd/system
sudo nano /etc/systemd/system/minimserver.service
[Unit] Description=MinimServer [Service] RemainAfterExit=true ExecStart=/opt/minimserver/bin/startd ExecStop=/opt/minimserver/bin/stopall [Install] WantedBy=network.target
7: Enable the newly created minimserver service
sudo systemctl enable minimserver.service
8: Create a BBCRadio.m3u file in minimplayer's media directory with contents as per here
9: Reboot your Raspberry Pi
10: Add individual stations via Moode's interface using urls like http://127.0.0.1:9790/minimstreamer/*/R1/
And then enjoy the BBC's 320kbps HLS streams via your Pi Zero and pHAT DAC.
MoOde has excellent support over on the DIYAudio forum
Sunday, January 18, 2015
H G Wells on Technological Unemployment
The concept of "Technological Unemployment" seems to be in fashion once again.
The sanest words I've read on the subject come from H. G. Wells in his 1914 work "The World Set Free":
He asked a passing stroller, and was told that the men had struck that day against the use of an atomic riveter that would have doubled the individual efficiency and halved the number of steel workers.
'Shouldn't wonder if they didn't get chucking bombs,' said Barnet's informant, hovered for a moment, and then went on his way to the Alhambra music hall.
Barnet became aware of an excitement in the newspaper kiosks at the corners of the square. Something very sensational had been flashed upon the transparencies. Forgetting for a moment his penniless condition, he made his way over a bridge to buy a paper, for in those days the papers, which were printed upon thin sheets of metallic foil, were sold at determinate points by specially licensed purveyors. Half over, he stopped short at a change in the traffic below; and was astonished to see that the police signals were restricting vehicles to the half roadway. When presently he got within sight of the transparencies that had replaced the placards of Victorian times, he read of the Great March of the Unemployed that was already in progress through the West End, and so without expenditure he was able to understand what was coming.
He watched, and his book describes this procession which the police had considered it unwise to prevent and which had been spontaneously organised in imitation of the Unemployed Processions of earlier times. He had expected a mob but there was a kind of sullen discipline about the procession when at last it arrived. What seemed for a time an unending column of men marched wearily, marched with a kind of implacable futility, along the roadway underneath him. He was, he says, moved to join them, but instead he remained watching. They were a dingy, shabby, ineffective-looking multitude, for the most part incapable of any but obsolete and superseded types of labour. They bore a few banners with the time-honoured inscription: 'Work, not Charity,' but otherwise their ranks were unadorned.
They were not singing, they were not even talking, there was nothing truculent nor aggressive in their bearing, they had no definite objective they were just marching and showing themselves in the more prosperous parts of London. They were a sample of that great mass of unskilled cheap labour which the now still cheaper mechanical powers had superseded for evermore. They were being 'scrapped'—as horses had been 'scrapped.'
Barnet leant over the parapet watching them, his mind quickened by his own precarious condition. For a time, he says, he felt nothing but despair at the sight; what should be done, what could be done for this gathering surplus of humanity? They were so manifestly useless—and incapable—and pitiful.
What were they asking for?
They had been overtaken by unexpected things. Nobody had foreseen——
It flashed suddenly into his mind just what the multitudinous shambling enigma below meant. It was an appeal against the unexpected, an appeal to those others who, more fortunate, seemed wiser and more powerful, for something—for INTELLIGENCE. This mute mass, weary footed, rank following rank, protested its persuasion that some of these others must have foreseen these dislocations—that anyhow they ought to have foreseen—and arranged.
That was what this crowd of wreckage was feeling and seeking so dumbly to assert.
'Things came to me like the turning on of a light in a darkened room,' he says. 'These men were praying to their fellow creatures as once they prayed to God! The last thing that men will realise about anything is that it is inanimate. They had transferred their animation to mankind. They still believed there was intelligence somewhere, even if it was careless or malignant.... It had only to be aroused to be conscience-stricken, to be moved to exertion.... And I saw, too, that as yet THERE WAS NO SUCH INTELLIGENCE. The world waits for intelligence. That intelligence has still to be made, that will for good and order has still to be gathered together, out of scraps of impulse and wandering seeds of benevolence and whatever is fine and creative in our souls, into a common purpose. It's something still to come....'
Wednesday, June 04, 2014
Deja Vu, Again
The Beeb's been at it again. Their finest journos have rehashed a press release from Premier Oil into yet another gushing news item about fossil fuels.
Just as in my "Here We Go Again" blog post from last June, there's no mention of climate change, not one word, apart from the "Climate" in the Department for Energy and (for) Climate Change's title.
Read it and weep.
Again.
Edited on: Wednesday, April 01, 2015 9:53 PM
Categories: Environment
Tweet
Sunday, October 13, 2013
Domestic LED lighting is Cost-effective Right Now
Of late I've been looking in the lighting sections of supermarkets in search of usable LED replacements for compact fluorescent and incandescent lamps for normal room lighting, but to no avail.
Until yesterday, that is, when I stumbled across this little beauty in my local Aldi store.
This is a Medion MD14536 11W LED lamp (equivalent to 75W incandescent, 18W compact fluorescent), 1055 lumens, 2700K colour temperature, rated lifetime of 35000 hours. Their light output is around 100 lumens per watt. And all for a princely £9.99. The only drawback is that they are not dimmable.
An equivalent Philips 18W compact fluorescent with a 6000 hour lifetime sells for a fiver on Amazon and gives around 60 lumens per watt.
10W (60W incandescent, 810 lumens) and 5.5W (40W incandescent, 470 lumens) are also available.
My experience of modern CFLs is that they rarely last their rated lifetime. I'm lucky if they get to half of it in my residence. CFLs I purchased over a decade ago were much more reliable.
LED lighting at this price is cost-effective right now. A real bargain!
Postscript, October 14th
My local Asda now has 12W (60W incandescent, 810 lumens, 25000 hour rated lifetime) LED lamps for £16. Not such a bargain.
Edited on: Wednesday, April 01, 2015 9:20 PM
Categories: Comment, Environment
Tweet
Friday, September 06, 2013
Scientific Illiteracy, Propaganda, or Both?
The Beeb are are being sloppy, ignorant, incompetent, or just plain malicious propaganda-spreaders once again.
A tweet by Greenpeace Nuclear @nukereaction alerted me this morning
Back in November, I commented on this nonsense:
The irony is that nuclear energy is the ultimate non-renewable, actually destroying matter.
Edited on: Wednesday, April 01, 2015 9:53 PM
Categories: Comment, Environment
Tweet