Demonstration of fractional delay function on real data
The meteorological station at Armagh Observatory, Northern Ireland closed in 2000. A lot of the data has been made online access via scanned logbooks and some digitised data, paid for primarily by lottery funds.
An unpublished version of the data is used as part of a fractional delay demonstration.
Earlier article providing a template and instructions is
The digitised data is not particularly accurate, containing mistakes and adjustments which are largely undocumented. I have corrected some of the mistakes but not so far fed all of this back.
One I have worked on but never shown is reworked soil temperature data wherein lies a tale.
The data is not very good over time with various causals, eg. a soil pipe filling un-noticed with earth, changing the depth.
Of note here the published data has been incorrectly adjusted where the intent was correction of an arbitrary decision in 1970 to change the measured depth to metric: 1 foot to 100mm and 4 feet to 1 metre.
It was considered to have a negligible effect on the 1 foot data but all pre-1970 data was altered and is covered in a published paper… except that omits to tell the whole story of what exactly was altered.
Fortunately by the very rare chance of general data availability a casual paper is available, less guarded which contains sufficient information for me to realise the adjustment is technically totally incorrect, meteorologists apparently not understanding heat flow.
I have been able to reverse engineer the published data to the original before this adjustment.
The ground acts as a bidirectional thermal delay line. Adding 0.1C or whatever to temperatures during a roughly summer period only as a compensation for depth change is wrong.
Soil data is unusual from several points of view.
the ground between the surface and sensing depth acts as a low pass filter, more nearly provides an integrating measurement so Nyquist is better met.
Stephan-Boltzmann connects with a surface, air is arbitrary.
On the other hand the equipment pre-modern fixed electronic probes is poor, such was bulb in wax ball, no temperature hold and has to be removed for measurement. This leads to reading errors.
The surface has to remain constant, often not the case, here for example there are problems over a path and over an iron soil pipe.
Debris is liable to fall into the tube.
Soil/ground temperature is a much better measure of planetary temperature than air temperature provided the measurements are accurate. Very few soil temperature datasets are available and most are in a terrible state. The better ones seem to be withheld.
I’m using a short portion of the whole data, from the start in 1904 through 1914 or so, 10 years. Data is daily but I computed monthly using the standard crude meteorological method. This is ideal for deducing a time delay smaller than the sample period and represents a clone of a typical real problem.
I used the XLT file published separately as a starting point, dropped the new data in place etc., added correlation, a control, and plots.
See figure 1.
You could experimentally delay for a whole year or two but this bring in other data variation so the match is worse. (shorten the sense period… end of dataset I warned about)
I’ve removed the control attached to cell D14 (non-portable) and here is the file as an XLS.
Have a play.
1. An AWS at a slightly different location was introduced but a fatal mistake was made by service personnel. Later a new AWS was installed and runs today. All AWS data is marked uncalibrated.
Soil data is not equivalent, the old site instead of being protected as SSSI was instead destroyed when a scented garden was built and the area covered in gravel etc.
http://star.arm.ac.uk/climate.html met site was behind the foreground tree.