The poor state of HadSST
What follows is first art. I have yet to work out a better pictorial method. For now a tortured spreadsheet will do well enough.
You can just make out continental land masses…
Data: Met Office HadSST3
I’ve known for a number of years of the dreadful state of climatic datasets, HadSST being one of those but did not have the pictorial evidence. Period from 2005 is an arbitary choice arising during software development.
When I first looked at the gridded SST data, some time before 2010 I noticed what seemed to be a mix of monthly and annual in cells, with many missing data. It also looked very dubious on coastline handling. This was noted but nothing further done.
The story, a clue on what exactly I am doing.
A few days ago Doug Proctor posed one of his hooks on the Talkshop Suggestions page. This tripped a quick look and I reported back no safe result.
Background here, early 2010 I knocked up a software suite which translates format for a variety of published gridded dataset and stores them in a common format SQLITE database. Then extracts can be done in a common format. This clones the published mean temperature time series but computed here from gridded and also produces area maps and hovmoller maps. Probably a few other items. (before anyone asks, excluding GISS)
I reckoned adding to computing wmean (global and hemispheric) temperature computation it ought to be able to produce a data presence map as a side effect, provided speed is not too much of an issue, it isn’t.
The routine creates a blank grid based on the dataset gridding, filled with zero data counts. Then for each actual data found at a data cell it increments the data count for that lattitude/longitude.
The result is dumped out as area data. A new trivial program reads this dump, does what I want with it; right now just writes out CSV, as you see.
Doug mentioned RSS, this as expected but not shown here, indicates the expected no data over Andes, Greenland and Himalaya, plus the Arctic/Antarctic limits. The authors do not hide this information.
Contrast with HADSST where the authors mention ship tracks but are coy on the effect. The data for 1850 .. 1860, ouch.
Just spent 5 minutes automating decadal, ran it, all very quick. WordPress doesn’t allow CSV or TXT so I’ll import by hand as pages to a portable XLS
XLS spreadsheet of decadal counts, 1850..1859, 1860..1869 and so on, **file here** (304kB). Just data. If you do anything useful with it, tell us.
Results for HADCRUT3 are as expected, primarily HADSST. Land coverage is non too good. I’ve never looked at HADCRUT4 gridded.
At least this shows honesty in sticking to actual stations.
Whether the introduction of isolated data which has no comparative reference is questionable. RA Fisher would spin in his grave.
Since I have the means to produce coastline maps a vector plot with counts and grid lines is feasible.
I’ll think on how to produce a weighting for RSS data so that an attempt at a like for like time series can be done.
Other timespans, other datasets, ask, might be able to help.