Epica Vostok resampled composite

I won’t say much here now, busy.

This seems to confirm a huge date mistmatch between the two sets of ice core data.

This is a deliberately large plot. Contact me if you need data or help.

Composite of signal processing resampled data from two ice cores

Simplest way to provide data is an export to XLS format of work, warts and all.

Contains usable resampled data and originals

Added later, easiest way to provide data is export to XLS, is scruffy workfile

epica-1

Vostok ice core, part 2

See part 1 if you haven’t.

Seemed a good test to see if I could reproduce the temperature vs. CO2 lead lag result but using signal processing, data resampling. Turns out  the CO2 data is even worse than the isotope ratio temperature data, fewer data points and sampled at different dates.

Easy. I applied identical processing to both datasets and then figured out how to time shift one of them. To my surprise there is a very high correlation, r2=0.82, at least given the preprocessing used. The quick and dirty way to do the time shift was apply an offset at the decimate stage, simply picks off data at a different point. (this is valid)

If I have done this right it is about 1,500 years for best fit of rise and fall. I then aligned the datasets and plotted (Y axis reacaled and offset CO2 by hand so the data roughly matches on one scale)  for an eyeball.

Time co-incident plot of temperature and CO2

There is obviously a lot going on but there it is visibly on one plot.

A net dig shows a work by Jo Nova (know the name, no idea who she is)

http://joannenova.com.au/global-warming/ice-core-graph/

That says 800 years and seems to cite others.

Ref

http://www.ncdc.noaa.gov/paleo/metadata/noaa-icecore-2453.html

ftp://ftp.ncdc.noaa.gov/pub/data/paleo/icecore/antarctica/vostok/co2nat.txt

Vostok ice core, part 1

An ongoing development is better handling irregularly sampled data. This is a very hard problem with no pure solution.

After a lot of investigation and experimentation I have concluded that NDFT/NDFT are of little use, solve nothing, kicks straight back into the input data must be good. Usually involved is approximating and other heuristics.

Looks hard. Run away.

Raw data overlaid with a resampled dataset

For what I am doing a good solution is fix up the dataset using signal processing, kind of trivial, although it will seem black magic to outsiders. (why no blue, pink, white, transparent magic?)

A key is keeping the human brain in the loop, each case is likely to be different with no one size fits all.

I’ve coded up the hard part for a human as an extension of one software package.

Seems to work nicely, as above, the test dataset. Vostok original data sampling ranges from 60 through 600 years.

An XLS with the original data and resample dataset is here vostoke-temperature-a

Reference

 http://www.ncdc.noaa.gov/paleo/metadata/noaa-icecore-2453.html

 ftp://ftp.ncdc.noaa.gov/pub/data/paleo/icecore/antarctica/vostok/deutnat.txt

The dataset is now clean and trivial for normal tools.

And for those who like first difference…

Hiding anything? Nope. The 1st diff does of course have high frequency noise but is surprisingly small. Only clear term in the part not shown is ~4044y. No idea what that is if anything.

More to be done, always is.

Length of Day, modelling the lunar and annual effect

As a result of helping someone out with lunar effects in Earth length of day I wondered if a slightly more comprehensive version would work.

The result  is an interactive spreadsheet which might be useful.

I would not usually  produce such a monster, this is an xls >30Mbyte but it does include reconstruction of the lunar LoD signal, subtraction from the raw LoD and decimation for plotting. It is live, you can turn on or off individual terms/factors and see the effect.

This needs a great deal of explanation, codas, and so on.

For now here is the file lod-work

Do not try this unless you have a fairly large computer. Checked it works with gnumeric, openoffice/libreoffice. Excel should not have a problem but recursion is used.

Circles and dimensions

Gold appeared during a detailed technical web search, complete wrong context for what I wanted but one of the strangest happened upons ever for me.

I will present the cross check first, this has a very real basis.

The Wiltshire Heritage Museum is based in Devizes, Wiltshire. Not so far from here and now I must visit when the opportunity arises. I know the henge area, where my father grew up, know this from the times where our heritage was ours and not stolen by the exploiters.

http://www.wiltshireheritage.org.uk/

A bizarre artefact held Is the Bush Barrow “lozenge”

http://www.wiltshireheritage.org.uk/galleries/index.php?Action=3&obID=89&prevID=9

A new zealander has written his interpretation of what this is

http://www.celticnz.co.nz/BBLOZ/BBLOZWEB1.htm

Enjoy.

Out of politeness here is his home page where there is a lot more in articles

http://www.celticnz.co.nz/

ERBS TSI

As part of an ongoing investigation I looked at the ERBS TSI data. This dataset is not particularly interesting and  as with all satellite data is far too short to say much. One interesting snippet did appear.

Creating a rough model posed some problems but in practice was simple.

First was the data has a Y2K corrupted date which was kindly sorted out by V.

The sampling is irregular (scattered time points) which the software can usually handle if slowly. The result is unremarkable.

I’ve shown a fore and hindcast which indicates the model is stable and sane. It will be somewhat wrong.

A surprise came when I looked at a paper associated with the dataset where the final sentence of the conclusion is “The fact that the measurements increased with time relative to the proxies  suggests the existence of a second TSI variability component with an amplitude greater than 0.04% (0.5 Wm-2), and with a period greater or equal to approximately 20 years.”
A minor model component: 20.0432969909    3.69480925972    0.124480125667

Period just over 20 years, amplitude 0.125 * 2 * sqrt(2) = 0.35W p-p

With such short data that will be way out and TSI certainly is far more complex in the long term.

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.81.834&rep=rep1&type=pdf

I’ve shown SORCE as well. This is not really TSI, is narrowband/is in my opinion incomplete [disputed, see comments]. It does however show the wide variation is measured values. Given the extreme difficulty in making these measurements the usual rule of thumb is sensible, the instrument must be at least an order more accurate and so for absolute maybe we know +-40W sq/m

Putting that into context 40W in 1360W is +-3% of absolute, not as easy as many will imagine and for a remote instrument is an intensely hostile environment would not be a surprise. Lets hope the instrument is returned to earth for post mission calibration checks. See the point?

“Validation of spacecraft active cavity radiometer total solar irradiance  [TSI] long-term measurement trends using proxy TSI least squares analyses

“aRobert Benjamin Lee III and bRobert S. Wilson
aNASA Langley Research Center, Atmospheric Sciences, MS 420, Hampton, VA 23681-2199
Science Application International Corp. (SAIC), One Enterprise Parkway, Hampton, VA 23666 “

http://www.acrim.com/

A lot of interesting material on TSI  can be found here http://www.leif.org/research/

RSS dataset update v3.2 to v3.3

So what is the difference?

Visually rendering shows a cleaner result, presumably a reduction in artefacts from the close to the knuckle high frequency response.

Not worked on time series.

Here we are for December 2010 (no old data version of Jan 2011)

December 2010 V3.2

December 2010 V3.3