Update to MATLAB code reading PSMSL zip files

2 August 2023

Several users of PSMSL data have reported issues with the matlab functions included in our zip files that read in the entire archive - we're glad to announce that we've now fixed the issue. The amended versions of the scripts will be included in future zip files, and are included at the bottom of this page. These functions will also work on previous versions of the zip file, but users should be aware we've also made some other modifications that could have a minor impact on your code.

The main bug fix

The issue occurred when the script tried to read in the data catalogue file included in the data directory (filelist.txt). This inventory includes the two fields that make up the old identifier we used to use for the sites: the coastline code and the old station code. For example, the site identified now using 202, Newlyn, was previously identified using the composite code 170/161.

Maintaining these old identifiers was quite time consuming, as they were roughly intended to follow coastlines, and we'd recently stopped allocating them to newly added sites. These missing values were replaced in filelist.txt with XXX. Unfortunately, the MATLAB code expected an integer rather than text, causing it to fail. The code now expects three characters and tries to convert them to a number, and will replace XXX with NaN for the new sites where no old identifiers exist.

Change to time variable

The "time" variable used in the monthly files wasn't correctly coded to provide the MATLAB datenum for the centre of the month, as described in the documentation, but was actually providing a number half a day earlier (so 0.5 too small). If you rely on the value of "time", you'll find all values have become 0.5 bigger.

Change to flag handling for MTL data in RLR files

We discovered for the functions reading RLR data, data recorded as mean tide level (MTL) was not being assigned the correct quality flag (the "dataflag" variable). The code assigned a dataflag value of 1 (indicating this data point should be treated with caution) to any value where the flag variable was not 000. This meant otherwise good data that was measured as MTL but adjusted to MSL using the methods described in the previous link was being flagged, for example, the period of 1807-1835 in the series at Brest.

We've fixed this issue by following the pattern used in the metric file reader, so the "dataflag" variable now only uses the final character flag for attention. MTL data is marked by the new variable "isMtl".

Remarks

Note that while we've fixed this code, it's still rather old, and uses some functions MATLAB no longer recommends (e.g. datenum), and doesn't take account of recent classes added to MATLAB to handle timeseries data (e.g. timetable). If created today, it would probably look quite different. We hope to provide newer code samples and data access methods in the future to make it easier for users to get our data and metadata into MATLAB, Python, R, Excel, or whatever software you use.

New files

Monthly metric: readMetricMonthly.m
Monthly RLR: readRlrMonthly.m
Annual RLR: readAnnual.m

About Us