Satellite Validation Match-Up Tools (Portable)
These validation match-up tools allows Users to
- find relevant OB.DAAC satellite granules from in situ points or a SeaBASS data file and
- generate coincident satellite validation match-ups for those points.
These tools are downloadable for Users to run individually, external to the official SeaBASS
validation search. These tools are designed to replicate some of SeaBASS validation search's core functionality, with the exception that these tools do NOT adjust in situ data to water-leaving values.
Getting Started
The satellite validation match-up tools are command-line tools served as part of the SeaDAS software package, which can be downloaded and installed from
here. The SeaDAS software package does NOT buiild these tools by default, but they can be added via a few extra steps described below.
Install SeaDAS
Follow the directions for your system architecture listed here.
Additional installation steps
With SeaDAS installed and running:
- From the "OCSSW" dropdown menu, select "Install OC Processors."
- Under the "Others" section, check the "Source Code" box.
- Click "Run" which will create the "ocssw" source code directory within your SeaDAS installation directory.
- Add $OCSSWROOT to your shell environment.
- If you use BASH, execute these lines of code OR add them to .bashrc or .profile:
export OCSSWROOT="[path_to_seadas_installation_folder]/ocssw"
source $OCSSWROOT/OCSSW_bash.env
- If you are using a shell other than BASH, talk to your system admin.
Software components
The specific scripts to use for creating satellite validation match-ups are:
NOTE: The mk_matchup.py script uses this binary:
- $OCSSWROOT/bin/val_extract
Ensure that it is compiled and on your $PATH, by typing 'which val_extract' on the command line.
Satellite Data Finder
- an OB.DAAC satellite/instrument name and
- a latitude, longitude, and time point or range,
or a SeaBASS file with latitude, longitude, and date-time information as field.
Output
- a list of OB.DAAC L2 satellite file granule names that contain the input criteria, per the CMR's records.
- a list of granule download links to fetch the matching satellite file granules, per the CMR's records.
Input arguments
-h, --help show this help message and exit
--sat {modisa,modist,viirsn,goci,meris,czcs,octs,seawifs}
String specifier for satellite platform/instrument
Valid options are:
-----------------
modisa = MODIS on AQUA
modist = MODIS on TERRA
viirsn = VIIRS on NPP
meris = MERIS on ENVISAT
goci = GOCI on COMS
czcs = CZCS on Nimbus-7
seawifs = SeaWiFS on OrbView-2
octs = OCTS on ADEOS-I
--data_type {oc,iop,sst}
OPTIONAL: String specifier for satellite data type
Default behavior returns all product suites
Valid options are:
-----------------
oc = Returns OC (ocean color) product suite
iop = Returns IOP (inherent optical properties) product suite
sst = Returns SST product suite (including SST4 where applicable)
--slat SLAT Starting latitude, south-most boundary
If used with --seabass_file, will override lats in the file
Valid values: (-90,90N)
--elat ELAT Ending latitude, north-most boundary
If used with --seabass_file and --slat, will override lats in the file
Valid values: (-90,90N)
--slon SLON Starting longitude, west-most boundary
If used with --seabass_file, will override lons in the file
Valid values: (-180,180E)
--elon ELON Ending longitude, east-most boundary
If used with --seabass_file and --slon, will override lons in the file
Valid values: (-180,180E)
--stime STIME Time (point) of interest in UTC
Default behavior: returns matches within +/- MAX_TIME_DIFF (default +/-3 hours) about this given time
If used with ETIME, this creates a search time window, between STIME and ETIME.
Valid format: string of the form: yyyy-mm-ddThh:mm:ssZ
OPTIONALLY: Use with --max_time_diff or --etime
--max_time_diff MAX_TIME_DIFF
Maximum time difference between satellite and in situ point
OPTIONAL: default value +/-3 hours
Valid values: decimal number of hours (0-36)
Use with --seabass_file OR --stime
--etime ETIME Maximum time (range) of interest in UTC
Valid format: string of the form: yyyy-mm-ddThh:mm:ssZ
Use with --stime
--seabass_file SEABASS_FILE [SEABASS_FILE ...]
Valid SeaBASS file name or list of file names
File must contain latitude, longitude, and date-time information as fields.
--get_data GET_DATA Flag to download all identified satellite granules.
Requires the use of an HTTP request.
Set to the desired output directory.
Usage examples
fd_matchup.py --sat=modist --slat=23.0 --slon=170.0 --stime=2015-11-16T09:00:00Z --time_window=8
fd_matchup.py --sat=modist --stime=2015-11-15T09:00:00Z --etime=2015-11-17T09:00:00Z --slat=23.0 --elat=25.0 --slon=170.0 --elon=175.0
fd_matchup.py --sat=modist --time_window=4 --seabass_file=[your SB file name].sb
This script creates satellite extracts for a given
SeaBASS file, saving the values passing the specified exclusion criteria to a
SeaBASS file. This script applies Level-2 satellite match-up methodology of Bailey and Werdell (2006), with a few exceptions:
- User-defined inupts to mk_matchup.py will override recommended defaults
- The maximum allowed solar zenith angle used here is 70-deg vs the Bailey & Werdell recommended 75-deg
- The Rrs and AOT data are only in the OC L2 satellite product suite, if using standard OBPG-distributed products. Other file_types (SST, SST4, IOP, etc) will not evaluate any maximum coefficient of variation threshold.
- The maximum coefficient of variation theshold is derived by taking the median, in spectral-space, of the band-specific CVs, derived by dividing the filtered standard deviation by the filtered mean of the Rrs and AOT from the, nominally, 5 by 5 extract. This is done for all Rrs bands between 405nm and 570nm (omitting the 469 and 555 land bands from MODIS) and for all AOT between 860nm and 900nm.
- For all SST L2 satellite product suite data, the qual_sst_max or qual_sst_mean fields should be used to screen the sst value quality. The qual_sst value varies between 0 (best) and 4 (worst), where qual_sst_mean (qual_sst_max) is the mean (max) of the 5 by 5 pixel satellite extract.
Output
- the original SeaBASS data
- coincident satellite products as additional columns into --out_file
OR if --out_file is not specified, --seabass_file with "_matchups.sb" appended
REQUIRED input arguments
- --sat_file SAT_FILE [SAT_FILE ...]
REQUIRED: input OB.DAAC Level-2 satellite netCDF file(s)
- --seabass_file SEABASS_FILE [SEABASS_FILE ...]
REQUIRED: input SeaBASS file(s)
Must be a valid SeaBASS file, passing FHCHECK with no errors.
Matched-up satellite variables will be appended as additional fields to the data matrix and relevant headers.
File must contain latitude and longitude and date-time expressed as FIELD entries.
OPTIONAL input arguments
-h, --help show this help message and exit
--box_size BOX_SIZE OPTIONAL: box size of the satellite data extract made around the in situ point
Valid values are odd numbers between 3 and 11, default = 5
--min_valid_sat_pix MIN_VALID_SAT_PIX
OPTIONAL: percent minimum valid satellite pixels required to create an extract
Valid value: (0.0 - 100.0), default = 50.0
--max_time_diff MAX_TIME_DIFF
OPTIONAL: maximum time difference between satellite and in situ point
Valid value: decimal number of hours (0 - 36 hours), default = 3
--max_coeff_variation MAX_COEFF_VARIATION
OPTIONAL: maximum coefficient of variation of satellite pixels within the satellite extract
Valid value: (0.0 - 1.0), default = 0.15
--slat SLAT OPTIONAL: Starting latitude, south-most boundary
If used with --seabass_file, will override lats in the file
Valid values: (-90,90N)
--elat ELAT OPTIONAL: Ending latitude, north-most boundary
If used with --seabass_file and --slat, will override lats in the file
Valid values: (-90,90N)
--slon SLON OPTIONAL: Starting longitude, west-most boundary
If used with --seabass_file, will override lons in the file
Valid values: (-180,180E)
--elon ELON OPTIONAL: Ending longitude, east-most boundary
If used with --seabass_file and --slon, will override lons in the file
Valid values: (-180,180E)
--verbose OPTIONAL: Displays reason for failed matchup for each in situ target called.
--no_header_comment OPTIONAL: Flag to NOT append exclusion criteria to the OFILE header. Useful when running script repeatedly.
Notes on OPTIONAL input arguments
- --slat AND --slon must be used together and will override any latitudes and longitudes in --seabass_file
- --elat AND --elon must be used together and with --slon and --slat
These will override any latitudes and longitudes in --seabass_file and use a latitude and longitude bounding box instead of --box_size
Usage examples
mk_matchup.py --sat_file=[file name].nc --seabass_file=[file name].sb --out_file=[OPTIONAL, file name].sb
mk_matchup.py --sat_file=[file name].nc --seabass_file=[file name].sb --slat=45.3 --slon=-157.4
mk_matchup.py --sat_file=[file name].nc --seabass_file=[file name].sb --slat=45.3 --elat=48.7 --slon=-157.4 --elon=-145.3
References
Bailey, S.W., and Werdell, P.J. (2006). A multi-sensor approach for the on-orbit validation of ocean color satellite data products. Rem. Sens. Environ. 102, 12-23.