OPENDAP Overview



OPeNDAP is a data connection tool that can be used within your favorite software program (MATLAB/PYTHON/R/IDL/EXCEL) to download the subsets of the MACA datasets. Some important steps in getting OPeNDAP to work for you are provided here:
  • Identifying the OPENDAP url for your files
  • Checking your software compatibility with OPENDAP
  • Example code for accessing data through OPENDAP
  • Identifying OPeNDAP URL

    In your software program, you will need to identify the OPeNDAP URL as a path to your files. The OPeNDAP URL is different than the URL you would use to download the file.


    STEP 1: navigate to the THREDDS catalog for your data

    See data catalogs. For OPeNDAP, you should use the 'Aggregated' files in this table, as they have longer year ranges(i.e.1950-2005 or 2006-2099) than the individual files (which only have year ranges of 5-10 years)

    STEP 2: find a desired file and view the record for it

    For example, here is a page from a data catalog for a file:


    STEP 3: find the path for the THREDDS server name



    STEP 4: find the THREDDS file name and directory

    The OPeNDAP directory and filename is listed under 'OPeNDAP':

    STEP 5: Form the OPeNDAP URL

    Concatenate the server name and file directory and name together to form the OPeNDAP URL.
    For example, from the screenshot it would be:

    myFile='http://thredds.northwestknowledge.net:8080//thredds/dodsC/macav2livneh_huss_BNU-ESM_r1i1p1_historical_1950-2005_CONUS_daily_aggregated.nc'

    Note: these filenames have since changed slightly so that the current correct filename is actually myFile='http://thredds.northwestknowledge.net:8080//thredds/dodsC/macav2livneh_huss_BNU-ESM_r1i1p1_historical_1950_2005_CONUS_daily_aggregated.nc'

    STEP 6: Double Check URL

    If you want to double check the URL you have formed, you can click on the link under OPeNDAP URL and look at the OPeDAP Dataset Access Form. There you will find the OPeNDAP URL in the box labeled 'DATA URL':

    OPeNDAP: check software setup

    • Check that you have an internet connection on your computer
    • Open your software program(MATLAB,python,R,IDL,Excel,etc)
    • Check that your software program has OPeNDAP support.
      • MATLAB (v2012a and later) has built-in OPeNDAP support
      • R includes support through the netCDF interface. However, the netcdf packages for Windows are not yet opendap-enabled. If using R, the ncdf4 package must be installed.The ncdf4 package can be downloaded at http://cirrus.ucsd.edu/~pierce/ncdf/
      • Python includes support through the netCDF/pydap libraries.
      • IDL(8-4 and later) include OPeNDAP support.
    • Find the OPeNDAP URL for one of our files and test that you can view some of the metadata for this file through OPeNDAP. For example,
      • myFile='http://thredds.northwestknowledge.net:8080/thredds/dodsC/agg_macav2metdata_huss_BNU-ESM_r1i1p1_historical_1950_2005_CONUS_daily.nc';
      • Matlab: ncdisp(myFile);
      • Python: from netCDF4 import Dataset; filehandle=Dataset(myFile,'r',format="NETCDF4");filehandle.variables
      • R: 1) library(ncdf4) 2) nc <- nc_open(myFile) 3) nc
    • If your OPeNDAP connection is successful, you should see metadata for the filename displayed in your software program.

    OPeNDAP Examples in MATLAB

    %Filename: OPeNDAPExample_TimeSeries_macav2metdata_SimpleExample.m
    %Author: Katherine Hegewisch (khegewisch@uidaho.edu)
    %Updated: 03/12/2017
    %Description: This script uses OPeNDAP to download the specified subset of the MACAv2-METDATA data
    %Requirements: This MATLAB script is run using MATLAB R2012a (which has native OPeNDAP support)
    % Older matlab versions need to get OpenEarthTools
    % For more information on using OPeNDAP in Matlab,
    % see http://www.mathworks.com/help/matlab/ref/ncread.html
    %=============================================
    % SET TARGET DATA
    %=============================================
    %day =1;
    lat_target=45.0;
    lon_target=-103.0+360;
    %=============================================
    % SET OPENDAP PATH
    %=============================================
    pathname = 'http://thredds.northwestknowledge.net:8080/thredds/dodsC/agg_macav2metdata_tasmax_BNU-ESM_r1i1p1_historical_1950_2005_CONUS_daily.nc';
    %Look at the contents of the file

    ncdisp(pathname)
    %=============================================
    % GET DATA SIZES
    %=============================================
    timeinfo = ncinfo(pathname,'time');
    timeSize =timeinfo.Size;
    loninfo = ncinfo(pathname,'lon');
    lonSize =loninfo.Size;
    latinfo = ncinfo(pathname,'lat');
    latSize =latinfo.Size;
    vinfo = ncinfo(pathname,'air_temperature'); %in tasmax file, the variable is called 'air_temperature'
    vSize =vinfo.Size;
    % the dimensions are:lon,lat,time
    %=============================================
    % GET DATA
    %=============================================
    lat =ncread(pathname,'lat');
    lon =ncread(pathname,'lon');
    %=============================================
    %find indices of target lat/lon/day
    [m,lat_index] = min(abs(lat-lat_target));
    [m,lon_index] = min(abs(lon-lon_target));
    lat=lat(lat_index);
    lon=lon(lon_index);
    %=============================================
    %extract time data
    time =ncread(pathname,'time');
    time_index = 1: length(time);
    %see what these dates correspond to...
    time = double(time); %necessary for next command
    [Y M D] =datevec(time+datenum(1900,1,1)); %the time variable says the units are 'days since 1900-01-01 00:00:00'
    %=============================================
    %data has 3 dimensions: time, lat,lon
    start=[lon_index(1) lat_index(1) time_index(1)];
    count=[length(lon) length(lat) length(time)];
    stride=[1 1 1]; %every day of data
    data=squeeze(ncread(pathname,'air_temperature',start,count,stride));
    %=============================================
    % MAKE A PLOT
    %=============================================
    yearref=1950;

    %plot only days in 1950
    f=find(Y==1950);
    time=time(f);
    days = 1:length(time);

    OPeNDAP Examples in R

    OPeNDAP Examples in IDL

    ;Filename: OPeNDAPExample_TimeSeries_SimpleExample.idl
    ;Author: Katherine Hegewisch (khegewisch@uidaho.edu) from John Mejia's Advanced Example
    ;Updated: 02/10/2015
    ;Description: This script uses OPeNDAP to download the specified subset of a MACAv2-METDATA file
    ;Requirements: This IDL script is run using IDL version 8.4. IDL version 8.2 didn't work with OPeNDAP.
    ;=============================================
    ; SET TARGET DATA
    ;=============================================
    ;day =1;
    lat_target=34.949852;
    lon_target=-117.887785+360;
    ;=============================================
    ; SET OPENDAP PATH, GET FILENAMES FROM CATALOG
    ;=============================================
    urlnetcdf = 'http://inside-dev1.nkn.uidaho.edu:8080/thredds/dodsC/agg_macav1metdata_huss_BNU-ESM_r1i1p1_historical_1950_2005_WUSA.nc'; %this is for macav1-metdata only cdfid = ncdf_open(urlnetcdf)
    ;Look at the contents of the file
    inq= ncdf_inquire(cdfid)
    ;=============================================
    ; GET LAT/LON/TIME
    ;=============================================
    ncdf_varget, cdfid, 'lon', xlon
    ncdf_varget, cdfid, 'lat', xlat
    ncdf_varget, cdfid, 'time', time
    ;=============================================
    ; FIND INDICES of CLOSEST GRID
    ;=============================================
    ;find indices of target lat/lon/day
    dist=abs(lat0-xlat)
    y0=where(dist eq min(dist))
    dist=abs(lon0-xlon)
    x0=where(dist eq min(dist))
    ;=============================================
    ; GET DATA
    ;=============================================
    ;data has 3 dimensions: lon,lat,time
    ncdf_varget, cdfid, mainvar,var,COUNT=[1, 1, n_elements(time)], OFFSET=[x0, y0, 0]
    ;=============================================
    ; CLOSE FILE
    ;=============================================
    ncdf_close,cdfid
    ;=============================================