OPeNDAP
   HOME

TheInfoList



OR:

OPeNDAP (Open-source Project for a Network Data Access Protocol) is an endeavor focused on enhancing the retrieval of remote,
structured data A data model is an abstract model that organizes elements of data and standardizes how they relate to one another and to the properties of real-world entities. For instance, a data model may specify that the data element representing a car be ...
through a
Web-based A web application (or web app) is application software that is created with web technologies and runs via a web browser. Web applications emerged during the late 1990s and allowed for the server to dynamically build a response to the request, ...
architecture and a discipline-neutral Data Access Protocol (DAP).


Project

Widely used, especially in
Earth science Earth science or geoscience includes all fields of natural science related to the planet Earth. This is a branch of science dealing with the physical, chemical, and biological complex constitutions and synergistic linkages of Earth's four spheres ...
, the protocol is layered on
HTTP HTTP (Hypertext Transfer Protocol) is an application layer protocol in the Internet protocol suite model for distributed, collaborative, hypermedia information systems. HTTP is the foundation of data communication for the World Wide Web, wher ...
, and its current specification is DAP4, though the previous DAP2 version remains broadly used. Developed and advanced (openly and collaboratively) by the non-profit OPeNDAP, Inc., DAP is intended to enable remote, selective data-retrieval as an easily invoked Web service. OPeNDAP, Inc. also develops and maintains zero-cost (reference) implementations of the DAP protocol in both server-side and client-side software. "OPeNDAP" often is used in place of "DAP" to denote the protocol but also may refer to an entire DAP-based data-retrieval architecture. Other DAP-centered architectures, such as THREDDS and ERDDAP, the NOAA GEO-IDE UAF ERDDAP exhibit significant interoperability with one another as well as with systems employing OPeNDAP's own (open-source) servers and software. A DAP client can be an ordinary browser or even a
spreadsheet A spreadsheet is a computer application for computation, organization, analysis and storage of data in tabular form. Spreadsheets were developed as computerized analogs of paper accounting worksheets. The program operates on data entered in c ...
, though with limited functionality (see OPeNDAP's Web page o
Available Client Software
. More typically, DAP clients are: * Data-analysis or data-visualization tools (such as
MATLAB MATLAB (an abbreviation of "MATrix LABoratory") is a proprietary multi-paradigm programming language and numeric computing environment developed by MathWorks. MATLAB allows matrix manipulations, plotting of functions and data, implementat ...
, IDL,
Panoply A panoply is a complete suit of armor. The word represents the Ancient Greek ('), where the word (') means "all", and (') means "arms". Thus, ''panoply'' refers to the full armor of a hoplite or heavily-armed soldier, i.e. the shield, breast ...
,
GrADS The Grid Analysis and Display System (GrADS) is an interactive desktop tool that is used for easy access, manipulation, and visualization of earth science data. The format of the data may be either binary, GRIB, NetCDF, or HDF-SDS (Scientific ...
, Integrated Data Viewer,
Ferret The ferret (''Mustela furo'') is a small, domesticated species belonging to the family Mustelidae. The ferret is most likely a domesticated form of the wild European polecat (''Mustela putorius''), as evidenced by the ferret's ability to inter ...
and ncBrowse) which their authors have adapted to enable DAP-based data input; * Similarly adapted Web applications (such as Dapper Data Viewer, aka DChart) * Similarly adapted end-user programs (in common languages) Regardless of their types, and whether developed commercially or by an end-user, clients almost universally link to DAP servers through ''libraries'' that implement the DAP2 or DAP4 protocol in one language or another. OPeNDAP offers open-source libraries in C++ and
Java Java is one of the Greater Sunda Islands in Indonesia. It is bordered by the Indian Ocean to the south and the Java Sea (a part of Pacific Ocean) to the north. With a population of 156.9 million people (including Madura) in mid 2024, proje ...
, but many clients rely on community developed libraries such a
PyDAP
or, especially, the
NetCDF NetCDF (Network Common Data Form) is a set of software libraries and self-describing, machine-independent data formats that support the creation, access, and sharing of array-oriented scientific data. The project homepage is hosted by the Unidat ...
suite. Developed and maintained by the Unidata Program at the UCAR in multiple programming languages, all
NetCDF NetCDF (Network Common Data Form) is a set of software libraries and self-describing, machine-independent data formats that support the creation, access, and sharing of array-oriented scientific data. The project homepage is hosted by the Unidat ...
libraries include embedded capabilities for retrieving (array-style) data from DAP servers. A data-using client references a
data set A data set (or dataset) is a collection of data. In the case of tabular data, a data set corresponds to one or more table (database), database tables, where every column (database), column of a table represents a particular Variable (computer sci ...
by its URL and requests metadata or content by issuing (usually through an embedded DAP library) an HTTP request to a DAP server. Content requests usually are ''preceded'' by requests for metadata describing the structure and other details about the referenced data set. With this information, the client may construct DAP constraint expressions to retrieve specific content (i.e., subsets) from the source. OPeNDAP servers offer various types of responses, depending on the specific form of the client's request, including XML, JSON, HTML and ASCII. In response to requests for ''content'', OPeNDAP servers can respond with multi-part mime documents that include a binary portion with
NetCDF NetCDF (Network Common Data Form) is a set of software libraries and self-describing, machine-independent data formats that support the creation, access, and sharing of array-oriented scientific data. The project homepage is hosted by the Unidat ...
or DAP-native encoding. (These binary forms offer compact means to deliver large volumes of content, and the DAP-native form may even be streamed if desired.) OPeNDAP's software for building DAP servers (on top of
Apache The Apache ( ) are several Southern Athabaskan language-speaking peoples of the Southwestern United States, Southwest, the Southern Plains and Northern Mexico. They are linguistically related to the Navajo. They migrated from the Athabascan ho ...
) is dubbed Hyrax and includes ''adapters'' that facilitate serving a wide variety of source data. DAP servers most frequently enable (remote) access to (large) HDF or NetCDF files, but the source data can exist in databases or other formats, including user-defined ones. When source data are organized as files, DAP retrievals enable, via
subsetting In research communities (for example, earth sciences, astronomy, business, and government), subsetting is the process of retrieving just the parts (a subset) of large files which are of interest for a specific purpose. This occurs usually in a clien ...
, finer-grained access than does the
FTP The File Transfer Protocol (FTP) is a standard communication protocol used for the transfer of computer files from a server to a client on a computer network. FTP is built on a client–server model architecture using separate control and dat ...
. Furthermore, OPeNDAP servers can aggregate subsets from multiple files for delivery in a single retrieval. Taken together, subsetting, aggregation and streaming can yield substantial data-access efficiencies, even in the presence of slow networks. OPeNDAP and other DAP servers are used operationally in government agencies, including
NASA The National Aeronautics and Space Administration (NASA ) is an independent agencies of the United States government, independent agency of the federal government of the United States, US federal government responsible for the United States ...
and
NOAA The National Oceanic and Atmospheric Administration (NOAA ) is an American scientific and regulatory agency charged with forecasting weather, monitoring oceanic and atmospheric conditions, charting the seas, conducting deep-sea exploratio ...
, for providing access to
Earth science Earth science or geoscience includes all fields of natural science related to the planet Earth. This is a branch of science dealing with the physical, chemical, and biological complex constitutions and synergistic linkages of Earth's four spheres ...
data, including satellite imagery and other high-volume information sources. The DAP data model embraces a comprehensive set of data structures, including multidimensional arrays and nested sequences (i.e., records), complemented by a correspondingly rich set of constraint expressions. Hence the OPeNDAP data-retrieval architecture has demonstrated utility across a broad range of scientific data types, including data generated via simulations and data generated via observations (whether remotely sensed or measured in situ).


References

{{Reflist


External links


OPeNDAP.orgTutorial on using OPeNDAP for data access at PO.DAAC (NASA's Distributed Active Archive Center for Physical Oceanography)
* ttps://web.archive.org/web/20141230230936/http://www.epic.noaa.gov/epic/software/dapper/ dapper - OPeNDAP server for in-situ databr>DChart - Web viewer for NOAA Observing System data (in-situ data)

ncBrowse - Java viewer for OPeNDAP netCDF files (supports wide range of netCDF conventions)netCDF Explorer
- netCDF Explorer is multi-platform graphical browser for netCDF files. netCDF Explorer can browse files locally or remotely, by means of OPeNDAP
NCAR Command Language - analysis and visualization softwareFerretPydap - client/server implementation written in Python
* ttps://web.archive.org/web/20011116220110/http://gcmd.gsfc.nasa.gov/Data/portals/dods/ NASA GCMD OPeNDAP portal Global Change Master Directory (GCMD)
Asia-Pacific Data Research Center -- a textbook example OPenDAP implementation
Earth sciences Web services Data analysis Technical communication Meteorological data and networks Earth sciences data formats