Saturday, November 17, 2018

Recent updates to OPUS (new OPUS ID format!)



  • There is now a link for Questions and Feedback in the top menu bar. Please use it!
  • The format of the OPUS ID (used to uniquely identify observations) has changed. The previous format was incompatible with the upcoming migration to PDS4. The new OPUS ID format should stay consistent for the foreseeable future. The old OPUS ID format is no longer supported. Some examples of the change:
        Old: COISS_1001-1294561143_1295221348-W1294561143_1
        New: co-iss-w129456114
        Old: NHLALO_x001-20060224_000310-lor_0003103486_0x630
        New: nh-lorri-lor_0003103486
  • We have removed the short-lived search fields Spatial Sampling, Wavelength Sampling, and Time Sampling and returned to a simpler way of describing observations called Observation Type. Current choices include:
    • Image (a 2-D array of intensity values)
    • Spectrum (a 1-D array of intensity vs. wavelength)
    • Spectral Image (similar to Image, but a grating or prism has been used to spread out the spectrum along one of the axes)
    • Spectral Cube (a 3-D array, similar to an Image but with each pixel value containing an entire spectrum)
    • Time Series (a 1-D array of intensity or optical depth vs. time, such as for an occultation)
    • Spectral Time Series (similar to Time Series, but containing a spectrum for each time step)
  • The Primary File Spec for Galileo SSI observations has been changed. The original format, which looked like "GO_0017:[J0.OPNAV.C034640]5900R.IMG", was based on an ancient filesystem constraint that required the actual filename to be split across multiple directories because it was too long. This resulted in the final filename, such as "5900R.IMG", being non-unique. We have changed the format to have a more modern, non-restricted format such as "GO_0017/J0/OPNAV/C0346405900R.IMG", where each filename is now unique.
  • When downloading observations that have been selected, the available products are now categorized and sorted by their product type, such as "Metadata Products", "Browse Products", and "Mission-Specific Products".
  • The definition of Planet has been changed for Cassini and New Horizons instruments. Previously, Planet was based on the Intended Target Name, so that an observation of Venus was marked as planet Venus. Now, consistent with other missions and instruments, the Planet field indicates approximately where the spacecraft was when the observation was taken. For most observations, this is a good starting point when looking for images of that particular planet, but if you want to be more specific use the Intended Target Name or Surface Geometry Target Name. The exception to this is the Hubble Space Telescope, which is always around Earth. In this case, the Planet field is still based on the Intended Target Name.
  • The definition of Measurement Quantity for Cassini UVIS has been changed. The choice between Optical Depth and Emission is no longer dependent on the instrument mode, but instead depends on the textual Note included with the observation. Any observation with a Note containing "OCCULTATION" is marked as Measurement Type Optical Depth. All other observations are marked as Emission.
  • We have added descriptive text to help users understand the difference between Intended Target Name and Surface Geometry Target Name.
  • For added clarity, we have renamed some targets:
    • J Rings → Jupiter Rings
    • S Rings → Saturn Rings
    • U Rings → Uranus Rings
    • N Rings → Neptune Rings
    • J Minor Sat Jupiter Minor Satellites
    • Cal Lamps Calibration Lamps
  • It is now possible to right-click on the View Detail link anywhere it appears and open OPUS in a new browser tab or window with the Detail tab selected.
  • Several annoying bugs were fixed. It is once again possible to sort in the table view on a field that contains multiple choices. There was a 3-order-of-magnitude error in the Observation Duration for certain instruments. Some HST NICMOS aperture names were listed twice, with only one working. Some observations were being treated as separate when they were really just separate products available for the same observation; fixing this results in a slightly smaller number of search results for some instruments. It is now possible to download all downlink copies of certain Cassini VIMS observations.



Thursday, August 30, 2018

New Hubble Images Added to OPUS


More than 6,000 solar system images taken by the Hubble Space Telescope have been added to OPUS. The majority of these images were taken by Hubble's Advanced Camera for Surveys (ACS) and Wide Field Camera 3 (WFC3), although there are also new images from NICMOS, STIS, and WFPC2. Most new images are of Jupiter and Saturn, but there are also new images of Uranus, Neptune, Pluto, and even Earth's Moon.


Thursday, August 23, 2018

Recent Additions to the OPUS API (/opus/api/metadata_v2)

The /opus/api/metadata/<OPUSID>.json API has long had a problem. While you request specific metadata using the "cols=" argument with slug names such as "opusid" or "SURFACEGEOtitancenterdistance", the returned data structure uses different names for those fields. This makes matching the returned values to the requested values difficult (they are at least given in the same order). On top of that, the returned names can be ambiguous. Take, for example, the query:
https://tools.pds-rings.seti.org/opus/api/metadata/COISS_2111-1866071296_1866225122-N1866071296_1.json?cols=opusid,instrument,planet,target,time1,observationduration,SURFACEGEOsaturncenterdistance,SURFACEGEOtitancenterdistance
This returns:
[{"opus_id": "COISS_2111-1866071296_1866225122-N1866071296_1"},
 {"instrument_id": "COISS"},
 {"planet_id": "SAT"},
 {"target_name": "TITAN"},
 {"time1": "2017-048T23:59:39.059"},
 {"observation_duration": 46.0},
 {"center_distance": 272061.836},
 {"center_distance": 272061.836}
]
Even though we queried on, for example, "opusid", we were returned a field named "opus_id". Worse yet, we have two returned fields called "center_distance" without a way to disambiguate them (and they are incorrectly the same value).

A second problem is that some of the returned values, such as "COISS" or "SAT", are internal representations, not the values normally shown to the user such as "Cassini ISS" or "Saturn".

To fix these problems while still providing backwards compatibility, we have introduced a second version of this API:
/opus/api/metadata_v2/<OPUSID>.json
This version fixes all of the problems mentioned above. For example, the similar query:
https://tools.pds-rings.seti.org/opus/api/metadata_v2/COISS_2111-1866071296_1866225122-N1866071296_1.json?cols=opusid,instrument,planet,target,time1,observationduration,SURFACEGEOsaturncenterdistance,SURFACEGEOtitancenterdistance
returns:
[{"opusid": "COISS_2111-1866071296_1866225122-N1866071296_1"},
 {"instrument": "Cassini ISS"},
 {"planet": "Saturn"},
 {"target": "Titan"},
 {"time1": "2017-048T23:59:39.059"},
 {"observationduration": 46.0},
 {"SURFACEGEOsaturncenterdistance": 1284796.671},
 {"SURFACEGEOtitancenterdistance": 272061.836}
]
Note that the labels are the same as the requested slugs, and the values are the same as seen in the main OPUS web UI.

We strongly suggest that you use the new version of the metadata API going forward to avoid these problems.

Cassini End-of-Mission ISS, UVIS, and VIMS Volumes (COISS_2115, COISS_2116, COUVIS_0060, COVIMS_0093, COVIMS_0094)


The final volumes from the Cassini End of Mission (July 1, 2017 through September 13, 2017) are now available in OPUS. These include COISS_2115, COISS_2116, COUVIS_0060, COVIMS_0093, and COVIMS_0094, all with enhanced geometric metadata.

See also how to interpret COUVIS preview images and how to interpret COVIMS preview images.

COCIRS is Again Available in OPUS



The PDS Ring-Moon Systems Node has a limited collection of Cassini CIRS observations available as separate, searchable data products. Due to the incomplete coverage, we removed CIRS from OPUS during our recent major upgrade. However, by popular demand, we are once again including CIRS in our search results. Please note that we only have CIRS data from June 1, 2004 through June 30, 2010. Complete coverage of CIRS will be available sometime in 2019.

See also how to interpret COCIRS preview images.

Sunday, August 5, 2018

Major OPUS Software and Data Release


We are pleased to announce a major update to both the OPUS software and the OPUS metadata.
  • All metadata has been re-imported to our database using a brand new import pipeline that dramatically improves accuracy and consistency. Much of the metadata has been improved, either through our own modifications as problems were found or through re-release of the metadata from the instrument teams.
  • Images from HST NICMOS and HST STIS have been added. All observations from Cassini CIRS have been removed pending an upcoming major effort to reformat the data. All data is up-to-date through the Q1'2018 release. We will add the Q2 (including Cassini end-of-mission) data shortly once it becomes available in our main repository. 
  • The layout of the OPUS search fields has changed somewhat, and some fields have been added or removed.
    • The original General Constraints has been split into General Constraints, which contains data about the mission, instrument, and targets, and PDS Constraints, which contains information only relevant to the internal workings of the PDS (volume ID, product creation time, etc.). For General Constraints:
      • Planet now contains Other for searching on observations that do not directly apply to a particular planet, and Intended Target Name includes Other as well, providing access to KBOs, asteroids, stars, etc. Earth has also been added, allowing searches on Earth as well as Earth's Moon.
      • Nominal Target Class has been expanded to include both Regular Satellite and Irregular Satellite.
      • Data Type has been removed and replaced with three new search fields: Spatial Sampling (Point, 1-D, 2-D), Wavelength Sampling (YesNo), and Time Sampling (Yes, No). For example, a COISS or VGISS image has 2-D spatial sampling, no wavelength sampling, and no time sampling. A COUVIS occultation has 1-D spatial sampling, no wavelength sampling, and time sampling. An HST image with a prism or grism has 2-D spatial sampling, wavelength sampling, and no time sampling.
    • For PDS Constraints:
      • There are new search fields Data Set ID, Product ID, and Product Creation Time.
    • The Image Constraints menu is now always available, making it easier to search on image attributes across all instruments, ignoring instruments that do not actually produce images.
    • The Surface Geometry Constraints and Ring Geometry Constraints menus contain the same information as before, but have been reorganized to be consistent with each other.
    • The Cassini Mission Constraints menu has new search fields Mission Phase, Cassini Target Code, and Sequence ID. It also now has the ability to search on Saturn Orbit Number either by numerical range or by checkboxes. Note that the numerical range understands the oddly named orbits 00A, 00B, and 00C.
    • The Cassini ISS Constraints menu has new search fields Instrument Mode, Target Description, and Image Number. The latter is particularly useful for searching by a range of image numbers without regard to the "N" or "W" prefix. Fields related to the inner workings of compression and telemetry have been removed.
    • The Cassini UVIS Constraints menu no longer has the High Voltage Power Supply State or ODC ID fields.
    • The Cassini VIMS Constraints menu has new search fields Swath Width, Swath Length, and Star Tracking.
    • The Galileo SSI Constraints menu no longer has the Data Set ID (moved to the PDS menu), Orbit Number (duplicate with Galileo Mission menu), Encoding Compression Ratio, or Processing History Text fields.
    • The Voyager Mission Constraints menu now has the Mission Phase search field, but has lost the Spacecraft Name field (which is redundant with Instrument Host Name in the General Constrains menu).
    • The Voyager ISS Constraints menu has new search fields Usable Lines and Usable Samples.
    • The Hubble Mission Constaints menu has new search fields Filter Type (Long Pass, Ramp, etc.), WFPC2 PC1/WF2/WF3/WF4 Flag, and Publication Date. The Hubble filter names are sorted by instrument, which results in a large number of checkboxes. We recommend some combination of searching by filter type and wavelength instead.
    • For consistency, there is a new New Horizons Constraints menu, which includes Mission Phase and Spacecraft Clock Count.
    • The New Horizons LORRI Constraints menu has lost the Nominal Target Center Resolution, Phase Angle, and Target Center Distance fields, as these are redundant with the superior information available in the surface geometry metadata. Likewise the New Horizons MVIC Constraints menu has lost the Phase Angle and Target Center Distance fields.
  • It is now possible to search on ranges of Spacecraft Clock Count in the encoded format (e.g. "08966:30:752").
  • We have attempted to keep the public API backwards compatible, while also improving it. There is a fleshed-out version of the API Guide available by clicking on "API" in the footer of the main OPUS page. This page will continue to be improved; suggestions are welcome.
  • The old field "ringobsid" is supported for backwards compatibility, but will not be available for new observations going forward. The new field is called "opusid" and is in a completely different format. We strongly recommend that you switch to using opusid as soon as possible and remove all references to ringobsid from your code.
  • There has been a major renaming of the "slugs" (URL elements) used for searching. This is things like "planet=Saturn&target=Pan&CASSINIrevno=202". The previous slugs were inconsistently named and had the potential for conflicts across missions and instruments. The new names include a prefix (CASSINI, COISS, SURFACEGEO) to keep the namespaces independent. At the bottom of the API Guide there is now a table that shows all available search terms sorted by where they show up in the OPUS web UI, along with the new and old (if different) slug names. Please start using the new slug names ASAP. 
  • A large number of bugs in the UI have been fixed, and the code has been reorganized to create a solid foundation for the future improvements we have planned. If you find any problems with content or usability or just have a request for a new feature, we strongly request that you drop us a note using the "Questions and Comments" link found at the bottom of the OPUS search page, or the "Help us improve OPUS" link found in the right column of this blog. You may also submit issue reports directly to our GitHub repository.

Thank you for using OPUS!


Tuesday, January 16, 2018

Latest Cassini ISS Volumes COISS_2110 and COISS_2111, Includes Calibrated Images


Cassini ISS Volumes COISS_2110 and COISS_2111 are now available.

Calibrated images are included in this release.