Tuesday, March 31, 2020

OPUS Update for March 2020 - Sort on Multiple Fields, Improved Mobile Support

As we continue in these challenging times, we hope you will enjoy some of the new features we've provided for you this month:

  • The ability to sort on more than one metadata field at a time
  • Improved support for small mobile devices and tablets
  • A minor change to the Multiple Target List field to make your life easier

Multiple-Field Sort

There are many cases where sorting on a single metadata field is not enough to give you the ordering you might want. For example, if you sort on Planet, you have no control over what order the observations for a particular planet are presented in. OPUS has always added OPUS ID as a second sort field to guarantee determinism in the results, but what if you wanted to search first on Planet, then for a given planet on Instrument Name, then for a given Planet+Instrument Name on Intended Target Name, and then for a given Planet+Instrument Name+Intended Target Name on Observation Start Time? The OPUS API has supported this ability for awhile, but now the UI does too!

The current sort fields are presented near the top of the Browse and Cart tabs. The same sort order is used for both tabs, and for both the Gallery and Table views.

Each sort field is shown in order; OPUS ID is always required and must be the last field, so it is shown to the right of a vertical divider. The sort direction for each field can be flipped between ascending and descending by clicking on the field name and the arrows show the direction chosen (down is ascending and up is descending, representing how the observations will be displayed). All sort fields except the final OPUS ID can removed by clicking on the X.

To add a new field (maximum 9), click on the +. You will be presented with a drop-down list of all of the unused fields you have currently selected using Select Metadata. The new field you select will be added at the end of the list (but before OPUS ID). If you want to sort on a field that isn't shown, you need to first add it to the available list using Select Metadata. Once you've added a field to the sort order, you can remove it from the displayed fields using Select Metadata if you'd rather not see the value in the slideshow or table. You can also reorder the fields with drag-and-drop. Using these techniques we can produce the sort order discussed above:

In the Table view, each table header field that is in the sort order has a small number inside a circle. This number represents the position of that field in the sort order list. If a field is in the sort order, you can click on the table header for that field to flip its sort direction. If a field is not in the sort order, you can click on the table header to sort on that field. If there are only two or fewer fields (including OPUS ID) currently being sorted on, clicking on a table header field will replace the current sort with that field. This gives the same behavior that OPUS had previously. However, if there are more than two fields, you will be asked if you want to append the field you clicked on to the end of the sort order, or if you want to replace the entire sort order with that single field. Using Ctrl+click on a table header field is a short cut to append the field to the end of the sort order.

If you would like to reset the sort order to the default (Observation Start Time, OPUS ID), simply click on Sort by or its icon.

Improved Mobile Support

OPUS is now much more responsive and is usable in smaller browsers, all the way down to the size of an iPhone 5. As browser size decreases, some features (such as download links and feedback) are inaccessible because they make the screen too crowded and are unlikely to be used on a small mobile device anyway. Note that Safari (the default browser) on iOS has "features" (such as hiding and showing the URL navigation bar, and allowing pull-to-refresh even when there's nothing to refresh) that make using OPUS more difficult, and there is no way around these "features". If you want to have a better experience using OPUS on an iOS device, we suggest using Chrome instead.

Here is what OPUS looks like now on an iPhone 7:

As always, we welcome your feedback. Stay safe everyone.

Friday, March 6, 2020

OPUS Update for February 2020 - Search by Regular Expression, New Multiple Target List Field

Happy February everyone! We at the PDS Ring-Moon Systems Node hope that you and your loved ones are safe and that your lives have not been unduly disrupted by recent events.

This month we have two big new features for you, as well as a couple of small ones:

  • The ability to search on string fields using regular expressions
  • A new Multiple Target List field for searching on targets in the field of view
  • Tooltips are now available on touchscreen-only devices
  • You can copy-and-paste the green hinting numbers

Regular Expressions

There are a variety of ways to search for strings:

  • contains - The metadata field contains the search string somewhere within it
  • begins - The metadata field starts with the search string
  • ends - The metadata field ends with the search string
  • matches - The metadata field matches the search string exactly
  • excludes - The metadata field does not contain the search string

To these we have added a new one:

  • regex - The metadata field matches the given regular expression

For example, let's say you would like to search for all COISS volumes 2013 thru 2018. Before you would've needed to use the recently-added multiple search functionality:

but now you can simplify it into a single search:

We implemented regular expressions using the full-featured ICU library, which supports just about every regular expression feature you could want. For example, you could use "(?!COISS_2015)COISS_201[3-8]" to search for every volume COISS_2013 thru COISS_2018 except for COISS_2015. Please note that, by default, regular expressions are not anchored and the search string can appear anywhere in the metadata field. If you want to make sure that the regular expression is the only thing in the field, you can start it with "^" (anchor left) and end it with "$" (anchor right): "^COISS_201[3-8]$". Also, string fields now support input validation just like numeric fields do. If you select the "regex" query type, then your regular expression will be validated as you type it. An illegal regular expression (for example mismatched parentheses) will have a yellow border until you fix it; a legal regular expression will have a green border.

The Multiple Target List Field

There is a new field under Surface Geometry Constraints called Multiple Target List. For every observation for which surface geometry is available (currently Cassini ISS, UVIS, and VIMS, New Horizons LORRI, and Voyager ISS) this field contains a comma-separated list of every body in the field of view in alphabetical order. For example, if an image contains Rhea, Tethys, and Saturn, the field will be "RHEA,SATURN,TETHYS". There are two main uses for this. The first is for easily displaying what body(ies) you are looking at in an observation. For this you can add Multiple Target List to your Selected Metadata. The second is for searching for observations that contain specific bodies. For example, there are Cassini ISS images containing only Rhea and Tethys. To search for them, you could use "RHEA,TETHYS" with the "matches" query type.

You can also combine the Multiple Target List field with regular expression searching to get more sophisticated searches. If you would like every image that contains exactly two bodies, you can use "^\w+,\w+$". If you want one of those bodies to be Rhea, you could do two searches, one on "^RHEA,\w+$" and the other on "^\w+,RHEA$". Here is an example of a search for all Cassini ISS images that contain Rhea plus one other body, where that other body is not Saturn, and where Rhea is reasonably-sized in the image (resolution 3-10 km/pixel):

As always, we welcome your feedback, comments, and questions and look forward to hearing from you.

Tuesday, January 28, 2020

OPUS Update for January 2020 - Add All to Cart, New API Guide, and Much More

This month we are pleased to introduce a host of improvements, including:

  • The ability to add all current results to the cart
  • Thumbnails in the table view
  • Improved handling of surface geometry target constraints
  • A new API Guide and minor changes to the API itself
  • Support for directly creating PDF files from the Help menu
  • New search fields for HST STIS and Cassini
  • A change to HST OPUS IDs

There's a lot to cover, so let's get started!

Add All Results to Cart

In addition to the current ways of adding a single observation to the cart (Ctrl+Click, the shopping cart icon in the gallery view / checkbox in the table view, or the Add to Cart menu option) or multiple observations to the cart (Shift+Click to select the start and end of the range or the Start Add Range menu option), it is now possible to add all of the observations in the current search results to the cart. This is useful if you have narrowed your search to just the observations you care about downloading. Note that there is a limit of 10,000 observations in the cart, so you must add search constraints to get your result set down below this number before attempting to add them.

There are two ways to add all of the current observations to the cart. The first is the Add all results to cart menu available on any thumbnail or table row; it doesn't matter which observation you use as they all do the same thing. The second is the shopping cart icon available at the left edge of the header of the table view. Both will ask for confirmation before performing the operation.

Thumbnails in the Table View

Rows in the table view now contain a small thumbnail to the right of the cart checkbox and menu. This thumbnail can be enlarged by simply hovering your mouse over it.

Improved Handling of Surface Geometry Constraints

Surface Geometry fields are available by selecting a Surface Geometry Target Name under the Surface Geometry Constraints category. Only one surface geometry target can be selected at a time. Now, when you change the surface geometry target, all of the search fields related to the previous target that you created will automatically switch to the new target while keeping their old search values. This is useful if, for example, you are looking for observations of a set of moons at a particular range of phase angles. Just choose your first target, set the phase angle range, and add all of those observations to your cart. Then change the target and the same phase angle range will automatically be used for the new target. Then you can add all of those observations to the cart as well.

In addition, when you change targets your list of selected metadata fields will automatically be updated. If you have any fields related to the current target, then when you change targets a new set of fields for the same metadata fill be added for the new target.

For example, let's start with high phase angle and high resolution images of Enceladus and select a few metadata fields:

Now let's change the target to Titan:

Note that when performing the search, all surface geometry targets other than the one selected have "--" as their result count. Normally a number is provided that tells you how many results will be available if you select the given option. However, in this case selecting an option actually changes other search fields, so such a number is much harder to compute and is confusing to interpret so it is not provided.

New API Guide and API Changes

OPUS provides a comprehensive API for third-party applications to search its database and retrieve information. The associated API Guide has been completely rewritten. It is now much more complete and easier to read. It is available under the Help menu or directly from https://tools.pds-rings.seti.org/opus/apiguide.pdf

The API itself has also undergone minor changes to make it more consistent. Most of these changes will not affect current users, and in many cases backwards compatibility is provided. Please refer to the API Guide for the most authoritative description of the API calls and the format of their return values.

Creation of PDF Files

Most documents available under the Help menu now allow you to create and download a PDF version. This PDF version is created live and is always up-to-date with the current version of OPUS.

New Search Fields for HST STIS and Cassini

There are two new search fields:
  • STIS Optical Element under Hubble Mission Constraints gives the optical element (grating, mirror, or prism) used for an HST STIS observation. It is NULL for all other instruments.
  • Cassini Original Target Name under Cassini Mission Constraints gives the original TARGET_NAME field supplied by the instrument teams. The meaning of this field varies by instrument and observation and should only be used by researchers familiar with these details. Normally, Intended Target Name gives a better representation of the actual primary target of the observation. OPUS creates Intended Target Name from a variety of sources to give the most useful results for most researchers.

Change to HST OPUS IDs

All HST OPUS IDs now include the full 9-character filename. This was done to make interaction with the MAST archive more seamless.

As always, we actively solicit all feedback, comments, and suggestions.

Thursday, December 19, 2019

OPUS Update for December 2019 - User-Selectable Units and More

This month we are pleased to introduce a major new feature: user-selectable units during search. We have also made some minor improvements to the Select Metadata dialog, made minor adjustments to the OPUS IDs for Hubble observations, and made minor changes to the product types available for download.

User-Selectable Units

Many search fields now allow the user to select which units to use for the input values. For example, Observed Ring Radius is usually in units of "km". However, it is now possible to select other distance units such as "m", "Rj" (Jupiter radii), or "Rs" (Saturn radii). Time fields (like Observation Duration) are normally in units of "secs". However, it is now possible to select other time units such as "msecs", "minutes", "hours", or "days". To select a unit, simply use the dropdown box at the top of the search box:

Preprogrammed ranges (Wavelength and Observed Ring Radius) will be presented in the selected units.

If there are already values entered into any of the input fields, they will automatically be converted when a new unit is selected. Only one unit can be selected for all of the input fields for a particular metadata field.

Note that when viewing metadata fields, such as on the Detail tab or in the slideshow or table views, the data will always be presented in its native units, not the units selected for search.

Select Metadata Enhancements

In the Select Metadata dialog, any metadata fields that have been selected on the Search tab will be presented first. This is an attempt to make your experience more efficient by assuming that if you're searching on a particular field, you probably also want to see that field displayed when you view the metadata. The fields presented under Current Search Fields will also be shown under their respective categories, and can be selected or deselected in either location.

In addition, it is now possible to download a CSV file directly from the Select Metadata dialog using the currently chosen fields, even if those fields haven't been "saved" as a permanent selection. This makes it easier to download a CSV file containing particular fields without pressing additional buttons, and also allows you to download a CSV file containing fields that you don't normally want to see in the OPUS UI (by downloading but not saving the field changes). If you are viewing the Browse tab, the CSV file will contain all of the current search results. If you are viewing the Cart tab, the CSV file will contain all items in the main cart (not in the recycle bin).

Hubble OPUS IDs

A small number of Hubble observations have multiple versions that were downlinked at different times. In these cases we make the filename portion of the OPUS ID 9 characters instead of 8 to allow the observations to be distinguished. This has slightly increased the number of NICMOS observations available.

Download Product Types

Some New Horizons observations have multiple versions that were downlinked in different formats but are otherwise identical. We provide the highest-quality versions as Raw Image or Calibrated Image and now provide all additional versions together as Raw Image Alternate Downlink or Calibrated Image Alternate Downlink. Also, the list of product types has been reversed, placing the most important types (e.g. actual mission data) first and secondary data (inventory files, preview images) last. 

Happy Holidays from the OPUS team (Rob, Dave, and Debby) and we look forward to serving you in the New Year. As always, we actively solicit all feedback, comments, and suggestions.

Tuesday, November 26, 2019

OPUS Updates for November 2019 - Recycle Bin and Multiple Search Fields

This month we are pleased to present two major improvements to OPUS: a new recycle bin concept and the ability to do multiple searches on the same search field.

The Recycle Bin

The cart acts as a convenient repository for observations you want to download or explore further. However, once you've added many observations to the cart, you may find that you have extraneous ones that you don't really need. Until now, the only way to remove these was to return to the Browse Results tab, find the observations, and remove them from the cart. Now you have another option: you can remove them from the cart directly on the Cart tab.

Removing an observation from the cart while viewing the Cart tab is performed in the same way as it is while viewing the Browse Results tab, but in this case the observation is not just removed - it is moved into a special recycle bin. Observations in the recycle bin are indicated by a large "recycle" icon:

or, in the Table View, by a green line drawn through the row:

You toggle whether an observation is in the cart or the recycle bin by using Ctrl+Click on a thumbnail or table row, the icon in the mini-toolbar inside the thumbnail (which now looks like a recycle or return glyph instead of a cart), the checkbox in the table view, or selecting an appropriate option from the "hamburger" menu. You can also add or remove an entire range of observations by using Shift+Click or one of the options in the "hamburger" menu.

Observations in the recycle bin are not included in any downloads (CSV or archive) and are not included in the statistics in the Download Data panel (Total Size, Total Files, or any of the Product Type counts). This allows you to transfer observations to and from the recycle bin while monitoring the size of the resulting download.

There are also now three options to manage the cart:

You can empty the cart, which empties both the main cart and the recycle bin. You can empty just the recycle bin, which will leave the main contents of your cart alone. And you can restore all of the observations in the recycle bin back to the cart. Note the number after the latter option, which indicates the total number of observations currently in the recycle bin. The number next to the Cart tab at the top indicates only the number of observations in the main cart, ignoring the recycle bin.

Finally, the download links, which were previously above the main cart gallery, have been moved to the Download Data panel.

Multiple Search Fields

It is now possible to perform multiple searches on the same numeric, string, or date/time search field. The searches are "OR"ed together. To add a new search, click on the "+(OR)" button to the right of an existing search. To remove a search, click on the trashcan icon.

For example, you could search for observations in volumes COISS_2001, COISS_2002, or COISS_2003:

You could also search for ring images that were not taken within 30 degrees of the ring plane, on either the lit or unlit side:

As always, we appreciate your use of OPUS and actively solicit feedback with comments, complaints, suggestions, or kudos!

Thursday, October 31, 2019

OPUS Updates for October 2019 - Citing OPUS and QR Codes

We have some major improvements coming in the next month, but this month we have a simple one for you - information on how to cite OPUS in your presentation, poster, or paper.

How to Cite OPUS (and QR Codes!)

The Help menu now has a new option: Citing OPUS. This new help page gives you information on how to cite OPUS in your presentation, poster, or paper, as well as a copy of the NASA PDS logo for your use. Finally, it includes customized QR codes for your poster or presentation that directly link to OPUS, your current search in OPUS, or your entire current state in OPUS (including which tab and observations you're looking at, etc.). Note that it does not include the contents of your cart. This is a perfect way to share your search with audience members in a way that they can easily reproduce.

As always, keep those suggestions and comments coming!

Monday, September 30, 2019

OPUS Updates for September 2019 - Preprogrammed Ranges

This month we are pleased to bring you a new feature that aids in searching on ring radius or wavelength.

Preprogrammed Ranges

Users often need to search for standard ranges for ring radius, such as the inner and outer limits of Saturn's A ring or the semimajor axis of Enceladus. A similar problem occurs for wavelength; a user may wish to search for "blue" images but have to look up exactly what wavelengths that corresponds to. Both of these problems are now solved by the existence of preprogrammed ranges. When you click on the "Min" input box of the Observed Ring Radius field (available under Ring Geometry Constraints), a dropdown box will appear with a large selection of satellite semimajor axes and ring limits for Jupiter, Saturn, Uranus, and Neptune. Simply clicking on one of these options will fill in both the minimum and maximum value:

Alternatively, you may start to type the name of a feature, and the list of options will be limited to those that contain the string you've typed:

If only a single result is left, hitting ENTER will automatically fill in the values.

A similar feature is available for the Wavelength field, with colors taken from the PDS4 dictionary, CRC, and Johnson UBV filters.

As always, we welcome your feedback.

Friday, August 30, 2019

OPUS Updates for August 2019 - Cart Improvements

This month we are pleased to bring you a number of updates to the cart and the selection of observations to put in the cart.

Selecting Ranges

There are several ways to select ranges of observations to put in the cart, or remove from the cart. To indicate the beginning of the range you can shift+click on a thumbnail or table row, or select "Start add range to cart here" or "Start remove range from cart here" from the hamburger menu at the bottom of each thumbnail or on the left side of each table row. If the observation you are starting with is not currently in the cart, you will be starting an "add range" operation; if the observation is already in the cart, you will be starting a "remove range" operation. To end the range, you either shift+click on a second observation, or select "End add range to cart here " or "End remove range from cart here", as appropriate.

To help you keep track of where the range started, the starting thumbnail or table row will show a flashing green square. In addition, there is now a small window in the lower left corner of the gallery or table views that gives you the relevant information. Then you can scroll away, or use the slider to jump much further away, and still know that you are in the middle of a range select operation and where that range started.

Details about Products Available for Download

The Cart tab now shows complete information about the data products available for download. For each product type (such as "Raw image"), you are given the number of observations in the cart that are providing that product type, the number of files (including data files, label files, and support files) that will be included in the archive, and the total size of those files. You may (de)select individual product types, entire categories, or all product types to limit the download to just those products you need. As you make your selections, the "Total Size" and "Total Files" fields at the top will update.

New Location for Download Links

When you select Download Data Archive or Download URL Archive, a special window will pop up in the lower right corner showing the link to use for the current download, plus a history of previous downloads you have requested during this session. You may toggle the presence of this window by clicking on the Download History button at the bottom (or dismiss it using the X in the corner) and you can remove all the links from the history by clicking the Clear All button. Click on one of the links to start the actual download operation in your browser.

Please keep those suggestions coming!

Tuesday, July 30, 2019

OPUS Updates for July 2019

This month we are pleased to bring you a number of features requested by our users (that's you!). As a reminder, feedback is always welcome and encouraged, and we put a high priority on implementing requests or fixing bugs that are affecting our users.

Create a New Search Based on an Existing Observation

  1. Using whatever method you like, find an observation that you are interested in.
  2. Display that observation in the Detail tab. There are two easy ways to do this: you can either hover your mouse over the thumbnail and click on the icon in the mini toolbar, or you can click on the "hamburger" menu and then select Show detail.

  3. On the Detail tab you will find a magnifying glass search icon next to every metadata field that contains a string or a multiple-choice value (indicated by checkboxes on the Search tab).
  4. Clicking on a search icon will create a new OPUS in a separate browser tab. This OPUS will contain exactly one search term: the particular metadata field you selected equal to the value for the observation you were looking at. For example, for the observation selected above:

    If you click on the magnifying glass next to "Observation Name: ISS_045RI_RDCOLSCNB001_PRIME" you will get a new OPUS with that single search:

Slider Now Available on Cart Tab

The new slider, which allows you to quickly position the thumbnail gallery or metadata table to a specific place in the result set, is now available on the Cart tab. This helps you more easily browse through the items in your cart if you have added hundreds or thousands of observations. We have also slightly reorganized the navigation buttons on the Cart tab, allowing you to more easily find those that will download your cart data. Look for many more improvements to the cart in a future release.

You Can Print Help Documents

All of the Help documents (such as the Getting Start Guide, the FAQ, and the API Guide) now include a button that will open the document in a new browser tab. At that point you can print the document to your printer or to a PDF file for future use. This is especially useful for the API Guide, which you can use as a reference while writing your own software to access the OPUS API.

Select Metadata Dialog Now Includes Help

The Select Metadata dialog now includes instructions for use:

Please keep those suggestions coming!

Thursday, June 13, 2019

Announcing the Release of OPUS3!

The Ring-Moon Systems Node of NASA's Planetary Data System is proud to announce the public release of OPUS3, commemorating the 10th anniversary of OPUS.

This new version of OPUS has been under development for the past 12 months, and features a modern look-and-feel, responsive design for desktop, laptop, tablet, and mobile devices, easier-to-use interface, improved documentation, and faster back-end server.

There are too many changes to OPUS to list here, but there are two guides available to help you transition to the new interface, both available under the new Help menu:
  • The Frequently Asked Questions (FAQ) section. In there you can click on What's new in OPUS3? for a brief discussion of the differences between the previous version of OPUS and this new release.
  • The Getting Started guide. This is a more comprehensive presentation of all of the major features in OPUS3.
Whether you are a new user or have been using OPUS since its initial release in 2009, we strongly encourage you to read through the Getting Started guide. There are likely many features of OPUS that you never knew existed.

As always, we welcome and encourage feedback. You can submit comments or questions through the Questions/Feedback link at the top of the OPUS3 screen, or you can contact us on Twitter at @pdsopus.

You can try it out here. We hope you enjoy the new version of OPUS!