Random Number Generation

Following typical practices, we refer to pseudorandom number generation and generators more generically as random number generation and random number generators (RNGs).

Please familiarize yourself with the RNG content in Random Number Generation before reading this section, especially the reference to the Jupyter book RNG example. Similarly, reviewing the historic record of surmise RNG requirements might be helpful to motivate the design and explain certain design decisions detailed here.

Our design and implementation should be informed by and, where possible, follow the advice and decisions reported in the scientific Python RNG specification.

Design

Design discussions originally considered two possible schemes for providing surmise code access to user-provided RNGs:

  1. Manual propagation scheme - all surmise code elements that use RNGs accept and require an RNG argument, which they use directly.

  2. Single “global” RNG scheme - users set into surmise a single RNG object that all surmise code elements that use RNGs access directly for direct use.

Since it was decided that the surmise design and use cases are consistent with users providing a single RNG for use by all surmise code, we adopted the latter access scheme.

surmise is effectively an application, and, therefore, we do not believe that this decision is contrary to the scientific Python RNG specification, which is geared toward libraries.

High-level

To adhere to the surmise RNG requirements related to easing implementation and maintenance by using only one, large statistics package with RNG capabilities, this design stipulates that all sampling of random numbers in surmise occur using either the

  • scipy.stats package (preferably using improved interface introduced at v1.15.0) or

  • scipy.stats RNG currently in use (e.g., using the RNG’s choice method).

If scipy.stats and its RNG both offer similar sampling functionality, prefer the use of scipy.stats functionality over the RNG’s functionality so that surmise code is decoupled as much as possible from the actual RNG object. In other words, users pass in an object and we simply pass it to scipy.stats without caring much what it is or what it can do.

In particular, no other packages, such as numpy.random, should be used in surmise even if the current RNG is compatible with that package. This decision is also motivated by the fact that

  • scipy.stats is considered to be sufficient for correct statistics-based modeling and simulation (but not more demanding than that),

  • the scipy.stats RNG can be used with cryptographically-strong seeds generated with secrets.randbits (as suggested by the generator documentation), and

  • the scipy.stats RNG supports the creation of sets of statistically independent RNGs (e.g., spawn()).

Note that, for simplicity’s sake, code that accepts an RNG will be constrained to only accept scipy.stats RNG objects rather than, as suggested by the scientific Python RNG specification, any object that is or could be used to construct such an RNG object.

RNG Access Pattern

We enforce the restriction of having at most one single RNG in existence at any time by designing the dedicated, internal _RandomNumberGenerator class using the Singleton pattern. In accordance with requirements, surmise code must

  • never set, change, or alter the single RNG,

  • access the single RNG through the Singleton interface and use it exclusively for random number generation,

  • be designed and implemented where possible so that results are identical when rerun with an identical random number generation scenario, and

  • be designed so that all related random number generation (e.g., performing a single calibration and generating a random reordering of the MCMC samples) are performed such that calling code cannot alter the single RNG during the middle of that computational process.

In this design, only external, user code should set the current, “global” RNG and no surmise code should store the current RNG for later use. Instead, upon each invocation surmise code should always acquire the current “global” RNG from the package.

A typical use of the RNG in a surmise routine might be

from ._RandomNumberGenerator import RandomNumberGenerator

def my_surmise_code(...):
    global_rng = RandomNumberGenerator().scipy_start_RNG
    scipy.stats.normal.rvs(..., random_state=global_rng)

To keep this class in the private package interface, the public interface includes the set_RNG() function, which accepts from the user the RNG object to be used and sets it into the Singleton object. See Random Number Generation for more information regarding the RNG public interface.

We recognize that in surmise the emulators and calibrators play a prominent, application-specific role and are, therefore, in the public interface. However, the MCMC samplers are general statistical tools that are hidden behind the calibrators. To decouple the samplers from surmise design decisions and enable their implementations to be more generic and widely useful, we exempt them from having to access the global RNG via the surmise RNG design, and instead require that the calibrators

  • access the global surmise RNG as specified,

  • determine the correct RNG usage for the larger calibration process including sampling, and

  • pass to its sampler the necessary scipy.stats-compatible RNG for its exclusive use directly and with scipy.stats for random number generation during its current invocation and only for that invocation.

Note that this design decision does effectively treat samplers as libraries so that this part of the design does follow the suggestions of the scientific Python RNG specification. However, rather than name the RNG arguments rng as suggested, we prefer to name them scipy_stats_rng so that the code explicitly reflects our design decision to use only one package.

External Packages

We plan to extend surmise so that external packages, including user-provided code, can be used as part of executing its work. For instance, surmise calibrators will hopefully use both surmise internal and external Bilby samplers. Since surmise cannot impose RNG use rules on external code, inclusion of external code must only be made official if the use of RNGs in the external code are compatible with surmise and allow for users to perform statistically correct studies. Users are responsible for determining if RNG use in their user-provided code is valid.