Open in Colab

Designers

This documentation will allow a developer to use the Designer API for typical algorithm design.

Installation and reference imports

!pip install google-vizier[jax,algorithms]

from typing import Optional, Sequence
import numpy as np

from vizier import algorithms as vza
from vizier import pythia
from vizier import pyvizier as vz
from vizier.algorithms import designers

Designers

The Designer API is an intuitive abstraction for writing and designing algorithms. It only requires two basic methods, update() and suggest(), shown below.

The source of truth for Designer can be found here.

class Designer(...):
  """Suggestion algorithm for sequential usage."""

  @abc.abstractmethod
  def update(self, completed: CompletedTrials, all_active: ActiveTrials) -> None:
    """Updates recently completed and ALL active trials into the designer's state."""

  @abc.abstractmethod
  def suggest(self, count: Optional[int] = None) -> Sequence[vz.TrialSuggestion]:
    """Make new suggestions."""

Every time update() is called, the Designer will get any newly COMPLETED trials since the last update() call, and will get all ACTIVE trials at the current moment in time.

Note: Trials which may have been provided as ACTIVE in previous update() calls, can be provided as COMPLETED in subsequent update() calls.

GP-Bandit Designer Example

The following example, using the default GP-Bandit algorithm, shows how to interact with Vizier designers.

# The problem statement (which parameters are being optimized)
problem = vz.ProblemStatement()
problem.search_space.root.add_float_param('x', 0.0, 1.0)
problem.search_space.root.add_float_param('y', 0.0, 1.0)
problem.metric_information.append(
    vz.MetricInformation(
        name='maximize_metric', goal=vz.ObjectiveMetricGoal.MAXIMIZE))

# Create a new designer object
designer = designers.VizierGPBandit(problem)
# Ask the designer for 2 suggestions
suggestions = designer.suggest(count=2)

In this case, since the designer was not update with any COMPLETED or ACTIVE trials, it will produce suggestions which will look like:

[TrialSuggestion(parameters=ParameterDict(_items={'x': 0.5, 'y': 0.5}), metadata=Metadata((namespace:, items: {'seeded': 'center'}), current_namespace=)),
 TrialSuggestion(parameters=ParameterDict(_items={'x': 0.10274669379450661, 'y': 0.10191725529767912}), metadata=Metadata((namespace:, items: {}), current_namespace=))]

Note that the first suggestion is seeded at the center of the search space, and the second suggestion is random. If we call designer.suggest() again before calling update(), the designer will produce an identical first suggestion at the center of the search space, and a second random suggestion.

Only when we call update(), will the designer update its internal state and generate different suggestions:

completed_trials = []
for suggestion in suggestions:
  metric_value = np.random.random()  # Make up a fake metric value.
  suggestion.to_trial().complete(
      vz.Measurement(metrics={'maximize_metric': metric_value})
  )

# Update the designer with the completed trials.
designer.update(vza.CompletedTrials(completed_trials), vza.ActiveTrials())

# Ask for more suggestions.
new_suggestions = designer.suggest(count=2)

Thus COMPLETED trials should be incrementally updated, while all ACTIVE trials are passed to the designer in every update() call.

A Designer can also be seeded with pre-existing data. Consider the following example:

# Make a fresh designer.
designer = designers.VizierGPBandit(problem)

# Create completed trials representing pre-existing training data.
trials = [vz.Trial(parameters={'x': 0.5, 'y': 0.6}).complete(vz.Measurement(metrics={'maximize_metric': 0.3}))]
designer.update(vza.CompletedTrials(trials), vza.ActiveTrials())

# As the designer for suggestions.
suggestions = designer.suggest(count=2)

In this case, the designer will not return a first trial seeded at the center of the search space, since it has been updated with completed trials. The new suggestions will look something like:

[TrialSuggestion(parameters=ParameterDict(_items={'x': 0.7199945005054509, 'y': 0.3800034493548722}), ...]

Additional References