Designers
This documentation will allow a developer to use the Designer API for typical algorithm design.
Installation and reference imports
!pip install google-vizier[jax,algorithms]
from typing import Optional, Sequence
import numpy as np
from vizier import algorithms as vza
from vizier import pythia
from vizier import pyvizier as vz
from vizier.algorithms import designers
Designers
The Designer
API is an intuitive abstraction for writing and designing algorithms. It only requires two basic methods, update()
and suggest()
, shown below.
The source of truth for Designer
can be found
here.
class Designer(...):
"""Suggestion algorithm for sequential usage."""
@abc.abstractmethod
def update(self, completed: CompletedTrials, all_active: ActiveTrials) -> None:
"""Updates recently completed and ALL active trials into the designer's state."""
@abc.abstractmethod
def suggest(self, count: Optional[int] = None) -> Sequence[vz.TrialSuggestion]:
"""Make new suggestions."""
Every time update()
is called, the Designer
will get any newly COMPLETED
trials since the last update()
call, and will get all ACTIVE
trials at the current moment in time.
Note: Trials which may have been provided as ACTIVE
in previous update()
calls, can be provided as COMPLETED
in subsequent update()
calls.
GP-Bandit Designer Example
The following example, using the default GP-Bandit algorithm, shows how to interact with Vizier designers.
# The problem statement (which parameters are being optimized)
problem = vz.ProblemStatement()
problem.search_space.root.add_float_param('x', 0.0, 1.0)
problem.search_space.root.add_float_param('y', 0.0, 1.0)
problem.metric_information.append(
vz.MetricInformation(
name='maximize_metric', goal=vz.ObjectiveMetricGoal.MAXIMIZE))
# Create a new designer object
designer = designers.VizierGPBandit(problem)
# Ask the designer for 2 suggestions
suggestions = designer.suggest(count=2)
In this case, since the designer was not update with any COMPLETED
or ACTIVE
trials, it will produce suggestions which will look like:
[TrialSuggestion(parameters=ParameterDict(_items={'x': 0.5, 'y': 0.5}), metadata=Metadata((namespace:, items: {'seeded': 'center'}), current_namespace=)),
TrialSuggestion(parameters=ParameterDict(_items={'x': 0.10274669379450661, 'y': 0.10191725529767912}), metadata=Metadata((namespace:, items: {}), current_namespace=))]
Note that the first suggestion is seeded at the center of the search space, and the second suggestion is random. If we call designer.suggest()
again before calling update()
, the designer will produce an identical first suggestion at the center of the search space, and a second random suggestion.
Only when we call update()
, will the designer update its internal state and generate different suggestions:
completed_trials = []
for suggestion in suggestions:
metric_value = np.random.random() # Make up a fake metric value.
suggestion.to_trial().complete(
vz.Measurement(metrics={'maximize_metric': metric_value})
)
# Update the designer with the completed trials.
designer.update(vza.CompletedTrials(completed_trials), vza.ActiveTrials())
# Ask for more suggestions.
new_suggestions = designer.suggest(count=2)
Thus COMPLETED
trials should be incrementally updated, while all ACTIVE
trials are passed to the designer in every update()
call.
A Designer
can also be seeded with pre-existing data. Consider the following example:
# Make a fresh designer.
designer = designers.VizierGPBandit(problem)
# Create completed trials representing pre-existing training data.
trials = [vz.Trial(parameters={'x': 0.5, 'y': 0.6}).complete(vz.Measurement(metrics={'maximize_metric': 0.3}))]
designer.update(vza.CompletedTrials(trials), vza.ActiveTrials())
# As the designer for suggestions.
suggestions = designer.suggest(count=2)
In this case, the designer will not return a first trial seeded at the center of the search space, since it has been updated with completed trials. The new suggestions will look something like:
[TrialSuggestion(parameters=ParameterDict(_items={'x': 0.7199945005054509, 'y': 0.3800034493548722}), ...]
Additional References
Our designers folder contains examples of designers.
Our evolution folder contains examples of creating evolutionary designers, such as NSGA2.
Our designer testing routine contains up-to-date examples on interacting with designers.