Converters

This documentation demonstrates how to use converters for representing PyVizier objects as NumPy arrays and vice-versa.

Installation and reference imports

!pip install google-vizier

from vizier import pyvizier as vz
from vizier.pyvizier import converters

Suppose we had a problem statement and some trials associated to the study.

# Setup search space
search_space = vz.SearchSpace()
root = search_space.root
root.add_float_param(name='double', min_value=0.0, max_value=1.0)
root.add_int_param(name='int', min_value=1, max_value=10)
root.add_discrete_param(name='discrete', feasible_values=[0.1, 0.3, 0.5])
root.add_categorical_param(name='categorical', feasible_values=['a', 'b', 'c'])

# Setup metric configurations
m1 = vz.MetricInformation(name='m1', goal=vz.ObjectiveMetricGoal.MAXIMIZE)
m2 = vz.MetricInformation(name='m2', goal=vz.ObjectiveMetricGoal.MINIMIZE)

# Final problem
problem = vz.ProblemStatement(search_space, metric_information=[m1, m2])

# Example trials
trial1 = vz.Trial(
    parameters={'double': 0.6, 'int': 2, 'discrete': 0.1, 'categorical': 'a'},
    final_measurement=vz.Measurement(metrics={'m1': 0.1, 'm2': 0.2}),
)
trial2 = vz.Trial(
    parameters={'double': 0.1, 'int': 6, 'discrete': 0.3, 'categorical': 'b'},
    final_measurement=vz.Measurement(metrics={'m1': -1.0, 'm2': 0.8}),
)

Quick Start

To use numerical models, both our x (parameters) and y (metrics) need to be formatted as numpy arrays. We can directly do so with TrialToArrayConverter:

t2a_converter = converters.TrialToArrayConverter.from_study_config(problem)
xs, ys = t2a_converter.to_xy([trial1, trial2])

We can also convert the xs back into PyVizier ParameterDicts:

t2a_converter.to_parameters(xs)

Behind the scenes, the TrialToArrayConverter actually uses a DefaultTrialConverter which first converts both trial parameters and metrics into dict[str, np.ndarray] and then concatenates the arrays together.

converter = converters.DefaultTrialConverter.from_study_config(problem)
xs_dict, ys_dict = converter.to_xy([trial1, trial2])

Trials can be recovered too:

original_trials = converter.to_trials(xs_dict, ys_dict)

Customization

There are multiple ways to convert parameters of specific types. For example, some common methods to convert the 'categorical' parameter (with feasible values ['a', 'b', 'c']) can be:

Integer Index: 'b' -> 1 since b has index 1 among feasible values.
One-Hot: 'b' -> [0, 1, 0] using one-hot encoding.

Additional considerations can be, for example:

Whether to scale continuous parameter values into [0,1]
Whether to always sign-flip metrics to assume maximization only.

These options can be specified when constructing both TrialToArrayConverter and DefaultTrialConverter (source code):

@classmethod
def from_study_config(
    cls,
    study_config: pyvizier.ProblemStatement,
    *,
    scale: bool = True,
    pad_oovs: bool = True,
    max_discrete_indices: int = 0,
    flip_sign_for_minimization_metrics: bool = True,
    dtype=np.float64,
):

For more fine-grained control over specific ParameterConfigs and MetricInformations, a user can specify individual arguments to each DefaultModelInputConverter and DefaultModelOutputConverter respectively.

# Only considers the 'double' parameter values.
double_pc = search_space.get('double')
double_converter = converters.DefaultModelInputConverter(double_pc, scale=True)
double_converter.convert([trial1, trial2])

# Only considers the 'categorical' parameter values.
categorical_pc = search_space.get('categorical')
categorial_converter = converters.DefaultModelInputConverter(categorical_pc, onehot_embed=True)
categorial_converter.convert([trial1, trial2])

# Only considers the 'm1' metric values.
m1_converter = converters.DefaultModelOutputConverter(m1)
m1_converter.convert([trial1.final_measurement, trial2.final_measurement])

These can be inserted into the DefaultTrialConverter:

parameter_converters = [double_converter, categorial_converter]
metric_converters = [m1_converter]

custom_converter = converters.DefaultTrialConverter(parameter_converters, metric_converters)
custom_converter.to_xy([trial1, trial2])  # Same array outputs as above.

For full customization, the user may create their own ModelInputConverters and ModelOutputConverters.

class ModelInputConverter(metaclass=abc.ABCMeta):
  """Interface for extracting inputs to the model."""

  @abc.abstractmethod
  def convert(self, trials: Sequence[vz.TrialSuggestion]) -> np.ndarray:
    """Returns an array of shape (number of trials, feature dimension)."""

  @property
  @abc.abstractmethod
  def output_spec(self) -> NumpyArraySpec:
    """Provides specification of the output from this converter."""

  @property
  @abc.abstractmethod
  def parameter_config(self):
    """Original ParameterConfig that this converter acts on."""

  @abc.abstractmethod
  def to_parameter_values(
      self, array: np.ndarray
  ) -> List[Optional[vz.ParameterValue]]:
    """Convert and clip to the nearest feasible parameter values."""

class ModelOutputConverter(metaclass=abc.ABCMeta):
  """Metric converter interface."""

  @abc.abstractmethod
  def convert(self, measurements: Sequence[vz.Measurement]) -> np.ndarray:
    """Returns N x 1 array."""
    pass

  @abc.abstractmethod
  def to_metrics(self, labels: np.ndarray) -> Sequence[Optional[vz.Metric]]:
    """Returns a list of pyvizier metrics."""

  @property
  @abc.abstractmethod
  def metric_information(self) -> vz.MetricInformation:
    """Describes the semantics of the return value from convert() method."""

  @property
  def output_shape(self) -> Tuple[None, int]:
    return (None, 1)