Converters
This documentation demonstrates how to use converters for representing PyVizier objects as NumPy arrays and vice-versa.
Installation and reference imports
!pip install google-vizier
from vizier import pyvizier as vz
from vizier.pyvizier import converters
Suppose we had a problem statement and some trials associated to the study.
# Setup search space
search_space = vz.SearchSpace()
root = search_space.root
root.add_float_param(name='double', min_value=0.0, max_value=1.0)
root.add_int_param(name='int', min_value=1, max_value=10)
root.add_discrete_param(name='discrete', feasible_values=[0.1, 0.3, 0.5])
root.add_categorical_param(name='categorical', feasible_values=['a', 'b', 'c'])
# Setup metric configurations
m1 = vz.MetricInformation(name='m1', goal=vz.ObjectiveMetricGoal.MAXIMIZE)
m2 = vz.MetricInformation(name='m2', goal=vz.ObjectiveMetricGoal.MINIMIZE)
# Final problem
problem = vz.ProblemStatement(search_space, metric_information=[m1, m2])
# Example trials
trial1 = vz.Trial(
parameters={'double': 0.6, 'int': 2, 'discrete': 0.1, 'categorical': 'a'},
final_measurement=vz.Measurement(metrics={'m1': 0.1, 'm2': 0.2}),
)
trial2 = vz.Trial(
parameters={'double': 0.1, 'int': 6, 'discrete': 0.3, 'categorical': 'b'},
final_measurement=vz.Measurement(metrics={'m1': -1.0, 'm2': 0.8}),
)
Quick Start
To use numerical models, both our x
(parameters) and y
(metrics) need to be formatted as numpy arrays. We can directly do so with TrialToArrayConverter
:
t2a_converter = converters.TrialToArrayConverter.from_study_config(problem)
xs, ys = t2a_converter.to_xy([trial1, trial2])
We can also convert the xs
back into PyVizier ParameterDict
s:
t2a_converter.to_parameters(xs)
Behind the scenes, the TrialToArrayConverter
actually uses a DefaultTrialConverter
which first converts both trial parameters and metrics into dict[str, np.ndarray]
and then concatenates the arrays together.
converter = converters.DefaultTrialConverter.from_study_config(problem)
xs_dict, ys_dict = converter.to_xy([trial1, trial2])
Trials can be recovered too:
original_trials = converter.to_trials(xs_dict, ys_dict)
Customization
There are multiple ways to convert parameters of specific types. For example,
some common methods to convert the 'categorical'
parameter (with feasible
values ['a', 'b', 'c']
) can be:
Integer Index:
'b' -> 1
sinceb
has index 1 among feasible values.One-Hot:
'b' -> [0, 1, 0]
using one-hot encoding.
Additional considerations can be, for example:
Whether to scale continuous parameter values into
[0,1]
Whether to always sign-flip metrics to assume maximization only.
These options can be specified when constructing both TrialToArrayConverter
and DefaultTrialConverter
(source code):
@classmethod
def from_study_config(
cls,
study_config: pyvizier.ProblemStatement,
*,
scale: bool = True,
pad_oovs: bool = True,
max_discrete_indices: int = 0,
flip_sign_for_minimization_metrics: bool = True,
dtype=np.float64,
):
For more fine-grained control over specific ParameterConfig
s and MetricInformation
s, a user can specify individual arguments to each DefaultModelInputConverter
and DefaultModelOutputConverter
respectively.
# Only considers the 'double' parameter values.
double_pc = search_space.get('double')
double_converter = converters.DefaultModelInputConverter(double_pc, scale=True)
double_converter.convert([trial1, trial2])
# Only considers the 'categorical' parameter values.
categorical_pc = search_space.get('categorical')
categorial_converter = converters.DefaultModelInputConverter(categorical_pc, onehot_embed=True)
categorial_converter.convert([trial1, trial2])
# Only considers the 'm1' metric values.
m1_converter = converters.DefaultModelOutputConverter(m1)
m1_converter.convert([trial1.final_measurement, trial2.final_measurement])
These can be inserted into the DefaultTrialConverter
:
parameter_converters = [double_converter, categorial_converter]
metric_converters = [m1_converter]
custom_converter = converters.DefaultTrialConverter(parameter_converters, metric_converters)
custom_converter.to_xy([trial1, trial2]) # Same array outputs as above.
For full customization, the user may create their own ModelInputConverter
s and ModelOutputConverter
s.
class ModelInputConverter(metaclass=abc.ABCMeta):
"""Interface for extracting inputs to the model."""
@abc.abstractmethod
def convert(self, trials: Sequence[vz.TrialSuggestion]) -> np.ndarray:
"""Returns an array of shape (number of trials, feature dimension)."""
@property
@abc.abstractmethod
def output_spec(self) -> NumpyArraySpec:
"""Provides specification of the output from this converter."""
@property
@abc.abstractmethod
def parameter_config(self):
"""Original ParameterConfig that this converter acts on."""
@abc.abstractmethod
def to_parameter_values(
self, array: np.ndarray
) -> List[Optional[vz.ParameterValue]]:
"""Convert and clip to the nearest feasible parameter values."""
class ModelOutputConverter(metaclass=abc.ABCMeta):
"""Metric converter interface."""
@abc.abstractmethod
def convert(self, measurements: Sequence[vz.Measurement]) -> np.ndarray:
"""Returns N x 1 array."""
pass
@abc.abstractmethod
def to_metrics(self, labels: np.ndarray) -> Sequence[Optional[vz.Metric]]:
"""Returns a list of pyvizier metrics."""
@property
@abc.abstractmethod
def metric_information(self) -> vz.MetricInformation:
"""Describes the semantics of the return value from convert() method."""
@property
def output_shape(self) -> Tuple[None, int]:
return (None, 1)