{ "cells": [ { "cell_type": "markdown", "metadata": { "id": "u4fiLarveGYu" }, "source": [ "[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/google/vizier/blob/main/docs/guides/developer/writing_algorithms.ipynb)\n", "\n", "\n", "# [Deprecated] Writing Algorithms and Pythia Policies\n", "This documentation will allow a developer to:\n", "\n", "* Understand the basic structure of a Pythia policy.\n", "* Use the Designer API for simplfying algorithm design.\n", "\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "id": "QhwKY4FDB2El" }, "source": [ "## Installation and reference imports" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "b_QlZv03vj1u" }, "outputs": [], "source": [ "!pip install google-vizier" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "1f_E4bKleQfG" }, "outputs": [], "source": [ "from typing import Optional, Sequence\n", "\n", "from vizier import pythia\n", "from vizier import pyvizier as vz\n", "from vizier import algorithms\n", "from vizier._src.algorithms.policies import designer_policy\n", "from vizier._src.algorithms.evolution import nsga2" ] }, { "cell_type": "markdown", "metadata": { "id": "t1lCW6L_fhIR" }, "source": [ "## Pythia Policies\n", "The Pythia Service maps algorithm names to `Policy` objects. All algorithms which need to be hosted on the server must eventually be wrapped into a `Policy`.\n", "\n", "Every `Policy` is injected with a `PolicySupporter`, which is a client used for fetching data from the datastore. This design choice serves two core purposes:\n", "\n", "1. The `Policy` is effectively stateless, and thus can be deleted and recovered at any time (e.g. due to a server preemption or failure).\n", "2. Consequently, this avoids needing to save an explicit and potentially complicated algorithm state. Instead, the \"algorithm state\" can be recovered purely from the entire study containing (`metadata`, `study_config`, `trials`).\n", "\n", "We show the `Policy` abstract class explicitly below. Exact class entrypoint can be found [here](https://github.com/google/vizier/blob/main/vizier/pythia.py).\n", "\n", "```python\n", "class Policy(abc.ABC):\n", " \"\"\"Interface for Pythia Policy subclasses.\"\"\"\n", "\n", " @abc.abstractmethod\n", " def suggest(self, request: SuggestRequest) -> SuggestDecision:\n", " \"\"\"Compute suggestions that Vizier will eventually hand to the user.\"\"\"\n", "\n", " @abc.abstractmethod\n", " def early_stop(self, request: EarlyStopRequest) -> EarlyStopDecisions:\n", " \"\"\"Decide which Trials Vizier should stop.\"\"\"\n", "\n", " @property\n", " def should_be_cached(self) -> bool:\n", " \"\"\"Returns True if it's safe & worthwhile to cache this Policy in RAM.\"\"\"\n", " return False\n", "```" ] }, { "cell_type": "markdown", "metadata": { "id": "2RxKvaxT_gDs" }, "source": [ "### Fundamental Rule of Service Pythia Policies\n", "For algorithms used in the Pythia Service, the fundamental rule is to assume that a Pythia policy class instance will only call once per user interaction:\n", "* `__init__`\n", "* `suggest()`\n", "\n", "and be immediately deleted afterwards. Thus a typical policy will use a `stateless_algorithm` and roughly look like:\n", "\n", "```python\n", "class TypicalPolicy(Policy):\n", "\n", " def __init__(self, policy_supporter: PolicySupporter):\n", " self._policy_supporter = policy_supporter\n", "\n", " def suggest(self, request: SuggestRequest) -> SuggestDecision:\n", " all_completed = policy_supporter.GetTrials(status_matches=COMPLETED)\n", " all_active = policy_supporter.GetTrials(status_matches=ACTIVE)\n", " suggestions = stateless_algorithm(all_completed, all_active)\n", " return SuggestDecision(suggestions)\n", "```" ] }, { "cell_type": "markdown", "metadata": { "id": "ftYM-7ymmWO1" }, "source": [ "## Example Pythia Policy\n", "Here, we write a toy policy, where we only act on `CATEGORICAL` parameters for simplicity. The `make_parameters` function will simply for-loop over every category and then cycle back." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "8PCogiUee3Zm" }, "outputs": [], "source": [ "def make_parameters(\n", " study_config: vz.StudyConfig, index: int\n", ") -> vz.ParameterDict:\n", " parameter_dict = vz.ParameterDict()\n", " for parameter_config in study_config.search_space.parameters:\n", " if parameter_config.type != vz.ParamterType.CATEGORICAL:\n", " raise ValueError(\"This function only supports CATEGORICAL parameters.\")\n", " feasible_values = parameter_config.feasible_values\n", " categorical_size = len(feasible_values)\n", " parameter_dict[parameter_config.name] = vz.ParameterValue(\n", " value=feasible_values[index % categorical_size]\n", " )\n", " return parameter_dict" ] }, { "cell_type": "markdown", "metadata": { "id": "0odJTgnwCtfz" }, "source": [ "To collect the `index` from the database, we will use the `PolicySupporter` to obtain the maximum trial ID based on completed and active trials." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "Be7V7ZTSC2As" }, "outputs": [], "source": [ "def get_next_index(policy_supporter: pythia.PolicySupporter):\n", " \"\"\"Returns current trial index.\"\"\"\n", " completed_and_active = policy_supporter.GetTrials(\n", " status_matches=vz.TrialStatus.COMPLETED\n", " ) + policy_supporter.GetTrials(status_matches=vz.TrialStatus.ACTIVE)\n", " trial_ids = [t.id for t in completed_and_active]\n", "\n", " if trial_ids:\n", " return max(trial_ids)\n", " return 0" ] }, { "cell_type": "markdown", "metadata": { "id": "se5g11DNC6NO" }, "source": [ "We can now put it all together into our Pythia Policy." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "dz5dzxCUC-AL" }, "outputs": [], "source": [ "class MyPolicy(pythia.Policy):\n", "\n", " def __init__(self, policy_supporter: pythia.PolicySupporter):\n", " self._policy_supporter = policy_supporter\n", "\n", " def suggest(self, request: pythia.SuggestRequest) -> pythia.SuggestDecision:\n", " \"\"\"Gets number of Trials to propose, and produces Trials.\"\"\"\n", " suggest_decision_list = []\n", " for _ in range(request.count):\n", " index = get_next_index(self._policy_supporter)\n", " parameters = make_parameters(request.study_config, index)\n", " suggest_decision_list.append(vz.TrialSuggestion(parameters=parameters))\n", " return pythia.SuggestDecision(\n", " suggestions=suggest_decision_list, metadata=vz.MetadataDelta()\n", " )" ] }, { "cell_type": "markdown", "metadata": { "id": "aVv_fmTug1cn" }, "source": [ "## Designers\n", "While Pythia policies are the proper interface for hosting algorithms on the\n", "server, we also provide the `Designer` API to simplify algorithm development.\n", "A `Policy` can use `PolicySupporter` to actively fetch trials from the datastore.\n", "Unlike a `Policy`, a `Designer` cannot actively fetch trials, but it can be updated with previously `COMPLETED` or `ACTIVE` trials.\n", "\n", "We display the `Designer` class below. The source of truth for `Designer` can be found\n", "[here](https://github.com/google/vizier/blob/main/vizier/algorithms/__init__.py).\n", "\n", "```python\n", "class Designer(...):\n", " \"\"\"Suggestion algorithm for sequential usage.\"\"\"\n", "\n", " @abc.abstractmethod\n", " def update(self, completed: CompletedTrials, all_active: ActiveTrials) -> None:\n", " \"\"\"Updates recently completed and ALL active trials into the designer's state.\"\"\"\n", "\n", " @abc.abstractmethod\n", " def suggest(self, count: Optional[int] = None) -> Sequence[vz.TrialSuggestion]:\n", " \"\"\"Make new suggestions.\"\"\"\n", "```" ] }, { "cell_type": "markdown", "metadata": { "id": "h43_ByEjgbb6" }, "source": [ "Every time `update()` is called, the `Designer` will get any newly `COMPLETED` trials since the last `update()` call, and will get all `ACTIVE` trials at the current moment in time.\n", "\n", "Note that trials which may have been provided as `ACTIVE` in previous `update()` calls, can be provided as `COMPLETED` in subsequent `update()` calls. " ] }, { "cell_type": "markdown", "metadata": { "id": "--eOH8TQDQnW" }, "source": [ "To implement our same algorithm above in a Designer, only the `update()` and `suggest()` methods need to be implemented using our previous `make_parameters` function. The designer class can now store completed trials inside its `self._completed_trials` attribute." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "rddElYlbDQD1" }, "outputs": [], "source": [ "class MyDesigner(algorithms.Designer):\n", "\n", " def __init__(self, study_config: vz.StudyConfig):\n", " self._study_config = study_config\n", " self._completed_trials = []\n", " self._active_trials = []\n", "\n", " def update(\n", " self,\n", " completed: algorithms.CompletedTrials,\n", " all_active: algorithms.ActiveTrials,\n", " ) -> None:\n", " self._completed_trials.extend(completed.trials)\n", " self._active_trials = all_active.trials\n", "\n", " def suggest(\n", " self, count: Optional[int] = None\n", " ) -> Sequence[vz.TrialSuggestion]:\n", " if count is None:\n", " return []\n", " trial_ids = [t.id for t in self._completed_trials + self._active_trials]\n", " current_index = max(trial_ids)\n", " return [\n", " make_parameters(self._study_config, current_index + i)\n", " for i in range(count)\n", " ]" ] }, { "cell_type": "markdown", "metadata": { "id": "3jgb_dKY3Bir" }, "source": [ "## Wrapping a `Designer` to a Pythia `Policy`\n", "Note that in the above implementation of `MyDesigner`, the entire algorithm (if deleted or preempted) can conveniently be recovered in just a **single** call of `update()` after `__init__`.\n", "\n", "Thus we may immediately wrap `MyDesigner` into a Pythia Policy with the following Pythia `suggest()` implementation:\n", "\n", "* Create the designer temporarily.\n", "* Update the temporary designer with **all** previously completed trials and active trials.\n", "* Obtain suggestions from the temporary designer.\n", "\n", "This is done conveniently with the `DesignerPolicy` wrapper ([code](https://github.com/google/vizier/blob/main/vizier/_src/algorithms/policies/designer_policy.py)):\n", "\n", "```python\n", "class DesignerPolicy(pythia.Policy):\n", " \"\"\"Wraps a Designer into a Pythia Policy.\"\"\"\n", "\n", " def __init__(self, supporter: pythia.PolicySupporter,\n", " designer_factory: Factory[vza.Designer]):\n", " self._supporter = supporter\n", " self._designer_factory = designer_factory\n", "\n", " def suggest(self, request: pythia.SuggestRequest) -> pythia.SuggestDecision:\n", " completed = self._supporter.GetTrials(status_matches=vz.TrialStatus.COMPLETED)\n", " active = self._supporter.GetTrials(status_matches=vz.TrialStatus.ACTIVE)\n", " designer.update(vza.CompletedTrials(completed), vza.ActiveTrials(active))\n", " return pythia.SuggestDecision(\n", " designer.suggest(request.count), metadata=vz.MetadataDelta())\n", "```" ] }, { "cell_type": "markdown", "metadata": { "id": "c4IZMHU2BQQt" }, "source": [ "Below is the actual act of wrapping:\n", "\n", "```python\n", "designer_factory = lambda study_config: MyDesigner(study_config)\n", "supporter: pythia.PolicySupporter = ... # Assume PolicySupporter was created.\n", "pythia_policy = designer_policy.DesignerPolicy(\n", " supporter=supporter, designer_factory=designer_factory)\n", "```" ] }, { "cell_type": "markdown", "metadata": { "id": "HcB_4mqYBt7Q" }, "source": [ "## Serializing algorithm states\n", "The above method can gradually become slower as the number of completed trials in the study increases.\n", "\n", "Thus we may consider storing a compressed representation of the algorithm state instead. Examples include:\n", "\n", "* The coordinate position in a grid search algorithm.\n", "* The population for evolutionary algorithms such as NSGA2.\n", "* Directory location for stored neural network weights.\n", "\n", "As a simple example, consider the case if our designer stores a `_counter` of **all** suggestions it has made:\n", "\n", "```python\n", "class CounterDesigner(algorithms.Designer):\n", "\n", " def __init__(self, ...):\n", " ...\n", " self._counter = 0\n", "\n", " def suggest(\n", " self, count: Optional[int] = None) -> Sequence[vz.TrialSuggestion]:\n", " ...\n", " self._counter += len(suggestions)\n", " return suggestions\n", "```" ] }, { "cell_type": "markdown", "metadata": { "id": "lw_-ta5CHeHq" }, "source": [ "Vizier offers\n", "[two Designer subclasses](https://github.com/google/vizier/blob/main/vizier/interfaces/serializable.py), both of which will use the `Metadata` primitive to store algorithm state data:\n", "\n", "* `SerializableDesigner` will use additional `recover`/`dump` methods and should be used if the entire algorithm state can be easily serialized and can be saved and restored in full.\n", "* `PartiallySerializableDesigner` will use additional `load`/`dump` methods and be used if the algorithm has subcomponents that are not easily serializable. State recovery will be handled by calling the Designer's `__init__` (with same arguments as before) and then `load`.\n", "\n", "They can also be converted into Pythia Policies using `SerializableDesignerPolicy` and `PartiallySerializableDesignerPolicy` respectively.\n", "\n", "Below is an example modifying our `CounterDesigner` into `CounterSerialDesigner` and `CounterPartialDesigner` respectively:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "kzDH-E2AKDyW" }, "outputs": [], "source": [ "class CounterSerialDesigner(algorithms.SerializableDesigner):\n", "\n", " def __init__(self, counter: int):\n", " self._counter = counter\n", "\n", " @classmethod\n", " def recover(cls, metadata: vz.Metadata) -> CounterSerialDesigner:\n", " return cls(metadata['counter'])\n", "\n", " def dump(self) -> vz.Metadata:\n", " metadata = vz.Metadata()\n", " metadata['counter'] = str(self._counter)\n", " return metadata\n", "\n", "\n", "class CounterPartialDesigner(algorithms.PartiallySerializableDesigner):\n", "\n", " def load(self, metadata: vz.Metadata) -> None:\n", " self._counter = int(metadata['counter'])\n", "\n", " def dump(self) -> vz.Metadata:\n", " metadata = vz.Metadata()\n", " metadata['counter'] = str(self._counter)\n", " return metadata" ] }, { "cell_type": "markdown", "metadata": { "id": "ZyyV8CfmHxUT" }, "source": [ "## Additional References\n", "* Our [policies folder](https://github.com/google/vizier/tree/main/vizier/_src/algorithms/policies) contains examples of Pythia policies.\n", "* Our [designers folder](https://github.com/google/vizier/tree/main/vizier/_src/algorithms/designers) contains examples of designers.\n", "* Our [evolution folder](https://github.com/google/vizier/blob/main/vizier/_src/algorithms/evolution) contains examples of creating evolutionary designers, such as [NSGA2](https://ieeexplore.ieee.org/document/996017/).\n", "* Our [designer testing routine](https://github.com/google/vizier/blob/main/vizier/_src/algorithms/testing/test_runners.py) contains up-to-date examples on interacting with designers." ] } ], "metadata": { "colab": { "last_runtime": { "build_target": "", "kind": "local" }, "name": "Writing Algorithms and Pythia Policies.ipynb", "private_outputs": true, "provenance": [] }, "gpuClass": "standard", "kernelspec": { "display_name": "Python 3", "name": "python3" }, "language_info": { "name": "python" } }, "nbformat": 4, "nbformat_minor": 0 }