ParamTools#

Define, update, and validate your model’s parameters.

Install using pip:

pip install paramtools

Install using conda:

conda install -c conda-forge paramtools

Usage#

Subclass paramtools.Parameters and define your model’s parameters:

import paramtools


class Params(paramtools.Parameters):
    defaults = {
        "schema": {
            "labels": {
                "date": {
                    "type": "date",
                    "validators": {
                        "range": {
                            "min": "2020-01-01",
                            "max": "2021-01-01",
                            "step": {"months": 1}
                        }
                    }
                }
            },
        },
        "a": {
            "title": "A",
            "type": "int",
            "value": [
                {"date": "2020-01-01", "value": 2},
                {"date": "2020-10-01", "value": 8},
            ],
            "validators": {
                "range" : {
                    "min": 0, "max": "b"
                }
            }
        },
        "b": {
            "title": "B",
            "type": "float",
            "value": [{"date": "2020-01-01", "value": 10.5}]
        }
    }

Access parameter values#

Access values using .sel:

params = Params()

params.sel["a"]
Values([
  {'value': 2, 'date': datetime.date(2020, 1, 1)},
  {'value': 8, 'date': datetime.date(2020, 10, 1)},
])

Look up parameter values using a pandas-like api:

from datetime import date

result = params.sel["a"]["date"] == date(2020, 1, 1)
result
QueryResult([
  {'value': 2, 'date': datetime.date(2020, 1, 1)}
])
result.isel[0]["value"]
2

Adjust and validate parameter values#

Add a new value:

params.adjust({"a": [{"date": "2020-11-01", "value": 22}]})

params.sel["a"]
Values([
  {'value': 2, 'date': datetime.date(2020, 1, 1)},
  {'value': 8, 'date': datetime.date(2020, 10, 1)},
  {'value': 22, 'date': datetime.date(2020, 11, 1)},
])

Update an existing value:

params.adjust({"a": [{"date": "2020-01-01", "value": 3}]})

params.sel["a"]
Values([
  {'value': 3, 'date': datetime.date(2020, 1, 1)},
  {'value': 8, 'date': datetime.date(2020, 10, 1)},
  {'value': 22, 'date': datetime.date(2020, 11, 1)},
])

Update all values:

params.adjust({"a": 7})

params.sel["a"]
Values([
  {'value': 7, 'date': datetime.date(2020, 1, 1)},
  {'value': 7, 'date': datetime.date(2020, 10, 1)},
  {'value': 7, 'date': datetime.date(2020, 11, 1)},
])

Errors on values that are out of range:

params.adjust({"a": -1})
---------------------------------------------------------------------------
ValidationError                           Traceback (most recent call last)
Cell In[8], line 1
----> 1 params.adjust({"a": -1})

File ~/work/ParamTools/ParamTools/paramtools/parameters.py:257, in Parameters.adjust(self, params_or_path, ignore_warnings, raise_errors, extend_adj, clobber)
    210 def adjust(
    211     self,
    212     params_or_path: Union[str, Mapping[str, List[ValueObject]]],
   (...)
    216     clobber: bool = True,
    217 ):
    218     """
    219     Deserialize and validate parameter adjustments. `params_or_path`
    220     can be a file path or a `dict` that has not been fully deserialized.
   (...)
    255         least one existing value item's corresponding label values.
    256     """
--> 257     return self._adjust(
    258         params_or_path,
    259         ignore_warnings=ignore_warnings,
    260         raise_errors=raise_errors,
    261         extend_adj=extend_adj,
    262         clobber=clobber,
    263     )

File ~/work/ParamTools/ParamTools/paramtools/parameters.py:375, in Parameters._adjust(self, params_or_path, ignore_warnings, raise_errors, extend_adj, deserialized, validate, clobber)
    371 # throw error if raise_errors is True or ignore_warnings is False
    372 if (raise_errors and has_errors) or (
    373     not ignore_warnings and has_warnings
    374 ):
--> 375     raise self.validation_error
    377 # Update attrs for params that were adjusted.
    378 self._set_state(params=parsed_params.keys())

ValidationError: {
    "errors": {
        "a": [
            "a -1 < min 0 "
        ]
    }
}
params = Params()

params.adjust({"a": [{"date": "2020-01-01", "value": 11}]})
---------------------------------------------------------------------------
ValidationError                           Traceback (most recent call last)
Cell In[9], line 3
      1 params = Params()
----> 3 params.adjust({"a": [{"date": "2020-01-01", "value": 11}]})

File ~/work/ParamTools/ParamTools/paramtools/parameters.py:257, in Parameters.adjust(self, params_or_path, ignore_warnings, raise_errors, extend_adj, clobber)
    210 def adjust(
    211     self,
    212     params_or_path: Union[str, Mapping[str, List[ValueObject]]],
   (...)
    216     clobber: bool = True,
    217 ):
    218     """
    219     Deserialize and validate parameter adjustments. `params_or_path`
    220     can be a file path or a `dict` that has not been fully deserialized.
   (...)
    255         least one existing value item's corresponding label values.
    256     """
--> 257     return self._adjust(
    258         params_or_path,
    259         ignore_warnings=ignore_warnings,
    260         raise_errors=raise_errors,
    261         extend_adj=extend_adj,
    262         clobber=clobber,
    263     )

File ~/work/ParamTools/ParamTools/paramtools/parameters.py:375, in Parameters._adjust(self, params_or_path, ignore_warnings, raise_errors, extend_adj, deserialized, validate, clobber)
    371 # throw error if raise_errors is True or ignore_warnings is False
    372 if (raise_errors and has_errors) or (
    373     not ignore_warnings and has_warnings
    374 ):
--> 375     raise self.validation_error
    377 # Update attrs for params that were adjusted.
    378 self._set_state(params=parsed_params.keys())

ValidationError: {
    "errors": {
        "a": [
            "a[date=2020-01-01] 11 > max 10.5 b[date=2020-01-01]"
        ]
    }
}

Errors on invalid values:

params = Params()

params.adjust({"b": "abc"})
---------------------------------------------------------------------------
ValidationError                           Traceback (most recent call last)
Cell In[10], line 3
      1 params = Params()
----> 3 params.adjust({"b": "abc"})

File ~/work/ParamTools/ParamTools/paramtools/parameters.py:257, in Parameters.adjust(self, params_or_path, ignore_warnings, raise_errors, extend_adj, clobber)
    210 def adjust(
    211     self,
    212     params_or_path: Union[str, Mapping[str, List[ValueObject]]],
   (...)
    216     clobber: bool = True,
    217 ):
    218     """
    219     Deserialize and validate parameter adjustments. `params_or_path`
    220     can be a file path or a `dict` that has not been fully deserialized.
   (...)
    255         least one existing value item's corresponding label values.
    256     """
--> 257     return self._adjust(
    258         params_or_path,
    259         ignore_warnings=ignore_warnings,
    260         raise_errors=raise_errors,
    261         extend_adj=extend_adj,
    262         clobber=clobber,
    263     )

File ~/work/ParamTools/ParamTools/paramtools/parameters.py:375, in Parameters._adjust(self, params_or_path, ignore_warnings, raise_errors, extend_adj, deserialized, validate, clobber)
    371 # throw error if raise_errors is True or ignore_warnings is False
    372 if (raise_errors and has_errors) or (
    373     not ignore_warnings and has_warnings
    374 ):
--> 375     raise self.validation_error
    377 # Update attrs for params that were adjusted.
    378 self._set_state(params=parsed_params.keys())

ValidationError: {
    "errors": {
        "b": [
            "Not a valid number: abc."
        ]
    }
}

Extend parameter values using label definitions#

Extend values using label_to_extend:

params = Params(label_to_extend="date")
params.sel["a"]
Values([
  {'value': 2, 'date': datetime.date(2020, 1, 1)},
  {'value': 2, 'date': datetime.date(2020, 2, 1), '_auto': True},
  {'value': 2, 'date': datetime.date(2020, 3, 1), '_auto': True},
  {'value': 2, 'date': datetime.date(2020, 4, 1), '_auto': True},
  {'value': 2, 'date': datetime.date(2020, 5, 1), '_auto': True},
  {'value': 2, 'date': datetime.date(2020, 6, 1), '_auto': True},
  {'value': 2, 'date': datetime.date(2020, 7, 1), '_auto': True},
  {'value': 2, 'date': datetime.date(2020, 8, 1), '_auto': True},
  {'value': 2, 'date': datetime.date(2020, 9, 1), '_auto': True},
  {'value': 8, 'date': datetime.date(2020, 10, 1)},
  {'value': 8, 'date': datetime.date(2020, 11, 1), '_auto': True},
  {'value': 8, 'date': datetime.date(2020, 12, 1), '_auto': True},
  {'value': 8, 'date': datetime.date(2021, 1, 1), '_auto': True},
])

Updates to values are carried through to future dates:

params.adjust({"a": [{"date": "2020-4-01", "value": 9}]})

params.sel["a"]
Values([
  {'value': 2, 'date': datetime.date(2020, 1, 1)},
  {'value': 2, 'date': datetime.date(2020, 2, 1), '_auto': True},
  {'value': 2, 'date': datetime.date(2020, 3, 1), '_auto': True},
  {'value': 9, 'date': datetime.date(2020, 4, 1)},
  {'value': 9, 'date': datetime.date(2020, 5, 1), '_auto': True},
  {'value': 9, 'date': datetime.date(2020, 6, 1), '_auto': True},
  {'value': 9, 'date': datetime.date(2020, 7, 1), '_auto': True},
  {'value': 9, 'date': datetime.date(2020, 8, 1), '_auto': True},
  {'value': 9, 'date': datetime.date(2020, 9, 1), '_auto': True},
  {'value': 9, 'date': datetime.date(2020, 10, 1), '_auto': True},
  {'value': 9, 'date': datetime.date(2020, 11, 1), '_auto': True},
  {'value': 9, 'date': datetime.date(2020, 12, 1), '_auto': True},
  {'value': 9, 'date': datetime.date(2021, 1, 1), '_auto': True},
])

Use clobber to only update values that were set automatically:

params = Params(label_to_extend="date")
params.adjust(
    {"a": [{"date": "2020-4-01", "value": 9}]},
    clobber=False,
)

# Sort parameter values by date for nicer output
params.sort_values()
params.sel["a"]
Values([
  {'value': 2, 'date': datetime.date(2020, 1, 1)},
  {'value': 2, 'date': datetime.date(2020, 2, 1), '_auto': True},
  {'value': 2, 'date': datetime.date(2020, 3, 1), '_auto': True},
  {'value': 9, 'date': datetime.date(2020, 4, 1)},
  {'value': 9, 'date': datetime.date(2020, 5, 1), '_auto': True},
  {'value': 9, 'date': datetime.date(2020, 6, 1), '_auto': True},
  {'value': 9, 'date': datetime.date(2020, 7, 1), '_auto': True},
  {'value': 9, 'date': datetime.date(2020, 8, 1), '_auto': True},
  {'value': 9, 'date': datetime.date(2020, 9, 1), '_auto': True},
  {'value': 8, 'date': datetime.date(2020, 10, 1)},
  {'value': 8, 'date': datetime.date(2020, 11, 1), '_auto': True},
  {'value': 8, 'date': datetime.date(2020, 12, 1), '_auto': True},
  {'value': 8, 'date': datetime.date(2021, 1, 1), '_auto': True},
])

NumPy integration#

Access values as NumPy arrays with array_first:

params = Params(label_to_extend="date", array_first=True)

params.a
array([2, 2, 2, 2, 2, 2, 2, 2, 2, 8, 8, 8, 8])
params.a * params.b
array([21., 21., 21., 21., 21., 21., 21., 21., 21., 84., 84., 84., 84.])

Only get the values that you want:

arr = params.to_array("a", date=["2020-01-01", "2020-11-01"])
arr
array([2, 8])

Go back to a list of dictionaries:

params.from_array("a", arr, date=["2020-01-01", "2020-11-01"])
[{'date': datetime.date(2020, 1, 1), 'value': 2},
 {'date': datetime.date(2020, 11, 1), 'value': 8}]

Documentation#

Full documentation available at paramtools.dev.

Contributing#

Contributions are welcome! Checkout CONTRIBUTING.md to get started.

Credits#

ParamTools is built on top of the excellent marshmallow JSON schema and validation framework. I encourage everyone to check out their repo and documentation. ParamTools was modeled off of Tax-Calculator’s parameter processing and validation engine due to its maturity and sophisticated capabilities.