Configurations¶
Configurations can be serialized to JSON dicts for storage and transfer (to for instance task servers). They can also be passed on to other configuration instances in a parent/child like relationship. Configurations have defaults which can be set when Django loads, in Django settings or a YAML file. These defaults are namespaced to prevent name clashes across apps.
Usually a request will set configurations during runtime to configure long running tasks. Configurations can also be used as a bag of properties. This is useful for Django models that have a very wide configuration range.
Getting started¶
In your apps using the ready method of the AppConfig you can register default configurations
using the register_defaults function
from django.apps import AppConfig
from datagrowth.configuration import register_defaults
class FooAppConfig(AppConfig):
name = 'your_app'
def ready(self):
register_defaults("foo_app", {
"foo": True,
"bar": False
})
Once defaults are specified you can create and use configurations with create_config anywhere in your code.
from datagrowth.configuration import create_config
config = create_config("your_app", {
"bar": "definitely!"
})
print(config.foo)
# out: True
print(config.bar)
# out: "definitely!"
print(config.get("bar"))
# out: "definitely!"
print(config.baz)
# raises ConfigurationNotFoundError
print(config.get("bar", default="absolutely!"))
# out: "absolutely!"
It’s also possible to iterate over all configurations using items.
Please note that default configurations get skipped.
from datagrowth.configuration import create_config
config = create_config("your_app", {
"bar": "definitely!"
})
for key, value in config.items():
print(key, value)
# out: bar definitely!
This is the simplest way to use Datagrowth configurations. Things get a little bit more interesting when you use Django field configurations.
Django fields¶
When you need a flexible way of configuration, without creating model fields for every option. Then you can consider the Datagrowth ConfigurationField. It can act as a bag-of-properties on your model. Or a family of models if you use it on an abstract model. Here is a basic usage example:
from django.db import models
from datagrowth.configuration import ConfigurationField
class FooModel(models.Model):
config = ConfigurationField(namespace="foo")
def foo(self):
print(self.config.bar)
Then somewhere in your code you can initialise this model with a configuration dictionary
instance = FooModel(config={"bar": True})
instance.foo()
# out: True
instance.save() # stores your model with the configuration and namespace
You can set defaults for ConfigurationField through the register_defaults function.
Pass a dictionary without prefixes to register_defaults
with foo as the namespace argument.
When models inherit from each other it is possible to change the namespace for the child class
by using CONFIG_NAMESPACE.
class BarModel(FooModel)
CONFIG_NAMESPACE = "baz"
instance = BarModel(config={})
instance.foo() # will look for a configuration named baz_bar in the defaults
The above model will look for default configurations prefixed with baz_.
If the configuration is not there and there is no default a ConfigurationNotFoundError exception gets raised.
Datagrowth configurations¶
Apart from creating your own configuration objects or using configuration fields,
it is also possible to leverage the DATAGROWTH_CONFIGURATION singleton.
The singleton lives in datagrowth.configuration and gives read access to the active
Datagrowth configuration values.
from pathlib import Path
from datagrowth.configuration import DATAGROWTH_CONFIGURATION
data_dir = DATAGROWTH_CONFIGURATION.DATA_DIR
if data_dir:
for path in Path(data_dir).iterdir():
print(path.name)
When a key does not exist, attribute access raises ConfigurationNotFoundError.
Use get with a default value when you need a fallback:
timeout = DATAGROWTH_CONFIGURATION.get("HTTP_RESOURCE_TIMEOUT", 30)
The package defaults are defined in datagrowth.yml. Use that file as the reference for available keys and default values.
Custom defaults¶
You can override package defaults by adding a datagrowth.yml file in your project root.
Datagrowth loads and merges this file over the package defaults.
The “global” namespace is the default and can be omitted when accessing a configuration.
Use nested namespaces in YAML that map to flat keys:
global:
data_dir: "/var/lib/datagrowth"
shell_resource:
bin_dir: "/usr/local/bin"
This example can be accessed as DATAGROWTH_CONFIGURATION.DATA_DIR and
DATAGROWTH_CONFIGURATION.SHELL_RESOURCE_BIN_DIR.
You can also override values from Django settings using DATAGROWTH_ variables.
The naming convention is:
DATAGROWTH_<NAMESPACE>_<KEY>for namespaced values (for exampleDATAGROWTH_SHELL_RESOURCE_BIN_DIR)DATAGROWTH_<KEY>for global values (for exampleDATAGROWTH_DATA_DIR)
Updating¶
You may need to update a configuration stored on a model based on information that comes from a different source.
For example with information from a request. This is possible using the updates method:
def foo_view(request, pk):
instance = FooModel.objects.get(id=pk)
request_configuration = request.GET.dict()
# Sanitize the user input to make sure no dangerous/unwanted configurations can take place
instance.update(request_configuration)
# Do something with the instance. It's configuration will be adjusted to the request.
Alternatively you can use the supplement method in the above example.
This method acts exactly like update, but it skips any configurations that are already in existance.
You can use supplement if you want to be sure that configurations set by the system are not overwritten by a user.
Serializers¶
Once you use and store configurations there is a good change you want to transfer them to for instance a task server. Or perhaps you’d like to initialize them from the command line.
Using the to_dict method will turn your configuration into a basic dictionary with some extra keys.
You can decorate your functions with load_config to receive these dictionaries as arguments.
The decorator will turn these dictionaries into proper configurations before passing it down to your function.
from datagrowth.configuration import create_config, load_config
config = create_config("your_app", {
"bar": "definitely!"
})
dict_config = config.to_dict()
@load_config()
def task_function(config, arg):
print(config.bar, arg)
task_function("baz", config=dict_config)
# out: definitely! baz
Alternatively you can turn your own dictionary configuration into proper configurations using from_dict
from datagrowth.configuration import ConfigurationType
config = ConfigurationType.from_dict(dict_config)
Last but not least it is possible to deserialize a configuration that is given to a Django command:
from django.core.management.base import BaseCommand
from datagrowth.configuration import DecodeConfigAction
class Command(BaseCommand):
def add_arguments(self, parser):
parser.add_argument('-c', '--config',
type=str,
action=DecodeConfigAction,
nargs="?",
default={}
)
def handle(*args, **options):
config = options["config"]
print(config.foo, config.bar)
To use this on the command line you would write:
python manage.py baz_command --config="foo=yes&bar=no"
# out: yes no
Legacy Django settings¶
Before version 0.21.x, Datagrowth had a different mechanism to be configured through Django settings.
To keep setting names predictable across YAML, environment variables and Django settings in new Datagrowth versions,
some legacy names were renamed to namespaced keys.
The table below shows how legacy settings (<0.21) map to new settings (>=0.21):
Legacy setting (<0.21) |
New setting (>=0.21) |
|---|---|
|
|
|
|
|
|
|
|
|
|