Merging dicts using singledispatch    Posted:


I am working on some code that reads config values from different sources. I would like to merge them in a way that values that are themselves dicts get the items from both sources, something like:

source_1 = {'foo': 10, 'options': {'a':1, 'b':1}
source_2 = {'foo': 20, 'options': {'a':2, 'c':3}

# merging source_1 and source_2 would give me:
{'foo': 20, 'options': {'a':2, 'b':1, 'c':3}

Notice how this is different from a simple dict.update() because options contains an item 'b'. Apart from that I would like list to be merged, that is merging [1, 2] and [3, 4] would result in [1 ,2, 3, 4].

I found some implementations but it seems that everybody has a different use-case for what "merge" should exactly do. So I decided to create a MergeDict that can be easily extended/configured.

singledispatch

The merge operation is specific depending on the type of the item. That makes a perfect opportunity to use singledispatch. Python 3.4 added support for single dispatch (see PEP 443). I am not using python 3.4 yet but there is also a package on pypi.

A single-dispatch is form of generic programming where you register different functions to be executed depending on the type of the first argument.

Here is a simple example:

from singledispatch import singledispatch

# the function to be executed by default gets a singledispatch decorator
@singledispatch
def fun(arg):
    print("default: {}".format(arg))

# register alternate function when argument is an int
@fun.register(int)
def _(arg):
    print("int: {}".format(arg))

# register alternate function when argument is a list
@fun.register(list)
def _(arg):
    print("list: {}".format(" ".join(str(a) for a in arg)))


fun('hi')
# default: hi

fun(1)
# int: 1

fun([1,2,3])
# list: 1 2 3

MergeDict

A MergeDict defines a merge() method. Each item is merged using the merge_value() function, by default it works the same as update() just replacing the value...

class MergeDict(dict):

    def merge(self, other):
        """merge other dict into self"""
        class Sentinel: pass
        for key, other_value in other.items():
            this_value = self.get(key, Sentinel)
            if this_value is Sentinel:
                self[key] = other_value
            else:
                self[key] = self.merge_value(this_value, other_value)

    @staticmethod
    def merge_value(this, other):
        """default merge operation, just replace the value"""
        return other

Applying singledispatch to class methods is a bit tricky, but ideally users would sub-class from MergeDict and define new merge() methods with a decorator similar to singledispatch.

The idea is to just annotate the methods but really register the dispatch only on object initialization.

class MergeDict(dict):

    def __init__(self, *args, **kwargs):
        super(MergeDict, self).__init__(*args, **kwargs)
        # register singlesingle dispatch methods
        self.merge_value = singledispatch(self.merge_value)
        for val in self.__class__.__dict__.values():
            _type = getattr(val, 'merge_dispatch', None)
            if _type:
                self.merge_value.register(_type, val)


    class dispatch:
        """decorator to mark methods as single dispatch functions."""
        def __init__(self, _type):
            self._type = _type
        def __call__(self, func):
            func.merge_dispatch = self._type
            return func

An example of a merge that sum up int values:

from mergedict import MergeDict

class SumDict(MergeDict):

    @MergeDict.dispatch(int)
    def merge_int(this, other):
        return this + other

a = SumDict({'a':2, 'b':3})
a.merge({'a':2, 'c':5})
print(a)
# {'a':4, 'b':3, 'c':5}

TL;DR;

Check mergedict on pypi.

Comments

github pull request workflow    Posted:


A quick reference so I don't need to re-learn these steps every time I do a pull-request...

  • clone the repo in github

  • clone the repo on your machine

git clone xxx
  • create a branch
git branch fix-something
  • use the branch
git checkout fix-something
  • hack something then commit
git commit -a
  • push new branch
git push origin fix-something
  • on github click create pull request (or something like that)

Comments

generating HTML with javascript    Posted:


I often hear people saying something like "HTML must be created using a template engine, anything else is a bad practice". I will try to demystify this dogma...

HTML

HTML is a text document. But browsers don't handle the text directly. It is first converted to DOM (document object model). So HTML is more like a declarative way to describe a user interface program.

This is a very important distinction, one is text the other is an object in a system.

Most components (HTML tags) simply describe how to display some data but other are actual UI components. For example a link (a tag) has an associated interaction with the user, but being part of the standard the browser already knows how to handle it. You don't need to write code for it, you just need to declare its href...

server-side templates

At some point the web moved from static HTML pages to dynamically generated HTML by the server side. After some time people realized that mixing text generation in the code could get quite messy. So they started using template engines.

A template engines combine a template with some data source to create a HTML text document.

Nowadays it is pretty much an established consensus that when generating HTML on the servers-side you should use a template engine, and I agree that's the right thing to do.

javascript templates

As the web evolved the code started to move from server to client. And the templates came together...

But wait. The server has no access to the DOM it can only create HTML text. Javascript has direct access to the DOM!

Should we always programmatically write some text in a declarative interface to create some UI components? Why not directly create these UI objects?

comparison

Lets compare different ways to modify the DOM using javascript to generate the content <div>Hello there</div>.

1) using DOM API

var myDiv = document.createElement("div");
var myContent = document.createTextNode("Hello there!");
myDiv.appendChild(myContent);
document.body.appendChild(myDiv);

Note this is not writing HTML text (although sometimes we may call it this way). It is manipulating the DOM directly. Oh, yes, it sucks :)

2) using jQuery

$('body').append($('<div>Hello there!</div>'));

That's funny, even that we are manipulating the DOM directly you must write HTML markup.

3) another DOM API (hoe.js)

$('body').append(div('Hello there!'));

That looks more reasonable.

4) using templates (Handlebars)

template = Handlebars.compile("<div>{{content}}</div>")
$('body').append(template({content: 'Hello there!'}));

This example is too contrived, normally you would put the template in a script tag.

declarative vs imperative

So which one is better? The answer is, of course, it depends...

Declarative is good for data. If you are just laying out the structure for some data, and using some existing UI component (HTML-tag). Using a template you will get a much more readable code.

example (handlebars):

<ul class="people_list">
  {{#each people}}
  <li>{{this}}</li>
  {{/each}}
</ul>

Declarative is also good if you are just using some custom component. In the example bellow a framework/library would look for elements with a class attribute indicating the widget to be used and the data- attributes to configure it.

<ul class="widget-XYZ" data-opt-a="5">
   some content
</ul>

Template has a big disadvantage, it produces text and loses the direct access to the DOM that is available from javascript.

When the web "moved" from server to the client the default UI elements provided by HTML were not enough to provide a rich user experience. So developers often need to create new UI components in javascript.

If you are creating a new UI component where you need to specify the behavior for a click and other actions you must manipulate the object in the DOM because you need to attach events.

example (hoe.js):

var myComponent = div('click me');
myComponent.click(function(){alert('Hello there!')});

So in one way or another you will have to get back a reference to the created object. If you use a template this is usually done by a CSS selector.

Using a CSS selector is bad for many reasons. It might be slow (need to search in the DOM). The code is error prone because it depends on attribute values that are far away from the code. The selectors have two meanings (styling with CSS and reference in javascript) that can gets confusing.

Another choice would be that your template/HTML reference the javascript code. For very simple things this was always possible using onclick and other attributes for events. Some frameworks provide more sophisticated mechanism like associating with a "controller", etc. The problem with this approach is that it doesn't scale as your component evolves. When it gets more complex you try to stuff more logic in the template. When your template can't handle it anymore you need to throw away your template and re-write it in javascript...

conclusion

If you are creating an UI component, manipulating the DOM from javascript is probably your best option.

If you are just filling some data and using existing components, templates are probably better for you.

Comments