Functional-style: flattening of lists of dictionaries (in Python) |
|
||||||||
(March 2012): A transcript from a Skype dialog A friend of mine is learning Python. So far, he has only worked with imperative-style languages like C and Ada, and Python is the first language he meets where the functional style of problem-solving is easily accessible - with immediate, tangible results. He asked me today (over Skype) for help with this code: ... def foo(): ... res={} for elem in t: res[elem.keys()[0]]=elem[elem.keys()[0]] return res ...My first reaction, was... head scratching. "What exactly is it that you want to do?", I asked. "You seem to be iterating over a list of dictionaries, reading the first element of the keys() of each one of them... and assigning its value... to another dictionary? You know, there's no order preservation in dictionaries - if you depended on elem.keys()[0] to give you the first element you inserted..." He replied: "No, each of the dictionaries in the list has only one [key,value] pair. I just want to "collapse" them all into one big dictionary." "And I imagine you can't do that when creating the list of smallish one-element ones?" "No, I can't." "If you are sure that only one pair is ever inserted, you could have just used a list of tuples, you know." "I know - but can you please answer my question? How would you write this code using a lambda?" "Well before going into lambda-stuff, even in imperative style, this would be much better:" ... def foo(): ... res={} for elem in t: res.update(elem) return res ... "I see - I didn't know about .update. And this inserts all elements of elem into res?" "Yes - and is much cleaner than an element assignment, no?" "Yes, indeed. But what about a lambda version?" (Looks like my functional-style preaching has raised his interest :-) "OK, let's look at this. You know about reduce, yes?" "Well..." "It's simple: you define a function/lambda that takes two params, and returns one: the result. You then pass it to reduce, alongside the list of things it will work on... look:" Python 2.6.6 (r266:84292, Dec 27 2010, 00:02:40) [GCC 4.4.5] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> listOfInts = range(10) >>> listOfInts [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] >>> print reduce(lambda a,b: a+b, listOfInts) 45 >>>"See? The lambda just adds the two parameters and returns the sum... reduce will therefore do (((0+1)+2)+3)... Makes sense?" "Sure - and what kind of lambda would work for my problem, the flattening of dictionaries?" "Well, can you think of any operator/member that is close to what we want to "reduce" with?" "Does '+' work on dictionaries?" "No. Anything else?" "Can we use .update?" "Let's see. Does .update return the "collapsed" dictionary?"
Python 2.6.6 (r266:84292, Dec 27 2010, 00:02:40)
...
>>> a={1:2}
>>> b={4:5}
>>> print a.update(b)
None
>>> a
{1: 2, 4: 5}
>>>
"...Nope, it returns "None". So we can't use it as-is - it modifies in-place the object we call it on. Any thoughts...?"
... "Remember the semantics of "a and b" and "c or d"?" ... "How if c is None..." "Of course! If c is None, the expression 'c or d' will return with 'd's result!" "Exactly! So we could write this:"
...
>>> c=[{1:2},{4:5},{8:9}]
>>> print reduce(lambda a,b: a.update(b) or a, c)
{8: 9, 1: 2, 4: 5}
...
"a.update(b) will fill 'a' with 'b's contents, return "None", so the "or a" will return 'a' - our accumulator"
"It worked!" "Yes :-) But there's one more thing. Now that that is finished, check the contents of c[0]."
>>> print c[0]
{8: 9, 1: 2, 4: 5}
"Remember that we used something that "mutates" its first argument - in this case, we call .update on parameter 'a'... which comes from where?"
"From the input list... so we mess up the input!" "Remember how I said that functional style works better with immutable inputs? This is a good example of why." "So we lose? There's no way?" "There is, actually. You can pass your own initial value for your "accumulator" to reduce:"
...
>>> c=[{1:2},{4:5},{8:9}]
>>> print reduce(lambda a,b: a.update(b) or a, c, {})
{8: 9, 1: 2, 4: 5}
>>> print c[0]
{1: 2}
...
"Now the mutating accumulator is the '{}' we passed, not any of the list elements. Cool?"
"Amazing." "And once you learn about reduce, you now know something important: just by looking at a piece of code, if you see reduce, you know that a list of data is "collapsed" into one result, via a function that "shrinks" two args into one - in some way. You see now how that is much clearer than mystic for-loops?" "Yep. Brilliant. Thanks!"
Next, I must introduce him to itertools :-)
|
|
||||||||