Creating a Map Reduce Function in Python from edge list

Question

I need some help creating a Map Reduce function in Python from an edge list.

Given the following list:

A,B
A,C
A,D
B,C
C,A
C,B
D,A

My code should follow the format below. My goal is to display a degree list along with the count of degrees.

map(key, value):
//key: document name; value: text of the document
    for each word w in value:
        emit(w, 1)

reduce(key, values):
//key: a word; value: an iterator over counts
    result = 0
    for each count v in values:
        result += v
        emit(key, result)

Loops have always been a struggle for me. Can someone point me in the right direction?

The output should be as follows:

Degree     Count
1           2
2           1
3           1

The code should work for any data set similarly formatted

So, as I understand it, I need to count the number of different values each letter is paired with--this would be the "degree" (A, for example, is degree 3), and then total the pairs according to the degree--this would be the "count".

Could the key be the pair of values?

map(key, value):
//key: document name; value: text of the document
    for each word w in value:
        emit([x,y], 1)

Seems like you posted the word count example. What are the column labels in your input and your desired output? — pault
– pault, Commented Nov 29, 2017 at 3:34

pault · Accepted Answer · 2019-03-13 14:47:37Z

1

I'm not sure if MapReduce is the best way to approach this problem, but I think the following makes sense.

First Map each Vertex-Edge pair to the (Vertex, 1). Then Reduce by summing the counts for each Vertex.

map(key, value):
//key: vertex; value: edge
    emit(key, 1)

reduce(key, values):
//key: vertex; value: an iterator over counts
    result = 0
    for each count v in values:
        result += v
        emit(key, result)

This assumes that the input rows are unique.

edited Mar 13, 2019 at 14:47

answered Nov 29, 2017 at 3:48

pault

43.7k17 gold badges121 silver badges161 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Creating a Map Reduce Function in Python from edge list

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related