Mam taką listę słowników: [{'topic_id': 1, 'average': 5.0, 'count': 1}, {'topic_id': 1, 'average': 8.0, 'count': 1}, {' topic_id': 2, 'average': 5.0, 'count': 1}] Chciałbym zmapować i ...

2
Gabriel Roger 23 czerwiec 2021, 16:47

4 odpowiedzi

Najlepsza odpowiedź

Możesz to osiągnąć za pomocą kilku wyrażeń, funkcji map i funkcji mean z wbudowanego modułu statistics.

from statistics import mean
data = [
    {
        'topic_id': 1, 
        'average': 5.0, 
        'count': 1
    }, {
        'topic_id': 1, 
        'average': 8.0, 
        'count': 1
    }, {
        'topic_id': 2, 
        'average': 5.0, 
        'count': 1
    }
]
# a set of unique topic_id's
keys = set(i['topic_id'] for i in data)
# a list of list of averages for each topic_id
averages = [[i['average'] for i in data if i['topic_id'] == j] for j in keys]
# a map of tuples of (counts, variances, averages) for each topic_id
stats = map(lambda x: (len(x), max(x) - min(x), mean(x)), averages)
# finally reconstruct it back into a list
result = [
    {
        'topic_id': key, 
        'count': count, 
        'variance': variance, 
        'global_average': average
    } for key, (count, variance, average) in zip(keys, stats)
]
print(result)

Zwroty

[{'topic_id': 1, 'count': 2, 'variance': 3.0, 'global_average': 6.5}, {'topic_id': 2, 'count': 1, 'variance': 0.0, 'global_average': 5.0}]
1
Axe319 23 czerwiec 2021, 15:41

Oto próba użycia itertools.groupby do pogrupowania danych na podstawie topic_id:

import itertools

data = [{'topic_id': 1, 'average': 5.0, 'count': 1}, {'topic_id': 1, 'average': 8.0, 'count': 1}, {'topic_id': 2, 'average': 5.0, 'count': 1}]

# groupby
grouper = itertools.groupby(data, key=lambda x: x['topic_id'])

# holder for output
output = []

# iterate over grouper to calculate things
for key, group in grouper:

    # variables for calculations
    count = 0
    maxi = -1
    mini = float('inf')
    total = 0

    # one pass over each dictionary
    for g in group:
        avg = g['average']
        maxi = avg if avg > maxi else maxi
        mini = avg if avg < mini else mini
        total += avg
        count += 1

    # write to output
    output.append({'total_id':key,
                   'count':count,
                   'variance':maxi-mini,
                   'global_average':total/count})

Dając to output:

[{'total_id': 1, 'count': 2, 'variance': 3.0, 'global_average': 6.5},
 {'total_id': 2, 'count': 1, 'variance': 0.0, 'global_average': 5.0}]

Zauważ, że 'variance' dla drugiej grupy to 0.0 tutaj zamiast 5.0; różni się to od oczekiwanego wyniku, ale myślę, że tego chcesz?

1
Tom 23 czerwiec 2021, 14:51

Jeśli chcesz użyć pand, wydaje się, że jest to odpowiedni przypadek użycia:

import pandas as pd

data = [{'topic_id': 1, 'average': 5.0, 'count': 1}, {'topic_id': 1, 'average': 8.0, 'count': 1}, {'topic_id': 2, 'average': 5.0, 'count': 1}]

# move to dataframe
df = pd.DataFrame(data)

# groupby and get all desired metrics
grouped = df.groupby('topic_id')['average'].describe()
grouped['variance'] = grouped['max'] - grouped['min']

# rename columns and remove unneeded ones
grouped = grouped.reset_index().loc[:, ['topic_id', 'count', 'mean', 'variance']].rename({'mean':'global_average'}, axis=1)

# back to list of dicts
output = grouped.to_dict('records')

output to:

[{'topic_id': 1, 'count': 2.0, 'global_average': 6.5, 'variance': 3.0},
 {'topic_id': 2, 'count': 1.0, 'global_average': 5.0, 'variance': 0.0}]
1
Tom 23 czerwiec 2021, 15:06

Możesz także spróbować użyć funkcji agg pandas dataframe w ten sposób

import pandas as pd

f = pd.DataFrame(d).set_index('topic_id')

def var(x):
    return x.max() - x.min()

out = f.groupby(level=0).agg(count=('count', 'sum'),
        global_average=('average', 'mean'),
        variance=('average', var))
1
Ivan Calderon 23 czerwiec 2021, 15:41