This is part of a series on mining your own Gmail data.
For this post I want to tackle a fun question: how long would it take to read all of my email if that's all I did, 24/7? It's one of those questions that should interest anyone who's concerned about information overload or is looking to pare down their information consumption: "Just how much of my time is theoretically committed to my inbox?"
First, the obvious: nobody actually does this. No one actually reads every email they receive from start to finish (as anyone who's dealt with email in a corporate environment knows all too well). Most of us have filters (both electronic and mental) set up to glean the info we need and skip the rest.
And I'll bet that a lot of email is written without any expectation that the whole thing will be read; the author may be well aware that different portions of the email are relevant to different recipients, or that the email itself will only be interesting to a subset of the mailing list (e.g., many marketing emails).
So it's not the one super-relevant data point that should make people completely re-think their information consumption habits or anything like that. But it is fun to think about, and as one data point among many others it might prove interesting or useful.
On to the fun part - actually coming up with a number!
Like most people, I'm getting new emails all the time. So technically I should be taking into account all the new emails I receive while I'm still reading through my old ones. But that's hard, so I'm not going to bother. Instead, I'm just going to assume I've stopped getting emails at all while I'm reading. Which means that getting a basic number is easy - I just have to count all the words in all my emails, divide that by the number of words per minute I read, and I've got the number of minutes it would take to read everything.
The first thing I need to do is go back to my PowerShell script and pull in the body of each email. This is where we hit the first snag - HTML emails.
For doing word counts, I really don't want to look at HTML emails, because there's a ton of junk in there which a human won't be reading. Luckily, most email clients which send HTML emails also include a text version; in those cases, we'll just extract that text portion of the email and ignore the HTML. Unfortunately, this isn't always the case; when there's not a text version available, we'll just have to get the HTML and figure out how to deal with it later.
As usual, MimeKit will be doing most of the work. This version of the script is pretty similar to our previous ones, except that we have to loop through the possible body formats for each message to figure out which formats are available. We always check for the 'Text' format first, because that's the one we really want. If that's not available, we run through the others until we find one that works.
The relevant changes are the hash of the possible formats, which we use for iteration and for tracking the number of emails of each type:
$formats = @{
[MimeKit.Text.TextFormat]::Text = 0;
[MimeKit.Text.TextFormat]::Flowed = 0;
[MimeKit.Text.TextFormat]::Html = 0;
[MimeKit.Text.TextFormat]::Enriched = 0;
[MimeKit.Text.TextFormat]::RichText = 0;
[MimeKit.Text.TextFormat]::CompressedRichText = 0
}
And the section where we determine what the actual format is and store it:
$bodyText = $null
$actualFormat = $null
# Run through all the enumeration values
# The pipe through sort ensures that we check them in the enum order,
# which is great because we prefer text over flowed over HTML, etc.
$formats.Keys | sort | % {
# try each Format until we find one that works
if($actualFormat -eq $null) {
# Try to get the body in the current format
$bodyText = $mimeMessage.GetTextBody($_)
if($bodyText) {
$actualFormat = $_
}
}
}
if($actualFormat -eq $null) {
$unknownFormat += 1;
$actualFormat = "Unknown"
} else {
$formats[$actualFormat] += 1;
}
You can find the full script here.
A couple of notes:
- This isn't perfect; sometimes MimeKit can't really figure out what the format is. For example, I have some Skype notification emails which MimeKit thinks are HTML only, but are in fact text. I'm not sure why MimeKit gets confused (probably incorrect headers in the original emails), but out of about 43,000 emails only a couple dozen seem to have issues, so I'm not going to worry about it.
- In all of my emails, the only two formats returned were Text and HTML. This might have something to do with what Gmail supports; I've seen some posts that suggest Gmail doesn't support Flowed, though those may be outdated. In any case, I'm only really dealing with Text and HTML in my word counts.
Once we've got the data, we can load it up in pandas and do some counting. Doing a naive count of the words in the plain text emails is trivial; we just define a method that uses Python's split method with None as the delimiter argument, and then look at the length of the returned list. Here's what textWordCount looks like:
def textWordCount(text):
if not(isinstance(text, str)):
return 0
return len(text.split(None))
But the HTML emails are problematic because most of the content is markup that the user will never actually read. So we need to strip all that markup out and just count the words in the text portions of the HTML. To do that, we create another method which parses the HTML email content using the amazing Beautiful Soup library, strips away the style, script, head, and title parts, and extracts the text from what's left using get_text(). Once we've got the actual human-readable text, we can run it through our usual word counting method:
def htmlWordCount(text):
if not(isinstance(text, str)):
return 0
soup = bsoup(text, 'html.parser')
if soup is None:
return 0
stripped = soup.get_text(" ", strip=True)
[s.extract() for s in soup(['style', 'script', 'head', 'title'])]
stripped = soup.get_text(" ", strip=True)
return textWordCount(stripped)
I took a couple of online tests to get an idea of how fast I read and came up with 350 words per minute. With that bit of data, we can now add some more columns to our data and figure out the total time to read all the emails:
def wordCount(row):
if(row['Format'] == 'Html'):
return htmlWordCount(row['Body'])
return textWordCount(row['Body'])
averageWordsPerMinute = 350
# Count the words in each message body
emails['WordCount'] = emails.apply(wordCount, axis=1)
emails['MinutesToRead'] = emails['WordCount'] / averageWordsPerMinute
# Get total number of minutes required to read all these emails
totalMinutes = emails['MinutesToRead'].sum()
# And convert that to a more human-readable timespan
timeToRead = humanfriendly.format_timespan(totalMinutes * 60)
The full script is here, if you're playing at home.
Running that against all of my Gmail gives me:
>>> timeToRead
'2 weeks, 6 days and 18 hours'
So if I sat down and read at my fastest speed 24/7 for three weeks straight with no breaks, no sleep, and never slowing down, I could finish reading every word of every email I've ever received in my Gmail account. If I only read them 8 hours a day, it'd take me about 9 weeks to finish.
That's actually less than I expected, though "two whole months of your life spent just reading your email" is a still a bit sobering.
Sobering enough that I'm not going to try to compute this for my other four email accounts, anyway.