Enough about Facebook. What’s your ISP done for you lately?

(4400 words)

We’ve been inundated lately by a deluge of disturbing news about the Silicon Valley Five. I say time for a bracing reminder about the real gatekeepers in digital life — your ISP. You can quit social media. But unless you’re going off the grid to embrace a 19th century lifestyle, you’re stuck at home with an access provider. Which is where the trouble starts. 

I’m going to open with a look at how astoundingly unpopular ISPs are in the US, and why that has a lot to do with chronic lack of competition in retail broadband. We’ll then dig into the FCC’s international comparison of broadband speeds and prices as they affect both Canadians and Americans — and compare those comparisons to what Canadian studies have found. We’ll close by looking at how a class assignment I launched a few years ago has given my students a hard-won understanding of the acutely anti-consumer spirit that rules the industry.

American Customer Satisfaction Index, 2018 Telecommunications Report: Comparative results for all US consumer industries

The unpopularity contest

The graph above shows the latest ranking for firms operating in the US consumer economy as compiled by the ACSI, the American Customer Satisfaction Index. You’ll notice that the industries occupying the two ranks at the very bottom are Internet service providers, ISPs, and their subscription TV services. Yes, ISPs are more unpopular than airlines, hospitals and banks — more than any other industry in the entire U.S. consumer economy.

This is the latest in a long series of unflattering portraits provided in the annual consumer rankings compiled by the ACSI, according to its 2018 Telecommunications Report, headlined Internet service providers hit an all-time low…

While video streaming services receive much better customer satisfaction scores than subscription TV, viewers still need internet access to get it. Unfortunately, internet service providers (ISPs), along with subscription TV, have the lowest customer satisfaction of all industries tracked by the ACSI (my emphasis).

Let’s clarify two points. First, for all their differences, the phone and cable companies are in exactly the same lines of business, especially broadband access to the global public Internet.

And those subscription TV services? The ones consumers really dislike are the ISP offerings, which include AT&T’s U-verse TV, Verizon Fios, DIRECTV and Optimum. The over-the-top, OTT, services like Netflix, Amazon Prime and Twitch, are much more popular — and have helped make the TV rankings better than they would have been without them:

“Video streaming services significantly outperformed subscription TV,” said David VanAmburg, Managing Director at the ACSI. “Streaming services don’t have the hidden fees and six-month rates that subscription TV does, not to mention they’re cheaper and simpler.”

The reasons for the ISP hate-on will be familiar to most American consumers, and probably to Canadians as well (I know of no Canadian data equivalent to the ASCI:

According to users, most aspects of ISPs are getting worse. Courtesy and helpfulness of staff has waned to 76, and in-store service is slower (74). Bills are more difficult to understand (-3% to 71) and customers aren’t happy with the variety of plans available (-3% to 64). 

Internet service is less reliable (69), more prone to outages (68), and performance during peak hours is worse (68). Video streaming quality is unchanged (68), but overall data transfer speed is lagging compared with a year ago (-3% to 67), as is the quality of email, storage, and security (-3% to 69). Call centers continue to be a sore spot for the industry, sinking 3% to a very low score of 59.

This abusive behavior is no accident. Market power corrupts. And market power stems from lack of competition — another crucial point to emerge from the ASCI report: 

ISPs are down 3.1% to 62, and while customers clearly aren’t satisfied with their service, more than half of Americans have only one choice for high-speed broadband. Every major ISP deteriorates this year except Xfinity, which remains unchanged (my emphasis).

Xavier Niel, founder of France’s “Free” ISP and competitive billionaire

Signs of this lack of competition show up everywhere, and it’s an old story. Back in 2015, the Center for Public Integrity did a study of Internet prices in five US cities and five comparable French cities. It found prices in the US were up to 3.5 times higher than those in France for similar service. Why would that be? Because consumers in those French cities have on average seven ISPs to choose from, as opposed to most of their American counterparts, who at best can choose from no more than two providers.

The press picked up numbers like these:

If you live in Columbus, Ohio, and you want fast internet service, you have two choices: Time Warner Cable’s $70-per-month plan or AT&T’s $61-per-month package, after any discount expires. If you live in Nice, France, it’s a different story. You can choose service from six internet providers, and you’ll pay between $19 and $40 a month for a comparable, or even faster, service.

The CPI draws unsettling conclusions about the social effects of lack of broadband affordability: 

The high prices and lack of competition and in towns like these — and there are many — add to a growing divide between the connected and unconnected. And for the unconnected, the increasing gap will be measured in fewer economic opportunities, less access to healthcare and other inequities.

Still other abuses are kept carefully hidden. The ASCI surveys don’t ask consumers (I assume) how they feel about their ISP spying on them, collecting data like their Web browsing history, which they are then free to sell to third parties. The Obama FCC saw online privacy as a crucial area for consumer protection. Then the Republican FCC killed the entire privacy initiative. Exactly what their industry supporters insisted on — calling on classic right-wing disingenuous nonsense: online privacy rules would “harm” consumers by… “confusing” them.

The anti-privacy goon squad (which, let us not forget, includes pretty much the entire Silicon Valley tech community) is inadvertently onto something. Consumers are indeed confused. But they’re not confused by rules to protect their privacy or their general welfare. They’re confused  by how ISPs operate, what they’re selling and how they sell it, and especially about how ISPs manage to make a living from making their customers miserable.

Information asymmetry: what you don’t know is definitely hurting you

In fact, there’s a common theme that runs through the problems noted above: information asymmetry. ISPs know everything about their subscribers and their services, whereas the subscribers themselves know little or nothing. This gap has a lot to do with the technical complexity of both broadband access and how it’s regulated. 

Yet the information asymmetry problem starts with the basics: most broadband subscribers don’t know what they’re paying for every month, i.e. how much “speed” they’re getting on their monthly plan, let alone that speed = bandwidth and it’s measured in multiples pf bits per second (like 150 megabits per second). I haven’t come across much hard data on this particular issue. But about six years ago, a Canadian advocacy group called the Public Interest Advocacy Centre did a study with this result:

75% of respondents to PIAC’s survey did not know the speed tier to which they subscribe even though 83% of consumers identified download speed as very important or somewhat important when choosing an ISP for their home. – Public Interest Advocacy Centre (PIAC), Ottawa, January 2013, Transparency in Broadband Advertising to Canadian Consumers (pdf)

In the interim, I’ve seen mountains of anecdotal evidence suggesting this gap hasn’t closed — including a great deal of discussion with students, who take my courses to find out what they’ve been missing. Not many consumers are motivated to follow this learning curve. But let’s not blame information asymmetry on consumer learning curves. The first and best reason to be confused lies with the jumble of service plans ISPs deploy for residential broadband — and how this jumble is handled by company reps when they’re out to sell you something. 

When consumers go looking for service, they have to weigh not just download and upload speeds against the monthly price, but also data caps, discounts and bundles that include TV and/or phone service. And since consumers don’t know what megabits per second or gigabytes are, access speeds have to be translated into activities like downloading a movie. Never mind the concept of “up to” speeds, a messy merge of the inevitable second-to-second fluctuations on any “best effort” network (like the public Internet), with the equally inevitable corner-cutting ISPs do to avoid spending money on network upgrades.

The incumbent ISPs have no qualms about taking advantage of this mess. I pulled the Bell graphic above at random, date unknown. But it points to a problem I’ve been hearing a lot about lately: Bell using its trade name “Fibe” to sow confusion among consumers about getting Internet access over fiberoptic technology (including from students doing research with Bell CSRs on the phone). 

Fiberoptic technology refers to the use of very fine glass strands to replace copper wire for providing residential access to the Internet. It’s orders of magnitude faster than either the old telco or cableco solutions, and that makes it the platform of the future.  

Unfortunately for unsuspecting shoppers, “Fibe” might be reminiscent of either fiber-to-the-node (FTTN) or fiber-to-the-home (FTTH) — if only they knew what those were, except vaguely, uh, really good. The display ad above is for a plan running at 25 Mbps. But real fiber can easily reach 1 Gbps — i.e. 1000 megabits per second, or 40 times faster than this Bell service.

It’s also capped at 100 gigabytes (Cisco reports that by 2022, the average Canadian  Internet user will be consuming 155 GB a month). Bell charges up to $100 a month if you go over the cap limit — meaning this monthly bill can turn out to be 2.5 times higher than posted ($165.95 vs $65.95). Even worse, data caps are a form of price-gouging that consumers are told are important for traffic management, which they aren’t — and research shows caps discourage online behavior in undesirable ways, especially among disadvantaged populations (see the Open Technology Institute’s “Artificial Scarcity: How Data Caps Harm Consumers and Innovation”). 

Okay, I’ll fess up. I checked today’s offers and Bell has come to its senses. You can now get so-called “Fibe Internet” running at twice the speed — 50 Mbps — without a data cap for a mere $77.95 a month. In the fine print, we’re told the upload speed will range from a minimum of 1.9 Mbps to a maximum of 10 Mbps.

Did I mention that for $9/month less ($68.95), I get three times more download speed (150 Mbps vs 50 Mbps) — and 15 Mbps upload. And no bullshit about maybe fiber, maybe not (as in TekSavvy Cable 150).

Whatever the particulars, this ad shows pretty compellingly why we have an information asymmetry problem. The convoluted decision-making we’re expected to tolerate is bad enough even if you think Bell is the only place to be. But the problems get much worse when you stray from one ISP’s walled garden and try to compare their plans with those from other providers. The tremendous difficulty posed by comparison shopping serves an important ISP business goal: it makes switching ISPs a total pain in the ass.

Ironically, while the apples-and-oranges approach is designed to discourage switching, it also gives the appearance of competition. If Bell and Rogers have lots of different plans, consumers may see this as having choice. That, of course, is an illusion since these choices are only meaningful if consumers can compare ISPs so as to find ways to save money and get better service. As we’ll now see from the unflattering details of the FCC’s international benchmarks, Canadians and Americans have no such prospects.

Sharing with our American friends: low speeds at high prices

Let’s now take a more systematic look at speeds and pricing, courtesy of the rankings  that appear in the regular assessments compiled by the FCC: its International Broadband Data Report, the 6th edition of which was released one year ago (pdf posted here). 

The FCC’s work is exhaustive in its pursuit of a fair and accurate representation of a very slippery set of “facts.” Lots of pitfalls await nternational comparisons of the performance of broadband access in the context of a “best effort” platform (the Internet), across the 29 OECD countries examined by the FCC. Just as there is no one “true” speed on a best effort network, no one broadband ranking methodology can be expected to corner the market. So the FCC comes at the problem from several different directions.

Speeds. Let’s look at speeds first, then prices. The data we’re covering here comprise fixed (wireline) and mobile, using measured not advertised speeds, over the 2014-16 period, for 29 OECD countries. If you’re uneasy about any of the FCC’s methodological choices, have a look at paras 5-6, pp.3-4 (on selection of countries); para 35, p.16 (on efforts to improve data); and para 12, p.22 (for caveats about use of Ookla data and reasons why that’s still a good idea).

Data are presented for download speeds in 2 FCC tables each for fixed and mobile, one showing the mean for each of the 28 countries, the other showing the median for each, all weighted by the number of city-level tests reported by Ookla. In the latest available year, 2016, the scores are as follows:

Mean Fixed Download Speed (2016: Table 2, p.24)

U.S. — 10th/28 (up from 14th prior year)

Canada — 17th/28 (up from 20th prior year)

One thing I’ve noticed over the years is that Canada’s speeds tend to get stuck in the 3rd quartile: the 3rd quartile range here is 15-21 (28 countries). So we’ve crept up from 20th, but not out of the 3rd quartile. It turns out that for download (but not upload) speeds for all countries, the Ookla means are consistently close to the median scores — a sign of data robustness (see Tables 3 and 8, pp.26 and 36).

The mean speeds for mobile show Canada stuck in almost the same rank as fixed — whereas US mobile is much worse than fixed:

Mean Mobile Download Speed (2016: Table 7, p.34) 

U.S. — 24th/28 (same as prior year)

Canada — 16th/28 (down from 10th the prior year)

As with fixed speeds, Canadian mobile downloads have dropped into the 3rd quartile. Mediocre is how I would describe Canada’s broadband performance on these measures.

Prices. Let’s move on to the FCC’s analysis of prices, where things get more interesting. The outline of their pricing methodologies occupies 11 densely argued pages in Appendix C. Compiling international speed rankings is a piece of cake compared to price rankings. I’m going to summarize what I see as the important points here:

First off, the usual procedure in pricing exercises like this is to smooth inter-country discrepancies by using the PPP, purchasing power parity formula, expressed in US dollars. But PPP can only account for some of the discrepancies, so the FCC takes this considerably further:

The pricing analysis in this Report is designed to account for: (1) the different costs of deploying and operating broadband networks; (2) demographic differences that affect demand for broadband service; (3) multi-product bundling in broadband pricing; (4) different product offerings in each country; and (5) the availability and quality of complementary content and applications (p.44, fn 6).

Second, they’ve collected data using stratified random sampling (sampling from sub-groups to reduce error), ending up with almost 3,000 fixed and mobile broadband plans from providers’ websites in the United States and the 28 other countries. 

Third, they’ve kept the methodology used in the past reports to generate rankings based on unweighted mean prices for standalone broadband plans within certain download speed ranges (I’m sticking with the fixed broadband analysis here to reduce complexity). But to account for the many factors that make such comparisons tricky, they add two other methodologies: a broadband price index and a hedonic price index. That allows them, for example, to calculate an implied price for broadband when bundled with multichannel video service, which is how 75% of US subscribers get their broadband service.

What we’re interested in here is how the FCC results compare across these three approaches, for Canada and the US in particular. Results from the old-fashioned method — unweighted mean monthly prices for standalone broadband — are shown in their Table 1b (p.55). The speed ranges used are a) below 10 Mbps, b) 10-25 Mbps, c) 25-100 Mbps, and d) above 100 Mbps. One thing you notice immediately is, while our two countries of interest have plans in all four ranges, many countries don’t show plans in the lower ranges (it’s not clear whether that was a sampling issue).

In Table A below (my tables are designated A-B-C-D to avoid confusion with the FCC tables), I’ve shown results for our two countries, starting with the ranks for each across the four speed ranges. They’re not dissimilar — the US is a little better in the two middle ranges. But what struck me wasn’t the rankings by themselves. Look at the figures in the columns headed “Ratio to all-country mean.”

In the far right column, I’ve posted the mean reported by the FCC for all the countries included in each speed range. Then I took the mean broadband prices for our two countries in the four speed ranges and calculated the ratio of those values to the all-country average. Think of it as the distance Canadian and American prices drift from the international average.

The striking trend in this table is how these ratios rise going from the slowest speeds to the highest speeds, for both countries. I’m going out on a limb and say this pattern tells us two things:

  • One, it’s consistent with the observation that as speeds go up, ISP choices go down since high-speed cable is taking over the high end in both countries (in large part thanks to how much better cable’s DOCSIS 3.x platform is than the telcos’ moribund DSL).
  • Two, it’s also consistent with the contempt ISPs have always shown for higher speeds, since satisfying demand means spending real money. And of course the future is higher speeds, thanks to high-def streaming and proliferating devices (Cisco reports that by 2022, Canadians will have 11 connected devices per capita, and over 3/4 of our online consumption will be video). 

Now the big question: how do these results stack up against those from the FCC’s two other, more advanced methodologies? Under the second, the broadband price index methodology, both Canada and the US still look terrible.

The second set of results differs from the first  in one basic way: it combines speed ranges rather than breaking them out, and also distinguishes between standalone and bundled broadband. But it turns out that doing it this way is still consistent with what we just saw — as shown in Table B below.

The best you can say for these results is that Americans enjoy broadband reasonably cheaply when bundled with video, because the bundle discounts are so deep. If we look back at Table A, Canada is down in the 4th quartile for the three higher speeds — exactly where Canada lands for all three results in Table B. 

Once the FCC gets to its third approach, the results get more complex because the authors use four distinct models that make adjustments for several different variables that would otherwise limit the veracity of the rankings. One complexity is that the inclusion of two particular variables — data usage and content quality — make a huge difference for certain countries, including the US, and to a much lesser degree, Canada, as we can see from Table C.

Now, just to make my overall point about Canada crystal clear… In Table D, I’ve combined all the rankings from the various FCC approaches, so you can see them in one place. And yes, the FCC warns several technical features make comparisons problematic. But I’m not comparing them. I’m simply showing that — no matter how you measure Canadian broadband prices against the other countries in the FCC set — we turn out looking like a banana republic.

And here’s the clincher. With the sole exception of the result I’ve marked in red — our rank under the slowest speed — every single one of the Canadian numbers falls into the fourth quartile.

TV vs broadband. Let’s take a moment to pause over the relationship between broadband and video — in particular the fact noted by the FCC that 75% of US subscribers get their broadband service in a bundle with video because the discounts are so deep. I’m going to assume the integrated cable ISPs offer such discounts because they’re still trying hard to sell old-fashioned TV to their customers. Yet it’s clear from a number of trends this strategy isn’t working.

First, consider the long-term prospects for the kind of TV sold by cable distributors versus those for the subscription services people actually like, such as Netflix (as the ACSI figures above indicate). Here’s one way to compare them…

Source: Rethink Technology Research

Rethink’s chart uses hours of tuning and how they’re falling dramatically for ad-based TV and rising for subscription video on demand (SVoD). We know Netflix and other OTT services are growing. We also know that subscribers to pay-TV in the US (what Canadians call cable-TV) have been cutting the cord relentlessly. In the last quarter of 2018, the five major US pay-TV distributors lost over one million subs. That may look bad for Comcast, AT&T/DirecTV, Charter, Dish and Verizon, and their counterparts in Canada. 

Turns out, however, this dump-cable trend is actually going to be much worse for many consumers. In the OTI report mentioned above on data caps, the authors include the  graph below…

Source: OTI, Artificial Scarcity (June 2015, p.3)

Starting over a decade ago, the integrated ISPs were compensated for a gradual decline in TV customers by acquiring more broadband customers. They can do this because they’re vertically integrated — combining a content business, TV, with a pipe business, Internet access. And they’re not just booking higher sub numbers — they’re also jacking up what Americans have to pay for their broadband service. In the case of Comcast, the content business happens to include billions of dollars of assets of their very own — like the NBC network and Universal Studios. 

Lucky them. Since their retail broadband rates aren’t regulated, and there’s no competition to discipline prices, the vertically integrated ISPs are laughing all the way to bank over cord-cutting. They’re screwing Americans over something they need a lot more than Hollywood movies — access to the Internet. Worse still, having their own content assets gives ISPs both the motive and the opportunity to screw with their customers, by making it cheaper to consume ISP-owned content than competitive content like Netflix.

This problem is going to get much worse as conventional TV dies. It’s also being made worse by regulators and courts that think vertical integration and concentration of ownership are good for the public interest. The latest development on that score comes from the US Court of Appeals for the District of Columbia Circuit, which this week rejected a Dept of Justice request to overturn AT&T’s $85.4 billion merger with Time Warner. When the Bellheads buy up content empires, bad things ensue… in this case, the resignation two days later of the long-time CEO of the TW crown jewel, HBO — aka “Emmy magnet” Richard Plepler.

Competing Canadian narratives on broadband

For all the effort applied by FCC staff to evaluating broadband performance internationally, we’re quite capable of doing our own assessments — which has meant a two-sided battle for the most part, pitting Ottawa’s official verdict against what the industry thinks of the official verdict.

Ottawa has conducted annual benchmarking studies to rate Canada’s telecom services — including fixed and wireless broadband — against performance in other countries since 2008. As has been observed over the years, however, it’s hard to take the reported results seriously because the basis of comparison has always been too narrow. Too few countries keep getting chosen for reasons that are never made clear. 

Two years ago, the usual reporting steward, the CRTC, handed the job to Innovation, Science and Economic Development Canada (ISED). ISED hired consultants and the consultants produced a report in October 2017 that, as far as I can see, has exactly the same problem these annual reports have always had. It says the following (p.23):

This year’s Study included seven foreign jurisdictions for the purpose of comparing telecommunications service prices with those offered in Canada. These seven jurisdictions included: the USA (four cities), the UK (London), France (Paris), Germany (Berlin), Italy (Rome), Japan (Tokyo) and Australia (Sydney). With the exception of the USA, price data were collected for the services available in each country’s largest city. 

So not only does this assessment confine itself to seven countries, chosen without any explicit rationale, it further confines the scope to a handful of cities within these countries. You could argue that comparisons with the vast FCC database would be invidious and unfair, given the resources they can call on.

But that doesn’t make the ISED report a useful tool or a good assessment or a wise use of public funds. Like so much of what Ottawa does on consumer technology, the public interest would be better served if they just didn’t wasrte the effort — like the CRTC’s ridiculous “public consultations,” the haphazard calls for comments on important proceedings, which have taken the place of actual survey research.

 In an unexpected plot twist, however, and much to the chagrin of the operators of our “world-class” telecom system, the 2017 report generated headlines like this one from Mobilesyrup:

Canadians still paying some of the highest wireless rates in the world: ISED

Fixed broadband doesn’t fare much better. Consistent with the point I made earlier about Canadian prices rising as fixed speeds go up, the Mobilesyrup article points out that “it’s in the higher speed brackets that one starts to notice how much more Canadians pay.” Among the eight countries surveyed, Canadians pay second least in the 3-9 Mbps range; third most in the 41-100 Mbps range; and in the 100-1,000 Mbps range, we pay second most, behind the ridiculously expensive prices Americans have to pay.

Unsurprisingly, the usual suspects came out in their usual defence of the industry — including one Christian Dippon of NERA Economic Consulting, who concludes not merely that the 2017 report got it all wrong, but that “all eleven iterations of this Study suffer from the same fatal flaws” (my emphasis). In an unusually confident, and question-begging, turn of phrase, the author titles his report “An Accurate Price Comparison of Communications Services in Canada and Select Foreign Jurisdictions” (my emphasis).

Did I mention NERA’s disclaimer? It was hired and paid by Telus, which has, to put it charitably, a vested interest in finding a consultant who can make them look better than the government’s consultants did.

It was this kind of endless nonsense about Canadian broadband that inspired me to start a little experiment with my students — and take the law into our own hands.

Continues in May 27 post…