Buying nothing: scoring a decade of ownership deception in Terms of Service

Quick question: what was the last movie you bought online?

Trick question. You didn't buy it. You clicked a button that said "Buy now" and what you got was a revocable licence the platform can pull whenever it feels like it.

And you're not careless for missing that. A 2017 study by Perzanowski and Hoofnagle found that 83% of people read "Buy now" as actual ownership. The button is doing exactly what it was designed to do.

That gap bugged me enough to measure it. For the Data and Society seminar at Saarland University I built DODI, the Digital Ownership Deception Index: a 0–100 score of how hard a platform's Terms of Service works to keep you from noticing what you didn't buy. Higher means more deceptive. Everything is public: scorer, corpus, figures.

No LLM. On purpose.

Here's the design choice I'll defend the hardest: DODI contains no model. No API calls. Nothing random. It's a plain function of the document text:

DODI = 0.25 × readability penalty
     + 0.50 × licence ratio
     + 0.25 × red-flag score

Three ingredients:

• Licence ratio (half the score). Count licence words ("license", "subscription", "access", "grant"…) against ownership words ("buy", "own", "purchase"…). This is the core trick (storefront says buy, contract says licence) so it gets the dominant weight.
• Readability (a quarter). Flesch–Kincaid grade level via textstat. Grade 8 or below scores 0, grade 16+ scores 100. Complexity enables deception but isn't deception itself.
• Red flags (a quarter). Counts of aggressive clauses: unilateral termination ("sole discretion", "without notice"), rights waivers ("class action", "arbitration", "indemnify") and data exploitation ("third parties", "sell your", "track").

Every term counts as a whole word, along with its real forms ("purchased", "licences", "terminated").

Why does the no-LLM part matter so much? Because the whole study compares 2015 documents against 2024 documents. If your measuring stick drifts, you can't tell instrument drift from document drift.

Same text in, same score out, forever. No LLM will promise you that.

Ten years of fine print, measured

The corpus: 40 documents. Ten major platforms, four ToS snapshots each (2015, 2018, 2021, 2024) pulled from the Wayback Machine.

The language got worse. This is the decade regulators started taking dark patterns seriously. The mean DODI score still rose from 77.5 to 81.7 and seven of ten platforms drifted toward more licence-heavy, less readable, more aggressive terms.

The score lands where it should at the honest end. GOG, the DRM-free store whose entire brand is "you actually own your games", scores lowest of all ten platforms in every single year. 27.4 in 2015. I didn't tune anything to make that happen; it's the sanity check the 25/50/25 weighting had to pass before I trusted it. The deceptive end is less clean. Adobe and Twitter score 86 to 100, but a licence-heavy contract isn't automatically a dishonest one. Adobe openly says "licensed, not sold" and prices everything per month, so is it really deceiving anyone? That gap between counting licence words and judging actual deception is the thing I went after next, in a separate write-up.

Netflix 2024 is my favourite data point. Ownership words in its ToS: 1. Licence words: 98. That's a ratio of 98, the highest in the corpus. Read that again the next time you "buy" something to watch.

Does it agree with actual humans?

Weakly. And I'd rather tell you that than dress it up.

I validated DODI against ToS;DR, the project where humans grade Terms of Service. On the ten documents with reviewed ToS;DR grades, DODI correlates at Spearman ρ = 0.45 (p = 0.19). Pearson r = 0.20. That's a weak signal and it isn't significant.

To be fair, part of the gap isn't error at all. ToS;DR grades general fairness. DODI grades ownership transparency. Those are different things.

Wikipedia is the clean example: it gets a good ToS;DR grade (B) but a high DODI score of 88. Its content is free under a copyleft license, so the text is packed with licensing language, which is exactly what DODI counts. Part of that is a real difference in what the two measure. Part of it is DODI's actual flaw: honest licensing looks like deception to a word counter.

Full disclosure on that example: I originally had this document labeled "WhatsApp" in my corpus. It's Wikipedia. The grade and the ToS;DR ID were right. The name on the file was wrong. My keyword scorer never noticed because it doesn't read, it counts. What caught it was a later pass where I had an LLM grade the same documents: it read the text, said "this is community-contributed copyleft content, not a messaging app." The mismatch fell out. That's the whole argument for adding a second kind of check. The deterministic scorer is reproducible and blind. The reader is fallible and awake. You want both.

Where DODI is weak (I'll say it before you do)

• Keyword counting can't read negation. "You do not own this content" counts as an ownership word. Spot checks suggest deceptive ToS rarely negate like this (they avoid ownership words entirely, which the ratio catches) but it's a real blind spot.
• n = 10 is small. ρ = 0.45 at p = 0.19 is a weak signal, not a result you should bet on.
• "Service" dominates the licence count. Every ToS says "Services" constantly, so about two thirds of all licence hits are that one word. The ratio partly measures how often a document names its own product.
• Wayback timing is coarse. A "2015" document is the nearest capture to that year, not January 1st.
• The score is gameable. A platform could stuff its ToS with ownership words without changing a single legal term. DODI measures language, not legal substance. Don't pretend otherwise if you use it.

What I'd build next

Negation-aware clause parsing with spaCy dependency trees would close the biggest blind spot. I've since checked DODI against three LLM judges from two vendors. They barely agree with it (write-up). Clause-level embeddings could trace how licence clauses copy themselves between platforms over time.

But the metric I actually want? The marketing-vs-ToS gap. The deception is largest exactly where the storefront screams "buy" and the contract whispers "licence". That gap is measurable. Someone should measure it at scale.

Everything here reproduces from the repo with three scripts and no GPU. Or skip the scripts entirely: paste any ToS into the live scorer and see for yourself. You won't like the number.