The space between the signal and the action

In the game-theoretic world, the gunner never shoots: the other side looks at the options down the game tree, realizes that one action will lead to his or her getting shot, and doesn’t take that action. In Game Theory textbooks, cases never go to court: both sides calculate the risk-adjusted expected payoff from trial, and if it is positive for one hyperrational side, then it is negative for the other hyperrational side, and a settlement can be calculated based on that. In both cases, knowledge that an event could occur largely has the same effect as the event itself.

But there was an eight-year glitch in the game-theoretic matrix, which has given us an interesting chance to see what happens between when a signal of a legal position is sent and when the position actually hits.

Here in 2014, the Supreme Court handed down a ruling regarding the patentability of software, Alice Corp v CLS Bank. The first draft of the ruling was published in 2006.

The question in the 2006 case of Labcorp v Metabolite was whether a patent of this form was valid:

1. draw the patient’s blood,
2. measure the level of a chemical in the blood, and
3. correlate that level to the risk of a disorder.

Steps 1 and 2 are by themselves not patentable, because they are physical processes but are entirely not-novel. Step 3 may be a novel correlation, but it is an abstract scientific discovery, and one may not patent abstract ideas or research results. So is the combination of the physical-but-quotidian steps 1+2 with the abstract-but-novel step 3 a physical and novel invention?

The Court’s decision (PDF) was one sentence long:

The writ of certiorari is dismissed as improvidently granted.

For those whose Latin is rusty, the ruling says that due to a procedural glitch, the Court should not have heard the case, and would not rule. However, Justice Breyer, with Stevens and Souter, wrote a 15-page dissent (see the above PDF). It was a hot one, with a real sense of urgency that allowing this patent to stand will cause real damage. It argued that the patent should not stand because abstractions are not patentable for good reason, and the physical steps in this patent are just a cursory attempt to get past the bar from patenting abstractions. In game-theoretic terms, this was the Justices signaling as well as they could given that technicalities precluded a ruling.

Years passed, and nothing further was said. No member of the Supreme Court has a blog. They make an effort to give rulings that are as concise as possible. Sure, they heard cases involving sketchy patents, but if the case was about applying patents across borders, then the ruling would take the patent as given and discuss the cross-border issue. If the question was about patenting business methods, then the ruling would be about whether it’s OK to patent business methods.

So during that period, the last signal the Supreme Court gave regarding the patents like the three-step patent above was the dissent in Labcorp. You can read tea leaves, but such faint signals pale compared to the strong language of the Labcorp dissent.

Let’s apply this to software. Here is another patent template:

1. buy a (physical but quotidian) stock computer;
2. load a (novel but abstract) program onto it.

This is the same format as in Labcorp in a different context, and the logic of the Labcorp dissent would invalidate this sort of patent as well. Nonetheless, during this period the US Patent and Trademark Office granted so many Labcorp-like patents that they had to move to a new (very attractive) office complex and implement a new telework infrastructure to handle all the new examiners. As will be discussed below, probably tens of thousands of these patents were completely invalid under the logic of the Labcorp dissent.

The PTO provides this handy PDF breaking down how many patents were granted by category. Between 1993 and 2004, there was growth in most fields, but typically a doubling or tripling. If you want to see the real growth, you have to skip down to the 700 series, where most of the load my novel algorithm on a stock computer patents lie.

Here’s a trimmed down table of patents granted in certain categories:

cat # field 1993 count 2013 count change
250 radiant energy 1887 3552 188%
267 spring devices 291 330 113%
273 amusement devices: games 430 538 122%
324 Electricity: Measuring and Testing 1272 2741 215%
340 communications: electrical 1332 4995 375%
704 Data Processing: Speech Signal Processing, Linguistics, Language Translation, and Audio Compression/Decompression 252 2053 815%
705 Data Processing: Financial, Business Practice, Management, or Cost/Price Determination 229 7897 3,448%
715 Data Processing: Presentation Processing of Document, Operator Interface Processing, and Screen Saver Display Processing 263 4350 1,654%
729 Information security 21 3810 18,143%
801 Multicellular Living Organisms and Unmodified Parts Thereof and Related 38 1536 4,402%
901 Robots 204 336 165%

Think back to 1993 for a minute: there were people doing research in speech therapy, there were devious financiers looking for an edge, there were people trying to make their presentations interesting, there was information that had to be secured, there were multicellular living organisms. The invention of the Internet (in the 1970s and 80s) certainly made all fields more efficient, but it’s hard to argue that financiers in 2013 were 34 times as inventive as the financiers of 1993. You get 375% growth in patent counts by having more engineers using better technology in their research, and you get 1,600% growth by making patentable what before had not been.

They did so with the support of the Court of Appeals for the Federal Circuit (CAFC), who presented a series of rulings pushing the boundaries of what is patentable far beyond the bounds signaled in Labcorp; more on them below.

In 2012, the case of Mayo v Prometheus was decided. The three-step medical test patent I described above in the Metabolite patent also describes the Prometheus patent. The ruling followed exactly what Breyer said in his Labcorp dissent: duct-taping together an obvious physical step to a novel abstract discovery does not make the discovery patentable. It was a unanimous ruling, closely following the 15-page sketch from 2006.

The count of rejections started to rise somehwat here, but there were judges in patent-centric courts who were still reluctant to apply this to computing patents, because they perceive that enviornment as too different from the medical test environment. See the majority ruling Judge Prout dissents from and Judge Michel’s amicus brief, below.

Here in 2014, Alice Corp v CLS Bank asked about a patent regarding arguably novel software on a stock computer, as per the two-part patent above. To summarize the ruling in Alice: directly apply the ruling in Mayo v Prometheus to find this patent invalid, for exactly the same reasons. This one was also unanimous.

Dennis Crouch, well-regarded legal scholar/blogger, describes the change brought about by Alice thusly:

Alice Corp. fits within that precedent as an incremental addition, but without rejecting or even modifying the Supreme Court’s own prior precedent. Rather, what Alice Corp. has rejected is the prior analysis of the Federal Circuit and US Patent Office that seemingly allowed for the patenting of systems and processes whose inventors relied upon only an iota of hardware to separate the patented invention from an underlying abstract idea or law of nature.

The new cohort of [Supreme Court rulings] collectively wipe-away thirty years of Federal Circuit precedent on eligibility, but at the same time, revive 150-years of Supreme Court doctrine on the topic.

The buzz when Alice came out was cautious and didn’t want to assume that much would change. [For those keeping score, here's the only thing I said about it.] But the present situation at the patent office and at the patent-centric courts is a patent bloodbath. Have a look at this article entitled “Software patents are crumbling, thanks to the Supreme Court“, or see “Already Alice Corp. v. CLS Bank Has Brought a Sea-Change in Patent-(In)eligibility“.

Back to the game-theoretic framing, if the Patent and Trademark Office and CAFC were seeking to maximize long-term stability, they would have looked down the game tree in 2006 and seen that the Labcorp ruling would eventually be precedential, so they might as well follow it immediately.

But that’s not what happened. Why?

Those who firmly believe that the Labcorp dissent was not a good position might have committed a host of confirmation biases and taken every piece of evidence hinting at another direction as proof that the Court would never implement Labcorp as a majority ruling. So that might be the story.

There were lots of patent rulings from other courts. As above, the majority on the Federal Circuit’s appeals court was especially active, but it was often swatted down by unanimous rulings by the Supreme Court over the course of this period. So if we commit the ecological fallacy and think The Law has a will and a course, then The Law, mostly steered by the Federal Circuit majority, was moving toward allowing Labcorp-style patents. But if we recognize the CAFC and SCOTUS as distinct bodies, SCOTUS was sending lots of signals that it (often unanimously) felt the Federal Circuit majority was doing it wrong. So there’s another way a not-perfectly-rational agent might misread things.

If the agents were fully rational, and knew Alice would come eventually, then we have to move on to the incentives of the agents. The Patent Office’s incentives push it to grant as many patents as the law allows. Department of Commerce employee evaluation forms rate employees on how well they protect intellectual property, not on how fairly they adjudicate the balance of what is or is not subject to IP protections. The head of the PTO through much of this period, David Kappos, had previously been the VP for intellectual property at IBM; IBM benefited immensely from having its Labcorp/Prometheus/Alice-failing patents for eight years, even if many of those patents are now worthless.

The judges on the CAFC have a more interesting utility function. They are effectively appointed for life, and you’d have to really be chummy with the litigants before there’s any buzz about removing a judge. So they have latitude to rule in terms of what they think is the correct law.

The short story for the CAFC v SCOTUS conflict is that the CAFC wants to find what is optimal for the patent system and the Supreme Court wants to find what is optimal for the United States. See Ars Technica or Cato for much discussion of the specialist court.

This has certainly led to conflict. But don’t take my word for it; here’s Judge Prout of the CAFC: “The majority [of CAFC judges who wrote this ruling] resists the Supreme Court’s unanimous directive to apply the patentable subject matter test with more vigor. [...] The majority has failed to follow the Supreme Court’s instructions [in Prometheus]—not just in its holding, but more importantly in its approach.” Or on the other side, Justice Roberts had a famous quip during the oral testimony for a patent case mostly about court procedure: “[Other courts] can’t say, I don’t like the Supreme Court rule so I’m not going to apply it, other than the Federal Circuit.” (p 18)

Judge Michel of the Federal Circuit wrote an amicus brief in Alice advising against restricting patentable subject matter. Even addressing the Supreme Court itself, Judge Michel calls a previous Court ruling as “aberrational” (p 6), describes the Court “as the creator and arbiter of the judicially-created ‘exceptions’ to statutory patent eligibility, including ‘abstract ideas,’ whatever that means” (p 7), and urges the Court to not rely on its own rulings (p 8).

So the majority at the Federal Circuit’s appeals court spent the `00s working hard on pushing the scope of patentability well beyond the framework signalled in Labcorp, based on what seems to be a heartfelt but insubordinate belief that the Supreme Court has been doing everything after 1981 wrong. Given that their goals led them to avoid the Labcorp rule as long as possible, the Supreme Court was forced to pull the trigger and set its rule down in a clear majority ruling.

How Presidential Appointees (or Lack Thereof) Matter

Originally posted on JOURNAL OF PUBLIC POLICY:



By William Resh (@billresh), University of Southern California

As Gary Hollibaugh, Jr. and colleagues plainly stated on the LSE blog, “Presidential appointees matter.” Of course, this is of little question when these positions are filled. Incompetent appointees cause deleterious consequences for both citizens and presidents. But, what about those positions reserved for Senate-confirmed presidential appointees (PAS) that remain empty?

A report last year by Pro Publica grouses that the Obama administration has been subject to more vacancies than previous administrations—in both independent and executive branch agencies (see Figure 1). Yet, occupancy is often legislatively required in independent commissions and agencies before agency action can be triggered. Therefore, outcomes (or a lack thereof) are easier to identify during periods of vacancy in those positions (at least anecdotally), whereas the effects vacancies have on agency performance in executive branch agencies generally may be less evident. The…

View original 2,489 more words

Bureaucrats at their desks

ImageDutch photographer Jan Banning has traveled the world documenting the consequences of war, the homeless and impoverished, and victims of human trafficking. Asked to photograph a story on the administration of international development aid, something he thought to be “un-photographable,” Banning and a journalist set out to visit hundreds of local government offices worldwide. Between 2003 and 2007, they met civil servants in eight countries on five continents. “Though there is a high degree of humour and absurdity in these photos,” Banning says, “they also show compassion with the inhabitants of the state’s paper labyrinth.”

Where European scholars are publishing in Public Administration and why we already knew this

The European Union recently commissioned a study to “reflect on the state of the discipline and general trends within the discipline and in practice” of public administration (brought to you by the EU’s “Coordinating for Cohesion in the Public Sector of the Future” Group–or COCOPS). The subsequent report produced a ranking of public administration/management journals through the results of a survey of European scholars, which asked the respondents to rank the order of preference for where they would submit a good paper.

At my own school, faculty have vigorously (and in a healthy manner, I might add) debated the relative importance of journal ranking. And, this debate is certainly not isolated to my current place of employment. But one might question whether any of this debate really matters. Once a given metric becomes an established point of reference among those judged on that metric, is there any reason to believe that any other metric (qualitative or quantitative) will adequately replace it?

For instance, the Journal Citations Report or Google Scholar Metrics are two rather widely accepted quantitative metrics for journal prominence in a given field. JCR, in particular, has been used for years and is prominently featured as the metric of choice on most social science journals’ websites.

Below, I show tables derived from the COCOPS study, JCR, and Google Scholar Metrics. I have eliminated distinctively “policy”-oriented journals from lists in the “Public Administration” category in both JCR and Google Scholar. Even keeping in mind the obvious European bias in the COCOPS report, an almost identical list would emerge based on five-year impact factor or Google Scholar metrics. In ALL three lists, the top five journals in the field of public administration are PA, PAR, JPART, Governance, and PMR.

Note that some journals do not yet have a 5-year impact factor score (e.g., IPMJ). Nonetheless, it seems to me that there are a couple things you could derive from the COCOPS report… (1) traditionally accepted quantitative rankings are endogenous to choice; or (2) they aren’t a bad rubric for some fields; or (3) both.

Continue reading

Airplanes and the intellectual commons

Today’s post is based on a great paper by Peter Meyer, on the invention of the airplane. He also has a set of slides summarizing the paper and offering lots of pictures of early plane designs.

The data set that he put together is of the writings, correspondences, and patents regarding air travel during the period before anybody worked out whether air travel is even possible.

He paints the picture of a real community of interacting researchers. Letters are sent, ideas are shared. Patents are obtained, but then immediately pledged to the world at large. We get a sense of a small community of people that everybody else thought was crazy (until they were proven right), and who longed to see flight happen. Some people, most notably Octave Chanute, worked hard on being an information hub to keep the conversation going.

And then, they stopped. Two members of the community, the Wright Brothers, were especially active until about 1902, at which point they realized that their design could actually fly, and they stopped sharing. By the next decade, the correspondences stop and the patent battles commence:

This rapid takeoff of the industry, unmoored from the original inventors, suggests that much of the key knowledge was widely available. There were great patent battles after 1906 in the U.S. (and after 1910 in Europe) and industrial competition, but the key knowledge necessary to fly was not in fact licensed from one place or closely tied to any particular patent.

Looking to somewhat more recent history, the software world followed a similar pattern. Before the mid-1990s, software was largely seen as not patentable. That was the period when people came up with word processors, spreadsheets, databases, compilers, scripting languages, windowed GUIs, email, chat clients, the WWW. Then, after a series of federal circuit rulings which I will not rehash now, patents showed up in the software industry. If Rip van Winkle fell asleep in 1994, he’d see modern computing technology as amazingly fast and tiny and beautiful, the product of ten thousand little incremental improvements, but a basically familiar elaboration on what was in the commons in 1994.

The 3D printing world has a different history, because the early innovations were deemed patentable from the start. Many authors characterize the 3D maker world as being in a holding pattern, because key patents from the mid-1990s claimed the fundamental technologies. For airplanes and software, the fundamental building blocks were out in the public before the lawyers showed up. For 3D printing, the patents came from the start, so it took the 17-year wait until their expiration for the common tools to become commonly available.

[By the way, I found that last link to be especially interesting. It lists 16 patents that the authors identify as key to 3D printing, though the authors refuse on principle to say that their being freed up will advance the industry. Five of the sixteen are listed as having "current assignment data unavailable", meaning that even if you wanted to license the described technology, the authors—a Partner and Clerk at an IP law firm—couldn't tell you who to contact to do so. Orphan works aren't exclusive to copyright.]

These are loose examples of broad industries, but they make good fodder for the steampunk alt history author in all of us. What would the 1910s and 1920s have looked like if airplanes were grounded in a patent thicket? What would our computer screens look like today if WordPerfect Corp had had a patent on the word processor? What would the last decade of our lives have looked like if the cheap 3D printer technology emerging today were patent-free then?

23andme and the FDA and me

23andMe provides a service wherein you send them a sample of your spit, they run it through a machine that detects 550,000 genetic markers, and then they express a likelihood that you are susceptible to certain genetic disorders.

To do this, they combed through the genetics literature for GWASes: genome-wide association studies, searching for the link between certain genes and certain disorders. I was on the team that did one of them: a study searching for genetic causes of bipolar disorder. Our study involved interviewing and drawing blood from over 2,500 subjects (a thousand at NIH and 1,500 at a German sister lab). As of this writing, Google Scholar says it’s been cited 418 times. If you have a 23andMe account, you can find our study as one of a few under the section on bipolar disorder.

The FDA considers this service—running spit through the Illumina machine, then comparing the data to correlations like those we reported—to be a “medical device”, and has ordered 23andMe to cease and desist from distributing this medical device.

After the genotyping itself, this is an informational product, so it gives us a chance to see how an institution built to handle food and drugs that people ingest will handle a product that is almost entirety a list of statistical correlations.

We have limited information about the storyline to date, and I’ll try to avoid hypothesizing about what’s not in the letter, which already gives us enough for discussion. The 23andMe people may have been uncooperative, or it might even be a simple case of clashing personalities, or who knows what else went on. The labeling regulations in the CFR go on for pages, and I won’t pretend to know whether 23andMe complies or not. [More useful trivia from 21 CFR 801(164): "artificial flavorings, artificial sweeteners, chemical preservatives, and color additives are not suitable ingredients in peanut butter." That's the law.]

Snake oil

The FDA should exist. It should be policing health claims. We joke about snake-oil salesmen, but that’s a phrase because there was once a time when people really did sell snake oil and people really were dumb enough to use it instead of seeing a real doctor (not that real doctors at the time were much better…). Pseudoscience lives to this day, and if the FDA didn’t exist, we’d see a lot of late-nite ads for guaranteed cures for cancer for the low low price of $19.95, which would cause people to delay seeking real treatment, which would kill people.

But there are differences. As above, the genotyping service is almost purely informational. You can go to the drug store and buy a thermometer, a bathroom scale, a mirror, or any of a number of other self-inspection tools to find out about yourself. While you’re at the drug store, you can check your blood pressure on that automated machine in the back. Then you can go home and compare your data to Wikipedia pages about hundreds of different maladies. As the first pass of several, Illumina’s genotyping machine is exactly like these other tools for measuring the body, apart from the fact that it is unabashedly a flippin’ miracle of modern science. Perhaps there was a time when people said the same thing about the thermometer.

Is the machine somehow unreliable? When working with the Illumina machines(*) what blew my mind is that the machine is very accurate, as in over 99.9% correct over 550,000 data points. So it’s like a thermometer, except the results are more accurate and reliable. A big chunk of academia’s publications are about surmounting problems in doing research, and along that vein it is easy to find abundant articles on the process and reliability of using the machines themselves. Its quirks are well understood.

[(*) They're BeadStations, but we always called them the Illumina. It's good to be reminded that the provision of cheap, universal genotyping is a plot by the Illuminati.]


But 23andMe provides more than just a list of SNPs (single-nucelotide polymorphisms, which I colloquially refer to as genetic markers here). They also provide the odds of being susceptible to certain diseases, using studies such as the one I worked on.

The FDA’s letter explains how this can cause problems:

For instance, if the BRCA-related risk assessment for breast or ovarian cancer reports a false positive, it could lead a patient to undergo prophylactic surgery, chemoprevention, intensive screening, or other morbidity-inducing actions, while a false negative could result in a failure to recognize an actual risk that may exist.

Here, it seems that the FDA is protecting against doctor incompetence, because providing a mastectomy or chemotherapy because somebody walked in to the office with a printout from 23andMe is not the action of a competent doctor. So there is an implication here that the FDA feels that doctors can’t be trusted. What research there is shows that the ability of MDs to do basic statistical analysis is not stellar. Perhaps the FDA should require that patients may only receive medical test results under the supervision of a trained statistician.

How it works

The false negative part is a little closer to the story of the snake-oil doses that prevent somebody with a real problem from seeking real treatment. However, here we run into a problem as well: the results from 23andMe do not report positives or negatives. They only report probabilities, binned into average, above-average, &c.

After all, our paper doesn’t report anything definite either. We report a statistical association between certain SNPs having certain values and the odds ratio for bipolar disorder.

Given a series of such odds ratios, we can apply Bayes’s rule. You may not remember it from Stats 101, but the gist is that we begin with a prior likelihood of a state (such as ‘the person who mailed in this spit has blue eyes’), then update that likelihood using the data available. If we have several studies, we can apply Bayes’s rule repeatedly. In reality, we would do this with probability distributions—typically bell curves indicating the mean likelihood of blue eyes and confidence bands around that mean. Repeated addition of new data, using multiple SNPs described in multiple studies, typically narrows the variance of the distribution, increasing our confidence in the result.

I don’t have the report in front of me, but they found my brother to be at over 50% risk for obesity. We thought it was funny: he lifts weights and can talk your ear off on all matters of diet. But that’s just how the system works. His behavior has had a greater influence on his odds of becoming obese than the genetic markers, but 23andMe just has a spit sample from a U.S. resident. In the United States, over 30% of males are obese, so if I only knew that a person is male and living in the U.S., I’d guess that he has 30% odds of being obese. That’s my prior, which could then be updated using the genetic information to produce a more personalized image. It looks like his genes raised the odds of obesity from baseline. Perhaps this percentage over 50% is the sort of “false positive” that the FDA letter referred to.

[To balance this "false positive", the tests also did a good job of picking up some rare risks that we know from our family's medical history to really be risks. It's probably TMI to go into a lot of detail here.]


To summarize the story so far, by my conception there are three steps to the genotyping service:

  1. Run spit through the Illumina machine.
  2. Gather data about marker/disorder correlations from the literature.
  3. Use statistical methods like Bayes’s rule to aggregate the data and studies to report the best guess at the odds of each disorder.

From the FDA letter: “we still do not have any assurance that the firm has analytically or clinically validated” the genotyping service.

Starting from step three, it is reasonable for the FDA to take a “guilty until proven innocent” attitude toward the data analysis and to require that 23andMe show its work to the FDA. But although the updating problem is far from trivial in practice, a review in good faith by both sides could verify or disprove the statisticians’ correctness in well under the five years that the FDA and 23andMe seem to have been bickering. Some of the results may not even need to go as far as applying Bayes’s rule, and may be simple application of a result from a paper.

The application of the FDA’s statement to steps one and two, where the bulk of the science happened, is the especially interesting part from a bureauphilic perspective. As above, the academic literature has more than enough on the analytic and clinical validity of step one (get good SNPs from the machine) and abundant studies such as the one I worked on that used 2,500 subjects to verify the analytic and clinical validity of step two (calculate disorder odds from SNP results). E.g., here is a survey of 1,350 GWASes.

Yet the authors of the FDA letter do not have “any assurance” of validity.

A paper’s admissability as evidence to the FDA depends on the “regulatory pathway” under which the study was done. If a study is done to approve a drug on a European market, that study is not admissible as evidence at the FDA. Evidently, our study done at the NIH is not admissible as evidence at the FDA. Perhaps I need to parse the sentence from the letter more carefully: “we still do not have any assurance that the firm has analytically or clinically validated” the genotyping service, where the use of “the firm” indicates that research from the NIH doesn’t count, but if the product vendor replicates the study under the explicit purview of the FDA (which, given the scale of some of these studies and their sheer number, is entirely impossible) then that does count.

It’s good that the FDA has standards on testing, because drug companies have strong incentives to put their thumb on the scale in every trial. It could indeed be the case that the FDA has determined the correct way to test medical devices and derive statistical results and, at the same time, everybody else is doing it wrong. Or it could be that the FDA as an institution has an acute case of NIH syndrome.