Sunday, July 1, 2018

The Horowitz IG Report: Lies, Damn Lies, and Statistics

The Horowitz IG Report: Lies, Damn Lies, and Statistics

The Horowitz IG Report: Lies, Damn Lies, and Statistics

The Inspector General of the FBI said that he found "no documentary evidence of bias" in the prosecutorial decisions made during the Hillary Clinton email investigation. The Left celebrates, and the Right explodes. As good Rabbis would suggest, perhaps the truth lies somewhere in between. But how do you find it?
There are two smoking guns. First, Peter Strzok famously texted his illicit lover that "We won't let [Donald Trump's election] happen." The other is found at 7:36 in Senator John Kennedy's questioning of the Inspector General. "Most" of the investigative decisions were made in Loretta Lynch's Department of Justice, not in the FBI. Since that body refused to pay any attention to demands by Republican investigative committees, it's no surprise that the investigation gave her a pass. But how do you prove it?
Inspector General Horowitz admitted that it was likely that various comments, such as Strzok's above, indicated bias. But when taken in isolation, he could not identify any prosecutorial decisions that would have come from bias, either for, or against Hillary Clinton. Notice that he did not offer any comment about the aggregate set of decisions. In other words, he did a very narrow analysis that was designed to avoid examining the possibility of bias in the whole investigation.
It is rare for a criminal to write down his intent to commit a crime. When Trey Gowdy questioned James Comey after his July 2016 press conference, he very effectively got Comey to explain that you show intent by a series of actions that indicate the willful nature of the crime. In Hillary Clinton's case, this included setting up a private email server after being briefed on the proper use of the State Department's secure system and the statutes regarding sensitive information. Then she transmitted over one hundred highly classified documents over that server. When confronted with a Congressional subpoena and preservation order, she directed her attorneys to destroy 33,000 emails. And she made false exculpatory statements about her server. This chain of events showed clear intent to perform illegal acts and consciousness of guilt.
But Hillary Clinton never wrote down, "I'm going to violate the law." So there is no "documentary evidence" of her intent. It is the chain of events that clearly shows intent. So we must now go where Inspector General Horowitz declined to go.
Statistics is often seen as some sort of dark art. Mathematical formulae for them get complex, even for the simpler tasks, such as Student's T-Test, which is used to “determine if means of two data sets differ significantly.”
The Inspector General listed a number of actions taken during the Clinton email investigation. If the investigation had been neutral, you'd expect that roughly half of the decisions would be in her favor and about half would be contrary to her. That would show that the Department of Justice was acting as an impartial agency.
But some of the decisions were starkly in Secretary Clinton's favor. For example, the exoneration letter was written months before seventeen witnesses were interviewed. (Is that one decision or seventeen?) Fact witnesses such as Cheryl Mills were allowed to sit in on Secretary Clinton's interview as if they were attorneys representing Mrs. Clinton. Secretary Clinton's interview was not under oath, and was neither recorded nor transcribed. Immunity agreements were reached before the content of proffered testimony was assessed. And devices containing documentary evidence were not searched.
These decisions were so favorable that even the Inspector General had to admit that they were "unprecedented." This sounds awful, and that emotional response clouds our thinking. So we have to do some math (OMG!) to sort things out. And we have to make some assumptions.
It is not possible at the moment to be sure that these hypothetical numbers are accurate.[i] For example, I did not see or hear of a single decision during the investigative process that was not in Secretary Clinton's favor. That doesn't mean there weren't any, just that I'm not aware of them.
But just to be fair, I set up a simple Student's t-test that included some decisions against her. In good statistical fashion, the test had to propose the "null hypothesis" that the investigation was neutral. A decision "for" Clinton was a "2," "against" Clinton was a "0", and "neutral" would be "1." I picked eighteen "for" and two "against," trying to match my impression of the facts on the ground.
The results were:
"The calculated t exceeds the critical value (9.4424>2.093), so the mean of data set is significantly different from μ0=1."
In this hypothetical the "null hypothesis" has been invalidated. In other words, the investigation was not neutral. Applying that t value to the test table allows us to assess the probability that an 18:2 ratio of for/against Hillary Clinton happened by chance.
Notice that the t value is far greater than the .001 probability that Hillary Clinton got all those decisions in her favor by chance. There is only one tenth of one percent chance that those decisions were not made without bias if this represents the decisions in the process.
But is 18:2 the right ratio? Suppose it was 16:4. This time the t value is 3.9829, again larger than the t  value for one tenth of one percent probability that America got a fair investigation of Hillary Clinton's email practices. We have to get to 14:6 to reach a point where we lose scientific certainty.
It's time to apply common sense to this issue. As my Dad used to say, "Figures don't lie, but liars can figure." We have to take even my work with a grain of salt. Statisticians might argue that another test is better than the t test. They might argue that we can't properly count the number of for/against decisions. They might argue that some decisions fall at 0.6 or 1.3 instead of the limited range I used. But common sense tells us we should use the Inspector General's language to describe what happened.
The favors granted to Hillary Clinton in her email investigation were strongly biased in her favor. In fact, they were so biased as to be "unprecedented."

[i] The format of the IG Report does not allow a clear enumeration of the specific decisions involved, since it does not list every specific decision in a format that would allow a ready count. For example, the Executive Summary states that the FBI, "Did not seek to obtain every device, including those of Clinton’s senior aides, or the contents of every email account through which a classified email may have traversed." This could be counted as a single decision for the reasons the ES later lists, or it could be counted as multiple decisions on a case-by-case basis.
Ultimately, it is necessary to create a hypothetical number in order to illustrate the folly of clearing the FBI of charges of bias by stating, "The question we considered was not whether a particular investigative decision was the ideal choice or one that could have been handled more effectively, but whether the circumstances surrounding the decision indicated that it was based on considerations other than the merits of the investigation. If a choice made by the investigative team was among two or more reasonable alternatives, we did not find that it was improper even if we believed that an alternative decision would have been more effective."
In short, as Inspector General Horowitz said in his testimony to Congress, each individual decision was considered in isolation. This precludes an assessment of the investigation as a whole, and forces a conclusion contrary to the known facts. But to evaluate the "circumstances surrounding the decision," an assessment of the whole investigation was required.
The Inspector General of the FBI said that he found "no documentary evidence of bias" in the prosecutorial decisions made during the Hillary Clinton email investigation. The Left celebrates, and the Right explodes. As good Rabbis would suggest, perhaps the truth lies somewhere in between. But how do you find it?

There are two smoking guns. First, Peter Strzok famously texted his illicit lover that "We won't let [Donald Trump's election] happen." The other is found at 7:36 in Senator John Kennedy's questioning of the Inspector General. "Most" of the investigative decisions were made in Loretta Lynch's Department of Justice, not in the FBI. Since that body refused to pay any attention to demands by Republican investigative committees, it's no surprise that the investigation gave her a pass. But how do you prove it?

Inspector General Horowitz admitted that it was likely that various comments, such as Strzok's above, indicated bias. But when taken in isolation, he could not identify any prosecutorial decisions that would have come from bias, either for, or against Hillary Clinton. Notice that he did not offer any comment about the aggregate set of decisions. In other words, he did a very narrow analysis that was designed to avoid examining the possibility of bias in the whole investigation.

It is rare for a criminal to write down his intent to commit a crime. When Trey Gowdy questioned James Comey after his July 2016 press conference, he very effectively got Comey to explain that you show intent by a series of actions that indicate the willful nature of the crime. In Hillary Clinton's case, this included setting up a private email server after being briefed on the proper use a the State Department's secure system and the statutes regarding sensitive information. Then she transmitted over one hundred highly classified documents over that server. When confronted with a Congressional subpoena and preservation order, she directed her attorneys to destroy 33,000 emails. And she made false exculpatory statements about her server. This chain of events showed clear intent to perform illegal acts and consciousness of guilt.

But Hillary Clinton never wrote down, "I'm going to violate the law." So there is no "documentary evidence" of her intent. It is the chain of events that clearly shows intent. So we must now go where Inspector General Horowitz declined to go.

Statistics is often seen as some sort of dark art. Mathematical formulae for them get complex, even for the simpler tasks, such as Student's T-Test, which is used to “determine if means of two data sets differ significantly.”
Students T-Test.PNG
Student's t-test

But one part of statistics is totally intuitive: the normal distribution. We expect certain outcomes to "center around a mean." In simple language, that means that when we flip a coin a twenty times, we expect to get ten heads and ten tails. If we get nine heads and eleven tails, that's not a big deal, since it's close to a 50:50 result. Getting all heads or all tails is almost as likely as talking your way out of that speeding ticket. Not happening. The curve looks like this, and we intuitively use it every day.

Normal Distribution.jpg

The Inspector General listed a number of actions taken during the Clinton email investigation. If the investigation had been neutral, you'd expect that roughly half of the decisions would be in her favor and about half would be contrary to her. That would show that the Department of Justice was acting as an impartial agency.

But some of the decisions were starkly in Secretary Clinton's favor. For example, the exoneration letter was written months before seventeen witnesses were interviewed. (Is that one decision or seventeen?) Fact witnesses such as Cheryl Mills were allowed to sit in on Secretary Clinton's interview as if they were attorneys representing Mrs. Clinton. Secretary Clinton's interview was not under oath, and was neither recorded nor transcribed. Immunity agreements were reached before the content of proffered testimony was assessed. And devices containing documentary evidence were not searched.

These decisions were so favorable that even the Inspector General had to admit that they were "unprecedented." This sounds awful, and that emotional response clouds our thinking. So we have to do some math (OMG!) to sort things out. And we have to make some assumptions.

It is not possible at the moment to be sure that these hypothetical numbers are accurate.[i] For example, I did not see or hear of a single decision during the investigative process that was not in Secretary Clinton's favor. That doesn't mean there weren't any, just that I'm not aware of them.

But just to be fair, I set up a simple Student's t-test that included some decisions against her. In good statistical fashion, the test had to propose the "null hypothesis" that the investigation was neutral. A decision "for" Clinton was a "2," "against" Clinton was a "0", and "neutral" would be "1." I picked eighteen "for" and two "against," trying to match my impression of the facts on the ground.

The results were:

"The calculated t exceeds the critical value (9.4424>2.093), so the mean of data set is significantly different from μ0=1."

In this hypothetical the "null hypothesis" has been invalidated. In other words, the investigation was not neutral. Applying that t value to the test table allows us to assess the probability that an 18:2 ratio of for/against Hillary Clinton happened by chance.

T Table.PNG

Notice that the t value is far greater than the .001 probability that Hillary Clinton got all those decisions in her favor by chance. There is only one tenth of one percent chance that those decisions were not made without bias if this represents the decisions in the process.

But is 18:2 the right ratio? Suppose it was 16:4. This time the t value is 3.9829, again larger than the t  value for one tenth of one percent probability that America got a fair investigation of Hillary Clinton's email practices. We have to get to 14:6 to reach a point where we lose scientific certainty.

It's time to apply common sense to this issue. As my Dad used to say, "Figures don't lie, but liars can figure." We have to take even my work with a grain of salt. Statisticians might argue that another test is better than the t test. They might argue that we can't properly count the number of for/against decisions. They might argue that some decisions fall at 0.6 or 1.3 instead of the limited range I used. But common sense tells us we should use the Inspector General's language to describe what happened.

The favors granted to Hillary Clinton in her email investigation were strongly biased in her favor. In fact, they were so biased as to be "unprecedented."
 
[i] The format of the IG Report does not allow a clear enumeration of the specific decisions involved, since it does not list every specific decision in a format that would allow a ready count. For example, the Executive Summary states that the FBI, "Did not seek to obtain every device, including those of Clinton’s senior aides, or the contents of every email account through which a classified email may have traversed." This could be counted as a single decision for the reasons the ES later lists, or it could be counted as multiple decisions on a case-by-case basis.

Ultimately, it is necessary to create a hypothetical number in order to illustrate the folly of clearing the FBI of charges of bias by stating, "The question we considered was not whether a particular investigative decision was the ideal choice or one that could have been handled more effectively, but whether the circumstances surrounding the decision indicated that it was based on considerations other than the merits of the investigation. If a choice made by the investigative team was among two or more reasonable alternatives, we did not find that it was improper even if we believed that an alternative decision would have been more effective."
In short, as Inspector General Horowitz said in his testimony to Congress, each individual decision was considered in isolation. This precludes an assessment of the investigation as a whole, and forces a conclusion contrary to the known facts. But to evaluate the "circumstances surrounding the decision," an assessment of the whole investigation was required.

No comments:

Post a Comment