Computer Evidence

Michael J L Turner MA FBCS CITP MAE FEWI RFP

e-mail Michael Turner

Print-friendly CV

Tell a colleague!

The case of Sergeant Gurpal Virdi, accused of sending racist hate mail to ethnic minority police officers including himself, casts light on current practice in the treatment of computer evidence. Michael Turner explains why it is instructive on the role of expert evidence on computer evidence, and on the thorny issue of Novel Science.

The incidents which form the basis of this article occurred at the time that the Macpherson inquiry into the death of Stephen Lawrence was investigating allegations of racism in the Metropolitan Police (MPS). As a result, this high profile case has had extensive coverage in the media, including a series of features on BBC2 Newsnight, and in more than ten parliamentary questions.

The Racist Letters

In December 1997 racist letters were received in the internal mail by a number of ethnic minority officers, including PS Virdi, in the Ealing Division of the MPS. In January 1998 a further set of racist letters were received in the internal mail by a number of ethnic minority civilian staff in the Ealing Division. Both sets of racist letters were individually addressed and comprised a single side of A4 containing a graphic image and offensive text ending with the letters 'NF'.

A police investigation was launched. No DNA or fingerprint evidence was found. Instead the investigation concentrated primarily on computer evidence. Back-up tapes and copies of log files were seized from the servers of the police OTIS computer system and analysed. PS Virdi's house, car, drains, loft and outbuildings were searched for seven hours. Dozens of police officers were interviewed.

Criminal Proceedings

PS Virdi, who had a previously unblemished disciplinary record, was arrested and charged with criminal offences relating to the production and dissemination of racist letters, and suspended from duty. He vigorously denied being responsible for the racist letters and had an alibi for the December 1997 documents.

When legal advice was taken the CPS declined to prosecute. The reasons for that decision have not been disclosed.

MPS Discipline Board Hearing

The same allegations formed the basis of charges brought against PS Virdi in a four-week MPS Discipline Board hearing in February 2000. MPS and PS Virdi were represented by counsel. The Board heard evidence from 51 witnesses.

Expert evidence was given for MPS that identified how and when the racist letters were produced. The technique used was:

1. to reconstruct the documents
2. to match print runs in event logs against reconstructed documents
3. to identify the logged on User IDs responsible for the print runs

This technique was subsequently christened Document Reconstruction.

The MPS Discipline Board did not hear any expert evidence for PS Virdi.

The MPS Discipline Board unanimously found PS Virdi guilty beyond reasonable doubt on 11 counts relating to the racist letters, including impersonating a colleague police officer by using her User ID and password on two occasions, using MPS computer systems to produce both sets of racist documents and distributing racist documents in the Ealing Division of MPS. PS Virdi was sacked.

Employment Tribunal

PS Virdi had made a complaint of racial discrimination against the MPS on a number of grounds under the Race Relations Act 1976. In the hearing of the first Employment Tribunal case¹ in July and August 2000 the complaint about his treatment relating to the racist letters was considered.

The Employment Tribunal decided it should first hear the computer evidence as a discrete matter. Evidence was given by two experts for PS Virdi and by the same two experts who had given evidence at the Discipline Board Hearing for MPS. In an Interim Decision dated 18 July 2000 the Tribunal found that on a balance of probabilities the two print-runs identified by the MPS expert witnesses were the racist letters.

As a result of that finding the Tribunal then heard the evidence from the investigating officers and some 30 police officer alibi witnesses. The Tribunal found that none of these officers had any particular reason to recall the alleged events of the early morning of Christmas Eve 1997 without the prompting of printouts from computer systems. The tribunal also heard evidence of password abuse.

The Tribunal found a number of anomalies arising from the factual evidence that threw into doubt its interim decision on the computer evidence. In particular, the evidence given on the internal mail system suggested that the second set of racist letters were posted two days before it was alleged that they had been printed. As a result, the assumption that the documents were produced in-house was seriously undermined.

Although the Tribunal did not reject the theory of the Document Reconstruction technique, this new evidence did call it into question.

The Tribunal noted that the MPS investigation had not checked whether the alleged computer evidence fitted in with what had actually happened and that MPS appeared to have proceeded on the basis that the computer evidence was unchallengeable.

In August 2000 the Tribunal found on the balance of probabilities that:

• there was no evidence that the racist letters were produced during the print runs identified by the MPS computer experts on 24 December 1997 and 18 January 1998
• there was no evidence that the racist letters distributed were produced by PS Virdi

and held that MPS had discriminated against PS Virdi on the grounds of his race, by treating him differently and detrimentally to a white WPC suspect.

Following that decision, the Metropolitan Police Authority announced an independent enquiry into the case. At a hearing of the MPS Discipline Board in November 2000 PS Virdi's appeal against dismissal was not opposed by MPS. He was reinstated on full pay and received a written apology.

At a Remedies hearing in December 2000 the Employment Tribunal awarded PS Virdi a record £150,000 compensation for injury to his feelings arising from discrimination, including aggravated damages and interest. The award for aggravated damages was in respect of the high-handed treatment of PS Virdi by MPS, particularly in its failure to apologise to PS Virdi until the end of November 2000.

A second Employment Tribunal case relating to the conduct of the MPS Discipline Board Hearing and his dismissal is pending.

The Computer Evidence

The case for MPS rested on a series of interrelated assumptions. It was assumed that there was only one way of creating a series of documents and that both sets of racist letters were:

• created and printed within the Ealing Division of MPS using MS Word
• printed as a consecutive sequence of unnamed MS Word documents.

All the evidence emanated from event logs on the MPS OTIS system servers. It was claimed that this system was secure; each user had a User ID and a password that was not meant to be disclosed to other users. Passwords had to be changed every 28 days and the same password could not be re-used within a 12-month period.

The Document Reconstruction technique was invented by the Assistant Systems Administrator for the Ealing Division. He thought that the January 1998 racist letters may have been produced in-house and to test this theory he set out to recreate the document. He produced a reconstructed document and noted the size of the print file (NB this is not the same as the file size) when it was printed. He then searched the event logs for a sequence of printed documents that matched that print file size. He identified a sequence of documents printed at a specific time using a specific user ID. It was these event log entries that formed the basis of MPS' evidence.

Expert Evidence on Computer Evidence

At the Employment Tribunal hearing, it was agreed that in a criminal investigation it was standard procedure for computers likely to contain evidence to be seized and copied using a non-destructive forensic-imaging process, so that all investigations could be conducted on write-protected copies of the evidence. It was also agreed that that procedure had not been followed in this case.

Two expert witnesses gave evidence for MPS. Mr A adopted the Document Reconstruction technique. He conducted his own Document Reconstructions and searches of the event logs, which identified the same two sequences of documents. His expert evidence was that if the racist letters had been produced in-house, then the event log evidence indicated the User IDs responsible for creating the racist letters.

Mr A had cracked the Password file and identified a highly relevant apparent breach of the 28-day password rule and/or of the rule that a password cannot be reused within 12 months.

MPS' second expert, Mr B, was asked by the investigating team to scrutinise the computer evidence. He adopted the Document Reconstruction technique and adopted the results of Mr A's reconstructions without doing his own. He analysed the event logs, and again identified the same two sequences of documents. His expert evidence was that his analysis of the event log evidence indicated beyond reasonable doubt that each series of racist letters was printed at a particular time and by a particular logged on User ID.

He also identified an anomaly in the relevant log evidence - the sequence of documents alleged to be the January 1998 racist letters were printed at a time when there were two concurrent logons using the same User ID on different workstations.

Both of the MPS' experts accepted that they had no prior or subsequent experience of Document Reconstruction as a forensic technique. Neither had conducted tests separate and apart from work on this case to establish the validity of the technique in its own right.

The expert evidence for PS Virdi identified anomalies and discrepancies in the computer evidence, including discontinuities in the security event logs, contamination of the security event logs and inconsistencies between server log entries.

Expert evidence for PS Virdi was given that the failure to secure and image the relevant servers in a timely fashion and the direct examination by uninformed investigators of the original computer evidence had irretrievably lost or contaminated highly relevant evidence. As a result both hearings had been deprived of evidence that could have helped determine who, if anybody, within the Ealing Division of MPS, created and printed the racist letters. Such loss or contamination would undoubtedly prevent PS Virdi from receiving a fair trial. There was no doubt that no criminal court would have allowed the case against PS Virdi to proceed. MPS' actions in failing to secure the evidence did not comply with the principles and practice guidelines set out in MPS' own Principles of Computer Based Evidence.²

The first expert for PS Virdi, Mr C, also identified evidence of relevant password abuse and questioned the reliability of all the relevant server log time-stamps on a number of grounds. In his opinion, all the crucial evidence was of poor reliability and of questionable admissibility.

Mr C challenged the whole basis of the Document Reconstruction technique. He identified 27 theoretical determinants of the size of a Windows print job file; many of those attributes having a very large range of permissible values. Without specifying all these variables (and they would not be specified for any test document reconstructed only from information available on the face of a printed document) then it would not be possible to guess the size of a Windows print job file with any accuracy at all. Mr B's evidence was that the value of some of those variables would be known if, as MPS alleged, the platform used to print the documents was known.

In Mr C's opinion the Document Reconstruction technique was untested, unverified, untestable, unverifiable, unaccepted and unscientific. It was based on a false premise - that it was possible to precisely determine the size of a Windows print job file by examining only the information available on the face of a printed document. In his opinion, the use of the Document Reconstruction technique was technical speculation and the results should be seen as the creation of evidence.

It was common ground between all the experts that they had no prior or subsequent experience of Document Reconstruction as a forensic technique. It was also agreed that there is no known formula for precisely determining the size of a print job file from the information available on the face of a printed document.

Document Reconstruction - Novel or Junk Science?

Counsel for MPS contended that Document Reconstruction was an example of a novel forensic technique and that it was legitimate for investigators to develop relevant novel techniques. Counsel for PS Virdi contended that Document Reconstruction was not a valid scientific method on the basis of any or all of the four tests set out in Daubert,³ the leading American case:

• whether the theory or technique can be or has been tested
• the error rate associated with the method
• publication in a peer-reviewed journal
• whether the technique has gained widespread acceptance.

In the USA such invalid forensic science techniques are known as Junk Science,⁴ and there is a substantial body of (largely pharmaceutical) case law on the admissibility of such expert evidence. The decision of the US Supreme Court in Kumho Tire⁵ extended this Gatekeeping role of the courts to evidence from 'engineers and other experts who are not scientists'. Yet in the present case neither counsel cited any relevant English case law.

Commentary

The real identity of the documents in the two sequences of printed documents identified by Document Reconstruction will never be known. In that sense, trying to match an event log print file size to a reconstructed document was never meaningful. The analogy was made with a blood sample found at the scene of a crime. When the forensic scientist seeks to match that sample with another sample of blood from a suspect, that is matching a piece of real evidence with an independent sample of real evidence. In this case, the investigators tried to match a trace of a document found on a computer with a "sample" that had been constructed solely for the purposes of matching.

This case illustrates many of the dangers inherent in over-reliance on computer evidence. Without the computer evidence there was no case against PS Virdi. Familiar topics such as password abuse and the reliability of time-stamps were very much in issue.

The case clearly demonstrates the risks of over-reliance on expert interpretation of computer evidence. Experts giving evidence on computer evidence have a special responsibility to distinguish clearly between facts and speculation, assumption, inference and opinion. It appears that distinction was not always made in this case. On that basis, it is suggested that this case was the first miscarriage of justice that resulted directly from the interpretation of computer evidence by experts.

The reversal of the Employment Tribunal's Interim Decision on the computer evidence is a welcome reminder to all experts that their opinion evidence plays second fiddle to evidence of fact. The case also illustrates how risky it is to rely on a single stream of computer evidence⁶ - all the computer evidence in this case came from event log files (two logs, but essentially one stream of evidence).

The handling of the computer evidence was inadequate. It is not known why the relevant servers were not seized and forensically imaged as soon as possible after the incidents.

Experts confronted with a novel forensic technique such as Document Reconstruction are in desperate need of legal guidance as to the limits of acceptable forensic science under English law.

Endnotes

1. Mr G S Virdi v The Commissioner of Police of the Metropolis, London North Employment Tribunal, Case Number 2202774/98.

2. Unpublished. Believed to be a version of the Association of Chief Police Officers (ACPO) Good Practice Guide for Computer Based Evidence, also unpublished.

3. Daubert v Merrell Dow (1993) 509 U.S. 579.

4. See for exampe Galileo's Revenge, Peter Huber, Basic Books, ISBN: 0465026249.

5. Kumho Tire Company Ltd v Patrick B Carmichael, 119 S.Ct 1167, 97-1709 Supreme Court Of The United States.

6. See 4.8 of Submission of The British Computer Society to Criminal Courts Review

Michael J L Turner is an independent computer evidence consultant who gave expert evidence for PS Virdi in the first Employment Tribunal case. expert@computerevidence.co.uk