Using data to pick the best patent prosecution professionals

Data increasingly illuminates a multitude of subjects that have historically been overshadowed by expert opinions and anecdotes. Patents are no exception to this. In the past, someone seeking to know whether their application would issue with the desired claim scope could trust only in the experience and competency of the expert they had selected to steer the matter through the USPTO. Several variables can influence this journey, which means that when most IP professionals are asked by their client whether the matter will make it successfully to issuance, the answer will typically be: “It depends.” Given the inherent uncertainties underpinning patent examination, if the outcome is less satisfactory than the putative patentee expected, the expert can blame generalised vagaries in the process: the USPTO, the examiner, the courts and the client’s lack of inventiveness, budget or fortitude to make it to the finish line – to name just a few.

Missing from this list is insight into what part is played by the competency of the professional. This is unfortunate because the quest for patenting success begins and may even be set when a client engages the person or team that will generate an application and shepherd it through the USPTO with the goal of obtaining protection that is aligned with its business goals.

Patent professional competence: why it matters (sometimes)

‘Patent professional competency’ is a relative term. A patent that includes legally valid claims that do not cover a company’s invention may be competently procured from a legal perspective, but its owner may not find the lawyer’s efforts competent in relation to its business needs. A high-volume filer may find that a team’s competence resides in its ability to generate many patents at a low cost per filing, with less emphasis on claim scope and validity because any one item in the portfolio has a low overall chance of being enforced. Those bringing to market products that comprise incremental improvements over previous customer offerings may need their IP expert to write patents with competently crafted claims intended only to deter competitors from direct copying. For other companies whose business involves placing large business bets on innovations that are intended to solve significant unmet needs in the market, competence will require not only legally valid claims that cover a specific product being sold to the customers, but also protection that is sufficiently broad to prevent competitors from offering goods that incorporate equivalent functionality.

For patentees that fall into this latter category – a relatively small sub-set of the total number of applicants – each filing should be considered a fundamental aspect of their business strategy, serving as a foundation for the decision-making process for any innovation investment that the company makes. That is, innovators take large risks to generate future revenues from technology solutions created in the present. Such income may be sustainable as a result of patents that competently protect both a specific product and the value proposition that the technology generates for the customer. It follows that the competency of the professional selected to help a company protect its innovation investment is a highly meaningful factor. Moreover, the choice must be correct out of the gate as there are few, if any, do-overs in the patent world.

For most people, competence in the realm of patent procurement is the proverbial black box – not least because the person choosing a competent professional is not themselves knowledgeable in this highly technical area. Typically, a non-patent expert will select an IP professional based on factors such as reputation, technical background, location, cost and gut feeling. While these criteria can be revisited post-issuance using readily available data (eg, overall application cost, time expended and generated claim scope), in our experience, such reviews are rarely, if ever, carried out.

If available past performance data is ignored, it follows that potential patentees do not seek out such information before making a hiring decision relevant to future filings. Indeed, to our knowledge, no one has sought to evaluate what is arguably the best record of an individual’s competency in generating patents – namely, USPTO data of past filings. We thus sought to evaluate this to determine whether such analysis could help businesses select their professionals for must-have matters.

USPTO data analysis at firm and practitioner levels: hypothesis, methodology and insights

For this article we hypothesised that patent information associated with USPTO customer number and practitioner registration number data could provide guidance into an individual’s performance metrics in various prosecution-related domains. We did not expect this to show whether an attorney or firm possessed track records of generating broad and business-relevant patents for companies requiring such protection. Nonetheless, we first needed to be able to drill down to the firm and individual level before attempting to derive insights into data that could illuminate apparent competencies. To this end, we used firm-level data – as identified by unique USPTO customer numbers and USPTO registration numbers assigned to individual patent agents and attorneys – to segregate relevant data so as to distil prosecution-related metrics for firms and individuals, respectively.

By way of explanation, a USPTO customer number for a firm typically includes a few or many USPTO registration numbers for each practitioner – usually attorneys, but also patent agents – working at a firm, with the firm information identified by a unique customer number. When a patent professional leaves a position, they are expected to notify the USPTO if they are practising at a new firm – their registration number, which does not change, will then be associated with a different customer number. We hypothesised that analysis of patent application prosecution activity using these numbers could provide insights into the performance of individuals at the firm and/or individual practitioner level. These might then be leveraged to separate, and therefore make available for selection, individual firms and professionals that performed better relative to their peers at different firms, as well as among peers at the same firm. Such information would not guarantee that a firm or individual identified as “above average” would successfully generate patents that align with the stated business goals of the client. However, knowing that one’s selected attorney’s or agent’s past performance was in line with a desired business outcome could improve the odds of successful strategy execution – or at least reduce the number of variables in the application process.

Unfortunately, from a broad analysis across USPTO customer number and registration number data, we found that it is not possible to distil useful information about firms and individuals that could aid in the selection of a patent professional from the data alone. In short, the data as available today is too noisy to allow the individuation of either firms or practitioners. We found several likely reasons for this. First, many firms do not allow their practitioners to sign USPTO documentation individually, which results in individual attorney/agent participation in a patent application matter not being visible. Applications also often include one or more change in power of attorney and/or correspondence address during prosecution, which results in multiple registration numbers and customer numbers even within a single record. Mergers and attendant name changes also make it difficult to observe firm-related activity from year to year because new customer numbers are often assigned to a law firm if it changes its name. Moreover, in a surprising number of cases, individual patent attorneys and agents failed to update their information in the USPTO roster when they changed firms, even though the registration number confers their status to practise before the office.

Although individual attorney/agent and firm data did not allow us to conduct a meaningful performance assessment among individual firms and/or practitioners across a generalised dataset, we did extract some performance data for professionals within a specific firm. The data capture for this inquiry was cumbersome and time consuming and is suitable only to provide a rough guide for someone seeking to filter attorney/agent performance at the selection point – that is, before the patent application is drafted. Moreover, data analysis will only be possible for law firms that maintain a patent practice of a size suitable to generate enough application information to populate a rich dataset. Notwithstanding these limitations, we believe that this specific firm data, which looks at patent professional performance within a single firm (described as ‘Firm X’ for confidentiality), could be useful to understand whether an attorney is ‘above average’ within a selected firm if the firm’s patent practice is large enough to generate a reasonably sized dataset.

Digging into the numbers

A specific firm, Firm X, was selected from which a dataset was generated. Application allowance rates and the average number of written responses needed per disposition were calculated for the firm as a whole (ie, by customer number(s)) and for the individual practitioners (ie, by registration number). Admittedly, the selection of Firm X was somewhat arbitrary; it was chosen in large part by the need for a reasonably sized dataset to test the data analysis hypothesis. Firm X has more than 50 registered USPTO practitioners, making it among the 100 largest law firms in terms of patent professionals in the United States, while at the same time it does not have too big a patent practice. The need to conduct optical character recognition on the firm’s patent application file records to extract relevant data for analysis required us to limit the number of applications incorporated into the dataset.

Before discussing the data, it bears repeating that the indicated categorisations for Firm X’s patent prosecution efforts are procedural and cannot address whether this firm or its practitioners have a track record of obtaining patents that align with clients’ business strategies. Nonetheless these metrics can illuminate how well (or not) a group of patent professionals at a firm have shepherded applications through the USPTO. In other words, these metrics enable someone who is not a patent expert to answer the question: “Does this attorney at this firm obtain patents for clients by moving a patent application efficiently through the USPTO?” This is not the same as the question: “Does this attorney obtain patents that hold the content and scope needed to effect the desired business outcome?” – but we have to start somewhere.

Analysis of Firm X’s USPTO prosecution data revealed that the firm’s overall allowance rate was about 85%, with 1.1 responses needed per application disposition. At the individual practitioner level, with each individual practitioner having a USPTO registration number with at least 100 application dispositions, allowance rates ranged from 60% to 97% per filed application, while the number of interactions with the patent examiner – generally referred to as ‘responses’ – per disposition ranged from 0.8 to 1.5. This is shown in Figure 1.

These results suggest that most of the professionals at Firm X are average in relation to their peers at this firm, but that some individual practitioners appear to be more or less effective in gaining application allowances for their clients. In particular, we identified one professional who exhibits a better-than-average allowance rate and a lower-than-average number of responses needed per disposition. This insight led us to infer that the patent professional, who we identified as a senior attorney at the firm and who we will denote ‘Attorney Y’, is much more efficient than the other professionals at Firm X. It can be expected that Attorney Y’s better application disposition rate may also translate to lower overall cost per patent obtained, as well as the quicker allowance of a filed patent. Of course, this professional may well have a much higher hourly rate than others at their firm, which could negate any cost savings, but the apparently faster allowance would nonetheless be significant in many patenting situations.

Before choosing a patent professional at Firm X on the basis of their being on the most efficient side of the scale, several other factors should be mentioned. For example, some of the professionals at Firm X might practise in areas for which patent allowances are harder to obtain. That is, a practitioner who labours in an area in the USPTO where patentability is more difficult to attain should not be penalised as being less efficient. It would thus be of interest to see how Firm X’s patent allowance rates vary as a function of technology by grouping according to the number of applications in a USPTO area.

To this end, we specifically examined patent application disposition rates in one of Firm X’s highest-volume technology practice areas – which is likely indicative of an area where it would exhibit more experience and, thus, enhanced competence. The dataset for Firm X revealed that the most experienced person in the firm’s most prevalent technology area, who is (probably unsurprisingly) Attorney Y, exhibited an application allowance rate of 99% (out of more than 50 application dispositions), whereas the rest of Firm X’s patent professionals averaged only about a 75% allowance in the same technology area. Moreover, Attorney Y exceeded both the USPTO average and Firm X’s overall average allowance rate in each of four technology areas. A comparison of Attorney Y’s efficiency with Firm X’s average for the firm’s four highest-volume patent application technology areas is illustrated in Figure 2.

A further interesting statistic is that the four technology areas in which Attorney Y appeared as the practitioner of record span different scientific subject matters. This could mean that Attorney Y is highly skilled at a variety of scientific subjects and/or that they are an excellent manager of other practitioners with differing technology practice areas for whom they serve as supervisor, as shown by the presence of their USPTO registration number. In either case, Attorney Y’s performance for clients is markedly better than those of other practitioners at Firm X whose USPTO registration numbers appear on clients’ patent applications.

From this data, we can see that Firm X – which is considered a prestige firm by both legal professionals and business leaders – employs one patent professional whose efforts for clients far exceeds those of their peers. In other words, while Firm X might be judged from the outside as being fully staffed by highly skilled patent professionals, the data shows that at least one of its professionals – namely, Attorney Y – appears to be much more effective for clients than others, who may not be particularly skilled at their jobs.

This exercise shows that a significant difference can exist among the attorneys at a big-name firm that operates a moderately sized patent practice. It is noteworthy that the selection of Firm X for data analysis and the conclusions from the dataset were conducted by two expert patent attorneys with many years of experience in prosecution and portfolio management. We were nonetheless stymied in our task of extracting broadly applicable, individualised patent practitioner information from USPTO data. Our conclusion is that USPTO data available today can be mined for firm and practitioner information, but the process is cumbersome and subject to various limitations such as those discussed above. In sum, we found the publicly available data can by no means provide a complete objective assessment of competency in relation to patent prosecution effectiveness.

Figure 1. Responses per disposition and allowance rates vary widely among practitioners at the same firm (each bubble corresponds to a different practitioner and the size correponds to the number of dispositions)

Figure 2. Responses per disposition and allowance rate: particular attorney versus firm average for four technology areas

Patent professional selection: a proposed approach to competency assessment

We found that there is no magic USPTO data bullet to enhance the patent professional selection process. The good news is that, in most cases, whether a patent was competently generated will not really matter to the client’s business goals, provided that the cost of obtaining it did not place an undue financial burden on its owner and prevent other key business tasks from being conducted. Put another way, a patent professional’s competence in most situations will be of little moment because patents do not often serve as a critical aspect of business strategy execution.

However, in some cases, the competence of a patent professional can be crucial to business success – for instance, when a company is betting on generating significant future value from innovation investments made today. For such companies, the scope and content of patent protection can greatly affect business outcomes, at least because when scalable revenue is identifiable from their innovative product offerings, competitors will likely seek to move into the validated customer markets. If the innovator’s patents are not crafted to protect more than a product offering, as is often the case, another company may be able to freely compete in the market once the existence of scalable customers is demonstrated. Such competition can be expected to reduce the revenues obtainable from the innovator’s investment, which in turn can reduce the financial return that may have justified the business strategy that initiated the innovation direction in the first order.

Moreover, early-stage companies relying on future revenue streams or corporate valuations from innovation investments and business decisions that must be made today will typically have only one shot at getting patent protection right, because timing is almost always critical for innovators who place business bets on unproven markets and customers. In these companies, business-relevant patent protection can operate as a market differentiator that can enhance outcomes when they are in competition with other companies for limited investor and acquisition opportunities. In other words, a company that obtains objectively strong and business-relevant patent protection can signal to potential investors and acquirers that its leadership team understands how patents can improve business outcomes when properly aligned with business strategy.

The leadership teams of early-stage technology companies are more likely to be managing a variety of tasks simultaneously, which means that patent efforts are often relegated to the selected law firm. Such outsourcing means that the company’s external professionals will often work somewhat autonomously, since the business management team will typically manage its patent matters on an ‘out of sight, out of mind’ basis. The lack of active management of business-critical patent matters can result in a misalignment between the company’s goals and the actions of the external patent team.

Competence of the selected patent professional to procure and manage patent matters for a company is therefore a critical aspect of the business activities of some organisations, even while the requisite expertise to make this decision is likely lacking on their leadership teams. If, as we have found, available USPTO data cannot assist non-experts in the patent professional selection process, how can an entrepreneur or business leader use analytics to better select a patent professional for must-have patent situations? While someday AI may be able to extract the necessary data and perform critical assessments of patents obtained by individual patent professionals, such methods remain out of reach today. Our conclusion is that companies for which patents comprise a critical aspect of business strategy execution should engage a human patent expert to assist in the selection and management of a patent prosecution professional. Indeed, this is exactly what corporations do when they hire in-house patent professionals. An internal expert is likely to better understand how a specific patent matter will align with the company’s business strategy, if only because they will be exposed to this information in the course of their job.

The fact that early-stage companies do not have the budget or matter volume to justify hiring a full-time patent expert does not abrogate the need to engage a competent professional who can generate suitably broad and business-relevant patent protection. It thus follows that those companies that also seek must-have patents could benefit from engaging a patent expert to work on their behalf from a business perspective, at least on a part-time or consultant basis. We believe that an external expert who is incentivised to select and manage another patent professional can ensure that the external practitioner is engaged to generate patents for a company for which meaningful protection is intended to provide enhanced valuations.

We understand that this recommendation might lead some readers to conclude that we are advocating something that will add unnecessary expense to a company’s patenting efforts. Certainly, this may be the case for those patentees that focus solely on cost when selecting a patent professional. While this is a relevant consideration, if it is the primary consideration, the leadership team has likely failed to make the connection between the generation of strong and market-relevant patent protection and successful execution of the company’s business strategy. Put simply, the recommendation to hire a second patent expert will not apply to these companies.

From a practical level, engaging a second expert to assess the appropriateness of the cost and quality of a patent professional’s efforts as required for a company’s needs should appear obvious to those seeking patents that underpin successful business strategy execution. That is, how does a non-patent expert know how much is too much for the scope and content of protection needed in a particular situation? And how do they know what subject matter the patents should cover to best protect the business?

In our experience, there are expensive patents that create no real business value for their owners, and relatively inexpensive patents that can greatly enhance corporate valuations. In short, patents rarely affect business fortunes but when they do, they do a lot. Engaging someone who can help a company’s leadership team identify both whether patents are relevant and, if so, how the protection can be crafted to best capture such relevance can be paramount to success. Moreover, having a second opinion can also help to reduce needless expenditures and wasted management time when it is determined that patent protection is not a relevant aspect of the company’s business strategy.

Alternatively, non-experts who are tasked with selecting a patent professional could employ the usual criteria of cost, reputation and gut feeling and hope for the best. However, for most business leaders, hope is not a viable business strategy. In other words, when a business needs an expert to complete a business-critical task, sometimes it really needs two.