14

Translating Fund Grades into Quantification

To value and measure risks for limited partnership funds, we have to overcome a series of problems. The quantitative analysis focuses on the financial strength and on the portfolio. But the relevance of the portfolio analysis follows the fund's lifecycle, with its importance increasing over time. As the investment is usually in a blind pool, any assessment has to rely – at least during the early years of the fund's life – to a high degree on qualitative criteria.

We follow on from the previous chapter and discuss a so-called fund grading system that draws upon analogies from established rating techniques for credit risk. The purpose of this discussion is not an exhaustive description of possible scoring techniques, which can be found in Crouhy et al. (2001) and Meyer and Mathonet (2005); instead, we focus on the question of how fund grades can be translated into a consistent quantification to determine ranges for growth rates as inputs for the cash flow projection models. This grading system comprises two components; that is (i) “expected performance” (P-A, P-B, P-C, P-D) and (ii) “operational status” (O-A, O-B, O-C, O-D) grades. The expected performance is assessed through benchmarking comparable funds with similar characteristics, and by identifying possible weaknesses in the fund's structure. This approach aims to provide a consistent framework for the ex-ante assessment, monitoring and ex-post performance measurement of partnerships. During the monitoring phase, the operational status grades aim to capture the risk that an unforeseen event (e.g., loss of a key person of the GP's investment team) can have a negative impact on the expected performance grades.

14.1 EXPECTED PERFORMANCE GRADES

The expected performance grades are assigned on the basis of both quantitative and qualitative criteria, the internal age to combine the two evaluations, a review and, if necessary, an adjustment of the grade. The grade reflects many attributes that are weighted depending on the specific stage within a fund's life. The quantitative score is derived through benchmarking against the fund's vintage-year peer group. The fund's internal age drives the model's sensitivity to qualitative inputs. Following this approach, the grading approach uses Bayesian inference in which evidence or observations are used to update the projections. Whereas projections for a young fund depend mainly on the subset of historical data as determined by the qualitative score, for funds with an internal age approaching 100% the model does not react to qualitative scores at all. See Figure 14.1.

Figure 14.1 Basic approach.

The model is essentially designed top-down, i.e. it is marked to comparable funds. As the fund progresses through its lifecycle and undrawn commitments decline, the quantitative score becomes increasingly important relative to qualitative factors. At the same time, increasing weight is attached to individual portfolio companies, resulting in a growing role for bottom-up analysis.

14.1.1 Determine quantitative score

While the IRR is used universally as a key measure in the evaluation of fund performance, it is highly dependent on both the NAV calculation and cash flow timing. Therefore, we treat a fund's lifetime as well as its TVPI at the end of its lifetime as dimensions that are assessed separately. The quantitative score is calculated by benchmarking a fund's ITVPI against the interim multiples of its peer-group population.

The peer group for a fund allows a LP to see how a particular fund is performing relative to other funds at a given point in time. To determine this peer group various data vendors, such as Preqin or Thomson VentureXpert, are used to define the sample of funds raised in the same vintage year, controlling for the stage and geographical focus of the partnerships in the sample. The comparison against the benchmark is translated into a quantitative score: consistent with the objective of determining a fund's quartile rank within the peer group, the fund's relative position within its benchmark is converted as a linear combination between the quartile limits, with the maximum being equal to 1, the minimum to 4, the median to 2.5, the top quartile minimum to 1.75 and, finally, the lower quartile maximum to 3.25.1

As far as smaller and less developed market segments are concerned, such as partnerships targeting distressed assets or growth capital deals in emerging markets, it may be challenging to identify a sample of benchmark funds with similar characteristics. While available databases vary in terms of their market coverage, in practice some markets are just too thin and in their infancy so it will be difficult to grade a fund against comparator funds. In such cases, an alternative solution might be sought in comparing a fund with similar funds in different markets, as long as their fundamental characteristics are sufficiently comparable. For instance, in grading a growth capital fund in emerging markets, a LP may decide to use comparable growth capital funds in other geographies as a benchmark sample.

Importantly, the relative performance of funds is found to vary substantially over their life. As far as venture capital is concerned, Schäli et al. (2002) report that only 14% of partnerships with an interim IRR in the first quartile after year 1 end in the first quartile upon maturity. After year 4, however, 50% of top-quartile funds based on interim IRRs actually end up in the first quartile upon maturity. This suggests that the best performers can be identified relatively late in the lifecycle of partnerships. Therefore, the interim top-quartile composition will change continuously – which underlines the importance of qualitative scoring.

14.1.2 Determine qualitative score

The objective of qualitative scoring is to determine a limited partnership fund's degree of compliance with industry standards applying to its closest peer group. It may be seen as a measure of how well adapted a fund is to the alternative asset market environment at a given point in time. Based on this, funds can be ranked according to their deviation from standard characteristics.

The qualitative scoring is used to evaluate several criteria related to the fund characteristics. There are multiple performance-relevant dimensions which the qualitative scoring aims to capture. The purpose of the qualitative scoring is to benchmark the fund against standard market practices. In assessing private equity funds, a scoring of the following dimensions (with several sub-dimensions) has proved to be useful (Meyer and Mathonet, 2005).

• Management team skills: private equity experience, operational experience, industry sector experience, country/regional experience, team size, team dynamics and key man, balance and coverage.
• Management team stability: cohesion of the management team, historical stability, sharing within the team, succession planning, financial stability.
• Management team motivation: incentive structure, reputation, team independence, outside activities, conflicts of interest, managers' investment into the fund.
• Fund strategy: deal flow strategy and sourcing, hands-on approach, investment focus and “sweet spot”, fund size, exit strategy, overall strategy fit.
• Fund structure: compliance with standards, cost of the structure, corporate governance.
• External validation: previous funds' track record analysis, performance of comparable funds, quality of co-investors, recurrent investors.

Points are awarded for the various criteria, depending on whether or not the fund is in compliance with standard industry practices and structures. To determine the scores, it is important to evaluate whether sufficient information is available and whether it is relevant for forming an opinion. Moreover, criteria such as the robustness of evidence (i.e., can it be observed over longer time periods and under alternative conditions?) or persistence (i.e., is this expected to continue?) need to be taken into account. As a matter of course, scores are not strictly additive, but within the “continuum” of operating funds that have attracted sufficient institutional investments, it appears to be a reasonable heuristics.

In parallel with the quantitative scoring, and assuming that n criteria are assessed, the following applies: if all n criteria are found to be in line with standards, the highest score of 1 is awarded; conversely, if none of the n criteria is fulfilled, the lowest score of 4 is given. The qualitative scoring is based on the assessment of an investment proposal's key dimensions and uses a peer-group population as a yardstick. Therefore, the first step is to identify the “peer-group” universe, which will be used as a reference.

Prior to the investment, the benchmarking peer group of the same vintage year is usually unknown. Its composition needs to be “estimated”, based on the current conditions and the prevailing standards in the fundraising market. The scoring is based on the assumption that the unknown future peer group will be comparable to the population of recent vintages. Even if the composition of the peer group is not known, to a high degree the investment decision is based on a list of criteria that is generally seen as consistent with the best-performing funds.2 In this context, we suggest differentiating between two peer groups. Because of the long investment cycles, statistics on the historical peer group for fully realized vintage years will be “stale”. Reliable quantitative information will relate to vintage-year cohorts that, in the extreme, date back more than 10 years. Therefore, qualitative scores are mainly based on interim data, anecdotal evidence and lessons learned from relatively young funds. The qualitative scoring cannot be seen as “static”. The scoring methodology needs to be continuously updated and calibrated as new mainstream characteristics emerge and industry players do not further apply certain established practices.

14.1.3 Combine the two scores, review and adjust

As the fund in question continues to mature, quantitative information becomes increasingly available. In the early years of the fund, quantitative information complements qualitative factors, but as the fund approaches the final phase of its life quantitative information eventually fully replaces the qualitative judgment. The method used to combine the qualitative and quantitative scores is based on a parameter that summarizes the predictive power of each. There are two endpoints, the start and the end of a fund's life. While at the beginning only the qualitative score is relevant, at the end of the fund's life the opposite is true. At this stage, the TVPI is known, making a qualitative assessment obsolete. Therefore, once the quantitative and qualitative scores have been calculated, they are aggregated weighted by the fund's internal age (IA):

As long as the fund is young, the performance grade is fully weighted towards the qualitative score. As the internal age approaches 1, which is equivalent to the fund having distributed the majority of its value, the performance grade becomes more heavily weighted towards the quantitative score. The expected performance grades are then determined as in Table 14.1.

Table 14.1 Linking scores to expected performance grades

The last step for determining a fund's expected performance grade is the review and adjustment of the grade. The grade can be adjusted based on such qualitative factors as diversification of a fund's portfolio or its operational status grade (O-grade). While these are all valid reasons, there is room for bias and, to some degree, abuse, since this adjustment can significantly change the projected performance of a fund. Introducing quality controls into this process would allow it to become less subjective and more consistent. Such quality controls could take a number of “red-flag” criteria into consideration. For example, is the fund's portfolio overdiversified or is too much exposure taken. Is the fund's remaining liquidity insufficient to support its investment strategy? And to what extent are there questions about the quality of portfolio companies?

If the sum of these “red flags” reaches a predefined threshold, the fund should be downgraded. For this purpose, a grading review policy should be put in place. It should, for example, incorporate the operational status grades or insights gained from the cross-checking of portfolio company valuations between various funds. Such a system of checks, not dissimilar to the qualitative scoring method, would remove some of the potential bias and ensure the integrity of the grading system.

Generally, it is important to note that the scoring should not be considered as an assessment of the various pieces of a puzzle. More often than not, the assessment of the various dimensions will not be clear-cut and therefore requires taking a look at the investment proposal's “big picture”. The overall fit of all these components is essential, notably the fit between the team and the fund strategy, but also the relation between the fund structure and the fund strategy. Finally, if too many dimensions cannot be assessed or too little evidence is found in the course of due diligence, this lack of completeness can set limits on the overall qualitative score assigned.

14.2 LINKING GRADES WITH QUANTIFICATIONS

After the fund's grade is determined, we would like to know the range of its projected TVPI. One approach lies in the collection of historical statistical data for funds according to their grades. This approach makes sense in situations where there is a sufficient amount of historical data, the environment is relatively stable and where categories do not change significantly. It is, however, problematic for alternative investments where historical data are rare and the investment environment is subject to material changes. Alternatively, classifications can be used to look for comparables with similar characteristics – either from the not too distant past or from the actual peer group – and take their observable quantitative characteristics as a reference.

14.2.1 Estimate likely TVPIs

We estimate a fund's likely TVPI by taking into account the fund's current grade and its internal age, and use a Monte Carlo simulation to select a scenario from historical TVPIs. For the simulation, historical TVPI figures are drawn out of the basket according to a schedule that reflects the grade of the fund and its internal age.3

Existing public databases on fund returns do not allow ex-ante conditions to be linked to outcomes. Additional risks, such as leverage or foreign exchange exposure, do not necessarily require specific modelling as the historical return statistics relate to funds that were leveraged and/or had foreign exchange exposure within the fund. However, details about the intended future strategy are generally not captured at inception, and there may be style drifts as a fund needs to adapt to a changing environment. Therefore, assumptions must be made regarding the relationship between the risks and rewards ex ante. Such assumptions should be sufficiently conservative in the sense that investors should be assumed not to be able to consistently pick above-average funds or avoid underperforming partnerships. In fact, it is important to be cautious about what is described as a “random pick” when selecting the portfolio of funds. The samples are drawn out of ex-post statistics for funds, i.e. funds that LPs invested in because they believed they would get “outperforming returns” thanks to rigorous due diligence.

Table 14.2 shows the weights of the baskets at a fund's internal age of 0 for all grades.5

Table 14.2 Weight of quartile basket for risk budgeting

Essentially, it is assumed that without superior selection skills, selecting a “standard” fund will result in average returns. “Experiment” funds, i.e. partnerships managed by emerging managers in new markets, should not be invested in unless financial resources are sufficient to absorb the poor returns or losses associated with a fourth-quartile performance.

However, this quantification is not taking the potential upside into consideration: conceptually, this is in line with a risk-budgeting approach. Alternatively, we could be neutral and assume that there are rewards commensurate with the amount of risk that is being taken. This could, for instance, be modelled with a matrix as shown in Table 14.3.

Table 14.3 Weight of quartile basket for risk/reward relationship

Calibrating the matrix as depicted in Table 14.3 assumes that for a commitment in a “standard fund” one could expect returns close to the average with a lower probability of extreme performance than a “mainstream” fund. For an “experiment” fund the expected outcome would be either extremely good or poor. Rather than predicting the portfolio's performance, such a calibration would be in line with an investment strategy that aims to give more weight to contrarian investing and searching for unexplored areas of the alternative asset market. In fact, it might be argued that portfolio management in the alternative investment arena should aim at maximizing the allocation to “experiments” and “niches” within the set limits that are based on this risk budget. In this sense, the assumed risk-taking behaviour is consistent with exploring new areas in the alternative asset space.

For any internal age 0 < t < 1, the weights given to the baskets are determined by the following matrix:

At the end of a fund's lifetime (with an internal age of 1), its grade is equivalent to its performance quartile within the benchmark (Table 14.4). Let s_i;j ≤ ft(t) be the fund's final TVPI (i.e., the multiple achieved at the end of the fund's lifetime) under a scenario i for a fund graded j at internal age t.

Table 14.4 Weight of quartile basket at end of fund's lifetime

To deal with the issue of the J-curve, we base our projections during the initial years of a fund on the TVPI statistics for realized historical funds only, rather than on the fund's interim multiples. Burgel (2000) reports that after 7 to 8 years, large changes in performance become unlikely and interim IRRs and final IRRs converge. After a few years, IIRRs can already provide a good approximation to the fund's overall return. To capture this behaviour, we foresee a trigger point, T, from which time onwards the interim multiple receives an increasing weight in the projections. As long as the fund's internal age is lower than a given T, the interim multiple is ignored entirely. If the internal age t ≤ T, the projected final TVPI for this scenario is

Only after the fund has reached an internal age higher than a set trigger do we start to take the interim TVPI into consideration. If t > T:

Thus, we now have a method that helps us determine the range of the final TVPIs and the range of the final life (discussed previously), which converge towards a fund's true final TVPI and its true lifetime.

14.2.2 Practical considerations

Qualitative scoring is not a substitute for due diligence, but instead is based on its results. The aim is to categorize and compare investment proposals. While there is a sort of “market view” of what represents an “ideal fund” in terms of structure, industry and geographical focus, team, etc., in this unregulated and opaque asset class there is ongoing innovation. Therefore, the definition of what constitutes “standard” is continuously evolving.

It is important to avoid a conservative bias in assigning grades. While such a bias would be an understandable reaction in an environment of high uncertainty, to be able to identify risks at the portfolio level, assessments need to be as unbiased as possible. It is certainly good investment management to be conservative in decision-making, but assessments have to be as free from biases as possible to allow the decision-maker to form an opinion.

Finally, the grade should not be confused with pricing. Even if a fund has a low grade, the price on the secondary market can be highly attractive. This is comparable to a bond rating, where the premium required by investors is not reflected in the rating. The two questions “what is the quality?” and “how much should we pay for?” are conceptually different.

14.3 OPERATIONAL STATUS GRADES

The expected performance grades are complemented by the operational status grades gained during the course of monitoring (Figure 14.2). The operational status grades capture information that is conceptually close to event risk. These events – unless a mitigating action follows within the short to medium timeframe – are expected to have a negative impact on a fund's performance.

Figure 14.2 Expected performance and operational status grades.

We also suggest four grading classes for the operational status grade depending on the severity of the operational issue (Table 14.5).

Table 14.5 Operational status grades

The operational status grade aims to identify these events and to form a judgment on its severity. In essence, the grades have two functions. One function is to alert the investor in cases where “red-flag” events could have such an adverse impact that they need to be addressed without delay. The second function concerns diagnosis, i.e. forming a judgment on the degree of potential impact resulting in a priority setting for monitoring corrective actions.

Assessing the severity of the event's impact is typically highly subjective. As there are all kinds of events possible depending on the investment area, no exhaustive list can be given. Accumulation of such events may be taken as a sign that a fund is veering off track and may show substandard performance. As these operational status grades can be indicative of a possible impairment, they should always be reflected in an updated expected performance grade and, therefore, tied into the fund's valuation.

14.4 CONCLUSIONS

As we discussed in Chapter 7, “objective” risk in illiquid asset classes is extremely difficult, if not impossible, to determine in the absence of observable market prices. Investment professionals therefore need to use subjective probabilities to operationally define perceived uncertainty. Under these circumstances, a more practical question is whether a risk metric is “useful”, in the sense that it promotes behaviour that is desirable in a given application. The purpose of the techniques presented in this chapter is mainly to encourage disciplined, meaningful investment behaviour and the efficient incorporation of relevant up-to-date information in the investment process.

The definition of the expected performance grades ensures that projections converge towards the fund's final lifetime and TVPI, and that the grades converge to the fund's final quartile within its benchmark population. The degree of the projections' precision depends on how far a fund is advanced in its lifecycle.

1 In cases where the fund's ITPVI is higher than the maximum ITVPI of the benchmark, the quantitative score remains at 1; likewise, in cases where the fund's ITPVI is lower than the minimum ITVPI of the benchmark, the quantitative score remains at 4.

2 Admittedly, it is difficult to perform consistent benchmarking focusing on particular industries and/or regions. In some market segments, relatively few funds are raised, and the sample in publicly available databases is likely to be even smaller as typically not all funds raised are actually captured. Thus, LPs may be confronted with a trade-off between using a larger sample with fewer similarities and employing a sample that shows a high degree of similar characteristics, which, however, includes only few funds.

3 Alternative schedules may be considered. However, in simulations we have found that the results are fairly insensitive to the schedule chosen.

4 The same problem can apply to bottom-up models. Also, here it can happen that one “case” has already been realized and it is no longer possible to employ the model with a scenario parameter for a different case.

5 For the rationale underlying these weights, see Meyer and Mathonet (2005).