Rely on Valid Compensation Survey Data

Methodology, salary dates, and study sample allow boards to judge reliable research.

December 9, 2010

Your board’s compensation decisions will continue to be a point of emphasis in today’s regulatory environment. Consider a policy that details your compensation philosophy and the quality research methods you rely on.

So how can you be sure the information that crosses the boardroom table is valid, regardless of who brought it to your attention—a director, your CEO, or the compensation committee chairman?

Directors NewsletterValid salary studies include:

  • The methodology of the study;
  • The effective date of the salary information; and
  • Information on the study sample.

Here’s a closer look, according to Beth Soltis, CUNA's senior research analyst in the Credit Union Directors Newsletter:

• Methodology. This tells you what methods were used to gather and analyze the salary data in the study. It also will tell you how many surveys were sent out and how many were completed, Soltis says. It clarifies who was asked to provide the salary information. “It’s preferable that managers or human resource staff provide salary information because they’re knowledgeable about compensation policies and practices as well as compensation levels in the organization,” she says.

Methodology also confirms how researchers handled the data, Soltis explains. For instance, outliers—salaries that are at the extreme ends of the spectrum—are often removed from the final data reported in salary studies.

Weighting is a standard survey analysis procedure, designed to increase the reliability of the survey results. It’s a process of adjusting data for the over- or under-represented groups to remove bias by specific groups, she says.

Weighting is often done by credit union asset size. For example, if there’s a higher percentage of credit unions in a particular asset size than there is in reality, the data can be skewed by this group. Since larger credit unions tend to pay higher salaries than smaller credit unions, an over-representation of large credit unions can distort the data, resulting in higher salaries.

• Effective date. Salary studies usually ask participants to provide salary information as of a specific date, or the “effective date.”This is done so that data is static, representing one point in time. Since the data is static, the salary information provided by the participants is comparable and that’s necessary to make reliable comparisons from year to year.

The effective date can be an indicator in and of itself, Soltis points out. “If the information is several years old, it’s outdated and not reliable.”


 Study sample. This is the group of study participants. They could be from one industry or several industries. The study sample tells you whether the participants reflect your competitive market.

There should be a sufficient number of participants for the data to be considered reliable. There’s no magic number because it depends on the total number of organizations in the population. For example, in CUNA’s annual study, the Complete Credit Union Staff Salary Survey, all affiliated credit unions with $1 million or more in assets are asked to participate in the study. That’s the population. Those that participate in the study make up the survey sample.

Although a larger sample size will be more reliable statistically than a smaller sample size, there doesn’t have to be a sizeable number of participants in the sample. It’s more important that the sample be representative of the population, Soltis notes. Also, there should be an adequate number of participants per position and per data cut (where data is broken out by a pay element, such as geographic area).

Compensation experts recommend using three different sources to determine executive compensation market pricing, Soltis says. But boards should consider the competitive market and competitive strategy to select the best salary sources for your credit union.

And finally, make sure these sources contain the data necessary for your competitive strategy, she says. Some salary studies report only the average (a simple mean) and/or the median (the midpoint—or halfway mark—if salaries are ranked from low to high). If you want to use market percentiles (standard points along the range of salaries when ranked from low to high), you should make sure the data source you use reports these percentiles.

Adapted from the December 2010 issue of Credit Union Directors Newsletter.