Innovation and organisational performance : A critical review of the instruments used to measure organisational performance

Organisational performance is an important indicator of organisational success (Stegerean & Gavrea 2010). Apart from organisational performance, organisational success also relates to employee skills levels, personnel development, quality of strategic planning and the ability to understand and adapt to the nature and dynamics of the business environment (Carvalho et al. 2016). However, organisational performance is arguably the most important indicator of organisational success and one of the most important variables in management research (Stegerean & Gavrea 2010).


Introduction
Organisational performance is an important indicator of organisational success (Stegerean & Gavrea 2010).Apart from organisational performance, organisational success also relates to employee skills levels, personnel development, quality of strategic planning and the ability to understand and adapt to the nature and dynamics of the business environment (Carvalho et al. 2016).However, organisational performance is arguably the most important indicator of organisational success and one of the most important variables in management research (Stegerean & Gavrea 2010).
An important aspect to consider when evaluating innovation efforts and organisational performance is the time factor, given that there is a time lag between innovation initiatives and the outcome that follows (Likar et al. 2014).In fact, O'Connor et al. (2008) state that the time lag between innovation and its impact on organisational performance ranges from 3 to 6 years.It is important to note this, as a focus on short term indicators (e.g. return on investment [ROI], sales growth and operating income) may be inappropriate and may indicate that innovation strategies are not working, while the effect may only be visible in the longer term (Ndregjoni & Elmazi 2012).
Although the study of organisational performance has been at the core of management research, very little has been done with regard to appropriate measures to assess the effectiveness of innovation initiatives.In addition, a cursory review of the literature shows that researchers focus on the discussion around typologies of organisational performance on financial and non-financial aspects, with very little attention to other dimensions, such as objective and subjective measures.The present study, therefore, aims primarily to investigate the most frequently used instruments.The results of this investigation will then be used as a lens through which to investigate which typologies (financial vs. non-financial; objective vs. subjective) of organisational performance were adopted and further to investigate whether the instruments selected played a role in the outcome of the study.This will result in the compilation of a more comprehensive and updated literature review that can form the basis for future research when selecting measures of organisational performance.

Problem statement and objective
The results of studies that investigate the relationship between innovation and organisational performance are inconclusive, with some studies (Carvalho et al. 2016;Cortez et al. 2015;Mafini 2015) showing a positive relationship, while others showed mixed results or no relationship with no definite conclusion (Hervas-Oliver, Sempere-Ripoll & Boronat-Moll 2014; Simachev, Kuzyk & Feygina 2015).This inconsistency has been attributed to a number of factors, including, among others, the measures used to evaluate organisational performance.
In an attempt to understand these inconsistences, Rubera and Kirca (2012) conducted a meta-analysis in a quest to better understand a firm-innovativeness-performance relationship, drawing on the chain-of-effects model as a unifying framework.The study revealed that the size of the firm, the sector in which the firm operates and the nature of innovation (radical innovation, for example) adopted can influence the relationship between innovation and organisational performance.However, although Rubera and Kirca's study is significant in many ways, the study did not investigate whether the type of instruments used to measure organisational performance can also influence the relationship between these constructs.This reveals a gap in the literature and shows the need for a critical review of the influence of the type of instruments used to measure organisational performance on the reported relationship between innovation and organisational performance.
Therefore, the objective of this study is twofold: firstly, the study seeks to investigate the most frequently used instruments and, secondly, the study will investigate whether the type of instruments used does influence the nature of the relationship between these constructs.

Measures of organisational performance
The construct of organisational performance is central to the understanding of organisational success and the factors responsible for that variation (Hoopes, Hadsen & Walker 2003).In order to get an accurate and comparative gauge of the variation mentioned, valid and reliable measures are necessary (Saunders 2012).Although several methods for measuring organisational performance exist, these methods can be classified into two main categories, namely financial and non-financial performance measurement (Maltz, Shenhar & Reilly 2003;Shin et al. 2015).

Financial performance measurement
Despite the general consensus among scholars that a firm's performance is a multidimensional construct, one of the most extensively used measures is the financial component -the fulfilment of the economic goal of the organisation (Gentry & Shen 2010).This is in line with Davidson's (2003) argument that the primary goal (aim) of management is to generate profit and to maximise shareholder value.Important to note is that scholars who embark on empirical studies employ a number of different measures to evaluate financial performance (Berger & Bonaccorsi di Patti 2003;Davidson 2003).
The literature research reveals that to assess the financial aspects of organisational performance, researchers generally use either accounting-based measures, such as profitability, sales growth, return on assets (ROA), return on sales (ROS), return on equity (ROE) and/or ROI, or stock market measures, such as Tobin's Q and price earning (P/E) ratio (Hult et al. 2008;Likar et al. 2014;Nawaz, Hassan & Shaukat 2014;Tsao & Lien 2013).
In the 1980s, researchers primarily used accounting-based measures of financial performance (Hoskisson et al. 1999).However, with the rise of shareholder activism in the late 1980s and early 1990s, organisations started adopting shareholder value maximisation as a measure of financial performance (Useem 1993).This paradigm shift promoted the adoption of market-based performance measures in management research (Hoskisson et al. 1999).
Despite its limitations, profit maximisation remains one of the key measures of organisational performance (Garg, Joubert & Pellissier 2004).Various researchers use growth as a sole measure of performance, while others choose to combine growth and profitability (Likar et al. 2014).However, most researchers prefer to combine ROS, ROA, ROE and ROI because they complement one another.The use of a single ratio generally does not provide sufficient information to allow investors to judge the overall performance of the firm (Marx 2004).For instance, ROA allows analysts to evaluate the effectiveness and efficiency of the firm's management and employees in generating profit by productively using assets (Firer et al. 2008).On the contrary, ROS allows analysts to evaluate the effectiveness and efficiency of the firm's management and employees in generating profit by means of sales (Karanja 2011;Marx 2004).
For the sake of clarity, a short explanation of the aforementioned measures has been provided in Table 1.
According to Campbell and Mínguez-Vera (2008), accountingbased measures are useful because they provide useful objective measures of organisational performance.However, various authors (Fernandez 2001;Frigo 2003;Smith 2007) argue that accounting measures only reflect the history, both in terms of income statements, which explain what happened in a certain year, and those of the balance sheet, which reflects the state of the firm's assets and liabilities at a certain point in time.As such, it is impossible for accounting-based measures to measure value creation.
The challenge of uncovering the true financial value of innovation is a result of practices such as international financial reporting standards (IFRS) not adequately reflecting innovation expenditure (Frigo 2003;Smith 2007).IFRS forces the recording of the immediate expense of investment and thus creates a challenge owing to the time lag between innovation expenditure and the effect it has on financial performance.This leads to a situation in which researchers will need to correlate initial expenditure with a product that will only emerge a few years later (Selby 2010).
Despite the need to measure the effects of innovation, Morris (2008) convincingly argues that measuring innovation presents a problem in itself, because innovation involves venturing into the unknown.Therefore, if one tries to pin down these unknowns too quickly, they may become harder to recognise.In addition, when measuring the impact of innovation, the innovation lifespan should also be put into perspective (Eggink 2011).For instance, sustaining innovation is continuous in nature and as such there is no beginning and no end to the innovation process (OECD/Eurostat 2005).Moreover, different types of innovation will have different lifespans.For example, some innovations will last for a very long time while others may have a short lifespan.
Several market-related measures are proposed in order to account for the long-tern benefits of innovation in an organisation.These include Tobin's Q and price earning (P/E): • Advocates of Tobin's Q argue that stock market measures incorporate all relevant information and thus, unlike accounting-based measures, are not limited to a single aspect of financial performance (Lubatkin & Shrieves 1986).Tobin's Q is a ratio that indicates the market value of the firm in relation to the replacement cost of the tangible assets (Tobin 1969).Tobin's Q is computed by dividing market capitalisation by the replacement cost of the firm's assets (Cho & Pucik 2005).Tobin's Q is based on the idea that stock markets, if the takeover market for companies was efficient, would operate at a Tobin's Q of 1 (Karanja 2011).In other words, the value of 1 for Tobin's Q indicates that the market value of the firm is greater than the value of the recorded assets in the book of accounts.High Tobin's Q value is an indication of higher capital investment.In contrast, a Tobin's Q value of less than 1 indicates that the market value of the firm is less than the recorded assets in the book of accounts.• Price earning (P/E), on the contrary, is calculated by dividing share price by earnings per share (EPS).In this method, the relationship between the market share price of a share of stock and the stock's current EPS is often stated in terms of P/E ratio (Garrison et al. 2008).The strength of the P/E ratio is its ability to use current and historical data to predict the future.Consequently, investors widely use the P/E ratio as an indicator of future prospects.A high P/E ratio means that investors are willing to pay a premium for a company stock, mainly because the company is expected to have higher than average future earnings growth.According to Selby (2010), when the company's outlook holds the likelihood of future profit, a generic investor will be more inclined to buy that stock.
Despite the intuitive appeal of the above-mentioned measures of the stock market (Lubatkin & Shrieves 1986), the assumption of market-efficiency has been questioned by prominent scholars in finance (Tobin 1969).Bettis (1983)

Instrument Description
Return on assets ROA is an accounting measure of a firm's financial performance based on income before tax and interest, and it indicates how profitable a firm is in relation to its assets (Alexander & Nobes 2010).It shows how effective managers are at generating revenue from the invested assets.
Return on sales ROS is a performance variable used to evaluate the firm's operational efficiency (Karanja 2011).It indicates how much profit is being generated for each rand of sales.
Return on investment ROI is defined as net operating income divided by average operating assets (Garrison, Noreen & Brewer 2008).ROI measures how efficiently the organisation utilises its available assets to generate income.Thus, the greater the return on investment, the better (Marx 2004).
Return on equity ROE, on the contrary, measures the return earned on the owner's investment.It relates to the return generated for shareholders with finance made available by the shareholders (Alexander & Nobes 2010).It is calculated by dividing the net profit after tax by the shareholders' equity.Generally, the owners are better off with a higher ROE.
market as valid for assessing organisational performance (Hoskisson et al. 1999).In support of this view, Shook et al. (2004) agree and argue that in order to improve the quality of construct measurement, a stream of management researchers prefer using multiple indicators to measure key constructs and then use the structural equation modelling technique (SEM) to do the analysis.For instance, Tsao and Lien (2013) used both ROA and Tobin's Q whereas Talke, Salomo and Kock (2011) and Padgett and Moura-Leite (2012) decided to use Tobin's Q exclusively, mainly because of its ability to capture the value of long-term investment such as innovation.

Non-financial performance measurement
According to Ndregjoni and Elmazi (2012) More recently, Gentry and Shen (2010) conducted an extensive literature review on organisational performance with the aim of contributing to the debate concerning appropriate measures of organisational performance.They concluded that the use of both financial and non-financial measures is the most appropriate and sound approach to measure organisational performance.However, the authors further argue that the use of financial aspects of performance as a sole measure is not necessarily wrong, but they emphasise that researchers should always clearly define which aspects of organisational performance they intend to study, and then develop and test the hypotheses around that.All of the above should be viewed against the background research against which organisational performance is measured, namely objectively and subjectively.

Objective versus subjective measures
Objective measures are the absolute values of a firm's actual performance (Battor & Battor 2010) and subjective measures generally ask respondents to assess their company's performance relative to that of their competitors (Greenley 1995).For instance, objective financial measures are audited financial data such as sales, profit or asset values (Rajan & Reichelstein 2009).By contrast, the term 'subjective measure' is used to mean that the company's performance is derived from direct observations by management, financial analysts or employee perceptions about organisational performance (Dawes 1999).By virtue of its nature, objective measures are verifiable whereas subjective measures cannot be verified (Rajan & Reichelstein 2009).

Method
This study adopted two generic steps central to the systematic review methodology (Nightingale 2009), namely defining the search strategy, and then selecting relevant studies by applying the inclusion and exclusion criteria.Originating in medical science, a systematic review differs from conversional reviews in that it aims at synthesising research in a systematic, transparent and reproducible manner (Tranfield, Denyer & Smart 2003).A systematic literature review uses explicit, thorough methods to identify, select, appraise and synthesise a set of research studies on a well-defined topic (Robson et al. 2007).The primary aim of this review was to identify and report on the instruments used in prior studies that investigated the relationship between innovation and organisational performance, and to identify the most frequently used instruments as well as the rationale behind choosing those instruments.
The keywords 'innovation' (innov*) and 'performance' (perform*) were used in the search.The options (criteria) selected for the search were full text, peer-reviewed and scholarly journals.Target articles needed to match both keywords in a title.Fifty-eight databases on the major database (presented in Box 1-A1), EBSCOhost, were searched for articles and 120 articles were retrieved.Articles whose abstract indicated that either financial or non-financial performance was used as a measure of organisational performance, which were published in English in the last 5 years and where the full text was available were included in the study.Only 71 articles (Table 1-A1) met these criteria.

Financial measures
The different instruments used to measure financial performance in the sample of 71 studies are presented in Table 2. From the sample of 71, a total of 16 financial instruments (profit, sales growth, ROA, ROI, turnover, ROE, ROS, Tobin's Q, operating costs, market to book, income, cash flow, basic earning power, long-term debt, inventory turnover and EPS) were used to measure financial performance.
In support of the argument by Cho and Pucik (2005), Table 2 shows that profitability, despite its weaknesses in measuring long-term investment, is by far the most preferred financial indicator used to measure financial performance, with a staggering 29 studies opting to use this measure, followed by sales growth with 28 studies.The most cited reason for using profitability and sales growth to measure organisational performance is twofold.Firstly, authors argue that innovative behaviour leads to improved operational performance such as cost efficiency, quality improvement and speed to market, which ultimately results in higher profitability and sales growth (Cambra-

Non-financial measures
Table 3 presents the instruments used to measure nonfinancial aspects of organisational performance when investigating the relationship between innovation and organisational performance.From the sample of 71 studies, a total of 10 instruments (market share, customer satisfaction, productivity, operational efficiency, employment growth, quality, competitiveness, reputation or branding, product attractiveness and quick to market) were used to measure non-financial aspects of organisational performance.Table 3 reveals that market share (14 studies), customer satisfaction and retention (12 studies) and productivity (10 studies) are the most popular instruments used to measure non-financial components of organisational performance.Interesting to note is that there are no reasons provided for why the measures were selected.However, one can infer that market dominance, customer satisfaction and productivity were chosen because they are easy to measure and they provide useful information to gauge whether a company is doing well or not., 3, 4, 8, 18, 24, 27, 33, 35, 38, 41, 53, 591

Subjective versus objective
Only three studies (Articles 50, 53 and 54) used both objective and subjective measures.In two studies (Articles 50 and 53), the results of the study revealed mixed results and in one study (Article 54), the results showed that innovation leads to superior organisational performance.Despite the importance of using both objective and subjective measures, a considerable number of studies adopted either subjective or objective measures of organisational performance.

Subjective measures
Table 4 presents the article reference number of studies (see Table 1-A1) that used the subjective measures of organisational performance and the findings of the studies that investigated the relationship between innovation and organisational performance.
As stated in the literature, subjective measures are perceived organisational performance, where respondents are requested to assess their company's performance relative to that of their competitors.Of the 71 studies that investigated the relationship between innovation and organisational performance, 43 studies used the subjective measures of organisational performance.
The findings provide overwhelming evidence (41 studies) indicating innovation is positively and significantly related to organisational performance.In contrast, two studies found mixed results.

Objective measures
Table 5 depicts authors and hypothesis results of studies that used objective measures of organisational performance on the relationship between innovation and organisational performance.Objective measures, the absolute values of a firm's actual performance, are generally sourced from an independent body such as a stock exchange.
Table 5 shows that when objective measures of organisational performance are used, the higher number of studies reveals mixed results.This suggests that the type of instrument used might also influence the results in studies that investigate the relationship between innovation and organisational performance.For example, the study conducted by Likar et al. (2014) showed innovation is significantly and positively related to performance when measured using ROE, whereas the same study revealed no relationship when ROS and ROA were used.Table 5 shows that, of the 25 studies that investigated the relationship between innovation and organisational performance, 13 found a positive relationship and 12 found mixed results.

Managerial implication
The primary purpose of this study is to report on the instruments used to measure organisational performance and investigate whether the type of instrument used influences the results of those studies that investigated the relationship between innovation and organisational performance.Using the systematic review methodology, this study finds that combining both financial and nonfinancial measures is touted as the most effective measure of organisational performance.In total, 37 studies use both financial and non-financial measures, which constitute 50.7% of the overall sample of articles.However, a substantial number of authors still prefer to use financial measures as the sole measure of organisational performance, with 29 studies focusing exclusively on the financial measures, which constitute 40.8% of the overall sample.The sole use of financial indicators as a proxy for organisational performance may be informed by the popular notion that ultimately the goal of the organisation is to maximise profit in the short term and to maximise shareholder value in the long-term.
In addition, the study provides evidence that profitability, sales growth, ROA, ROS, ROI, ROE and turnover are the most preferred accounting measures for financial performance.
Similarly, the study further reveals that Tobin's Q is the most favoured market-related measure used by innovation scholars to measure financial aspects of organisational performance.

Total -25
The use of any specific measure of organisational performance is not implicitly wrong, but Gentry and Shen (2010) urge that researchers should always be cautious in their approach and clearly define which aspects of organisational performance they intend to study, and then develop and test hypotheses around that defined area.
When findings were studied, this study showed that organisations that practise innovative behaviour generally exhibit superior organisational performance relative to organisations with less innovative behaviour.The study showed that 54 studies, which constitute 76% of the overall sample, supported the hypothesis that innovation leads to superior organisational performance.In addition, the findings also showed that 60.6% of the overall sample used the subjective measures of organisational performance, relative to only 35.2% which used objective measures of organisational performance.When objective measures were used, the findings reveal that a higher number of studies (48%) showed mixed results, no relationship or negative relationship, relative to 0.05% which showed mixed results, no relationship or negative relationship when subjective measures are used.This finding suggests that the selection of the instruments to measure organisational performance does influence the outcome of the results, as shown in studies that investigate the relationship between innovation and organisational performance.
Thus, the implications of the research for both researchers and practitioners can be divided into two main areas: • Firstly, the study revealed the measurement instrument favoured by researchers.But of significance is that the reasons for selecting the instruments are generally based on the popularity of the instrument in this domain, and not necessarily based on the objective of the study.This observation suggests that researchers should be more cautious when selecting the instrument to measure organisational performance because the instrument has a direct impact on the outcome of the study.• Secondly, the finding shows that the method in which the instruments is used can affect the outcome of the research.In other words, when subjective measures of organisational performance are used, the outcome of the results is easily predictable.In contrast, when objective measures are used, the extent of variability of the results increases.In other words, the outcome of the results is not easily predictable when objective measures are used.As such, researchers and practitioners should be more alert to the possible false inferences that may be the result of using a specific method to measure organisational performance, particularly the use of subjective measures.

Conclusion
In conclusion, this finding supports the argument put forward by Gentry and Shen (2010), which states that a thorough literature study should be central to decision-making when selecting measures of organisational performance, as the types of measures seemingly influence the outcome of the enquiry.

Recommendation for future research
This study should serve as stimulus for future studies to explore all the possible factors that influence findings related to the relationship between innovation and organisational performance.Future studies that investigate the relationship between innovation and organisational performance should try to isolate the role of innovation on organisations, and eliminate the cloud created by factors such as measurement tools, by selecting the instrument(s) based on the objective of the study.
http://www.sajesbm.co.zaOpen Access argues that, even if the market-efficiency theory holds, stock price does not necessarily reflect its fundamental value because it is influenced by what management chooses to disclose to the investors.Acknowledging that neither accounting nor market-based measures are perfect, management researchers have accepted measures based on both accounting and stock
, non-financial measures must also be assessed in order to evaluate overall performance, for two main reasons.Firstly, several interest groups are involved in the business and they all have particular goals and expectations related to the organisation.

TABLE 2 :
Financial instruments used to measure organisational performance.
Note: Numbers in brackets represent studies that exclusively used financial measures.

TABLE 3 :
Non-financial instruments used to measure organisational performance.

TABLE 4 :
Subjective measures of organisational performance.

TABLE 5 :
Objective measures of organisational performance.

TABLE 1 -
A1: Chronological list of articles selected for the research.Lai, Kao & Chen Target costing, business model innovation and firm performance: an empirical analysis of Chinese firms 45 2012 Mazzola, Bruccoleri & Errone The effect of inbound, outbound and coupled innovation on performance Appendix 1 continues on the next page →