Henry Ford once famously remarked that, if he had asked his customers what they wanted, they would have said a faster horse. He might have added that, if he had asked his marketing team what they thought their customers wanted, they may well have said the same slow horse with a slightly different coloured forelock.
New product development (NPD) in any sector has to be fine-tuned to meet consumer needs. A lot of the time those may be unarticulated needs, but they should never be imaginary ones dreamt up by researchers, product managers or development specialists.
That is just one of the lines that product development teams have to tread as they attempt to maximise the success rate of (and revenue from) new products. Back in 2004, in a report on innovation and profitable growth, Deloitte predicted that new product revenue as a share of sales would, across all sectors, rise from 21% in 1998 to 35% in 2007.
Bryan Seyfarth, solutions marketing director at software company Sopheon, says: "A client such as Cadbury will come to us with an objective, regarding where they are today with revenue from new products and where they want to be." The paradox is that, while a multinational like Cadbury Schweppes may be willing to put its money where its mouth is, many other brand-owners are not investing in NPD.
But if the widely-reported (but rarely substantiated) 70-80% failure rate in food and drink NPD is correct, clearly companies need to up their game. The million-dollar question is, where do these dramatic improvements need to take place?
Jeffrey Hyman, chairman of the UK-based Food and Drink Innovation Network starts from the premise that the success of a new product depends on whether it really does meet consumer needs. It is how that wisdom is interpreted that lets many companies down, he alleges.
"It means that you have to spend a lot of quality time with your consumers," Hyman explains. "Surprisingly perhaps, for a lot of marketers, this is a novel concept."
Too often, he says, qualitative research consists of focus groups of randomly-selected consumers. "You should get in a number of groups of heartland consumers and frequent users and spend a morning with them to gain real insights."
That approach should be standard practice. "But when you look around many of our supermarkets at what's new, it seems that this essential work with heartland consumers hasn't been done," Hyman says. Too much of what passes for innovation is simply variation on a theme. There is also the nagging suspicion that much of it was developed from ideas first hatched by a product or category manager, he suggests.
But surely there is nothing wrong with range extensions as an aspect of innovation? "There can be a lot wrong with them," he counters. "They appear to up the rate of innovation. But very often all they're doing is adding complexity and cannibalising the existing range."
Hyman argues that even once an idea has been taken through to the proposition and quality prototyping phase, trials should again be with these groups of heartland consumers. Only at that stage should the development team begin trials with other groups of consumers.
Sopheon also focuses on the early stages of the development process, though not to the exclusion of other stages. It makes the bold claim that its Accolade innovation process management system can reduce new product failure rates by between 55% and 60%.
"Many companies feel the need to make this process more efficient," says Seyfarth. "New Zealand King Salmon (NZKS) is an example of a company which has said it has no problem coming up with ideas, but it wants to ensure it is gathering the right kind of ideas." He adds: "We provide the capability not only to capture them but also evaluate them as early as possible."
Sopheon says that another client reduced its portfolio by 20% in six months. Richard Smith, technical manager at NZKS, says: "Accolade assists us by comparing metrics on each project, enabling the portfolio of projects to be ranked with ease." Importantly, he adds, it prevents weaker projects from competing for resources with stronger alternatives.
Efficient allocation of resources is a major consideration for many large food companies. According to Sopheon, when SABMiller came looking for a management solution, it was particularly keen to avoid duplication of R&D effort on its different sites around the world.
But few believe an automated system is some sort of magic bullet when it comes to liquidating NPD failures. Head of product development at British Bakels Gary Gibbs says: "Many products fail, but they fail for a huge number of reasons. And it's not always easy to say that there's a system that would prevent that." As he points out, as well as consumer preferences, it could come down to a mistake with the size of packaging in the supply chain, or another aspect of marketing. Or it could be to do with the NPD process, and an oversight with declarations or claims.
Gibbs believes that for larger companies, automated systems are likely to be more beneficial. But the filters provided by software-based packages are likely, in any case, to be replicated in other forms by a company's own process. So face-to-face meetings at prearranged intervals will involve, for example, key managers from manufacturing once a product is due to go live. He also contends: "Stage Gate-style processes seem to work well for managing commercial risk, but are not usually so good at assessing technical risks."
Kirsten Hoskisson, NPD manager at Jordan's Cereals, has worked with automated and paper-based systems. In her experience, they can be equally effective. Unlike Gibbs, she sees that integrated systems may have advantages for companies without the resources to put into developing their own processes, so typically smaller manufacturers.
An effective system will allow teams to store and access ideas which are not ready for launch. "It's good to have a catalogue of concepts that have had the initial work done on them, and which are ready to switch on and run with," she says.
Timing a launch to coincide with evolving consumer demand can be a huge benefit. "At Jordan's, we've been beating ourselves up for around 30 years trying to source natural ingredients, such as apricots without sulphur dioxide," she says. "Now, at last, our time has come, and at least some of the market is recognising that."
Other aspects of 'timing' can be even more important. As pressure increases to maximise returns from new products, the temptation may be to accelerate the NPD process. "There's always a risk with speed, especially if you haven't got a robust system in place," says Hoskisson. "There's a danger that some crucial piece of research could be missed, or a mistake made with the ingredients."
At the Food and Drink Innovation Network, Hyman is as impatient with terms like'Fast-track NPD' and 'Kickstart NPD' as he is with 'Blue-sky NPD'. "They usually come about because those particular companies don't have systematic development processes," he argues. "And that, in turn, usually happens when there's no board-level NPD director to strategically manage innovation."
Gibbs at Bakels explains that an own-label manufacturer may not always be responsible for the development timetable on a new product. "The process is under someone else's control, and you usually have to work to their timescale."
Even in these cases, when working for one of the larger retailers it is still important to retain influence over the process. There is one skill, says Gibbs, that no amount of matrices will replace: "People have to get better at saying 'no'."
Will this kind of 'no-can-do' attitude be appreciated by retail managers? "It will, as long as you are saying 'yes' to the projects that genuinely add value," he maintains.