Massey-developed robots will soon be working in New Zealand orchards

It was a bulletin on Radio New Zealand’s 6.30am rural report that made Garth Atkinson pause. The kiwifruit industry was going through one of its perennial crises. The fruit were ripening on the vines, but growers could not find the labour to pick them.

By Lindsey Birnie and Malcolm Wood.

During the course of the 13-week season, the newsreader said, 100 million kiwifruit would be picked by hand.

By hand, thought Atkinson. In an age of automation the idea seemed an anachronism Perhaps, he thought, Massey could do something about that, and as a business development manager with the School of Engineering and Advanced Technology he had an idea about who to approach.
Dr Rory Flemmer, the expert in robotics to whom he took the problem, was more certain.
With what he knew and the expertise he had to hand, an automated picker was definitely do-able.

In a industrial workshop on Massey’s Palmerston North campus the proof of Flemmer’s assertion is taking shape. At a series of workstations postgraduate students are designing the specialised circuitry that will link the picker’s servomotors to its central controller. In a forecourt, PhD student (and project managing director) Alistair Scarfe is fabricating the four articulated arms that will do the picking. Close by is the gleaming metal chassis with its oversize Tonka-toyish tyres on which the picker will manoeuvre.

Some time in mid-2008 the picker will despatch itself on its maiden voyage out into an orchard. Its instructions, says Flemmer, will be simple: go to a set of GPS coordinates; identify a row of vines; pick that row and successive rows; if your bin is full, go and change it; and when you have finished picking, come back and stop.

“The picker is completely autonomous,” says Flemmer. “It will go out and follow thinking behaviour patterns.”

The lineage of industrial robots goes back almost half a century to 1961 when Unimate, the first of its kind, unobtrusively entered service with General Motors. Unimate was first used as a pick-and-place robot to transfer hot metal parts from a die-casting machine. Six years later it was being used for spot welding, and in 1970 the carmaker opened an automated spot welding line of 28 robots.

In the early 1980s, when the Japanese became converts to the new technology, the field began to take off.

Since then the numbers of industrial robots has burgeoned. A survey published in 2004 estimated that at least 800,000 were in use within industry worldwide, 350,000 of them in Japan, close to 250,000 in Europe, and about 112,000 in North America.

In the automobile industries of Japan, Italy and Germany the ratio is one robot for every 10 production workers.

It is not unusual for an industrial robot to have a pay-back period of as little as one to two years.
“The modern economy is completely dependent on robotics,” says Flemmer. Everything you purchase is made by robots. If it weren’t made by a robot you couldn’t own it – it would be too expensive. If you had a car that was made by hand it would cost you hundreds of thousands of dollars. And in fact it couldn’t be made by hand. All of the electronic aspects are made by robots because the components are too small to see, never mind place and solder.

“We think [a product] is made in China by people; it’s not, it’s made in China by robots.”

But those robots are largely in foreign factories. We don’t see them.

Picture this then. You are driving through the green Bay of Plenty countryside when you see a far off movement among the vines.

Manoeuvring itself through the kiwifruit trellises at a measured walking pace, pausing when it needs to, is an extraordinary machine, its four grey powder-coated arms dancing from vine to bin and back. The only sound is the gentle pulse of a generator. Elsewhere another picker goes about its business.
Even from the distance of your car window, these machines seem to behaving oddly. There is an apparent intelligence to their actions. Choices and decisions are being made.

The picker only pauses to return its bin to the orchard headland and collect another. The uncanniness of the sight becomes still stronger when, after a while, you may notice that the two pickers appear to be coordinating their picking paths.

And if you were to approach the machines, the detail of their actions would come in to focus. The pickers are choosing particular fruit: fruit that is neither over- nor under-ripe; fruit that falls in a given size range; fruit without blemish. Four kiwifruit a second are being placed in the bin. That’s 14,000 kiwifruit-an-hour-per-machine, picked from the moment the dew dries on the vines until late into the night.

During spring these same machines carrying modified arms will be used to dispense puffs of pollen into the waiting flowers. And in the off season? The machines will be packed into containers and shipped to the kiwifruit orchards of Italy.

It is all a far cry from the traditional bucolic images of harvest time, of good-spirited communal labour. But then those times are long past anyway. Nowadays it is a struggle to find enough people to pick.
Kiwifruit Growers Incorporated general manager Mike Chapman sketches the industry’s labour needs. “Through winter into spring we have repacking and winter pruning, and we have about 11,000 full-time workers. We need about 9000 extra workers for the kiwifruit season, starting from the end of March to a finish mid-to-late June. Every year it’s a challenge finding those additional workers. We expect this year 5000 of that 9000 will be from existing staff, and approaching 2000 brought into New Zealand under a recognised seasonal employers scheme. The rest are people on working holidays – backpackers.”

A fruit picker in New Zealand may not earn a fortune, but he or she earns far more than someone in China or Chile and the cost of wages, a large component of the cost of production, must be passed on to the producer and ultimately the consumer.

Then there is the problem of quality assurance: making sure the fruit is picked in optimum condition and carefully handled from then on. Like so much of the work for which robots are well suited (the word robot comes from the Czech word for drudgery), picking is highly repetitive, even tedious, work. Humans are notoriously intolerant of boredom; sometimes there are lapses of attention.

Previous attempts at mechanisation have proved difficult in the orchard, Chapman says, but he’s “interested” in how Massey’s kiwipicker pans out.

“Anything is useful that will improve the dynamic and the quality of work done, anything that maintains the quality of fruit is most welcome, anything that makes utilisation of labour better.”

Why is a kiwifruit picker so late in arriving on the scene? Why, when there are tens of thousands of robots welding car components, are so few used in agriculture and horticulture?

Flemmer believes it has to do with one problem in particular – realising effective artificial vision.
Robots may have become smarter, faster, stronger, smaller, cheaper and more accurate, he says, “but you don’t hire blind workers. Robots really need to see what they are doing.”

And artificial vision is complicated and demanding.

This is Massey’s competitive advantage. Flemmer and his wife Claire understand artificial vision to a highly unusual degree, the result, he says, of a 20-year pedigree in building commercial industrial robots.
 

The kiwifruit picker employs a panoply of cameras: two are mounted looking forward and enable the picker to make its way around the orchard; two are mounted looking toward the rear “because sometimes it has to go find itself a kiwifruit bin, figure out its orientation and then drive in to pick it up with its forks”, and there are “a number of cameras that look up at the canopy to see what is going on – and of course there has to be hand-eye coordination with the picking robots”.

A particular technical problem is dealing with the extreme fish-eye effect of the very short focal length wide-angle lens the picker uses when identifying and picking fruit.

The short focal length allows the lens to take in a usefully wide span – around 120 degrees – but an uncorrected image looks like a reflection from a fun-house mirror. To locate objects such as fruit in three dimensional space the picker must reconcile the information from two of these lenses and hence two of these highly distorted images in real time.

Solving this problem has been Claire Flemmer’s particular interest. Flemmer says he and his wife have a balance of strengths: “She is more theoretical; I am more mechanical.”

The kiwifruit picker has been carefully designed to avoid any risk to the people or animals that enter its operating space.

It can detect the movement of people or animals using infrared sensors and the picker will instantly stop if the soft bumper system running round the edge of the picker encounters an unexpected obstacle.

Then there is the conventional stop mechanism: the red button. The picker will have a number of strategically placed red buttons. You can even stop the picker by telling it to – the picker’s control module allows for voice control.

Flemmer and Atkinson are now eyeing other automation projects. Work has already begun on automating the apple industry – which will, in fact, install their packing machines before the kiwifruit industry.

Flemmer anticipates having eight packing cells working this packing season and a specialised apple picker ready for the season beyond.

Apples are New Zealand’s third most profitable horticultural export, coming after kiwifruit and wine.
Atkinson says strawberries would be “easy”.

“They are just sitting there so all you’d need is the machine to see a flash of red…”

For more on this story go to http://www.massey.ac.nz/massey/about-us/news/article.cfm?mnarticle=leader-of-the-pick-06-05-2008