Shaping tomorrow: Robots, digital and virtual reality

Andrew Harris is the director of Laing O’Rourke’s future engineering and innovation consultancy, the Engineering Excellence Group. Laing O’Rourke is Australia’s largest private engineering and construction business, with a local turnover of $3 billion per year.

Harris trained as a chemist and chemical engineer, and today is a Professor of Chemical and Biomolecular Engineering at the University of Sydney. He is also a non-executive director of ASX-listed clean tech company Hazer Group, and a board member of the Australian Research Council Centre for Robotic Vision. In 2016, he was recognised as one of Australia’s 50 most innovative engineers by peak body Engineer Australia.

How long have you been engaged in the disruptive innovation stream?

I’ve been looking at innovation broadly defined with Laing O’Rourke for the past eight years. That’s basically how long I’ve been looking at the construction sector and the upstream and downstream value chain, including materials and design. I’ve been trying to come up with ways of improving how things are done and making sure that we’re future-proof, which includes being disrupted from the outside.

What does Laing O’Rourke’s Engineering Excellence Group (EEG) offer the construction and extractive industries?

Ray O’Rourke, the founder and chief executive of Laing O’Rourke, set up the EEG. He was interested in disrupting his business from the inside – that was the whole point of the EEG – and it was set up with discipline-based experts, structural, heavy civil, chemical, electrical and chemical process – I was the chemical process bit. The group today actually runs the entire innovation centre for Laing O’Rourke.

{{quote-A:R-W:300-I:2-Q:"I don’t see robots as a replacement, I see them as augmenting the existing worker, making them more productive, safer, streamlined."-WHO:Andrew Harris, EEG director of Laing O’Rourke}}We designed the innovation program for the business, we do work in digital, broadly defined – whether that’s digital models, construction or manufacture, in big data analytics, in robotics, automation, computer vision, in IoT. The stuff that we build has sensors embedded into them that makes them intelligent and they start thinking about what they’re doing and how they work.

So, an example recently was a bridge in Sydney. A truck driver at two in the morning had his crane up and drove into this bridge. I’m interested in the bridge being able to tell us how damaged it is, whether it needs to close itself, whether it can stay open, and eventually whether the bridge can fix itself too – although that’s a fair way down the track!

In the mining sector, this is called predictive maintenance. The sensors in the big, fully automated 300-tonne dumpers that BHP and Rio Tinto run in the Pilbara can tell the operators in their NASA-like control room at Perth Airport when there’s going to be a failure – eg a truck tyre is going to blow within a couple of hours. There is no reason at all why a road or a piece of infrastructure that we build today shouldn’t have the same capability.

That’s an example of EEG’s work. We also have a broad-ranging interest in materials – there is a whole untapped paradigm in construction materials. In the construction industry, we use a very limited pallet – steel, concrete and its derivatives, aggregates, glass, timber and the odd polymer, that’s basically it. However, if you look at other fields – in medicine, health, in automotive/Formula 1/aerospace – the materials that are used there and the properties that you get from them are quite extraordinary.

So I see us as an industry taking lessons from those other industries, also from defence and even space travel, and we will slowly start to pick up some of the material innovations that come out of there and apply them in the construction and constructive spaces. And the EEG does a bit of work in there as well – graphene, carbon nanotubes, unusual polymer composites with self-repairing, treating properties.

What is the key message you are hoping to impart to the audience at the conference?

Lots of people talk about innovation and the future, and I have heard a lot of talks by experts that have just left me hugely irritated – just because they never show you how. So what I’m hoping to do is use the construction industry as a case study for what is possible and what the future might look like, and give some real concrete examples of what is happening and what’s likely to happen, so people can see that there is innovation going on and that disruption is coming, how it will affect them and why they need to prepare for it now, to take advantage of it. That is the number one message.

What are some of the disruptive technologies and innovations that you believe will become essential parts of industry in the coming years?

It starts with digital – a digital model first enables everything else. Some people call this a “digital twin”. There’s developments in off-site manufacturing, prefabrication, data analytics, robotics/ automation technologies – that automate some of the most dangerous tasks to focus on safety. And then there’s some of the materials innovations that are going on in parallel industries – aerospace, automotive, Formula 1, even the space industries – that could be applied in the construction space and give you better value.

There’s an Australian company called Eden Energy that has an admixture for concrete that has carbon nanotubes in it and that improves the properties of the set concrete in a whole bunch of unexpected ways. They’re doing really good work in the US and the tech they are developing is absolutely first class. So it’s about how you take advantage of what’s cutting edge in the universities and the research sector and applying those to deliver value in the supply chain.

How can these disruptors assist aggregate and construction materials producers to improve their business models and production processes?

{{image3-a:r-w:300}}We get lots of ideas from outside of the industry. I like doing that because it de-risks how you can implement them. If somebody has done it successfully somewhere else, it’s not such a big bite that you’re trying to chew off. One of the things that we’ve developed here in Australia was the world’s largest 3D printer. It was all designed and developed here but actually the first case use for it was in the UK and we built it there. This printer was 30m long, 12m wide and 6m high.

I think there’s a whole bunch of material implications that come out of printing stuff, as opposed to casting it or using a more traditional approach to assembly, and there’s a whole bunch of work that needs to be done there to take advantage of that and make sure it is compliant.

But the potential is extraordinary because you end up in a space where you can have some grand design that historically has been unbuildable that you now just press a button and out it spits. So we won’t have so many rectilinear-looking cities any longer, you’ll be able to have complexity and it won’t cost more – in fact it should be cheaper.

How would you see that working in the way a quarry manages its production plant? Is there the potential for a quarry to use a 3D printer as part of their plant and equipment and streamline processes?

In terms of somebody who operates a quarry, I’d see that as a value-added service actually. You’re providing an aggregate mix or similar material and you wing it in such a way that it is more printable than some other way. If you do that, you’re able to charge a premium because you’re differentiating your product set in non-traditional ways.

You’re always going to need the raw materials so I see demand going up because of these different delivery technologies but what it does is give you an opportunity to differentiate your product lines, so you can have more premium, higher value product lines that are used in specific end use cases.


How would disruptors improve environmental and health and safety processes?

One of the things we’ve developed is a Toolbox Spotter. It’s a small, low cost camera that we’ve made intelligent so that it can recognise what it’s looking at.

In the construction sector, there are three major causes of serious injury at work. We classify them as severe and fatal risks, so it’s heights (either falling from heights or something falling from heights) electricity/electrocution, or a man/machine or a machine/machine interface, that is, a human coming into contact with a machine, or, for example, cars crashing.

What we’ve set up the Toolbox Spotter to do is to focus on the man/machine interfaces. When a human comes into harm’s way, the camera can recognise that there is likely to be an issue because it tracks the speed at which the person is walking. Imagine someone walks behind an excavator and the operator can’t see them, and they’re reversing. The spotter says: “Hey, there’s a bloke behind you and they’re at risk!” And it might alert you in a different way – it could vibrate your watch or flash a light or send a sound into your cabin, something really, really simple.

We’ve done almost 50,000 hours of testing on these now, in different use cases, in rail environments, warehouses and factories. We have a good understanding of the psychology of how the spotter works, and we think we have something that is really valuable now. And the guys on the ground are saying, “This technology is really helpful, it will save lives.”

It’s a really clever piece of tech, it’s a camera with artificial intelligence buried in there, but it’s just the size of a can of Coke, and it sits on the back of an excavator, or on a ute or on a locomotive, or people can wear them. We’re finding real demand for that, we’re going to spin it out soon, and turn it into a product offering for the whole industry.

Has the Toolbox Spotter been tested in a quarrying or mine environment?

We’ve certainly tested it in borrow pits. Up on the north coast of New South Wales, Laing O’Rourke has been running tests of the spotter on behalf of the NSW Government on the Pacific Highway project. We’ve done lots of testing of it up there, certainly on our rail jobs. I don’t think we’ve ever done it in a quarry but I don’t see why it wouldn’t work in that environment.

The extractive industry has traditionally been reliant on traditional labour and manual processing tasks. Is there a risk that disruptors such as IoT and robotics could create too much reliance in workers on remote devices and automation at the expense of human intuition?

There’s been a lot of interesting questions about this more broadly, eg “the computer ate my job” or “I’ve been replaced by a robot”. There are lots of good pieces of research on this now. The benchmark study was completed for the US by two academics – the Frey/Osborne Study1, it’s very, very famous – and it’s been duplicated in Australia a couple of times.

{{image5-a:r-w:300}}What it looks at is the likelihood that your job will be replaced by a computer program or a robot, and it turns out that there are lots of manual handling and labour jobs that are more difficult to automate than you might expect, eg they require quite dexterous skill. An excavator operator is quite skilled and the good ones are really creative about it, and it’s really difficult to replace creativity with a computer or a robot, in fact it’s almost impossible for them today. So that’s my first point.

My second is that I don’t see robots as a replacement, I see them as augmenting the existing worker, making them more productive, safer, streamlined. Robotics is an assisting tool rather than a replacing tool. In the psychology of how this works, you just can’t replace a whole bunch of people with a robot. It’s just a terrible outcome, it’s terrible all round for communities, for jobs, for companies, it just doesn’t work, it’s not right.

There’s a really good example again elsewhere. In the aerospace industry, they have coupled a little industrial robot to each individual worker that takes away one or two dangerous tasks.
The robot follows them around like a little puppy dog, and the employees absolutely love it. Productivity and safety have gone through the roof, and quality has improved out of sight. I can see that happening in the construction sector as well.

My third and final point is that there was a recent analysis of British census data going back to the start of records in the 1700s. The researchers looked at the impact of technology on jobs and job losses, and the outcome of that study was that many, many more jobs are created through new technologies than are destroyed by them. So it’s very unlikely that a robot will eat your job, it’s much more likely they will make you more productive, safer, with better quality. The key skill is how good and creative you are. There is a degree of creativity in your job that is the key.

What could be the pros and cons of robotics and virtual reality in a construction materials setting?

We looked at exoskeletons – putting a worker inside a robotic frame that can lift a bit more weight. We looked at a commercial robotics lab in Japan that sells a commercial exoskeleton and it was genuinely awful – I’ll probably show a video of it to everybody at the conference just to highlight how rubbish it was!

Technology is not the thing here, it’s psychology. If you give a typical worker an “Iron Man” suit that lets them lift 300 kilos, somebody is going to do something stupid with it. A better alternative is an unpowered exoskeleton that supports a specific joint or helps you with a repetitive task to minimise repetitive strain injuries, or it holds a tool for you in the right place.

So there’s quite a subtlety around the psychology of robotics and robots, and also digital tools, virtual reality, for example. You’re never going to replace someone’s training with a fully virtual environment. It’s about making the training a bit more realistic, and to add some more authenticity to it, but it’s not about replacing the whole thing.

Is there a possibility that virtual reality/simulation technology may not equip workers adequately to operate machinery in the field, making such equipment patently unsafe?

The best example I can think of here is in medicine, when surgeons are being trained to do a really complex surgery. The worker in this case is a very highly skilled, very highly trained doctor but they’re unfamiliar with the surgery, and it’s incredibly complex. You can’t just have a crack at it first time around because you’re not going to get the best outcomes for the patient, so they undertake a virtual scenario first, and then someone who has done it before can coach the surgeons through the operation. It is found that when you do that, the outcomes are better. So it provides a missing link between the two extremes – the basic training and the skills required to do this advanced operation.

{{image6-a:r-w:300}}I see a parallel in the operation or maintenance of machinery – if you’re doing something complex, you have a base level of skills and if you haven’t done that specific task before, having a crack in a virtual environment is a really good way to learn. And again drawing parallels from other places, I know that NASA has simulators that use the same educational philosophy for pilots to manage fuel burn in re-entry for a shuttle. You can’t just have a crack because the first time you do it will be the first time anybody has done it. So they do this virtual training, and in that type of environment, it’s really beneficial.

I see the same thing for a machinery operator who might have done their basic training. Maybe they can drive a forklift but they’ve never been able to use a circular saw on an excavator to cut sandstone. And virtually where you position the blade, the angle you use, how much washwater you use, you can have a virtual go and you can get better at it virtually, and then you’ll be better equipped to do the real thing.

What are the pros and cons of 3D printing? If aggregate producers, for example, are being actively encouraged to 3D print spare parts for their machines (even to professional standards), could this create any safety risks?

There are a whole bunch of compliance issues here. I know that if you 3D print replacement parts, then they don’t comply with the relevant standards as they exist today, and there are risks. There is a government-funded research agency called the National Energy Resources Australia – NERA – that is sponsoring a project for the big oil and gas producers, which really struggle to get spare parts for high pressure gaslines in remote areas quickly.

NERA is sponsoring some excellent research to update the standards so that new technologies – such as 3D printers – could be used in this situation. If they can, then production isn’t down for two weeks while the spare valve is shipped in from Houston. You can turn it around in a matter of hours and print it on-site. There are challenges today around compliance but I think in the future it can be resolved and you’ll get better outcomes as a result of that.

Aided by these disruptors, what would you forecast the extractive and construction materials industries to look like in the short-term?

In the next five years, I expect a much more closely integrated digital supply chain, so materials providers will be able to ensure what they’re providing absolutely fits the requirements of the project, and it will be easier to do that in terms of specifications because it will all be digitally enabled.

I also see opportunities with new materials. Companies that develop construction materials – and the extractive industry that supports that – will move towards enhanced polymers and away from more traditional materials like brick, stone, concrete. These new polymers will give you the same performance properties, but they will be stronger, lighter, easier to install, and more sustainable.

I think that the way you go about installing these component-type materials now will be more automatic. They will use a robotic system, and the quality control around them will be digitised and fed back into this model at the end. I think we’re seeing all of those things beginning to happen now, I just think they will happen a lot faster into the future.

Professor Andrew Harris will deliver his presentation at CMIC 18 at the International Convention Centre, Sydney on 20 September, 2018.

Reference & further reading

1. Frey CB, Osborne MA. The future of employment: How susceptible are jobs to computerisation? Paper presented at the Machines and Employment workshop, courtesy of the Oxford University Engineering Sciences Department and the Oxford Martin Programme on the Impacts of Future Technology, on 17 September, 2013. Future_of_Employment.pdf

Leave a Reply