Member Login

ARTICLE: Navigating Augmented Reality: User Interface and UX in Cobotics

Written by Postdoctoral Research Fellow, Dr Alan Burden from the Designing Socio-technical Robotic Systems research program in the Centre.  

The rise of collaborative robots (cobots) is a game-changer for various industries. These robots are designed to work alongside humans, enhancing productivity and efficiency. However, the real challenge lies in making this human-robot interaction as seamless as possible. Augmented Reality (AR) is a technology that has the potential to revolutionise this space by overlaying digital information onto our physical environment.

The Shift in Cobot Interfaces

Traditionally, human-cobot interactions have been facilitated through screen-based interfaces or specialised hardware. While these methods are functional, they often require a strenuous learning curve and can be less intuitive. Augmented Reality offers a paradigm shift. By overlaying digital guides, data, or even real-time analytics onto a workspace, AR can make the interaction with cobots more straightforward and efficient. This reduces the time needed for task completion and makes the process more intuitive, reducing the need for extensive training. As we move forward, we are poised to transition from digital 2D interfaces to more immersive 3D interfaces, further enhancing the user experience.

UX Design Principles in AR

User Experience (UX) design is pivotal in making AR-based cobot interaction effective. The objective is to create interfaces that are not just visually appealing but also user-friendly and functional. This involves a deep understanding of the user’s needs, their tasks with the cobot, and the environmental factors at play. For example, an AR interface for a cobot in a medical lab would need to consider sterility and precision. At the same time, one in a manufacturing setting might focus on speed and durability. The design process should be iterative, continually involving users in testing to refine the interface.

User Journey Mapping

Mapping the user’s journey is an invaluable tool in this design process. It involves creating a visual representation of all the interaction points between the user and the cobot facilitated by the AR interface. This helps identify potential issues, bottlenecks, or areas for improvement in the interaction process. For instance, if users find it challenging to access certain information quickly, the interface can be tweaked to make that data more readily available. The ultimate aim is to make the AR interface a tool that enhances, rather than hinders, productivity and user satisfaction.

Safety and Ethics

While AR offers many advantages, it raises important ethical and safety considerations. Data privacy is a significant concern, especially when sensitive or proprietary information is displayed in a shared workspace. The AR interface must also be designed to minimise distractions that could lead to safety hazards. For example, overly flashy or intrusive graphics could divert the user’s attention from critical tasks, leading to accidents. Therefore, ethical guidelines and safety protocols must be integrated into the design process.

What’s Next?

As AR technology continues to evolve, the possibilities for its application in cobotics are virtually limitless. Future developments could include gesture-based controls, adaptive learning algorithms that tailor the interface to individual user preferences, and even real-time collaboration features that allow multiple users to interact with a single cobot. These advancements will make the interaction more seamless and open new avenues for automation and efficiency in various industries.

As we stand on the brink of a new era in human-robot collaboration, enabled by the transformative power of Augmented Reality, we must pause to consider some critical questions.

Will AR interfaces become the new standard in cobotics, making traditional interfaces obsolete?

If we integrate more advanced features like gesture controls and adaptive learning algorithms, are we also prepared to address the complex ethical and safety considerations that come with them?

These questions serve as a reminder that while technology offers immense potential for improvement and innovation, it also demands a level of responsibility and foresight. As we navigate this exciting frontier, let’s ensure our approach is technologically advanced, ethically sound, and user-centric.

 

Congratulations Dr Stine Johansen on her ECR Grant

Congratulations to our postdoc, Dr Stine Johansen on her ECR Grant from the QUT Centre for Robotics!

QUT Postdoctoral Research Fellow, Dr Stine Johansen has been awarded a $20,000 ECR grant from the QUT Centre for Robotics for her Robotics blended sonification project. This project intends to design and evaluate a robotic blended sonification system. The system will enable operators to collaborate with a collaborative robot arm without relying on visual feedback but instead using sound as a feedback modality.

Project 2: Robotic Blended Sonification
Chief Investigator: Dr. Stine Johansen
Abstract: This project introduces a revolutionary approach to human-robot collaboration through sound. Instead of relying on visual feedback, the project focuses on capturing and modifying the sounds robots naturally produce. By processing these sounds based on contextual information, the system aims to reduce the cognitive load of operators and enhance their ability to oversee multiple robots effectively. This innovative approach taps into tacit knowledge and aims to create a prototype for further research and development.

Read more HERE

Meet our E.P.I.C. Researcher, Nadimul Haque

Nadimul Haque is a PhD researcher based at the University of Technology Sydney and his project is part of the Biomimic Cobots Program at the Australian Cobotics Centre.
His research interests lie in the applications of deep reinforcement learning in robotics.

We interviewed Nadimul recently to find out more about why he does what he does.

  • Tell us a bit about yourself and your research with the Centre?

I graduated from the University of Dhaka from the Department of Robotics and Mechatronics Engineering in 2020, just before the pandemic hit. While doing my bachelor’s, I was parallelly working as a research assistant on a funded project on agricultural automation, which I continued till June 2022.

The research I am undertaking under ACC is on the effective manipulation of collaborative robots with learning frameworks. I want to create a generalised cobotic control system for complex manipulation tasks. I envision making a learning framework that will allow the cobot to adapt quickly to any scenario and, hopefully, any task. Current systems are generally optimised to work on a particular task under very specific conditions. My research will look to unlock the potential of generalised learning frameworks that will facilitate fast adaptation to the changing environment. This will eventually be tested and applied to industrial scenarios where a cobot can be counted on to perform effectively with humans in a dynamic environment.

  • Why did you decide to be a part of the Australian Cobotics Centre?

I have always wanted to conduct research in robotics that will have a real-world impact. ACC provides the perfect opportunity for me to do that. There is a persistent fear amongst the general masses that robots will replace the human workforce. The center’s ideal of using collaborative robots in industrial spaces could alleviate this issue. I believe that the only sustainable move forward towards an automated industry would be cobots and humans working together. The center will play a pivotal role in this aspect.

The match in ideals is supplemented by the center’s collaborations with established industry partners. The fact that the robotic systems developed will actively be adapted to the industry makes it the ideal playground for a robotics enthusiast.

  • What project are you most proud of throughout your career and why?

It was a simple project where I, with another group member, created a line-following robot equipped with reinforcement learning. The idea was that rather than hard coding the robot to follow a line, the robot would learn how to traverse any path, with the signals from simple IR sensors. Of course, it was not anywhere near as efficient as an optimised LFR, it was exhilarating to watch it learn and slowly but surely, get better. Although most of the projects I have taken on so far have yielded more tangible results, I am most proud of this one as it got me hooked on robotics and reinforcement learning.

  • What do you hope the long-term impact of your work will be?

I hope that my research with the center will pave the way towards generalised robotic controls that can be redeployed into any situation, preferably for multiple different tasks.

  • Aside from your research, what topic could you give an hour-long presentation on with little to no preparation?

Football (The one you play with your feet)

ARTICLE: How to ensure quality assurance when integrating a cobot

Written by Postdoctoral Research Fellow, Dr. Anushani Bibile and Research Program Co-Lead, Dr. Michelle Dunn, both from SUT

A collaborative robot (or cobot) is designed to work side-by-side with people and can support applications from welding, pick and place, injection moulding, CNC, packaging, palletising, assembly, machine tending and materials handling. The integration of cobots enables the delegation of many human-based skill activities, with cobots able to undertake a range of repetitious tasks, whilst offering high flexibility and increased productivity.

A collaborative robot arm is compact, occupying a smaller floorspace than a conventional robot and can offer great flexibility for ‘low-volume, high-mix’ production, or high specialisation environments.

It is easier to re-program and re-tool a cobot to undertake a range of actions, providing greater agility as well as reductions in cost of operation. As cobots are also designed to work safely side-by-side with human operators, reduced safety measures are required when compared with a conventional robot.

If you are thinking of integrating a cobot into your manufacturing process it is important to look at the quality assurance of your system. When implementing a conventional robot, you would ensure the quality assurance was satisfied during initial setup, but when you use a cobot, which can be reconfigured for different processes, you need to consider the quality assurance every time you make a change. Changes to the code by non-experts, will have to be checked and verified very closely and safety always needs to be considered. Therefore, quality assurance is critical for human-cobot systems in automated processes as it ensures that the products or services produced meet the required specifications and are safe for use.

Why are continuous quality assurance checks important for human-cobot systems?

  • Productivity: Quality assurance measures can help optimise the performance of a human-cobot system, improve productivity and reducing waste. This can include monitoring and controlling the system to ensure that it is working efficiently and identifying areas where improvements can be made.
  • Safety: Safety is a critical concern when it comes to human-cobot systems. A cobot does not need to be caged, therefore a malfunctioning or improperly programmed cobot can cause serious injury or damage to humans or equipment. Quality assurance measures help ensure that the cobot system is designed and programmed correctly, and that it is safe for use.
  • Compliance: Quality assurance measures can help ensure that a human-cobot system meets regulatory and industry standards. This can include performing audits and inspections to ensure that the system is operating within the required parameters and that all safety regulations are being followed.

If proper quality assurance measures are not in place, there are potential risks associated with human-cobot systems. Some of these risks include:

  • Malfunctioning: A cobot that is not properly programmed or maintained can malfunction, causing damage or injury to humans or equipment.
  • Inaccuracy: A poorly calibrated or inaccurate cobot can produce defective products or services, leading to waste, customer dissatisfaction, and potentially legal liabilities.
  • Cybersecurity: Human-cobot systems are susceptible to cyber threats, which can lead to system failures, data breaches, and other security issues. Quality assurance measures can help ensure that the system is secure and that appropriate cybersecurity protocols are in place.

Design of safety mechanisms must meet the corresponding industrial standards which are exemplified in the figure below. First, a cobot must meet the relevant safety requirements, laws and directives for general machinery such as the European Machinery Directive (2006/42/EC). Basic safety rules and regulations (known as Type A standards) must also be met. Specific applications of a cobot system must meet type B standards. Finally, the cobots as products must meet type C standards.

Safety Assurance standards and regulations for human and machine collaboration [1]

Finally, it is important to regularly review and update quality assurance protocols to keep pace with evolving technologies and changing workplace conditions. By remaining vigilant and proactive in preserving the quality assurance of cobots in automated processes, organisations can reap the benefits of cobot automation while minimising risks and maximising productivity.

[1]Bi, Z. M., et al. (2021). “Safety assurance mechanisms of collaborative robotic systems in  manufacturing.” Robotics and Computer-Integrated Manufacturing 67.

[2] Vicentini, F. (2021). “Collaborative Robotics: A Survey.” Journal of Mechanical Design 143(4).

[3] Cobot – Wikipedia

 

New PhD Researcher, Justin Botha

Let’s introduce Justin Botha, our newest team member. Justin is a PhD researcher at QUT (Queensland University of Technology), actively involved in the Human-Robot-Interaction program and the Interactive (and Collaborative) Robot Programming using Language project.

Programming robots to carry out desired tasks is difficult and time-consuming. This PhD project focuses on collaborative and instructional dialogue agents to help human operators program robot tasks. We are pleased to welcome Justin to the team and anticipate his valuable contributions. Join us in welcoming Justin aboard!

Welcome Justin!

PhD Project Introductions

Collaboration and sharing of information are vital for the success of our Centre. To support this, we ask our PhD Researchers to give a brief introduction to their projects within the initial 6 months.

During our latest seminar, QUT (Queensland University of Technology)‘s Phuong Anh TranJasper Vermeulen and Yuan Liu provided an outline of their projects’ objectives, methodology, and anticipated outcomes.

As they continue their research, we’ll keep you posted on their progress. Meanwhile, you can learn more about their research updates HERE.

ARTICLE: Cobots in manufacturing: Good for skill shortages and much more.

Written by Research Program Co-Lead Professor Greg Hearn and PhD Candidate Nisar Ahmed Channa both from the Human Robot Workforce research program in the Centre.  

In this era of rapidly evolving technology landscape, almost every industry sector needs to keep pace with technological advancements to prosper and remain competitive. However, many companies struggle to develop or even adopt innovations in technologies, processes, or business models. COVID-19 is one recent example where manufacturing companies found it extremely challenging to generate an innovative response to address labor shortages caused by lockdowns and movement restrictions across many countries. As a result, many production units of large as well as small to medium manufacturing companies shut down for significant time periods. This negatively affected global supply chains in many other sectors because manufacturing industries provide input in the form of usable goods and services to many other industries. Soon after the global economic crises, the manufacturing companies were facing issues like increasing production costs caused by unavailability of raw material and increased labour costs. Covid-19 pandemic further fuelled these issues due to disruptions in global supply chains and restricted movement of human workers. Even after the pandemic, various countries are still facing issues like increased labour costs, and shortages of skilled labour. Resultantly, companies are now investing huge financial resources to future proof their manufacturing potential and reduce input and increase output.

One of the innovative solutions to these labor and skills shortages on which academics and industry experts are working is the adoption of collaborative robots (Cobots) in manufacturing. A Cobot is a special kind of robot, with context awareness, which can safely share a workspace with other Cobots or with human operators. Recent research suggests that Cobots can be used as alternatives to skilled human workers and thus can supplement the shortage of workers across the industries. For instance, to cope with labour shortages caused by pandemic and to meet increased demand, manufacturing companies in North America spent around 2 billion USD in 2021 to acquire 40,000 robots[i],[ii],[iii]. Similarly, rising labour costs, and an aging workforce, has led to an increase in the demand for Cobots in the automobile sectors of Europe and the Asia–Pacific region iii.

Some labour economists believe that the introduction of technologies like artificial intelligence (AI) and robots increases production and efficiency in manufacturing through the displacement of jobs traditionally being performed by human workers. However, under certain conditions, these technologies can create new jobs and upskill other jobs across the ecosystem of the related suppliers and services providers.

In line with the priorities for Australian manufacturing formulated by the Australian Advanced Manufacturing Growth Centre[iv], we argue Cobots could be “creatively productive” for Australian manufacturing not only because of their potential to reduce production cost efficiencies but also to enhance value differentiation, and potentially open up new revenue segments including through export[v]. Efficiencies can be achieved through optimisation of human-robot workflow design; accelerating workforce acceptance of robotic driven process efficiencies; reducing human errors in automation documentation; and by reducing downtime through enhanced work safety.  Value differentiation can be achieved by integration of Cobots in product design for rapid prototyping; by developing autonomous systems of quality assurance and better data analytics as value adding services; by improving capabilities for just-in-time and mass customisation products in existing markets; and by upskilling the manufacturing workforce for innovation leadership which in itself is a value differentiator. The fact that Cobots are designed to work alongside and close to people to perform their jobs and responsibilities can help companies to integrate and digitalise their business operations without compromising on lacking human aspects of the job. In many respects, Cobots are the hardware equivalent of augmented intelligence, rather than replacing people with autonomous equivalents. Cobots can supplement and improve human skills with super-strength, accuracy, and data capabilities, allowing them to perform more and add more value to the production process and to final product itself. It aids in creating strategic business value and improves efficiency, resulting in better, quicker delivery of products to customers in market.

[i] North American companies send in the robots, even as productivity slumps | Reuters

[ii] Robots marched on in 2021, with record orders by North American firms | Reuters

[iii] Rise of The Cobots in Automotive Manufacturing | GEP

[iv] https://www.amgc.org.au/our-purpose/about-advanced-manufacturing/

[v] Microsoft Word – Hearn et al ACRA Final Submission.docx (linklings.net)