Author: Merryn Ballantyne

Postdoctoral Research Fellow, Dr Stine Johansen was at the IEEE Human Robot Interaction Conference this week where she presented her paper, Illustrating Robot Motions.

Stine created a video presentation of the paper which gives an overview of the survey and examples of findings relating to how robot movements are illustrated in the ACM/IEEE Conference on Human-Robot Interaction proceedings from 2016 to 2022.

Download the paper and Stine’s video here: Illustrating Robot Movements | Proceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction

We congratulate Stine, Jared and Markus on their paper acceptance especially given the conference’s average acceptance rate of 24% (last 5 years 24%).

At the 2022 annual Australian conference for the Computer-Human Interaction Special Interest Group, OzCHI, 20 researchers from leading Australian robotics research environments met up for the workshop “Empowering People in Human-Robot Collaboration”. This workshop was co-organised by members from the Australian Cobotics Centre and the Collaborative Intelligence Future Science Platform (CSIRO). In this short overview, we offer insights from the workshop that show challenges and opportunities for building this research in Australia.

An Australian Perspective on Human-Robot Collaboration

Discussions that considered “The Tyranny of Distance”, the classic account of how Australia’s geographical remoteness has been central to shaping the people’s values and available resources, identified the lack of robotics manufacturing facilities as one major challenge. While this limits the opportunity to conduct fundamental research that contributes to the hardware design of robots, this workshop discussed how this limitation enables Australian HRC researchers to focus more on application-specific and ‘in the wild’ research. Discussions around the cultural mindset that tends to lack trust in automation highlighted that this mindset can be used in answering critical research questions such as what work should and should not be automated. Workshop participants also called attention to one of the most fundamental problems that is not only relevant to Australian context or HRC research. Currently in research gatherings, we focus on sharing success stories. We need to start facilitating “failure tracks” to share the lessons learnt from failures if we want to truly expedite the advancement of HRC research.

Beyond the Australian perspective, the researchers offered several directions for future work in human-robot collaboration:

Human role in HRC: Participants discussed the terms we use for people in human-robot collaboration. Suggestions included collaborator, partner, and helper. Similarly, the role of the robot in relation to the person was discussed, e.g., robot as leader, follower, co-artist, etc. Examples of appropriate and creative ways of partnering included mimicking and training.

Improving awareness: An emerging research interest in enhancing robots’ and human collaborators’ awareness of each others’ states and actions was observed. Specifically, some participants were interested in studying how to model human internal states (e.g., cognitive workload) and individual differences using multimodal analytics, so robots can use related outcomes to adapt to human needs and augment humans. Another area of interest was to implement intelligent user interfaces to assist humans in predicting states and actions of robots and understanding reasons behind robots’ decisions.

Empathy for robots: While the focus of human-robot collaborative applications has mostly been on fulfilling human needs and achieving human goals, participants also highlighted that putting people in the ‘shoes’ of robots is also important. Topics discussed in this space included interface requirements of robots (considering robot-human as well as robot-robot interactions) and the need to prevent human abuse of robots.

Ethical considerations: Discussion covered the ethical considerations associated with the purpose, design and intended users of cobots, and the ways these may interact. Certain purposes, such as intimate engagements or persuasion, have ethical implications yet to be fully explored or resolved. Given the purpose of many collaborative robots is social, there are special considerations that need to be made when identifying and responding to the needs of vulnerable populations, such as children, elderly people, and people with disabilities. Design matters with broad relevance to ethical robotics include safety, appropriate trust, and transparency. Participants suggested special consideration should be given to the potential consequences of anthropomorphism of robots, including mistreatment of robots, attribution of moral agency, and uncanny valley backlash.

HRC design aims: There is a need for designers to consider how the design of the robot complements people. One example is the opportunity for robots to extend abilities of, or provide access for, people with disabilities. Related to this, participants noted the barriers of robots complementing people, including building and negotiating trust, making explainable robots, and designing collaboration in iterative steps.

HRC design approaches: Several discussions revolved around materials and prototyping approaches for design of collaborative robots. This was supported by questions around how to involve end users in prototyping processes and what kinds of wizard of oz methods could be appropriate. Another discussion around design approaches was centered on robot creativity and how to support human-robot interaction for creative purposes.

Evaluating HRC: A challenge for the research field is how to evaluate human-robot collaboration outcomes. Participants noted that this was mainly due to the dynamic nature of the collaboration. An example is trust which changes over the duration of the collaboration.

We thank all the participants for their valuable perspective, and CINTEL for a great workshop collaboration. Let’s continue to build this research community in Australia.

Technology permeates all aspects of modern society, from using social media to communicate with friends to relying on machine automation for mundane tasks. As an integral part of our everyday lives, technology drives how we live, work, and interact with each other – this includes the emergence of new technology woven into our social lives. These combinations of social elements interacting with technology are called a socio-technical system (STS). Despite being at the centre of our daily routines, not many people know about them or the effect they will have on our future technologies. This article presents a (very) brief introduction to the broad subject of socio-technical systems and how approaching our interactions with technology from a holistic viewpoint allows us to design better for our futures.  

Essentially, the aim of an STS is to bring people and technology together to achieve a common goal or purpose. Social media platforms are an example of a modern STS allowing people to communicate via technical software, algorithms, and data analysis. Other well-known examples include email, E-commerce systems (e.g. Amazon and eBay), modern healthcare databases, transportation systems such as logistics, public transport or ridesharing, and energy grids that many of us rely on for electricity. 

The first ideas of the STS were developed in the mid-20th Century to understand the growing complexity and interdependence between people, machines, organisations, and their environments in industries like mining, manufacturing, and agriculture. This socio-technical theory revolutionised modern society by increasing connectivity, transforming how work is performed, and disrupting traditional industries. They have changed consumer behaviour and increased dependence on technology, leading to new opportunities and challenges for individuals and organisations. 

Currently, the concept of STSs often draws upon design philosophies that recognise the social and technical parts but still regard the system holistically as symbiotic, i.e. each system element should benefit from its interaction with another element. Design approaches and problem-solving techniques, such as human-centred design (HCD), cover a broad context that includes a system’s human, social, technological, and environmental aspects. In this approach, the design process starts with understanding the human users’ needs, wants, and limitations and then incorporates these into the design of the technology. Ultimately HCD provides us with a toolkit that optimises STS interactions to achieve a better outcome and to produce an effective system. The goal is to create a system that supports and augments human capabilities while also meeting technical requirements, resulting in a solution that is both efficient and user-friendly. This approach is commonly used in fields such as ergonomics, user experience (UX), human-computer interaction (HCI), and collaborative robotics (aka cobotics) – where the interaction between humans and technology is critical to the success of the system. 

In robotics, STSs have significantly impacted robot development and deployment, especially in collaborative environments shared between people and robots. Cloud computing, Internet-of-Things (IoT) connectivity, and remote collaboration tools have enabled faster and more efficient robotics development by allowing experts to work together on projects regardless of their location. The vast amounts of data generated by these systems have advanced artificial intelligence and machine learning technologies, making robots more autonomous, interactive, and gradually capable of increasingly complex tasks. The ease of access to robotics technology via STS approaches has increased adoption, particularly in manufacturing, where robots traditionally automate repetitive tasks – but are now envisioned to assist with more complex labour tasks. Interaction between humans and robots has also been impacted by STSs, with new methods such as natural language processing, mixed-reality interfaces, and voice or gesture commands being developed to enable more seamless and intuitive human-robot interaction. 

However, here lies one central challenge – balancing technical components against social ones often proves difficult. Finding the correct ratio that can meet various goals, like promoting efficiency while maintaining human responsibility, adds complexity towards overall issues such as community involvement, a person’s health, and the environment. It is essential for organisations to carefully consider the ethical and safety implications of collaboration between humans and robots before implementing them. STSs effectively create solutions that work well and consider social aspects such as ethics or user experience. By combining technical elements with social ones – robots with humans – we balance what is technically possible and socially acceptable, creating something beneficial for everyone involved. The goal of this combination isn’t just efficiency but rather something that can be used safely while being more productive than ever before. That is why including human-centred factors is paramount to achieving correct integration and guarantees excellent outcomes for everyone. 

In summary, human-robot collaboration in socio-technical systems can bring several benefits, including increased efficiency, improved safety, enhanced productivity, better outcomes, and cost savings. However, it is important to consider this collaboration’s ethical and safety implications to ensure it is used responsibly. Future technological developments will bring challenges and opportunities for organisations involved in human-robot collaboration within socio-technical systems. It is up to these organisations to carefully consider the implications of their actions and strive towards creating effective and socially acceptable solutions. 

Australian Cobotics Centre researchers have three papers accepted for publication at the upcoming International Conference on Robotics and Automation (ICRA) 2023 in London. ICRA is the IEEE Robotics and Automation Society’s flagship conference and the premier international forum for robotics researchers to present and discuss their work.

Postdoctoral Research Fellow, Dr Fouad Sukkar gave is a brief summary of two of the papers appearing at the conference mid this year.

“Guided Learning from Demonstration for Robust Transferability”, by Fouad Sukkar, Victor Hernandez Moreno, Teresa Vidal-Calleja and Jochen Deuse, contributes to the idea of humans teaching robots how to carry out tasks in the real world. A common approach is to physically move the robot around and record its motions. However, this can be inconvenient if the robot is already busy doing work and even dangerous/ not possible in the case of large industrial arms. Ideally, we should be able to demonstrate the task more naturally using the original tool, such as a welder, without the physical robot present. This is where the proposed guided learning from demonstration (LFD) framework comes in handy. It visually displays regions in the workspace that the robot is capable of moving within to the demonstrator. That way the user can be confident that the robot will be able to execute the demonstrated task and avoid having to carry out repeated trial and error. To validate their method, a user study was carried out which involved users teaching a collaborative robot (cobot) how to carry out a welding task with obstacles in the environment. Results showed that with the visual guidance users were always able to achieve a successful execution of the task on the robot in comparison to only 44% of the time without it. While the framework is in a promising proof of concept state, the authors strongly believe this idea will be important for industry as they adopt more flexible methods for manufacturing. As for future work, they are currently working towards testing their framework in real world applications and exploring different guidance mediums such augmented reality and haptic feedback. A preprint for the paper can be found here: https://arxiv.org/abs/2302.03901.

“Optimal Workpiece Placement Based on Robot Reach, Manipulability and Joint Torques”, by Baris Balci, Jared Donovan and Peter Corke, looks at how to best place a workpiece for a robot to physically interact with. This is particularly important for cobots, such as the Universal Robot’s UR arms, which are designed to be safe around humans. Cobots are sensitive to external forces in order to be safe around people, however, this sensitivity complicates cobots’ physical interaction with objects for manufacturing purposes. The arm will deviate away from where you tell it to go due to the external forces that are required for the manufacturing processes. What the authors in the paper show is that depending on how you place a workpiece, it is possible to increase cobots’ physical interaction performance. They present a novel method for choosing this placement and demonstrate it works effectively in a simulated surface finishing application for a few different objects with interesting surface geometries.