LEARNING FROM PEOPLE: INTEGRATING ROBOTS INTO UNSTRUCTURED ENVIRONMENTS
LEARNING FROM PEOPLE: INTEGRATING ROBOTS INTO UNSTRUCTURED ENVIRONMENTS
Title: Learning from People, Integrating Robots Into Unstructured Environments Author: Vivian Chu, CTO and Co-Founder Date: January 7, 2019
Until recently, most robots that have been created are pre-programmed industrial robots that execute repetitive activities in structured, controlled environments where few to no humans also work like manufacturing factories. The goal of industrial robots is to automate the work in place of humans having to do it.
But the future requires more than just behind-the-scenes brute (and mute) automated task execution; it requires robots that don’t replace human jobs, but enhance them, by being robot teammates that do the parts of work that can be automated to give people more time for the vital, human-centric parts. The future requires robots that can operate in unstructured, variable and cluttered environments where robots have to continuously learn and be able to react to unexpected situations.
While cobots (collaborative robots that work with people) present significantly more complex technical challenges that industrial robots, the opportunities for cobots to enhance our human workplaces are limitless.
For cobots to fulfill their mission as trusted, reliable members of human/robot teams, they must be able to operate in fast-paced and chaotic human environments. This requires the cobot to have a technical framework that allows it to adapt to variability using a variety of sensory inputs. In particular, we believe that multi-sensory input that integrates visual, auditory and haptic (touch) data is crucial for cobots to operate in human environments. Cobots that can modify their actions based on the environment have the skill of what we call “adaptive object manipulation”.
The second quality cobots must have is the ability to assist with the highly precise tasks that humans have to do such as picking up items of tiny sizes on varying shelf heights — or what is known as “dexterous manipulation.” Our nurse helper robot, Moxi, for example, has a high degree-of-freedom arm and sturdy gripper hand that allows it to pick up objects of various sizes.
And most importantly, we believe the key technical component cobots must have is the ability (and motivation) to learn from human teachers — or what we call “human-guided learning” or “ human-guided exploration.” In human environments, things are always changing and evolving. Just as humans are always learning how to navigate our changing environments and complete new tasks, cobots also need to have the framework that allows them to learn from their environment and from humans.
Examples of situations that cobots need advanced, learning-oriented technical framework are (using real situations Moxi has faced):
Moxi hears (using auditory input) unexpected ambulatory situation coming up behind, so Moxi turns around and sees (using visual data) a patient on a gurney being quickly wheeled towards it on the left side of the hallway; Moxi needs to learn that it needs to quickly move to the right side of hallway out of the way
Moxi needs to pick up a gauze pad to deliver to room 429, but realizes that it has never learned to pick up a gauze pad. Moxi then requests to be taught the crucial information to identify and manipulate a gauze pad. The teacher shows Moxi (using human-guided exploration) how to grasp the gauze pad. Meanwhile, Moxi records key information such as the weight (haptic), color (visual), and sound (audio) of the object as it picks up the gauze pad. Now whenever Moxi needs to deliver a gauze pad to any room, it knows what to expect. Furthermore, if Moxi were to ever see a white pad that looks like a gauze pad but is not quite the right weight, it can use this information to either adapt (picked up too many) or request information (the gauze pad has been replaced with wound dressings).
While we lose structure in human environments (which makes tasks very difficult for robots), we gain the help of people. When human teachers, the richest source of information, are added to the equation, tasks that previously were impossible for robots become possible. Understanding how human teachers can guide cobots and how cobots can learn from human teachers is the key ingredient to successfully integrating human-supporting cobots in the real world.
Humans have irreplaceable qualities that robots can never match; emotions, passions, intuition, drive, creativity, analytical skills, and the ability to build meaningful human relationships.
At Diligent, we get to see humans’ amazing ability to connect with others and problem solve every day. We see nurses comfort patients who are confused, frustrated and in pain. We see nurses simultaneously connect with their patients regardless of how shy or loud while also making sure everything is as expected according to their medical chart. We see nurses go the extra mile to check on even the smallest issues that they suspect might be a problem because they have what robots do not — human warmth and intuition.
The ability to deal with the unexpected, problem solve, and connect with others is what make us human; things technology cannot and should not replace. However, socially intelligent cobots like Moxi, can help humans with our increasing loads of busy work (nurses spend about 30% of their time on non-patient facing logistical work) so that people can spend more time on people (nurses with their patients).
Imagine if a cobot could help you with your busy work. And imagine if the cobot learned from you directly, the human it’s helping, to make sure it’s really helping you in the most efficient way.
What would you do with that extra time? Who could you have a positive, deeper impact on?
We’re proud to be part of a movement helping people have more time to utilize their important human-centric skills; starting with helping nurses.
Vivian Chu (CTO and Co-Founder): Vivian Chu is a leading innovator in robotics, applying AI and social intelligence in the development of robots, with honors as an Google Anita Borg Memorial Scholar, Stanford University’s EECS Rising Stars, and Robohub’s “25 women in robotics you need to know about” in 2016. During her university studies, Vivian explored various engineering disciplines and discovered her perfect fit with robotics when she had the opportunity to program a Roomba to climb up a ramp while at UC Berkeley. She loved seeing the physical manifestation of code and how robots can move in the real world. After completing her Ph.D. at Georgia Tech, she help found Diligent Robotics to develop socially intelligent robots to work autonomously yet collaboratively in a team. Vivian has over 10 years of experience with AI, machine learning, and natural language processing and is an expert in applying machine learning algorithms with multimodal data to concrete tasks across various robotic platforms such as the PR2, Meka Robot, and Kinova Jaco2. She has worked at several industry labs including IBM, Google X, and Honda Research Institute.
NURSES: THE HEART OF PATIENT CARE
NURSES: THE HEART OF PATIENT CARE
Title: Nurses, The Heart of Patient Care Author: Agata Rozga, Head of Product Date: November 16, 2018
There are 3 million individuals who have chosen and worked to become registered nurses (RN’s) in the US as of 2017. While their reasons for choosing this profession vary, there is one foundational trait that most nurses share: an innate desire to care for other people.
Nurses face significant challenges, including overcoming the barrier to entry (minimum of 6 years of professional education, training and certification), adapting to the job’s increasing complexities and often receiving less recognition than other hospital professionals such as doctors.
In other words, most nurses don’t choose their profession for the money, glory or ease of the job. Most chose to be nurses because their passion and care for other people outweighs the challenges they face entering and staying in the nursing role.
Nurses spend more time interacting with patients than any other healthcare professional. Their role requires them to juggle multiple responsibilities including ongoing patient bedside support and monitoring (physical, emotional, mental), administering of medications, educating patients and communicating with their families, discharge planning, follow-up care and bringing all of that patient data together to drive care. Amidst these clinical tasks, nurses are also gathering the supplies and equipment needed for patient care and may need to perform other logistical tasks such as delivering lab samples or light housekeeping.
“The two most important parts of a nurse’s role are their consistency in presence with patients and their ability to not only gather in-person patient data and emotion, but to qualitatively examine ‘what does all of this data mean,’” says Joyce Batcheller, Nurse Executive Advisor of Leadership at The Center for the Advancement of Healthcare Professionals. “With the growing complexities of the nursing role, nurses are spending much more time doing logistical tasks away from patients and have less time for the thoughtful, human analysis of the patient’s bigger picture.”
The healthcare industry is changing at a record-breaking pace in attempt to deliver better patient care while controlling costs and improving operational inefficiencies. These changes include institutional overhauls and introducing new technologies like electronic medical records (EMRs) and patient online portals. The US population is aging, insurance policies are changing and the typical patient on a hospital medical/surgical unit is more acute than ever before. Thus, nurses have to navigate operational changes while also providing high quality care to patients who increasingly demand more of their time.
EMRs are digital versions of patient paper charts in healthcare provider’s offices and hospitals. EMRs are intended to help clinical staff provide more efficient and accurate (based on patient prior data) care to patients. While EMRs have drastically improved the way our healthcare industry manages patient data, they also have introduced significant time and workflow challenges. In a 2018 study by Applied Nursing Research examining nursing burnout and EMRs, 50% of advanced practice registered nurses (APRNs) indicated that EMRs added to their daily frustration and 97% said they had insufficient amount of time for EMR documentation.
In addition to new operational workflow changes, our country is facing an aging population, longer life expectancies, health insurance changes and overcrowding in hospitals resulting in nurses having to care for more patients with more acute needs over shorter periods of time. In contrast to chronic, long-term care, acute care refers to short term care for unexpected, emergency situations like severe injury, urgent conditions and severe illness. Acute care patient settings are much faster-paced, stressful, and have higher volumes of patients. Increasingly, chronic care is being delivered outside of hospitals, in specialized treatment centers. One result of this transition to less chronic care taking place within hospitals is that hospital nurses are caring for more acute patients, while still coordinating with chronic care facilities as an integrated partner.
Nurses are the ones who catch the small patient changes both physically and emotionally that are vital in understanding the patient’s needs and overall health. With higher intensity acute care workflows and increased volumes of patients, nurses may have less time for the level of patient bedside surveillance and interaction than they used to.
Nurses are indispensable members of the hospital clinical care team, coordinating a patient’s care, providing emotional support, educating patients about their condition and facilitating their transition to home or other care arrangements. Yet, studies find that they may spend up to 30% of their precious and valuable time on non-patient-facing logistical tasks such as locating supplies and equipment needed for patient care or delivering specimens to the lab. This is problematic when you consider that studies also find that that the amount of time nurses spend in direct patient care is a key determinant of patient satisfaction, better patient outcomes, and shorter lengths of stay.
At Diligent Robotics, we realized we had an important opportunity to use our technology background to create a product that would return time for nurses at the bedside and enable them to remain focused on patient care. Moxi is our friendly social intelligence hospital robot that supports nurses by carrying out logistical tasks such gathering and delivering equipment and supplies needed for patient care so that nurses do not have to leave the patient bedside. We created Moxi to empower nurses by doing the parts of their jobs that they don’t want — or should — have to do.
“Moxi gives nurses a third arm they can rely on,” says Aliya Aaron, Principal at AMR Healthcare Consulting. “Nursing burnout comes from increased amounts of multi-tasking that nurses need to balance with a lack of support. On average nurses are responsible for 5–7 patients, each patient with about 20 tasks, 30% of which are non-patient facing. If Moxi delivers supplies in advance of the nurse needing them, it could save the nurse 10 minutes on each task, which they could then use for patient facing care. Finding outside resources that care about nursing workflow and are working to create new efficiencies is the key to helping address nursing burnout.”
Human interaction is crucial in an industry where healing is the focus. Moxi will never try or be able to replace the skillful care that nurses provide; their ability to emotionally connect to patients, to critically analyze all aspects of their progress, and to coordinate their care. But we hope Moxi will be a valued member of the care team that positively supports nurses on the back-end.
Moxi is just one of many efforts aimed at supporting the 3 million individuals who’ve become nurses and dedicate their lives to helping others.
Join us in our journey to thank and help those who help us!
Agata Rozga (Head of Product): Agata Rozga is a psychologist and the chief researcher responsible for overseeing observational studies to determine how new technologies can assist in healthcare environments. Having previously worked with the founders at Georgia Tech, she accepted the opportunity to continue the research on how robots could support members of clinical care teams. During this process, her appreciation deepened for the challenges the frontline healthcare staff face every day and the opportunities to address their needs by introducing robots which can do many of the tasks that divert them from patient care. She earned her Ph.D. from UCLA and has spent over 15 years leading observational studies of social behavior. At Georgia Tech, she joined computer scientists and engineers to explore how technology could be used to develop new tools to measure these behaviors. She was also a postdoctoral fellow in the Center for Behavioral Neuroscience at Georgia State University.
MEET MOXI: OUR SOCIALLY INTELLIGENT ROBOT SUPPORTING HEALTHCARE TEAMS
MEET MOXI: OUR SOCIALLY INTELLIGENT ROBOT SUPPORTING HEALTHCARE TEAMS
Title: Meet Moxi, Our Socially Intelligent Robot Supporting Healthcare Teams
Author: Andrea Thomaz, CEO and Co-Founder
Date: September 15, 2018
Burnout in the healthcare industry feels inevitable. Currently, nurses face significant physical, mental and emotional challenges in increasingly overburdened work environments. The average turnover rate for Registered Nurses (RNs) in hospitals hovers around 20 percent within their first year in the field.
Clinical staff spend a reported average of 30 percent of their time on non-care activities like gathering medical supplies or restocking supply rooms. Staff clock up to eight to ten miles a day running back and forth to supply rooms, further diverting them from direct patient care. The top five things that nurses wish they had more time for include: emotional support, education, care coordination and discharge planning, care planning, and timeliness of care.
With the ongoing complexities of attending to the changing conditions of patients combined with the demanding back-end tasks expected in healthcare shift work, there’s less and less time for meaningful face to face contact with patients.
But with the right help, this can change.
Enter AI. To address these types of cognitive overloads many industries face, my focus as a technologist has been to utilize technology that learns, adapts and has limitless potential. We need to utilize the technology that has the power to not just create something new, but address something existing. And utilize the one that has the power to work with everyday people to address an existing problem or inefficiency.
We decided to start with healthcare and today are announcing Moxi, our socially intelligent robot that supports clinical staff as an important and trusted member of the team.
Moxi’s goal as a robot is not to replace the jobs of people, but quite the opposite: to support people in their roles. Moxi supports clinical staff by augmenting logistical tasks that limit valuable patient care time. By executing non-patient facing, logistical tasks that clinical teams are responsible for, Moxi creates a more efficient and thoughtful environment allowing for better patient care.
As a friendly, sensitive and intuitive robot, Moxi not only alleviates clinical staff of routine tasks but does so in a non-threatening and supportive way that encourages positive relationships between humans and robots, further enhancing clinical staff’s ability to and interest in leveraging AI in the healthcare industry. Created with a face to visually communicate social cues and able to show its intention before moving to the next task, Moxi is built to foster trust between patients and staff alike, setting the stage for future innovation and partnerships with developing technology. Moxi’s specific tasks and responsibilities at each hospital will be tailored to fit each hospital’s needs.
The first trials with Moxi begin this week at several hospitals including Texas Health Dallas, The University of Texas Medical Branch (UTMB Health) and Houston Methodist Hospital.
We founded Diligent Robotics in 2016 with the vision of changing the way people feel about and utilize technology.
We believe that the unity and harmony between people and AI has the potential to change the way we take care of one another. We’re specifically focusing on the development of socially intelligent robots that function in care-oriented environments such as hospitals. With robots as trusted and reliable members of a team, we hope to inspire people to use their ingenuity, passion and skills to address bigger and more pressing challenges.
Care is a team effort, we hope you join us.
Andrea Thomaz (CEO and Co-Founder): Dr. Andrea Thomaz is a renowned social robotics expert and currently the CEO of Diligent Robotics. Her accolades include being recognized by the National Academy of Science as a Kavli Fellow, the US President’s Council of Advisors on Science and Tech, Popular Science’s Brilliant 10 list with her robot Curi appearing on the cover and MIT Technology Review’s Next Gen of 35 Innovators U 35. Andrea’s passion for social robotics began during her work at MIT Media Lab where she realized the opportunity to utilize AI to develop machines that address existing human needs, rather than developing machines before understanding how humans could benefit from them. She co-founded Diligent Robotics to bring her vision of machines helping humans mainstream with socially intelligent robots that collaborate with humans by doing their routine tasks so they have more time for human experiences. Andrea earned her PhD from MIT, BS in Electrical and Computer Engineering from UT Austin and was a Robotics Professor at Georgia Tech and UT Austin.
Great write up in IEEE Spectrum by Evan Akerman, giving a nice technical perspective on our work. "Diligent Robotics, a startup founded by Andrea Thomaz and Vivian Chu, is undaunted by the challenge of autonomous mobile manipulation in semi-structured environments." [full article]
Today we are announcing our $2.1 million seed investment, led by Silicon-Valley-based True Ventures. We are honored to be joining the True portfolio, and to have Rohit Sharma on our board to help navigate the early days. Also investing is Pathbreaker Ventures, with Ryan Gembala asking all the right questions about service robotics. Boom Capital, with Celestine Schnugg who we could see from our first meeting, is going to bring a unique perspective. And Next Coast Ventures is a local investor, who not only joined our seed round but are letting us work out of their new office space in downtown Austin as we ramp up. We are incredibly excited to have assembled an amazing set of people backing the vision of Diligent Robotics, and we are ready to hit the ground running to bring that vision to reality. We are looking forward to an incredible 2018, as we hire our founding team and deploy robots with the first set of hospital customers in our early adopters program.
We have been awarded a $500k Small Business Innovation Research grant from the National Science Foundation. This is a Phase II grant for research on “Mobile Manipulation Hospital Service Robots” and is follow-up to the Phase I grant of $225k awarded in 2016. The 6-month Phase I grant allowed us to build up a small team for a sprint to prototype, and resulted in 1-week technical feasibility tests at three different Austin-area hospitals (Poli is seen here with the nursing team at Seton Medical Center Austin). We look forward to building on these early results during Phase II of the project, building our next prototypes and focusing on long term pilot testing with hospitals to establish the technical and commercial feasibility of hospital service robots for acute care units. Our goal is to develop service robots that hospital staff view as a competent and useful member of the care team.
The Dell Seton Medical Center at The University of Texas, is Seton Healthcare’s newest teaching hospital in Austin. We were invited to demo our hospital service robot in their opening day celebration. Poli performed fetch and deliver tasks all afternoon in the ICU, showing visitors how Seton is taking the lead in technology innovation to improve patient care. “Time is precious and we want to spend our time with our patients, not on non-clinical tasks that a robot can do.The community trusts us to provide person-centered care, and this is just one more way we are doing that. We’re constantly working to improve the patient’s experience,” -- Kristi Henderson, VP of Innovation and Virtual Care, Seton Healthcare