Research

Robots can save us if they can see us: Heckman receives CAREER award

Emily Adams — Tue, 15 Oct 2024 20:07:19 +0000

Robots can save us if they can see us: Heckman receives CAREER award Emily Adams Tue, 10/15/2024 - 14:07

Grace Wilson

Autonomous robots could save human lives more easily if they could “see” and react better in adverse environmental conditions. By pursuing the possibilities of using millimeter wave radar for robotic perception, Christoffer Heckman is making this fundamental shift possible.

An associate professor of computer science at CU Boulder, Heckman will receive $600,000 over the next five years through the National Science Foundation's CAREER award for this research.

Currently, most robots use sensors based on the visible spectrum of light, like cameras or lasers. In environments with smoke, fog or dust, however, visible light bounces off these particles.

Robots, like humans, can't plan their movements accurately if they don't know where they are or what is around them.

"Humans operating in a visually degraded environment are in trouble. We cannot solve that problem, but incorporating millimeter wave radar could enable our robots to do things that even humans can't do," Heckman said.

This is because millimeter waves pass through smoke, fog and dust.

A new path

Traditionally, Heckman explained, radar has been viewed with skepticism for these kinds of tasks. The sensors have been too large and energy-intensive for agile robots. The long wavelength of radar creates complex, confused signals.

With the advent of new, smaller system-on-a-chip radar sensors, the traditional energy and size limitations have been removed. This leaves the complexity of radar waveform signals.

"This is a fascinating problem," Heckman explained. "People really understand how radar works, down to equations that have existed for almost a century, but radar can be difficult to precisely interpret in cluttered environments. It bounces around within an enclosed area, and can pass right through small objects."

Heckman's solution is to fuse the knowledge we have about electromagnetic waves with supervised machine learning.

Datasets from high-fidelity optical sensors are paired with low-fidelity radar signals of the same scene. Machine learning then cleans the radar signal to match the high-fidelity scene. This training then can be used to build clear radar reconstructions of environments where optical sensors are obscured.

This powerful synthesis of physics and computer science stands to dramatically improve the capability of radar as a perception sensor.

Beyond sensing

Heckman has further plans as well. He wants to use this advance to support quick and accurate actions and replanning for autonomous systems.

Robotic thinking has traditionally followed the saying "sense, plan, act." A robot understands a scene, plans its route according to its inputs, and acts on that plan. Segmenting these activities, however, can lead to slow movement and inability to react to changes.

Heckman seeks to use radar in conjunction with optical and lidar sensors to improve navigation strategies as a robot is navigating a space, allowing it to respond more quickly to changes.

Robots that can plan for themselves better and can see into obscured spaces have a valuable role in search-and-rescue, firefighting and space missions.

Heckman's MARBLE team has used robots to explore dark caves through the DARPA Subterranean Challenge and as a firefighting assistant finding active embers. As the research advances made possible by this CAREER Award take shape, where will robots be able to see next?

Humans operating in a visually degraded environment are in trouble. We cannot solve that problem, but incorporating millimeter wave radar could enable our robots to do things that even humans can't do." - Chris Heckman

Radar breakthrough in robotic sensing to help systems see and act in smoke, darkness recognized by $600,000 National Science Foundation award.

Off

Traditional

A SPOT robot with a light enters a dark mine tunnel.

White

Video - ChatGPT: Fear, Hype, or Hope? Education and research practices and ethics in the generative AI era

Anonymous — Thu, 20 Apr 2023 23:26:18 +0000

Video - ChatGPT: Fear, Hype, or Hope? Education and research practices and ethics in the generative AI era Anonymous (not verified) Thu, 04/20/2023 - 17:26

Grace Wilson

Three leading experts discuss how the disruptive and powerful elements of ChatGPT and other generative AI stand to transform our world. Jim Martin clarifies what a large language model like ChatGPT actually is, Diane Sieber urges the creation of norms around the usage of these tools while Tom Yeh focuses on its potential impacts on education.

[video:https://youtu.be/6nkKFmFOoOE]

Panelists:

Jim Martin (Department of Computer Science and Institute of Cognitive Science), an expert on natural language processing and large language models
Diane Sieber (Herbst Program for Engineering, Ethics & Society), a pioneer in education bridging technology, humanities and arts
Tom Yeh (computer science), a leading researcher in human-computer interaction who has studied the use of generative AI in introductory programming and K-12 settings.

Moderator:

Bobby Schnabel, external chair of computer science, founding director of the ATLAS Institute and former CEO of the Association for Computing Machinery.

Through this panel discussion attended by over 300 people from the university and general public, hear from leading experts on the technical areas underlying ChatGPT and other generative AI, the uses of generative AI in university and K-12 education, and the ethical and societal issues associated with generative AI tools.

Off

Traditional

White

AI has social consequences, but who pays the price? Tech companies’ problem with ‘ethical debt’

Anonymous — Thu, 20 Apr 2023 23:05:20 +0000

AI has social consequences, but who pays the price? Tech companies’ problem with ‘ethical debt’ Anonymous (not verified) Thu, 04/20/2023 - 17:05

Casey Fiesler, Associate Professor of Information Science with a courtesy appointment in Computer Science, writes for The Conversation about how we can tackle possible negative consequences and societal harms from AI development. Links to external article.

Off

Traditional

White

Computer science PhD student leads research into secrets of honeycomb formation

Anonymous — Thu, 20 Apr 2023 22:37:56 +0000

Computer science PhD student leads research into secrets of honeycomb formation Anonymous (not verified) Thu, 04/20/2023 - 16:37

Josh Rhoten

Researchers in the Department of Computer Science and BioFrontiers Institute are studying honeycomb formation in bees with the hope of one day recreating the same intricate and impressive hexagonal structures for other uses.

Off

Traditional

White

Talking with the fireflies: Orit Peleg receives CAREER Award and Alfred P. Sloan Fellowship

Anonymous — Fri, 17 Feb 2023 17:36:27 +0000

Talking with the fireflies: Orit Peleg receives CAREER Award and Alfred P. Sloan Fellowship Anonymous (not verified) Fri, 02/17/2023 - 10:36

Grace Wilson

If we as humans can understand how fireflies in swarms synchronize, we can understand their communication more deeply. Their on-off flickers of light are more similar to our binary computational logic than the tremulous variations of a frog croak or pitches of bugling elk calls.

Fireflies' elegant, distributed communication systems could eventually help us with our own telecommunications through new ideas about compressing information and distributed networks.

Assistant Professor of computer science Orit Peleg has just received $900,000 over the next five years to learn how fireflies in a swarm synchronize their lighting displays. The funding was provided by a National Science Foundation CAREER award, a highly prestigious early-career grant for junior faculty members.

Peleg, a member of the Biofrontiers Institute and Department of Computer Science, seeks to create testable theories about animal communication with her lab by merging tools from physics, biology, math and computer science.

Female fireflies judge the blinking displays of males to determine their mate. When there are swarms of small, blinking insects that can stretch over miles in the dark, any way to cut down on the visual clutter is important, and this is where synchronizing their flash patterns comes together.

"There are some really interesting questions about how really similar signals on the level of individual fireflies can result in a different collective signal," Peleg said.

Some groups of fireflies begin to flash in a burst together and stop the burst together, while others flash in a matching pattern but offset in time from one another so that the overall swarm always has some members flashing.

Currently our telecommunication networks and other manmade technology require synchronization to a central clock, which means that the network is really sensitive to failures of that clock.

"But that's not how biology does it. Biology achieves synchronization in a distributed way, which is more robust to failures of individual nodes," Peleg said

Changing the Working Model

We currently describe fireflies' brilliant displays of synchronizing bioluminescence through mathematical models that don't fully account for the possible agency of fireflies to change their lighting patterns in response to other fireflies.

The models are also tied to limited experimental data, giving Peleg and her lab rich ground for research and experimentation.

In the field, the lab sets up dark tents in the middle of the firefly swarms. They bring fireflies inside and use LED lights to mimic their signal patterns. They've already seen that, by changing the LED flashes, they can change the responses of living fireflies.

"They kind of treat it as if it were another firefly. It's really fun to watch how the firefly responds and communicates back to the artificial light," Peleg said.

They have also had success in training fireflies to create light patterns they have never been observed making in the wild in response to the LED light's patterns.

Outside of the carefully controlled environment in the tent, the lab will also use low-cost video equipment throughout the swarms to create robust 3-D video that can be analyzed and turned into 3D models that match the observed behavior of the fireflies.

By creating a model from field-data, Peleg can create more testable and verifiable theories around the ways that fireflies manage their synchronization.

The field data can be used to create simulations of firefly lighting patterns where the actions of real fireflies placed in the system impact the response of the program's lights, which involves advanced image processing and hardware.

In addition, Peleg is excited to use the data created from these field recordings to offer a class called "Physics, Artificial Intelligence, and Generative Art of Agent-Based Models," where undergraduate and graduate students will be able to craft their own visualizations of the data captured by the cameras in the field. By bringing experiential learning to the forefront, Peleg hopes to support their joy of learning.

Assistant Professor of computer science Orit Peleg has just received $900,000 over the next five years to learn how fireflies in a swarm synchronize their lighting displays. She's using LEDs, VR and big tents in the wilderness to signal to the fireflies... and they're signaling back.

Off

Traditional

White

Keeping the unseen safe: Improving digital privacy for blind people

Anonymous — Fri, 19 Nov 2021 18:19:51 +0000

Keeping the unseen safe: Improving digital privacy for blind people Anonymous (not verified) Fri, 11/19/2021 - 11:19

Grace Wilson

Associate Professor Danna Gurari

Blind people, like sighted people, post on Instagram, swipe on Tinder, and text photos of their children to a group chat. They also use photos to learn about their visual surroundings.

Blind users often share images with identification software such as Microsoft's Seeing AI, Be My Eyes and TapTapSee to learn about their visual surroundings. There's a high demand too. Seeing AI, for instance, has been used over 20 million times.

When blind people share photos, however, there is an added risk that they could unknowingly capture information considered private, such as a pregnancy test or a return address.

To Assistant Professor Danna Gurari, this shouldn't have to be a concern.

Gurari, the founding director of the Image and Video Computing group in the Department of Computer Science, is part of a cross-institutional team that has been awarded over $1 million through a Safe and Trustworthy Cyberspace (SaTC) grant from the National Science Foundation to study the issue.

Currently, blind people must either trust friends or family members to vet their images for private information before sharing publicly, which can have social repercussions of its own. Or they can accept the risk to their privacy when they post.

The goal of the team's four-year interdisciplinary project is to create a novel system that can alert users when private information is present in an image and, if the blind person wants to, obscure it.

Working with human-centered computing expert Leah Findlater from the University of Washington and privacy expert Yang Wang from the University of Illinois at Urbana-Champaign, Gurari's group is leading the automatic analysis of images for the project. Their goal is to turn the desires of users and theories of private information into actionable knowledge.

This comes with a number of challenges, both technical and philosophical.

Because AI makes mistakes, you have to be careful how certain you make an analysis sound.

"We really want to endow the appropriate level of trust but also give decision-making power," Gurari said.

The Image and Video Computing group is creating ways to share what private information might be present in an image and let the user decide to use the image as-is, discard it, or obscure the private information and then share it.

The other problem to solve for Gurari's group is how to determine what the most prominent object in an image is and obscure everything else.

Because blind people often share photos for object identification, this feature could reduce the amount of private information introduced during this straight-forward task.

Illustration of envisioned user interaction pipeline for empowering users to safeguard private content in their pictures and videos. (a) For the general use case, the tool will notify the user about what private content is detected and then provide a choice to either discard the media, share it as-is, or share an edited version where private content (teel mask overlaid on image) is obfuscated. (b) For the scenario where a user wants assistance to learn about an object, the tool will share an edited version with all content outside of the foreground object (teel mask overlaid on image) obfuscated.

Gurari's team will be focusing on creating algorithms robust enough to counteract image blur and other properties common for images taken by blind photographers. The team must also craft algorithms that don’t need to be trained on specific objects to see them as important.

This object identification riddle is one that has haunted much of computer vision's history, termed the "long-tail problem" for the graph it produces.

Computers usually have very low levels of object detection accuracy until they have been trained on thousands of images, but here they must understand the significance of an object in only a few frames.

And, as with other assistive technologies, the benefits of these algorithms could go far beyond their original purpose. From product photography, which is built on isolating prominent objects, to alerting sighted users of private information they didn't notice, the project has great potential benefit for building a safe and trustworthy cyberspace for all.

Blind people, like sighted people, publicly share photos, but have limited options for checking if they've captured something they consider private. Assistant Professor Danna Gurari, as a co-lead for an over-$1 million NSF grant, is working to change this.

Off

Traditional

White