Feed aggregator
Wildfire smoke causes only ‘modest reduction’ in solar power — study
Calif. legislative analysts blame climate programs, wildfires for high power bills
Biden admin approves California’s zero-emission ferry rule
What this week’s winter wallop means for US farmers
Trump looks to shatter icebreaker pact with Canada, Finland
Teaching AI to communicate sounds like humans do
Whether you’re describing the sound of your faulty car engine or meowing like your neighbor’s cat, imitating sounds with your voice can be a helpful way to relay a concept when words don’t do the trick.
Vocal imitation is the sonic equivalent of doodling a quick picture to communicate something you saw — except that instead of using a pencil to illustrate an image, you use your vocal tract to express a sound. This might seem difficult, but it’s something we all do intuitively: To experience it for yourself, try using your voice to mirror the sound of an ambulance siren, a crow, or a bell being struck.
Inspired by the cognitive science of how we communicate, MIT Computer Science and Artificial Intelligence Laboratory (CSAIL) researchers have developed an AI system that can produce human-like vocal imitations with no training, and without ever having "heard" a human vocal impression before.
To achieve this, the researchers engineered their system to produce and interpret sounds much like we do. They started by building a model of the human vocal tract that simulates how vibrations from the voice box are shaped by the throat, tongue, and lips. Then, they used a cognitively-inspired AI algorithm to control this vocal tract model and make it produce imitations, taking into consideration the context-specific ways that humans choose to communicate sound.
The model can effectively take many sounds from the world and generate a human-like imitation of them — including noises like leaves rustling, a snake’s hiss, and an approaching ambulance siren. Their model can also be run in reverse to guess real-world sounds from human vocal imitations, similar to how some computer vision systems can retrieve high-quality images based on sketches. For instance, the model can correctly distinguish the sound of a human imitating a cat’s “meow” versus its “hiss.”
In the future, this model could potentially lead to more intuitive “imitation-based” interfaces for sound designers, more human-like AI characters in virtual reality, and even methods to help students learn new languages.
The co-lead authors — MIT CSAIL PhD students Kartik Chandra SM ’23 and Karima Ma, and undergraduate researcher Matthew Caren — note that computer graphics researchers have long recognized that realism is rarely the ultimate goal of visual expression. For example, an abstract painting or a child’s crayon doodle can be just as expressive as a photograph.
“Over the past few decades, advances in sketching algorithms have led to new tools for artists, advances in AI and computer vision, and even a deeper understanding of human cognition,” notes Chandra. “In the same way that a sketch is an abstract, non-photorealistic representation of an image, our method captures the abstract, non-phono-realistic ways humans express the sounds they hear. This teaches us about the process of auditory abstraction.”
The art of imitation, in three parts
The team developed three increasingly nuanced versions of the model to compare to human vocal imitations. First, they created a baseline model that simply aimed to generate imitations that were as similar to real-world sounds as possible — but this model didn’t match human behavior very well.
The researchers then designed a second “communicative” model. According to Caren, this model considers what’s distinctive about a sound to a listener. For instance, you’d likely imitate the sound of a motorboat by mimicking the rumble of its engine, since that’s its most distinctive auditory feature, even if it’s not the loudest aspect of the sound (compared to, say, the water splashing). This second model created imitations that were better than the baseline, but the team wanted to improve it even more.
To take their method a step further, the researchers added a final layer of reasoning to the model. “Vocal imitations can sound different based on the amount of effort you put into them. It costs time and energy to produce sounds that are perfectly accurate,” says Chandra. The researchers’ full model accounts for this by trying to avoid utterances that are very rapid, loud, or high- or low-pitched, which people are less likely to use in a conversation. The result: more human-like imitations that closely match many of the decisions that humans make when imitating the same sounds.
After building this model, the team conducted a behavioral experiment to see whether the AI- or human-generated vocal imitations were perceived as better by human judges. Notably, participants in the experiment favored the AI model 25 percent of the time in general, and as much as 75 percent for an imitation of a motorboat and 50 percent for an imitation of a gunshot.
Toward more expressive sound technology
Passionate about technology for music and art, Caren envisions that this model could help artists better communicate sounds to computational systems and assist filmmakers and other content creators with generating AI sounds that are more nuanced to a specific context. It could also enable a musician to rapidly search a sound database by imitating a noise that is difficult to describe in, say, a text prompt.
In the meantime, Caren, Chandra, and Ma are looking at the implications of their model in other domains, including the development of language, how infants learn to talk, and even imitation behaviors in birds like parrots and songbirds.
The team still has work to do with the current iteration of their model: It struggles with some consonants, like “z,” which led to inaccurate impressions of some sounds, like bees buzzing. They also can’t yet replicate how humans imitate speech, music, or sounds that are imitated differently across different languages, like a heartbeat.
Stanford University linguistics professor Robert Hawkins says that language is full of onomatopoeia and words that mimic but don’t fully replicate the things they describe, like the “meow” sound that very inexactly approximates the sound that cats make. “The processes that get us from the sound of a real cat to a word like ‘meow’ reveal a lot about the intricate interplay between physiology, social reasoning, and communication in the evolution of language,” says Hawkins, who wasn’t involved in the CSAIL research. “This model presents an exciting step toward formalizing and testing theories of those processes, demonstrating that both physical constraints from the human vocal tract and social pressures from communication are needed to explain the distribution of vocal imitations.”
Caren, Chandra, and Ma wrote the paper with two other CSAIL affiliates: Jonathan Ragan-Kelley, MIT Department of Electrical Engineering and Computer Science associate professor, and Joshua Tenenbaum, MIT Brain and Cognitive Sciences professor and Center for Brains, Minds, and Machines member. Their work was supported, in part, by the Hertz Foundation and the National Science Foundation. It was presented at SIGGRAPH Asia in early December.
Monitoring space traffic
If there’s a through line in Sydney Dolan’s pursuits, it’s a fervent belief in being a good steward — both in space and on Earth.
As a doctoral student in the MIT Department of Aeronautics and Astronautics (AeroAstro), Dolan is developing a model that aims to mitigate satellite collisions. They see space as a public good, a resource for everyone. “There’s a real concern that you could be potentially desecrating a whole orbit if enough collisions were to happen,” they say. “We have to be very thoughtful about trying maintain people’s access, to be able to use space for all the different applications that it has today.”
Here on the Blue Planet, Dolan is passionate about building community and ensuring that students in the department have what they need to succeed. To that end, they have been deeply invested in mentoring other students; leading and participating in affinity groups for women and the LGBTQ+ community; and creating communications resources to help students navigate grad school.
Launching into new territories
Dolan’s interest in aerospace began as a high school student in Centerville, Virginia. A close friend asked them to go to a model rocket club meeting because she didn’t want to go alone. “I ended up going with her and really liking it, and it ended up becoming more of my thing than her kind of thing!” they say with a laugh. Building rockets and launching them in rural Virginia gave Dolan formative, hands-on experience in aerospace engineering and convinced them to pursue the field in college.
They attended Purdue University, lured by the beautiful aerospace building and the school’s stature as a leading producer of astronauts. While they’re grateful for the education they received at Purdue, the dearth of other women in the department was glaring.
That gender imbalance motivated Dolan to launch Purdue Women in Aerospace, to facilitate connections and work on changing the department’s culture. The group worked to make study spaces more welcoming to women and planned the inaugural Amelia Earhart Summit to celebrate women’s contributions to the field. Several hundred students, alumni, and others gathered for a full day of inspiring speakers, academic and industry panels, and networking opportunities.
During their junior year, Dolan was accepted into the Matthew Isakowitz Fellowship Program, which places students with a commercial space company and pairs them with a career mentor. They interned at Nanoracks over the summer, developing a small cubesat payload that went on the International Space Station. Through the internship they met an MIT AeroAstro PhD alumna, Natalya Bailey ’14. Since Dolan was leaning toward going to graduate school, Bailey provided valuable advice about where to consider applying and what goes into an application package — as well as a plug for MIT.
Although they applied to other schools, MIT stood out. “At the time, I really wasn’t sure if I wanted to be more in systems engineering or if I wanted to specialize more in guidance, navigation, controls, and autonomy. And I really like that the program at MIT has strength in both of those areas,” Dolan explains, adding that few schools have both specialties. That way, they would always have the option to switch from one to the other if their interests changed.
Being a good space actor
That option would come in handy. For their master’s degree, they conducted two research projects in systems engineering. In their first year, they joined the Engineering Systems Laboratory, comparing lunar and Martian mission architectures to identify which technologies could be successfully deployed both on the moon and Mars to, as Dolan says, “get our bang for the buck.” Next, they worked on the Media Lab’s TESSERAE project, which aims to create tiles that can autonomously self-assemble to form science labs, zero-gravity habitats, and other applications in space. Dolan worked on the controls for the tiles and the feasibility of using computer vision for them.
Ultimately, Dolan decided to switch their focus to autonomy for their PhD, with a focus on satellite traffic applications. They joined the DINaMo Research Group, working with Hamsa Balakrishnan, associate dean of the School of Engineering and the William Leonhard (1940) Professor of Aeronautics and Astronautics.
Managing space traffic has become increasingly complex. As the cost to get to space decreases and new launch providers like SpaceX have spun up, the number of satellites has grown over the last few decades — as well as the risk of collisions. Traveling at approximately 17,000 miles per hour, satellites can cause catastrophic damage and create debris that, in turn, poses an additional hazard. The European Space Agency has estimated that there are roughly 11,500 satellites in orbit (2,500 of which are not active) and over 35,000 pieces of debris larger than 10 centimeters. Last February, there was a near-collision — missing by only 33 feet — between a NASA satellite and a non-operational Russian spy satellite.
Despite these risks, there’s no centralized governing body monitoring satellite maneuvers, and many operators are reluctant to share their satellite’s exact location, although they will provide limited information, Dolan says. Their doctoral thesis aims to address these issues through a model that enables satellites to independently make decisions on maneuvers to avoid collisions, using information they glean from nearby satellites. Dolan’s approach is interdisciplinary, using reinforcement learning, game theory, and optimal control to abstract a graph representation of the space environment.
Dolan sees the model as a potential tool that could provide decentralized oversight and inform policy: “I’m largely just all in favor of being a good space actor, thinking of space as a protected resource, just like the national parks. And here’s a mathematical tool we can use to really validate that this sort of information would be helpful.”
Finding a natural fit
Now wrapping up their fifth year, Dolan has been deeply involved in the MIT AeroAstro community since arriving in 2019. They have served as a peer mediator in the dREFS program (Department Resources for Easing Friction and Stress); mentored other women students; and served as co-president of the Graduate Women in Aerospace Engineering group. As a communication fellow in the AeroAstro Communications Lab, Dolan has created and offered workshops, coaching, and other resources to help students with journal articles, fellowship applications, posters, resumes, and other forms of science communications. “I just believe so firmly that all people should have the same resources to succeed in grad school,” Dolan says. “MIT does a really great job providing a lot of resources, but sometimes it can be daunting to figure out what they are and who to ask.”
In 2020, they helped found an LGBTQ+ affinity group called QuASAR (Queer Advocacy Space in AeroAstro). Unlike most MIT clubs, QuASAR is open to everyone in the department — undergraduate and graduate students, faculty, and staff. Members gather several times a year for social events, and QuASAR has hosted academic and industry panels to better reflect the variety of identities in the aerospace field.
In their spare time Dolan loves ultrarunning — that is, running distances greater than a marathon. To date, they’ve run 50-kilometer and 50-mile races, and recently, a whopping 120 miles in a backyard ultramarathon (“basically, run ’til you drop,” Dolan says). It’s a great antidote to stress, and, curiously, they’ve noticed there are a lot of PhD students in ultrarunning. “I was talking with my advisor about it one time and she’s like, ‘Sydney, you’re crazy, why on Earth would you do anything like that?’ She said this respectfully! And I’m like, ‘Yeah, why would I ever want to do a task that has an ambiguous end date and that requires a lot of work and discipline?’” Dolan says, grinning.
Their hard work and discipline will pay off as they prepare to complete their MIT journey. After wrapping up their degree program, Dolan hopes to land a faculty position at a college or university. Being a professor feels like a natural fit, they say, combining their fascination with aerospace engineering with their passion for teaching and mentoring. As to where they will end up, Dolan waxes philosophical: “I’m throwing a lot of darts at the wall, and we’ll see … it’s with the universe now.”
Images that transform through heat
Researchers in MIT Professor Stefanie Mueller’s group have spent much of the last decade developing a variety of computing techniques aimed at reimagining how products and systems are designed. Much in the way that platforms like Instagram allow users to modify 2-D photographs with filters, Mueller imagines a world where we can do the same thing for a wide array of physical objects.
In a new open-access paper, her team at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) has demonstrated a novel printing technique along these lines — which they call “Thermochromorph” — that produces images that can change colors when heated up.
Led by first author and MIT electrical engineering and computer science doctoral student Ticha Melody Sethapakdi SM '22, the researchers say that they could imagine their method being applied in ways that are both artistic and functional, like a coffee-cup that warns if the liquid is too hot, or packaging for medicines or perishable foods that could indicate if the product has been stored at a safe temperature.
So-called “thermochromic” materials that visually change with temperature are not new — you can see examples with consumer beverages like Coke and Coors Light that reveal “ready to drink” labeling when refrigerated. But such instances in product marketing have traditionally been limited to a single color. By using inks with complementary characteristics — with one set that goes from clear to colored, and another from colored to clear — Sethapakdi says that she and her colleagues are “finally taking advantage of full-color process printing, which opens up a lot of possibilities for designing with thermochromic materials.”
The researchers worked with several visual artists to teach them to use Thermochromorph, and then solicited feedback and brainstorming about new narrative concepts and techniques unlocked by the tool, like color-changing postcards that could tell sequential stories in more compact, dynamic ways. One participant even plans to use Thermochromorph to make an educational science kit aimed at teaching students about sea creatures that change color.
The team developed their method to be applied specifically to “relief printing,” an early form of printmaking that involves carving a design into a block of material, applying ink or pigment to it, and then transferring the image onto paper or another surface.
Sethapakdi says that, compared to techniques like screen printing, relief printing is “more lightweight” and can be done with less setup and fewer materials, enabling a faster, lower-stakes iteration process. Artists that include the likes of Pablo Picasso and Salvador Dalí have used a range of related approaches in their work, such as woodcut and linocut printing.
“Our key contribution is applying these new materials to a traditional artistic process, and exploring how artists might be able to use it as part of their practice,” says Sethapakdi, lead author on a related paper that was recently presented at SIGGRAPH Asia in Tokyo.
The color-changing component also need not come from an active external heating or cooling source like, say, a fridge or a hot plate; using thermochromic inks with lower activation temperatures can allow for more subtle thermal changes brought about by human touch. Sethapakdi says she could even imagine applying this new process to create interactive surfaces or dynamic analog “interfaces” that visually change in response to touch.
Thermochromorph combines digital and analog processes in the form of, on the one hand, CMYK imaging and laser cutting, and, on the other, manual printmaking and thermochromic inks. Fabrication involves four core steps:
- Block preparation: Solid hardwood blocks are used for Thermochromorph. The blocks are laser cut and engraved with the desired design, and then rinsed with water to remove any leftover particles.
- Inking the block: First, a thin layer of ink is spread evenly onto a plate using a rubber brayer. Then, the ink is transferred from the brayer to the woodblock.
- Registration: A registration jig is used to position the woodblock to ensure the different ink layers are aligned correctly. The printing surface, such as paper, is then placed on top of the block and secured.
- Printing the images: A printing press is used to apply even pressure across the printing surface and transfer the ink from the block to the surface. The hot image is printed first, followed by the cold image. (If necessary, additional ink can be applied to specific areas of the block to touch up the print.)
The three prints the team used to demonstrate their technique were a set of frames from a Batman comic, a label depicting a fish and its underlying skeleton, and an image of a male subject both in profile and viewed from the front. (For the latter, as the temperature changes, the viewpoint gradually shifts, giving the effect of motion.)
It’s worth noting that Thermochromorph does have some potential limitations related to image resolution and print quality. Specifically, image resolution is constrained by the smallest dot size that the team’s laser cutter can engrave. Techniques like screen printing would offset this, but with the additional drawback of needing more time and materials. In terms of print quality, the pigments are not entirely invisible in their ‘clear’ states, which means that the clarity of the transitions depends on how thickly the ink layers were applied during printmaking. While this issue is intrinsic to the properties of the pigments, Sethapakdi says that for future iterations the team plans to explore different image-processing techniques to modify the overlay of halftone patterns for the hot and cold images, which may help to reduce these visual artifacts.
Sethapakdi and Mueller co-authored the new paper alongside Juliana Covarrubias ’24, MIT graduate student in media arts and sciences Paris Myers, University of California at Berkeley PhD student Tianyu Yu, and Adobe Research Scientist Mackenzie Leake.
Meta invites climate disinformation by tossing fact-checkers
Treasury finalizes long-awaited tax rules for boosting renewables
It will be a busy year for carbon markets. Here’s what to expect.
Oregon finalizes new wildfire maps after backlash
‘Hidden’ killers from floods include heart attacks, infections
California high-speed rail CEO says project can withstand 4 years of Trump
JPMorgan quits climate finance group, following Citi, Bank of America
Artificial intelligence energy demand is driving climate tech investing
Severe flood warning issued as the UK faces more weather disruptions
Insights into political outsiders
As the old saw has it, 90 percent of politics is just showing up. Which is fine for people who are already engaged in the political system and expect to influence it. What about everyone else? The U.S. has millions and millions of people who typically do not vote or participate in politics. Is there a way into political life for those who are normally disconnected from it?
This is a topic MIT political scientist Ariel White has been studying closely over the last decade. White conducts careful empirical research on typically overlooked subjects, such as the relationship between incarceration and political participation; the way people interact with government administrators; and how a variety of factors, from media coverage to income inequality, influence engagement with politics.
While the media heavily cover the views of frequent voters in certain areas, there is very little attention paid to citizens who do not vote regularly but could. To grasp U.S. politics, it might help us to better understand such people.
“I think there is a much broader story to be told here,” says White, an associate professor in MIT’s Department of Political Science.
Study by study, her research has been telling that story. Even short, misdemeanor-linked jail terms, White has found, reduce the likelihood that people will vote — and lower the propensity of family members to vote as well. When people are convicted of felonies, they often lose their right to vote, but they also vote at low rates when eligible. Other studies by White also suggest that an 8 percent minimum wage increase leads to an increase in turnout of about one-third of 1 percent, and that those receiving public benefits are far less likely to vote than those who do not.
These issues are often viewed in partisan terms, although the reality, White thinks, is considerably more complex. When evaluating infrequent or disconnected voters, we do not know enough to make assumptions about these matters.
“Getting people with past criminal convictions registered and voting, when they are eligible, is not a surefire partisan advantage for anybody,” White says. “There’s a lot of heterogeneity in this group, which is not what people assume. Legislators tend to treat this as a partisan issue, but at the mass public level you see less polarization, and more people are willing to support a path for others back into daily life.”
Experiences matter
White grew up near Rochester, New York, and majored in economics and government at Cornell University. She says that initially she never considered entering academia, and tried her hand at a few jobs after graduation. One of them, working as an Americorps-funded paralegal in a legal services office, had a lasting influence; she started thinking more about the nature of government-citizen interactions in these settings.
“It really stuck in my mind the way people’s experiences, one-on-one with a person who is representing government, when trying to get benefits, really shapes people’s views about how government is going to operate and see them, and what they can expect from the state,” White says. “People’s experiences with government matter for what they do politically.”
Before long, White was accepted into the doctoral program at Harvard University, where she earned an MA in 2012 and her PhD in 2016. White then joined the MIT faculty, also in 2016, and has remained at the Institute ever since.
White’s first published paper, in 2015, co-authored with Julie Faller and Noah Nathan, found that government officials tended to have different levels of responsiveness when providing voting information to people of apparently different ethnicities. It won an award from the American Political Science Association. (Nathan is now also a faculty member at MIT.)
Since then, White has published a string of papers examining how many factors interact with voting propensities. In one study focused in Pennsylvania, she found that public benefits recipients made up 20 percent of eligible voters in 2020 but just 12 percent of those who voted. When examining the criminal justice system, White has found that even short-term jail time leads to a turnout drop of several percentage points among the incarcerated. Family members of those serving even short jail sentences are less likely to vote in the near term too, although their participation rebounds over time.
“People don’t often think of incarceration as a thing they connect with politics,” White says. “Descriptively, with many people who have had the experience of incarceration or criminal convictions, or who are living in families or neighborhoods with a lot of it, we don’t see a lot of political action, and we see low levels of voting. Given how widespread incarceration is in the U.S., it seems like one of the most common and impactful things the government can do. But for a long time it was left to sociology to study.”
How to reach people?
Having determined that citizens are less likely to vote in many circumstances, White’s research is now evolving toward a related question: What are the most viable ways of changing that? To be sure, nothing is likely to create a tsunami of new voters. Even where people convicted of felonies can vote from prison, she found in still another study, they do so at single-digit rates. People who are used to not voting are not going to start voting at high rates, on aggregate.
Still, this fall, White led a new field experiment about getting unregistered voters to both register and vote. In this case, she and some colleagues created a study designed to see if friends of unregistered voters might be especially able to get their networks to join the voter rolls. The results are still under review. But for White, it is a new area where many kinds of experiments and studies seem possible.
“Political science in general and the world of actual practicing political campaigns knows an awful lot about how to get registered voters to turn out to vote,” White says. “There’s so much work on get-out-the-vote activities, mailers and calls and texts. We know way, way less about the 1-in-4 or so eligible voters who are simply not registered at all, and are in a very real sense invisible in the political landscape. Overwhelmingly, the people I’m curious about fall into that category.”
It is also a subject that she hopes will sustain the interest of her students. White’s classes tend to be filled by students with many different registered majors but an abiding interest in civic life. White wants them to come away with a more informed sense of their civic landscape, as well as new tools for conducting clean empirical studies. And, who knows? Like White herself, some of her students may end up making a career out of political engagement, even if they don’t know it yet.
“I really like working with MIT students,” White says. “I do hope my students gain some key understandings about what we know about political life, and how we can know about it, which I think are likely to be helpful to them in a variety of realms. My hope is they take a fundamental understanding of social science research, and some big questions, and some big concepts, out into the world.”