Ashutosh Trivedi

Holding tax software accountable

Anonymous — Mon, 11 Mar 2024 19:03:19 +0000

Holding tax software accountable Anonymous (not verified) Mon, 03/11/2024 - 13:03

Grace Wilson

Tax preparation software is critical for our society, but if the software is incorrect, people using it are responsible for accuracy-related penalties."
- Ashutosh Trivedi

When you file your taxes, you are responsible for any errors, even those created by the software you trust to compute your tax return. Since over 93 percent of individual tax returns were filed electronically in 2023, many in the United States are vulnerable to bugs in these systems.

This didn't sit right with Associate Professor of Computer Science Ashutosh Trivedi, and his work is getting positive attention from the IRS.

"Tax preparation software is critical for our society, but if the software is incorrect, people using it are responsible for accuracy-related penalties" he said.

This led Trivedi and fellow researchers to seek a way to verify the correctness of tax software, but how to do it?

No 'oracles'

"Tax software has what is commonly called an oracle problem. It means that you can think of the input, but you don't know what the ideal output is for that situation," he said.

Unlike a simple calculator where you can add two numbers and have a guaranteed output, the U.S. tax system is highly complex and expressed in potentially ambiguous natural language.

There are gaps and loopholes that leave more of the code up for interpretation than you might think.

The United States tax code, notes included, is also over 9,000 pages. As the code changes every year, software can unintentionally miss new requirements.

The researchers' solution? Legal precedent. In United States law, similar cases should have similar outcomes. This can be replicated in a software engineering approach called "metamorphic testing."

It's the little things

"In metamorphic testing, you present two inputs to the system that differ from each other so that the correct output of the program for these inputs must be in a certain predictable relationship," Trivedi said.

For instance, one may not know someone's exact tax return, but they can expect that another individual, who has the same taxable characteristics except that their spouse is blind, must receive a higher standard deduction.

Due to privacy concerns, there is no accessible dataset of taxpayer answers to forms, so it was necessary for Trivedi and his team to delve deep into the tax documentation and create fictional personas based on edge and corner cases. These variations could then be tested next to each other.

By creating personas with very similar taxable characteristics, you can test whether people who have similar inputs are getting similar results, or if something has gone wrong.

What they found was that there were real bugs in open-source software, especially when returns were close to zero dollars, or when a taxpayer was disabled. They then created simple, easy-to-follow flow charts explaining where the errors occurred. This test can be applied to any tax preparation software.

Law into logic

Trivedi sees the power of this approach as going beyond taxpayer software. What if interested parties could transform the natural language limitations of law into code upon which experiments could be conducted?

"For the first time you have software that, if proven correct, could stand in for the text that the government releases," he said. Trivedi said he believes code would be easier to interrogate for fairness and discrimination than natural language, potentially increasing the impartiality of law.

The project has intrigued the IRS, who have invited the team behind the research to present at the IRS-TPC Joint Research Conference on Tax Administration in June.

Errors in tax software could keep you from your best return, or worse, leave you liable for inaccuracies. Now, computer science researchers at CU Boulder are working to ensure the correctness of tax software.

Off

Traditional

White

Trivedi seeks to democratize artificial intelligence through CAREER award

Anonymous — Thu, 23 Jun 2022 20:05:18 +0000

Trivedi seeks to democratize artificial intelligence through CAREER award Anonymous (not verified) Thu, 06/23/2022 - 14:05

Grace Wilson

Ashutosh Trivedi, an assistant professor in the Department of Computer Science at CU Boulder, is working to democratize artificial intelligence by making machine learning more programmable, trustworthy and accessible to everyone.

He has just been presented with a prestigious CAREER award from the National Science Foundation to do so. The award supports the research and educational activities of early career faculty members who have the potential to become leaders in their field. Trivedi's provides $600,000 over the next five years. Six faculty members within the College of Engineering and Applied Science received CAREER Awards from the National Science Foundation in 2022.

Trivedi will use the award to improve abstraction as an alternative to traditional neural networks, which have huge energy and data requirements, and to build our ability to trust and understand machine learning.

Trivedi, who was born in India and raised during the country's intensive focus on computers in the 80s and 90s, knows how essential access is. Reminiscing about this pivotal moment in India's history, Trivedi said, "by having the power to talk to computers, we transformed not only our own lives, but those around us."

If people everywhere aren't given access to artificial intelligence now, he said, it will remain confined to applications with high capital investment, rather than being a vehicle for widespread innovative problem-solving.

"Computers can be little engines of creativity and they can co-create with us. Humans are not the only sources of beauty and ingenuity," Trivedi said.

To understand the fundamental change Trivedi is pursuing in his research, we must first understand what machine learning looks like right now. This current moment, he said, is as transformational as the shift from computers that took up several rooms to those sitting in a wrist-watch today.

Right now, many applications rely on neural networks and reinforcement learning.

Neural networks are large computer programs that, given huge amounts of data, create working definitions for what they have been trained on. For example, after viewing a large number of cat images, the machine "learns" to see a grouping of pixels as a cat.

But, Trivedi said, there's a problem with that. The program can't explain what a cat is beyond that grouping of pixels.

“If you train something with a neural network, you do not know what has been learned. We cannot explain why something is a cat or not a cat," he said. The machine's learning process and reasoning is hidden from us.

The high skill-floor and intense resource demand of these large neural networks can also keep machine learning away from passionate but under-resourced programmers who can’t afford the massive costs of creating and maintaining the networks.

Reinforcement learning, especially when combined with neural networks, is a promising machine learning approach to problem solving if it could be made more trustworthy and capable of solving complex problems.

Reinforcement learning is similar to training a dog, Trivedi said. By rewarding good behaviors over and over and chastising the bad ones, you can slowly train a dog to do many tricks, like shake hands or heel.

But when and how should a reward be given? In computer science, this is a question programmers must answer each time they create a reinforcement learning application. If you reward at the wrong time, you could cause a program to learn the wrong thing, like mis-training a dog to bark when it sees food.

A programmer could create a program that inadvertently damages a power grid or makes racially-biased decisions on who can access a home loan due to a bad internal reward system.

Trivedi's CAREER proposal focuses on building tools for reinforcement learning that free the programmer from the burden of translating desired outcomes to specific rewards. Instead of using a gut-feeling for the reward, programmers would now have a rigorous, formal system to assist them that they can trust.

But, even if rewards are given correctly, reinforcement learning traditionally doesn't work as well for the large, complicated problems machine learning has so much promise for.

So Trivedi wants to be able to increase the scale of tasks that reinforcement learning can be used for by exploiting modularity – a design principle that emphasizes breaking apart an overall system into simpler, well-defined parts.

Trivedi will use "recursive Markov decision processes" to describe the larger system as a collection of simpler systems, then build the overall solution to the complicated problem by combining the solutions to those simpler subtasks.

These subtasks are less energy and time-intensive to solve and the resulting modularity promotes reusability and makes explanations easier.

Through reinforcement learning that has a rigorous, formal system and that supports modularity, Trivedi's CAREER award opens a new path for complex artificial intelligence alongside neural networks, one that promises to be trustworthy, powerful and accessible to all.

Trivedi is working to democratize artificial intelligence by making machine learning more programmable, trustworthy and accessible to everyone through a prestigious CAREER award from the National Science Foundation to do so.

Off

Traditional

White

NSF grants aim to improve security and safety of autonomous cars and systems

Anonymous — Fri, 30 Oct 2020 00:00:00 +0000

NSF grants aim to improve security and safety of autonomous cars and systems Anonymous (not verified) Thu, 10/29/2020 - 18:00

Researchers at CU Boulder, including Majid Zamani and Ashutosh Trivedi from computer science, are leading four new NSF-funded projects.

Off

Traditional

White