Creating fairer tests and surveys

Mathematics researcher Ivy Liu uses categorical data analysis to improve tests and surveys so they are fairer and more accurate.

Associate Professor Ivy Liu, School of Mathematics and Statistics

Associate Professor Ivy (I-Ming) Liu researches an area of mathematics and statistics that crosses into almost all areas of life: making tests and surveys fairer, more accurate and less likely to contain bias, and identifying groups of people with similar patterns.

Deputy Head of Victoria University of Wellington’s School of Mathematics and Statistics, Associate Professor Liu specialises in categorical data analysis. Often used in mathematical and scientific data collection, categorical data analysis is about analysing information that has been organised into groups.

Associate Professor Liu chose the subject initially because she hoped it would enable her to collaborate professionally with her husband, who was planning to study sociology.

“After I chose my topic, my husband decided not to continue with sociology. We’ve never been able to work together, but it was still the right choice,” she says.

“When I gained my PhD, there was a high demand for people who worked in categorical data analysis, which made it easy for me to get a job.”

Associate Professor Liu grew up in Taiwan, gaining her PhD at the University of Florida in the United States. She came to New Zealand on a casual visit and decided it would be a great place to live and work.

Her multidisciplinary research falls into three main areas:

  • differential item functioning (DIF)
  • dimension reduction for mixed type multivariate data
  • multiple response data.

DIF analysis is about ensuring educational tests are fair to everyone taking the test, regardless of gender, ethnicity or any other factor. If a test were to be taken by both domestic and international students, for example, it would be necessary to look at each student’s ability to determine whether the test was fair. If 80 percent of top domestic students and 80 percent of top international students get a question right, it suggests there is no bias in the test.

DIF analysis was originally used to discover whether a question was answered correctly or not, but the statistical method Associate Professor Liu developed with a colleague enables researchers to find out how well it was answered: perfectly, not so well or very badly. The methodology is not only useful for detecting biases in tests; it can also be applied in a clinical trial to find drug effects.

Associate Professor Liu has just finished a project on the ‘cluster’ element of categorical data, which provides researchers with information on how to cluster data into groups to reduce the number of questions in a test or survey.

In 2016, Associate Professor Liu was awarded a $550,000 Marsden Fund grant from Government funding managed by the Royal Society of New Zealand to carry out a further three years of research into clustering to reduce dimension of data. She says the research can potentially be applied to many fields, from categorising the risk of self-harm among groups of patients at a doctor’s surgery to classifying shoppers according to their buying patterns.

Associate Professor Liu’s work on multiple response data, her third research interest, involves looking at the best ways to analyse data derived from surveys that ask respondents to tick as many responses as they want to. For example, people might be given a list of movie genres, from drama to romance, and asked to tick all the genres they like.

“All my research interests are linked, and we use our existing work to solve new problems,” says Associate Professor Liu.

“Multiple response data links to the work we’ve done on DIF analysis. For DIF analysis, rather than focusing on one question in a test, you are interested in several questions simultaneously. Multiple response data also looks at several questions at the same time.”

She says she often discusses her research projects with students. She has taught her DIF research to her second-year applied statistics students, many of whom are studying psychology or biology and can see how useful the work could be to their disciplines.

“I really like the combination of teaching and research. You feel excited when you are able to talk to students about the area you’re researching,” she says.

Associate Professor Liu says she’s glad she decided to join the University’s School of Mathematics and Statistics and feels fortunate to be in an academic environment that gives its staff so much support to tap into sources of research funding.

“We’re also encouraged to attend international conferences and to invite international scholars in our field to visit us,” she says.

“I’ve found that international scholars are always happy to visit us here because the research coming from our group is so strong.”