Half the beneficiaries investigated for fraud by the Ministry of Social Development (MSD) have to answer questions about their relationship status.
Being in a marriage-like relationship can mean changes to someone's benefit entitlements - and their payments may end up being reduced.
Figures from the ministry show that in the 2016/17 year, there were close to 6000 fraud investigations completed.
About half of the investigations closed in that same year involved questions about relationship status.
However, of the total number of investigations completed, overpayments were established in just 1800 cases.
And there were only 431 successful prosecutions.
Child Poverty Action Group economic spokesperson Susan St John told Morning Report the welfare system needs to better reflect 21st century relationships.
She said the term benefit fraud should be saved for genuine fraud, and instead these cases should be referred to as disputed overpayments.
The relationship status aspect of the welfare system is "very problematic" and there needs to be a fundamental rethinking of this, she said.
"Good relationships are good for people, they're good for children, we should be encouraging them."
She said the system penalises people trying to get into good relationships and couples on benefits are the poorest in the system.
Ms St John said the single and married benefit rate needs to be aligned.
Lawyer Frances Joychild QC said the figures suggested too many beneficiaries were being needlessly scrutinised.
There should be clear grounds before investigators asked people about their personal lives, she said.
"When you are in a vulnerable position of being on a benefit, it is very unpleasant and stressful to have a fraud investigator contact you and inquire about your relationships with other people.
"It shouldn't happen unless there is reason to believe that there is something amiss."
Beneficiary Advocacy and Information Services worker Karen Pattie said the biggest problem was people don't know what being in a marriage-like relationship means.
"Clients, especially single mothers, are too afraid to even date someone or go out for dinner because does this mean a relationship? Does this mean I'll be investigated? If I talk to my case manager, does this mean I have to continue this relationship because I've declared it? Do I ring them and say, 'oh, we're not dating anymore'."
But MSD deputy chief executive of service delivery Viv Rickard said no one's benefit should be affected by going on a couple of dates.
"Can I unequivocally say having a few dates is nothing to do with this organisation and shouldn't impact on anything to do with what people are entitled to."
When allegations were made without supporting evidence, Mr Rickard said it was more than likely no further action would be taken.
However, he said his staff do need to take suggestions of potential fraud seriously.
"Clearly there are a small number of people who will try things on, and the Ministry of Social Development, we administer $24 billion, so I'm not going to open up the gate to the king's gold.
"But we do need to ensure when there are allegations, we consider them appropriately and if there are matters to investigate we will."
The small number of prosecutions the ministry took showed that sort of action was only taken in the most serious cases, Mr Rickard said.
He said the vast majority of Work and Income's 276,000 clients were following the rules and do get in touch when their circumstances change.