Under the deal, 4000 refugees will be resettled in Australia while 800 asylum seekers will be sent to Malaysia.
The first group of refugees are expected to arrive in Australia later this week.
Yasmeen Fatima, from the Burmese Rohingya Association of Australia, says around 90 per cent of the 90,000 refugees and asylum seekers in Malaysia are Burmese.
She says many are from a Rohingya background, along with other Burmese ethnic minorities such as the Chin and Karen.
Ms Fatima says whilst the Rohingyas are a large group, community contacts in Malaysia say no Rohingyas have been picked to go to Australia in the refugee swap.
"I think this is really unfair because it is well known that in Malaysia the Rohingyas are the first burmese refugee," she said.
A spokesman for the Australian immigration department says the makeup of the 4,000 refugees to be swapped is yet to be determined, with the United Nations High Commission for Refugees making that selection.
Source: Radio Australia News