There are several important details here, so it's best to share some additional information on how those mailboxes were created, which migration method you are using, is Directory sync involved and so on.
To give you some generic instructions, you have few options:
- If migrating from on-premises Exchange, use the method outlined in this article
- If the mailbox was created/licensed in Exchange Online, remove the Exchange license and wait
- As an alternative, destructive option, consider removing the user object in Office 365. Do make sure to backup any and all data you might need from it, if you decide to go this route.