Hello Test User,
Thank you for reaching out to Microsoft Support!
After testing, as you said, even if folderId is not correct, but contactId is correct, it can be successfully deleted, which is recommended that you give feedback here.
Hope this helps.
If the answer is helpful, please click Accept Answer and kindly upvote it. If you have any further questions about this answer, please click Comment.