No, that's not the expected behavior. Files to which your application has not explicitly been granted access should result in an error, when you're only using the Files.SelectedOperations.Selected scope. In your scenario I would suggest checking the permissions on any "parent" entry, including folders, lists and sites. Make sure that none of them has explicit permission entry for the app.
As a quick test, you can register a new application and try to access any of the same files via it.