Hi,
What you are seeing is normal and expected.
Azure SQL Database is a multi-tenant Platform-as-a-Service (PaaS) and as a result you are not connecting to a public IP address for a dedicated instance of SQL Server. Instead it is shared by many different customers, some of which allow access via public Internet.
When you telnet from external network you are making an initial, unauthenticated TCP connection to shared Gateway. At this point in the connection process the service doesn't have enough information to decide if the connection should be allowed or not based on SQL firewall settings.
When you attempt a full SQL connection using an appropriate client, the connection fails (as you noted in your question) as soon as the service has enough information to know that it should be blocked due to SQL firewall settings.
Please see article below for details on Azure SQL Database connectivity architecture.
Azure SQL Database and Azure Synapse Analytics connectivity architecture
Please click Accept Answer and upvote if the above was helpful.
Thanks.
-TP