Great question @Ken Nagata
By the way, exactly yesterday I configured my personal site to use SSL so it is fresh 🙄
is it there any way to set up the certificate in this way ? or I need to actually buy a new domain from a provider?
The short answer is yes, but I have no idea what AWS offers.
You can use a single SSL certificate for all subdomain using Wildcard configuration of the certificate.
In this case you will need to copy the certificate to all servers which might be a bit complex if you are no familiar with the topic. For most cases if you have only few sub-domains then you are probably better off simply purchasing separate certificates.
it is already registered in AWS *.domain.com and its working
I don't understand if this is working then what is not working?!?
I am not familiar with the procedure of configure SSL certificate at AWS.
I can tell you that you can get a free certificate from letsencrypt, which is nonprofit free, automated, and open certificate authority (CA), This is is the service which I use usually.
There are a large number of ACME clients available which support letsencrypt certification including in the Azure (check the link).