Chapter 15: Using Windows HTTP Services

The Hilo Browser application allows you to upload photos to the Flickr online photo sharing application. To do this, Hilo uses Windows HTTP Services. This chapter will explore how this library is used in the Hilo Browser to implement its photo sharing feature.

Flickr Share Dialog

When Hilo uploads a photo to Flickr, it makes several calls to the Flickr web server. These calls are made to authenticate the Hilo application (and obtain a session token called a frob); to authorize the access of the Hilo Flickr application to upload a photo to a Flickr account (and obtain an access token) and then to upload the photo. These calls are made across the network, and potentially they can take a noticeable amount of time. Hilo has to wait for responses from the Flickr web server in such a way that the user is kept informed. This is the purpose of the Share dialog.

The Hilo Browser’s user interface provides a button, labeled Share (Figure 1). When you click on this button you will see the Share dialog (Figure 2), which is implemented by the ShareDialog class. This class has static methods and allows you to upload either the selected photo, or all photos in the current folder. The Share dialog has effectively three sets of controls reflecting your progress through the mechanism of uploading photos. The first set of controls is shown in Figure 2. When you click the Upload button a progress bar is shown under the radio buttons (Figure 3) to display the progress of the upload; and when the upload is complete all the initial controls are hidden except for the Cancel button which is relabeled Close, and the View Photos link control is displayed (Figure 4). The same class is used for all versions of this dialog.

Figure 1 The Hilo Browser Share button

Figure 2 The initial display of the Share dialog

Figure 3 The display of the Share dialog when uploading photos

Figure 4 The display of the Share dialog when uploading has completed

When the user clicks the Upload button, the ShareDialog::UploadImages method creates an instance of the FlickrUploader class to perform the network access. First, the ShareDialog::UploadImages method calls the FlickrUploader::Connect method to obtain the session key (the frob value) and launch the system registered browser to show the Flickr logon page. The user logs on to the Flickr account where Hilo will upload the photos. Next the UploadImages method calls the FlickrUploader::GetToken method to get the access token associated with the account where the photos will be uploaded. Finally, a new thread is created to run the SharedDialog::ImageUploadThreadProc method asynchronously. This method uploads each photo by calling the FlickrUploader.UploadPhotos method passing the token and the path to the photo. As each photo is uploaded the progress bar is updated by one position. When each photo is uploaded Flickr returns an ID and once all photos have been uploaded the SharedDialog::ImageUploadThreadProc method creates a URL to display the uploaded images. The format of this URL is shown in Listing 1 where the ids parameter is a comma separated list of the IDs of the photos to show. This URL is provided as the link of the View Photos link control in Figure 4.

Listing 1 Uploaded photos URL

https://www.flickr.com/photos/upload/edit/?ids=[comma separated list]

At several stages in the upload process the code has to pause for user input. The first time this happens is when the ShareDialog::UploadImages method obtains the frob from Flickr. The UploadImages method calls the FlickrUploader::Connect method which launches the system registered browser to show the Flickr logon page and the user task switches to this browser page. During this time the UploadImages method must be paused until the logon process has completed. To do this the UploadImages method displays two Task Dialog windows (created by calling the TaskDialogIndirect function).

Figure 5 Dialog informing the user that they must authorize Hilo

The first dialog (Figure 5) informs the user that they will have to authorize the Hilo application’s access to their account. This lets the user know what is about to happen, and gives the user the opportunity to cancel the operation. When the user clicks the Authorize button on this first dialog, the FlickrUploader::Connect method is called to display the logon page and the second dialog (Figure 6) is shown in the background. This modal dialog blocks the UploadImages method and the user will see this dialog when they have finished the logon procedure and switched back to Browser. At this point the user can click the Authorization Complete button to unblock the UploadImages method and allow it to upload the photos by calling SharedDialog:: ImageUploadThreadProc method on a worker thread.

Figure 6 Dialog used to block Browser until the user has authorized the Hilo Flickr application

Uploading a Photograph

A photo can be several megabytes of data. The Flickr API for uploading photos involves a multi-part HTTP POST request to the https://api.flickr.com/services/upload/ URI. Each part of the message is one of the arguments that describe the upload action. The action must be authenticated and authorized, and so the API key and the access token must be provided along with a message digest generated from them. The message digest (Flickr calls it the signature) is a hash generated from the method parameters and the secret known only by the Flickr application and Flickr. The digest is used by Flickr to determine the integrity of the parameters so that it can detect if the request has been altered en route, either maliciously or by accident.

The final part of the message is the photo provided as binary data. Listing 2 shows the general format of the POST request (the items in square brackets will be replaced with actual data).

Listing 2 Example POST message to upload a photo

POST /services/upload/ HTTP/1.1
Content-Type: multipart/form-data; boundary=--EBA799EB-D9A2-472B-AE86-568D4645707E
Host: api.flickr.com
Content-Length: [data_length]

--EBA799EB-D9A2-472B-AE86-568D4645707E
Content-Disposition: form-data; name="api_key"

[api_key_value]

--EBA799EB-D9A2-472B-AE86-568D4645707E
Content-Disposition: form-data; name="auth_token"

[token_value]

--EBA799EB-D9A2-472B-AE86-568D4645707E
Content-Disposition: form-data; name="api_sig"

[api_sig_value]

--EBA799EB-D9A2-472B-AE86-568D4645707E
Content-Disposition: form-data; name="photo"; filename="[filename]"

[image_binary_data]

--EBA799EB-D9A2-472B-AE86-568D4645707E

Using Windows HTTP Services

The network access to Flickr is performed by methods on the FlickrUploader class. This class makes two types of calls: calls to Web Services methods where the data is formatted and transmitted according to the SOAP protocol, and low level calls over HTTP to the Flickr website. The next chapter, Chapter 16, will describe the Windows 7 Web Services API code in Hilo. Windows HTTP Services provides access to web servers over the HTTP protocol and Hilo uses this API to upload photos to Flickr using the POST request shown in Listing 2. To use this API you have to include the Winhttp.h header file and link to the Winhttp.lib library.

Before you call any of the HTTP Services functions you must call the WinHttpOpen function. You pass to this function the name of the user agent that will be sent during future web requests, information about the proxy (if any) that will be used, and whether the web calls will be synchronous or asynchronous. The WinHttpOpen function returns a session handle (HINTERNET) that, like all the HTTP Services handles, must be closed with the WinHttpCloseHandle function when you have finished. Next you call the WinHttpConnect function to obtain a connection handle. For this function, you provide the session handle and the server name and port. This function does not make a network connection; it just prepares the internal connection settings. The next task is to create a request, to do this you call the WinHttpOpenRequest function, passing the connection handle and information about the request to be made. The WinHttpOpenRequest function returns a request handle which is used to store all the RFC822, MIME, and HTTP headers to be sent as part of the request. You add the actual headers for the request through a call to the WinHttpAddRequestHeaders function passing the request handle and a string that contains all the headers separated by return/linefeed pairs.

The actual network call is made when you call the WinHttpSendRequest function passing the request handle. This function can be used to provide additional HTTP headers not provided by calls to WinHtttpAddRequestHeaders and any additional data required when the call is made for a PUT or POST request. The server will respond to the request, and you call the WinHttpReceiveResponse function with the request handle for the system to read the response headers from the server (which can be obtained by calling the WinHttpQueryHeaders function).

Once the server has responded you can receive data from the server. To do this you call the WinHttpQueryDataAvailable function with the request handle to receive the number of bytes that can be downloaded and then call the WinHttpReadData function to read the data. If the request is made synchronously, the WinHttpQueryDataAvailable function will block until there is data available.

Using HTTP Services in Hilo

Hilo uses HTTP Web Services when it uploads photos to Flickr. Listing 2 gives the general format of the POST request that is made to upload a photo and Listing 3 shows the FlickrUploader::UploadPhotos method that makes this call. The first few lines are straightforward. The code calls the WinHttpOpen function to indicate that the user agent is Hilo/1.0 and that at this point no proxy will be specified. Next the code calls the WinHttpConnect function indicating that the host name is api.flickr.com and access will be made on the default HTTP port, the TCP port 80 and then calls the WinHttpOpenRequest function to specify that the request is a POST call to the /Services/Upload/ resource.

Listing 3 Uploading photos to Flickr

std::wstring FlickrUploader::UploadPhotos(
   const std::wstring& token, const std::wstring& fileName, bool* errorFound)
{
   std::wstring outputString;
   HINTERNET session = nullptr;
   HINTERNET connect = nullptr;
   HINTERNET request = nullptr;

   WINHTTP_AUTOPROXY OPTIONS  autoProxyOptions;
   WINHTTP_PROXY_INFO proxyInfo;
   unsigned long proxyInfoSize = sizeof(proxyInfo);
   ZeroMemory(&autoProxyOptions, sizeof(autoProxyOptions));
   ZeroMemory(&proxyInfo, sizeof(proxyInfo));

   // Create the WinHTTP session.
   session = ::WinHttpOpen( 
      L"Hilo/1.0", WINHTTP_ACCESS_TYPE_NO_PROXY, 
      WINHTTP_NO_PROXY_NAME, WINHTTP_NO_PROXY_BYPASS, 0);
   connect = ::WinHttpConnect(session, L"api.flickr.com", INTERNET_DEFAULT_HTTP_PORT, 0);
   request = ::WinHttpOpenRequest(
      connect, L"POST", L"/services/upload/", L"HTTP/1.1", 
      WINHTTP_NO_REFERER, WINHTTP_DEFAULT_ACCEPT_TYPES, 0);
   autoProxyOptions.dwFlags = WINHTTP_AUTOPROXY_AUTO_DETECT;
   // Use DHCP and DNS-based auto-detection.
   autoProxyOptions.dwAutoDetectFlags = 
      WINHTTP_AUTO_DETECT_TYPE_DHCP | WINHTTP_AUTO_DETECT_TYPE_DNS_A;
   // If obtaining the PAC script requires NTLM/Negotiate authentication,
   // then automatically supply the client domain credentials.
   autoProxyOptions.fAutoLogonIfChallenged = true;

   if (FALSE != ::WinHttpGetProxyForUrl(
         session, L"https://api.flickr.com/services/upload/", &autoProxyOptions, &proxyInfo))
   {
      // A proxy configuration was found, set it on the request handle.
      ::WinHttpSetOption(request, WINHTTP_OPTION_PROXY, &proxyInfo, proxyInfoSize);
   }

   SendWebRequest(&request, token, fileName);
   outputString = GetPhotoId(&request, errorFound);

   // Clean up
   if (proxyInfo.lpszProxy)
   {
      GlobalFree(proxyInfo.lpszProxy);
   }
   if (proxyInfo.lpszProxyBypass)
   {
      GlobalFree(proxyInfo.lpszProxyBypass);
   }

   WinHttpCloseHandle(request);
   WinHttpCloseHandle(connect);
   WinHttpCloseHandle(session);
   return outputString;
}

The FlickrUploader class will upload the photo though a web call that will be made through a proxy if one is set up for the local network. To do this, the UploadPhotos method calls the WinHttpGetProxyForUrl function to receive the contents of the Proxy Auto-Configuration (PAC) file discovered using DHCP and DNS queries. The results are returned in the proxyInfo variable, which has a proxy server list and a proxy bypass list, that are allocated on the system global heap.These fields are freed with calls to the GlobalFree function when the method completes. The proxy information is associated with the request by calling the WinHttpSetOption method. At this point the method can make the web request by calling the FlickrUploader::SendWebRequest method shown in Listing 4.

The majority of the SendWebRequest method is to construct the headers of the HTTP POST request. The upload parameters have to be signed with the Flickr API secret and so the first part of the SendWebRequest method concatenates the secret, the API key, and the token. It then generates a MD5 hash of this string by calling the FlickrUploader::CreateMD5Hash method (which is described later). This header is added to the request by calling WinHttpAddRequestHeaders.

Listing 4 Making the web request

int FlickrUploader::SendWebRequest(
   const HINTERNET *request, const std::wstring& token, const std::wstring& fileName)
{
   static const char* mimeBoundary = "EBA799EB-D9A2-472B-AE86-568D4645707E";
   static const wchar_t* contentType = 
      L"Content-Type: multipart/form-data; boundary=EBA799EB-D9A2-472B-AE86-568D4645707E\r\n";

   // Parameters put in alphabetical order to generate the api_sig
   std::wstring params = flickr_secret;
   params += L"api_key";
   params += flickr_api_key;
   params += L"auth_token";
   params += token;

   std::wstring api_sig = CalculateMD5Hash(params);
   int result = ::WinHttpAddRequestHeaders(
      *request, contentType, (unsigned long)-1, WINHTTP_ADDREQ_FLAG_ADD);
   if (result)
   {
      std::wostringstream sb;

      sb << L"--" << mimeBoundary << L"\r\n";
      sb << L"Content-Disposition: form-data; name=\"api_key\"\r\n";
      sb << L"\r\n" << flickr_api_key << L"\r\n";

      sb << L"--" << mimeBoundary << L"\r\n";
      sb << L"Content-Disposition: form-data; name=\"auth_token\"\r\n";
      sb << L"\r\n" << token << L"\r\n";

      sb << L"--" << mimeBoundary << L"\r\n";
      sb << L"Content-Disposition: form-data; name=\"api_sig\"\r\n";
      sb << L"\r\n" << api_sig.c_str() << L"\r\n";

      sb << L"--" << mimeBoundary << L"\r\n";
      sb << L"Content-Disposition: form-data; name=\"photo\"; filename=\"" << fileName << L"\"\r\n\r\n";

      // Convert wstring to string
      std::wstring wideString = sb.str();
      int stringSize = WideCharToMultiByte(CP_ACP, 0, wideString.c_str(), -1, nullptr, 0, nullptr, nullptr);
      char* temp = new char[stringSize];
      WideCharToMultiByte(CP_ACP, 0, wideString.c_str(), -1, temp, stringSize, nullptr, nullptr);
      std::string str = temp;
      delete [] temp;

      // Add the photo to the stream
      std::ifstream f(fileName, std::ios::binary);
      std::ostringstream sb_ascii;
      sb_ascii << str;
      sb_ascii << f.rdbuf();
      sb_ascii << "\r\n--" << mimeBoundary << "\r\n";
      str = sb_ascii.str();
      result = WinHttpSendRequest(
          *request, WINHTTP_NO_ADDITIONAL_HEADERS,  0, (void*)str.c_str(),
          static_cast<unsigned long>(str.length()), static_cast<unsigned long>(str.length()), 0);
   }
   return result;
}

The remainder of the method adds the parameters to the request, a multi-part MIME message built as an ASCII string. First, the parts that indicate the API key, the token, and the signature are added using a Unicode buffer that is converted to an ASCII buffer before the contents of the photo are added as binary data. Finally the actual request is made to the server by calling the WinHttpSendRequest function.

Once the FlickrUploader::UploadPhotos method has made the request to the server it calls FlickrUploader::GetPhotoId to read the response from the server as shown in Listing 5. The first action is to call the WinHttpReceiveResponse method, which will block until the server sends the response. The GetPhotoId method then calls the WinHttpQueryHeaders function to get all the response headers. These headers are read in a single buffer because they are not needed, what is needed is the data that follows the response headers and these are obtained by calling the WinHttpReadData function.

Listing 5 Retrieving data from a web request

std::wstring FlickrUploader::GetPhotoId(const HINTERNET *request, bool* errorFound)
{
   std::wstring outputString;
   int result = ::WinHttpReceiveResponse(*request, nullptr);
   unsigned long dwSize = sizeof(unsigned long);
   if (result)
   {
      wchar_t headers[1024];
      dwSize = ARRAYSIZE(headers) * sizeof(wchar_t);
      result = ::WinHttpQueryHeaders(
         *request, WINHTTP_QUERY_RAW_HEADERS, nullptr, headers, &dwSize, nullptr);
   }
   if (result)
   {
      char resultText[1024] = {0};
      unsigned long bytesRead;
      dwSize = ARRAYSIZE(resultText) * sizeof(char);
      result =::WinHttpReadData(*request, resultText, dwSize, &bytesRead);
      if (result)
      {
         // Convert string to wstring
         int wideSize = MultiByteToWideChar(CP_UTF8, 0, resultText, -1, 0, 0);
         wchar_t* wideString = new wchar_t[wideSize];
         result = MultiByteToWideChar(CP_UTF8, 0, resultText, -1, wideString, wideSize);
         if (result)
         {
            std::wstring photoId = GetXmlElementValueByName(wideString, L"photoid", errorFound);
            if (!(*errorFound))
            {
               outputString = photoId;
            }
         }
         delete [] wideString;
      }
   }
   return outputString;
}

The response from the server will be an XML string. The FlickrUploader::GetXmlElementValueByName method uses the XMLLite API to create an XML reader object by calling the CreateXmlReader function and then iterating over every element until it finds the photoid element and returns the value of the element as the ID of the photo just uploaded. The SharedDialog::ImageUploadThreadProc method stores these IDs, and then when all photos have been uploaded it creates a URL to the page that display the uploaded images.

Using Cryptographic API: Next Generation

Whenever you make a call to a Flickr service method you must pass a message digest along with the method parameters. The message digest allows Flickr to determine the integrity of the parameters so that it can detect if the request has been altered during transmission, either maliciously or by accident. The digest is generated from a string that is the concatenation of the Hilo Flickr application secret, the named parameters in alphabetic order and the Web Service method name. The digest is the hex string value of the MD5 hash of this parameter string.

The digest is not used for privacy, indeed, the parameters are passed cleartext along with the message digest. When Flickr receives a request it obtains the secret for the Flickr client application (the Hilo Browser)making the request and using this and the parameters it can also create an MD5 hash. Flickr compares the hash it created with the message digest of the request, and if the two are the same then it indicates that the message has not been tampered with or corrupted en route.

To create a MD5 hash Hilo uses the Windows Cryptography API: Next Generation (CNG) library that was introduced with Windows Vista. Earlier versions of Windows provided cryptographic functions through the CryptoAPI. CNG is much improved compared to CryptoAPI: it provides more cryptographic providers, including the newer algorithms that are part of the National Security Agency (NSA) Suite B, and the API is more logically factored. All features of CNG are accessed using the same steps: open the provider, get or set properties of the algorithm, perform the cryptographic action and then close the provider.

To open a provider you call the BCryptOpenAlgorithmProvider function and provide the names of the algorithm, the provider, and any appropriate flags. The function returns a BCRYPT_ALG_HANDLE handle, which you can use to get or set the properties of the algorithm. When you have finished with the provider you must close it by passing the handle to the BCryptCloseAlgorithmProvider function.

You can alter how a cryptographic algorithm works, or obtain information about the algorithm through properties. To obtain a property you call the BCryptGetProperty function, passing the handle for an open provider, the name of the property, and a pointer to a caller allocated buffer to receive the property value. The function also has an in/out parameter for the size of the buffer. Some properties are of a known size (for example BCRYPT_KEY_LENGTH which returns the size of the cryptographic key as a 32-bit value). Others are not known and so you can call BCryptGetProperty with a NULL value for the pointer to the buffer and the function will return the required buffer size through the pointer used to indicate the buffer size. Changing a property is similar: you call the BCryptSetProperty function passing the provider handle, the name of the property, and a pointer to the buffer containing the new property value and use a parameter to indicate the size of this buffer.

Once you have set the properties of the algorithm you may call one of the CNG functions to perform the cryptographic action. If you wish to encrypt or decrypt data then you will need to provide a key. To do this you can either create a key with a call to the BCryptGenerateSymmetricKey function to create a symmetric (or secret) key or the BCryptGenerateKeyPair functions to create the asymmetric (or public-private) key pair. These functions return a BCRYPT_KEY_HANDLE handle and when you have finished with the handle you release the associated resources by calling the BCryptDestroyKey function. The key can then be passed, along with buffers for the input and output values to BCryptEncrypt to encrypt data or to BCryptDecrypt to decrypt data.

Creating a hash is a cryptographic operation and you can provide a key to be used by the algorithm, but this is optional. Before you can hash data you have to create a hash object by calling the BCryptCreateHash function, which returns a hash object handle that must be closed with a call to the BCryptDestroyHash function when you have finished performing the hash operation. You perform the hash by calling the BCryptHashData function passing the handle to the hash object and a pointer to a buffer with the data to hash (and a parameter indicating the size of the buffer). This function does not return the hashed data, in fact the data passed to the function is not modified, instead, the hash is maintained in memory by the hash object. The reason for this is you can call the BCryptHashData function more than once to hash the hashed data. To obtain the hashed value you call the BCryptFinishHash function passing the handle to the hash object and a user allocated buffer and its size, the function copies the hash from the hash object into the buffer. The size of the buffer is provided through the BCRYPT_HASH_LENGTH property.

Hashing Data in Hilo

Hilo provides a method called FlickrUploader::CalculateMD5Hash to generate an MD5 hash from a string passed as a parameter. The hash will be a binary value so the string returned from the CalculateMD5Hash method is the hex encoded value. Listing 6 shows the first half of this method, which creates the hash object. First the code calls the BCryptOpenAlgorithmProvider function to open the default MD5 provider. Next the method calls the BCryptGetProperty twice: the first time to get the size of the hash object and the second time to get the size of the buffer needed for the generated hash. These two buffers are created on the process heap by calling the HeapAlloc function but they could be allocated using any suitable memory allocator. Finally the CalculateMD5Hash method creates the hash object by calling the BCryptCreateHash function which initializes the buffer allocated for the hash object.

Listing 6 Creating a hash object

std::wstring FlickrUploader::CalculateMD5Hash(const std::wstring& buffer)
{
   // Convert wstring to string
   std::string byteString(buffer.begin(), buffer.end());

   // Open an algorithm handle
   BCRYPT_ALG_HANDLE algorithm = nullptr;
   BCryptOpenAlgorithmProvider(&algorithm, BCRYPT_MD5_ALGORITHM, nullptr, 0);

   // Calculate the size of the buffer to hold the hash object
   unsigned long dataSize = 0;
   unsigned long hashObjectSize = 0;
   BCryptGetProperty(
      algorithm, BCRYPT_OBJECT_LENGTH, (unsigned char*)&hashObjectSize, sizeof(unsigned long), &dataSize, 0);

   // Allocate the hash object on the heap
   unsigned char* hashObject = nullptr;
   hashObject = (unsigned char*) HeapAlloc(GetProcessHeap (), 0, hashObjectSize);

   // Calculate the length of the hash
   unsigned long  hashSize = 0;
   BCryptGetProperty(
      algorithm, BCRYPT_HASH_LENGTH, (unsigned char*)&hashSize, sizeof(unsigned long), &dataSize, 0);

   // Allocate the hash buffer on the heap
   unsigned char* hash = nullptr;
   hash = (unsigned char*)HeapAlloc (GetProcessHeap(), 0, hashSize);

   // Create a hash
   BCRYPT_HASH_HANDLE cryptHash = nullptr;
   BCryptCreateHash(algorithm, &cryptHash, hashObject, hashObjectSize, nullptr, 0, 0);

The next part of the CalculateMD5Hash method is shown in Listing 7. This code calls the BCryptHashData function to hash the data passed as the second parameter. The hash object retains the actual hash which is then obtained by calling the BCryptFinishHash function passing the buffer previously allocated to hold the hash. Finally this binary data is converted to a hex string.

Listing 7 Hashing the data

   // Hash data
   BCryptHashData(
      cryptHash, (unsigned char*)byteString.c_str(), static_cast<unsigned long>(byteString.length()), 0);

   // Close the hash and get hash data
   BCryptFinishHash(cryptHash, hash, hashSize, 0);

   std::wstring resultString;
   // If no issues, then copy the bytes to the output string 
   std::wostringstream hexString;
   for (unsigned short i = 0; i < hashSize; i++)
   {
      hexString << std::setfill(L'0') << std::setw(2) << std::hex << hash[i];
   }
   resultString =  hexString.str();

The last part of the CalculateMD5Hash method is shown in Listing 8. This shows the cleanup that is needed to release the hash object and the cryptographic provider and to release the previously allocated memory.

Listing 8 Cleaning up the objects and buffers used by the method

   // Cleanup
   BCryptCloseAlgorithmProvider(algorithm, 0);
   BCryptDestroyHash(cryptHash);
   HeapFree(GetProcessHeap(), 0, hashObject);
   HeapFree(GetProcessHeap(), 0, hash);

   return resultString;
}

Conclusion

In this chapter you have seen how the Hilo Browser uploads photos to the Flickr web site using the HTTP Services API to make a HTTP POST request, and how Hilo uses task dialogs to provide visual feedback about the upload operation to the user. You have also seen how the CNG functions can be used to create cryptographic hashes used by Flickr to verify the integrity of the parameters passed to it by a client. In the next chapter you will see how Hilo uses the Windows 7 Web Services API to call the Flickr Web Service to obtain an authentication session key and an access token.

Next | Previous | Home