Cloud Cache Overview
Cloud Cache is a feature that works with Profile and ODFC containers to provide resiliency and high availability. Cloud Cache uses the locally mounted container to provide periodic updates to the remote storage providers. Cloud Cache is designed to insulate users from short-term or intermittent local (inner-region, close proximity) storage issues. Based on the configuration, it can also be used as part of a Business Continuity or Disaster Recovery (BCDR) plan when using remote storage providers in different regions. Using Cloud Cache puts a performance and storage requirement on the virtual machine to accommodate the extra I/O operations and storage required by the local cache.
Cloud Cache considerations:
- Cloud Cache uses storage providers based on the order of entries in
CCDLocations. - Storage providers should be listed in order of proximity, then preference.
- Only one (1) provider is used when data is hydrated from the storage provider.
- Data is written to all storage providers regardless of which provider is used during data hydration.
- Performance (latency, utilization, bottlenecks) of a storage provider affects its sync status with the local copy.
- Having one (1) or more provider that is behind in updates from local cache could be an indicator of an under performing storage provider.
PingorTest-NetConnectioncommand results are not the same as transactional I/O and are bad indicators of how a storage provider will or can perform.
Figure 1: Detailed diagram showing Cloud Cache components
Cloud Cache Components
Local Cache
Cloud Cache is able to insulate a user from connectivity issues to the remote storage providers because the container used for the user's profile is created and stored local to the virtual machine (local cache). During a first-time sign-in, FSLogix creates the container for the user in C:\ProgramData\FSLogix\Cache and stores data read from the remote storage provider and data written from the local machine. Next, FSLogix sets up all the necessary redirections for the user's profile. Then the User Profile Service creates the user's profile into the local cache.
During a 2nd or Nth sign-in, FSLogix attempts to find and mount any previous local cache VHD(s) stored on the virtual machine. Looking for a local cache is the default configuration setting and may not be desired as it may lead to low disk space events. Review the Cloud Cache settings reference page for more settings. If no local cache VHD(s) exist, a Proxy file is registered and a local cache VHD created. Next, FSLogix will set up the necessary redirections for the user's profile. Finally, FSLogix will use one (1) of the remote storage providers as the read source to populate the local cache as requested by the operating system during the sign-in process to complete the sign-in.
Cache Objects
Queue files
The *.queue files are created for each remote storage provider and keeps track of the *.index files that are not yet flushed to the local cache and remote storage providers.
Index files
The *.index files contain batches of block-level changes that need to be written to the local cache and remote storage providers.
Figure 2: Cloud Cache Local Cache
Remote Storage Providers (hydrate, flush, clone)
Cloud Cache operates the user's profile from the local cache during the user's session and must be configured with one or more remote storage providers as specified in CCDLocations. These remote storage providers store full copies of the local cache and are used during the current session and for subsequent sign ins. If all providers become unhealthy during the user's session, the local cache continues to operate and grow1 until one or more provider returns to a healthy state.
Warning
If a read is requested from the user's profile and Cloud Cache is unable to hydrate the data from any of the remote storage providers, this will cause a session hang or worse a system crash (BSOD).
1 The local cache will only grow up to the maximum size of the container as specified in the SizeInMBs setting.
Hydrate
When the local cache doesn't contain the data requested by the file system, Cloud Cache hydrates (reads and copies) the data from 1 of the remote storage providers to the local cache. This operation is also part of the sign in process when populating the local cache for the user's profile.
Flush
The flush operation commonly occurs in three ways.
- On a lazy asynchronous operation, Cloud Cache flushes the changes to all storage providers simultaneously as each provider is flushed on its own thread. FSLogix doesn't throttle this operation and utilizes as much throughput as the system will allow.
- During sign out when one or more providers don't contain all the updates, the user's sign out is delayed2 until all providers are at the same sequence.
- During a user's session when the connection to any storage provider becomes unhealthy, FSLogix queues up all the changes and then flushes them to the providers when they return to a healthy state.
2 A user's sign out is delayed based on how Cloud Cache is configured using the HealthyProvidersRequiredForUnregister and CcdUnregisterTimeout value.
Clone
A full VHD(x) clone is performed when Cloud Cache determines, at sign in, that a storage provider isn't at the same sequence. During this operation, all pending writes are kept in the local cache until all storage providers are at the same sequence. Once complete, the flush operation begins sending new data to the storage providers.
Proxy File
Cloud Cache uses the concept of a proxy file, represented as Profile_%username%.vhd though it isn't a true VHD file. The proxy file is mounted to the local machine as the registered VHD representing the user's container. The proxy file is used as a means to process all I/O writes destined for the local cache. This includes missing data in the local cache that is hydrated from a remote storage provider. The I/O writes are buffered on disk and tracked via the proxy file before they're written out as *.index cache objects in the cache directory. While the proxy file has the same size as the local cache file, the actual size on disk is zero, as no data is written to this file.
Figure 3: Cloud Cache Proxy File
Auxiliary Files
Cloud Cache makes use of two (2) auxiliary files in order to maintain control and sequence of the local cache.
Note
These auxiliary files are used by FSLogix and are not meant to be opened or used outside of the product. Any relevant information in those files will be made available through our log files or event log entries.
Lock File
The lock file is what its name might imply, a file use to determine which virtual machine has an I/O lock on the container. Cloud Cache uses this information to determine ownership of the container for a given provider. The lock file mechanism is critical when using Cloud Cache with ProfileType set to '3' (Multiple or concurrent sessions). This file only exists on the remote storage providers.
Meta File
The meta file is a multi-purpose file where we track the state of the container. Inside the meta file, Cloud Cache uses a sequence numbering system to determine which provider has the most recent data. This file is both local to the virtual machine and sync'ed to the remote storage providers.
Storage Providers
FSLogix isn't a storage provider, rather we rely on underlying architecture of the storage provider(s). For more information on the storage providers FSLogix supports, review our Container Storage Options.