Share via

Long Paths in .NET, Part 3 of 3 Redux [Kim Hamilton]

  • Article

My original part 3 blog caused confusion, mostly because it didn’t tie together loose ends and explain that an immediate .NET “solution” is at best partial. To minimize confusion and answer questions in the comments, I decided to do a complete overhaul and link to the original (here).

Win32 file-naming conventions include the MAX_PATH (260 character) restriction. A subset of Win32 APIs allow you to work around the MAX_PATH restriction by adding the \\?\ prefix.

Solutions based on the \\?\ prefix

Recall that these are the problems with the \\?\ prefix:

  1. Other apps may not support this prefix, and may not be able to use these files
  2. Not all Win32 APIs allow this prefix, meaning that the file will not necessarily work with other .NET APIs

#2 isn’t obvious to many people so let’s focus on that.

.NET dependence on Win32

.NET relies heavily on Win32 and isn’t always able to provide clean workarounds to Win32 restrictions. If we introduce a concept that has partial Win32 support, we either:

  1. Open up a new set of potential failures, or
  2. Add kludges on top of existing functions to make things work

An example of (a) is: you move a file to a location enabled with the \\?\ prefix, and then pass that path to another .NET API, which results in a call to a Win32 API that doesn’t support \\?\. (e.g. LoadLibrary). This sequence of operations would fail.

An example of (b) is: we try to make things work for Win32 functions that don’t support \\?\. This would be a major undertaking of rebuilding Win32 functionality or re-building a file system in the .NET layer (for example, we associate with shorter paths that can be accessed). Either approach is extremely risky and error prone, and we don’t see this as an option at the moment.

.NET solutions based on \\?\, given Win32 restrictions

Currently, .NET doesn’t allow you to use the \\?\ prefix at all. Let’s consider how .NET might enable it:

  1. Supported by default: .NET adds the \\?\ prefix behind the scenes
    1. Because of limited support of the \\?\ prefix, this will at times force you into awkward coding practices, like having to explicitly check size to make sure the file names are supported by your other apps or other functions.
    2. Also, this would be a maintenance burden going forward if Win32 provides a solution that isn’t based around \\?\
  2. Supported on an opt-in basis: callers have to opt in, such as by adding the \\?\ prefix
    1. This is just like if we had never blocked out the option from Win32
    2. Because this turns off canonicalization, this would be associated with full trust demand, which seems reasonable for applications that want to use long paths
    3. Could add convenience API such as Path.ToLongPath(file), which would add the \\?\ prefix for you, before you pass to System.IO APIs.

Other Solutions: Softening Impact

We also considered off-by-one attempts to soften the MAX_PATH limit, such as auto-shrinking which is done in the Windows shell. It’s compelling because such a path can be supported by other Win32 APIs and the shell. However, based on feedback, we think this just makes the story more confusing. The lack of consistency makes it useful to only corner-case scenarios, and certainly apps wouldn’t want to depend on it.

Proposed “Solution” (Best attempt for now)

We won’t be able to provide seamless long path support throughout .NET until something is done on the Win32 side to broaden support of long paths throughout their APIs.

As a mitigation to at least provide parity with Win32, we propose to allow use of the \\?\ prefix (described in #2 above) to avoid the MAX_PATH limitation. The caller must explicitly use this prefix, at best aided with a helper API as follows:

 String longPathEnabledName = Path.ToLongPath(@"C:\veryLongPath...\veryLongName.txt");
FileStream fs = new FileStream(longPathEnabledName);

This enables use of long paths throughout the System.IO namespace. Support of long path files outside of System.IO will be limited by support of underlying Win32 APIs.

This is not a real solution, but this provides core file (reading, writing, etc) functionality for long path files, which is compelling for certain types of apps. Such apps are best described as closed-system, file reading/writing-focused apps.

Ideal Solution

Some possibilities include:

  1. A Win32 solution, supported throughout Win32 APIs for full parity
  2. Going forward, less dependence on Strings as paths. This was touched on in my comments to long paths part 1 and is a very large discussion, for some later time.

Comments

  • Anonymous
    July 07, 2008
    PingBack from http://blogs.msdn.com/bclteam/archive/2007/02/13/long-paths-in-net-part-1-of-3-kim-hamilton.aspx

  • Anonymous
    July 07, 2008
    I'd rather see paths abstracted out as a class rather than being strings. I know this would be a monumental change, but it would make working with paths so much easier e.g. getting at the individual parts of a path, not having to worry about separator charactors, or whether you have to trim trailing slashes before appending, etc. Plus you wouln't have to worry about what's going on under the hood - as long as everything in the BCL accepts a path object parameter (rather than a string) then the ? format could be used at the very last step internally by the BLC. There's still the problem of underlying Win32 APIs not supporting ? though.  That's definitely a pain!

  • Anonymous
    July 08, 2008
    >> I'd rather see paths abstracted out as a class rather than being strings. I completely agree. It would confine the many problems of working with paths as strings to as small an area as possible.   You're right that it would be a monumental change to introduce this in the public surface area, but any progress towards this -- at least internally in the BCL -- would help. I'm definitely interested in that.

  • Anonymous
    July 08, 2008
    I would expect use of long paths outside of System.IO to throw an exception where such use is inappropriate.

  • Anonymous
    July 09, 2008
    Your proposed solution looks quite good to me.  Thanks!  It would be very nice if Path.ToLongPath accepts relative paths, too, and automatically prepend the current directory.  (It should accept any kind of path, including UNC, and "do the right thing".) Regarding the ideal solution, I do hope that you are asking (strongly) the Windows team to add full support for long paths in Windows 7. An abstract class for paths would be very welcome.  It could be something like Perl's File::Spec (but better). The is one more thing you should do: document a clear set of best practices for developers who want to support long paths using .Net.

  • Anonymous
    July 10, 2008
    The comment has been removed

  • Anonymous
    July 12, 2008
    Wow, thanks for the update Kim!  Kudos to you and the BCL Team for your thoughtfulness and hard work on this issue! Any idea when and what Framework version the proposed solution would show up in the BCL?

  • Anonymous
    July 13, 2008
    Thanks for the feedback! We're still working out the scheduling issues so I can't give a timeframe yet. I'll definitely look into where this feedback can be given on the win32 side (regarding the ideal solution). I'll get back soon.

  • Anonymous
    July 14, 2008
    Looks like a great solution, thank you! Another method you might consider adding would be "static bool Path.IsLongPath(string)" In an ideal world it would be nice if developers didn't need to know if something was a long path, but in might be a good stop-gap solution until more Win32 functions support long paths. Thanks again! I really appreciate all the hard work that has been done - definitely not an easy problem to solve!

  • Anonymous
    February 20, 2009
    Finding Path lengths greater than MAX_PATH

  • Anonymous
    February 20, 2009
    Finding Path lengths greater than MAX_PATH

  • Anonymous
    February 21, 2009
    Finding Path lengths greater than MAX_PATH

  • Anonymous
    February 21, 2009
    Finding Path lengths greater than MAX_PATH