This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
While reading a UTF-8 text file in a Unicode C++ build, the CStdioFile::ReadString method fails to read certain Unicode characters correctly. For example, “John W. Gates” Day, is read as âJohn W. Gatesâ Day.
In memory I see this:
0x000001B291ADC258 e2 00 80 00 9c 00 4a 00 6f 00 68 00 6e 00 20 00 â.€.œ.J.o.h.n. .
0x000001B291ADC268 57 00 2e 00 20 00 47 00 61 00 74 00 65 00 73 00 W... .G.a.t.e.s.
0x000001B291ADC278 e2 00 80 00 9d 00 20 00 44 00 61 00 79 00 2c 00 â.€... .D.a.y
I tried opening the file with CFile::typeUnicode flag instead of CFile::typeText, but that makes things worse because the text is converted to 8 bit ASCII which is completely unreadable in a Unicode environment.
After reading the text into an edit box, I can paste in the correct text, and it displays correctly, so the problem is strictly with reading the text file.
Am I doing something wrong or does this call just not support UTF-8?
I think that an UTF-8 file will contain this: E2 80 9C 4A 6F 68 6E 20 57 2E 20 47 61 74 65 73 E2 80 9D 20 44 61 79 2C, optionally with EF BB BF signature at the beginning.. Maybe the file was created incorrectly. You can use "Open With..., Binary Editor" in Solution Explorer to check the contents of the file, if it is included into project.
If the UTF-8 file is correctly encoded, then try this approach:
FILE* f;
errno_t e = fopen_s( &f, "C:\\MyFile.txt", "rt, ccs=UTF-8" ); // opened successfully if e==0
CStdioFile file( f );
CString text;
file.ReadString( text );
. . .
file.Close( ); // also close the FILE*
@Viorel Thank you for the quick response. I was able to take your suggestion with a few minor changes that were required since I am working in a Unicode environment (fopen_s was changed to __wfopen__s for example). Here is the code I added to a static helper class to read all of the lines of a text file into a vector::
/////////////////////////////////////////////////////////////////////////////
// Reads text files in various encodings and returns a vector of strings
// for each line of text read
// Parameter csCCS allowed values are:
// UNICODE,
// UTF-8, and
// UTF-16LE
// If no value is specified for csCCS, ReadText uses ANSI encoding.
static inline vector<CString> ReadText
(
CString csPath, CString csCCS = L"UTF-8"
)
{
vector<CString> value;
FILE* pFile = nullptr;
CString csMode;
csMode.Format( L"rt, ccs=%s", csCCS );
errno_t e = _wfopen_s( &pFile, csPath, csMode );
if ( e == 0 )
{
CStdioFile file( pFile );
CString csLine;
while ( file.ReadString( csLine ))
{
value.push_back( csLine );
}
}
return value;
}
My application is used to add labels to photographs and, in this screen shot you can see the smart quotes are displaying correctly. Thanks again for your quick response.
If the pFile is not needed for other operations, it should be released using file.Close or fclose. (The destructor of CStdioFile does not seem to close it in this particular case).